Abstract

The lack of gold standard diagnostic criteria for cognitive impairment in the absence of dementia has resulted in variable nomenclature, case definitions, outcomes, risk factors, and prognostic utilities. Our objective was to elucidate the clinical correlates of conversion to dementia in a longitudinal population-based sample. Using data from the Canadian Study of Health and Aging, a machine learning algorithm was used to identify symptoms that best differentiated converting from nonconverting cognitively impaired not demented participants. Poor retrieval was the sole predictor of conversion to dementia over 5 years. This finding suggests that patients with impaired retrieval are at greater risk for progression to dementia at follow-up. Employing significant predictors as markers for ongoing monitoring and assessment, rather than as clinical markers of conversion, is recommended given the less than optimal specificity of the predictive algorithms.

Introduction

The age-associated increase in the prevalence of dementia (CSHA Working Group, 1994) highlights the importance of accurately identifying individuals who are at-risk for developing dementia. Cognitive impairment, no dementia (CIND) is a broad-based cognitive classification of cognitive impairment of insufficient magnitude to warrant a diagnosis of dementia. Unlike other cognitive classifications, such as age-associated cognitive impairment (Levy, 1994) and mild cognitive impairment (MCI; Winblad et al., 2004), the sole exclusionary criterion for CIND is a diagnosis of dementia. As a result, CIND is a classification of inclusion with a prevalence that exceeds that of MCI (Ritchie, 2008) and dementia (Graham et al., 1997). Moreover, CIND is notably heterogeneous (Peters, Graf, Hayden, & Feldman, 2004; Tuokko & Frerichs, 2000; Tuokko, Frerichs, & Kristjansson, 2001) and is hypothesized to capture a broader representation of individuals at-risk for future dementia (Peters et al., 2004). For example, in a longitudinal Canadian study, the probability of developing dementia was five times greater for individuals identified as CIND, compared with individuals with no cognitive impairment (NCI; Tuokko et al., 2003). Similarly, in the Kungsholmen Project, participants with CIND exhibited a 3-fold risk for the development of Alzheimer's disease (AD) after 3 years, compared with cognitively intact participants (Monastero, Palmer, Qiu, Winblad, & Fratiglioni, 2007). The CIND classification therefore identifies a significant number of persons at-risk for progressive pathological cognitive decline, and the study of this population may elucidate the clinical correlates of progression to dementia.

Although many studies examining the risk factors for cognitive decline focus on one or two clinical domains, Monastero and colleagues (2007) examined four research hypotheses (each reflecting a separate pathophysiological mechanism) with the goal of identifying factors related to CIND and the progression to dementia. The frailty hypothesis proposed that frailty-related factors (e.g., sensory deficits, functional dependence, chronic disease) are related to cognitive impairment. The vascular hypothesis held that cognitive impairment is related to vascular disease factors (e.g., hypertension, diabetes). The neuropsychiatric hypothesis proposed that depression, psychotropic drugs, and psychosis are risk factors for cognitive impairment. Finally, the social hypothesis held that cognitive impairment is related to limited social and physical activities. Each hypothesis was evaluated separately using logistic regression. An increased risk of CIND was reported for psychosis, polypharmacy, and hip fracture. Participants identified as CIND were found to be at-risk for progression to dementia. The authors concluded that CIND is associated with heterogeneous risk factors. These results, also, likely reflect the heterogeneous nature of the CIND case definition itself.

Despite evidence identifying a variety of risk and protective factors, much research has focused on differences in the prevalence and conversion rates for various definitions of MCI and has been limited to the examination of cognitive and functional abilities. In the current study, we have extended beyond previous research predicting conversion to dementia from CIND (i.e., Peters et al., 2004) through the unique application of a machine learning algorithm to clinical variables addressing multiple clinical domains in a population-based sample. Similarly, rather than predicting conversion to AD, we examined the clinical correlates of progression from CIND to dementia in general. We followed the lead of Monastero and colleagues (2007) in our examination of the characteristics and conditions that best discriminate between stable and progressing (i.e., to dementia) CIND participants. The purpose of the study was to identify the factors that predict different outcomes, with the goal of informing clinical practice and research toward primary and secondary interventions.

Materials and Methods

All human data included in this manuscript were obtained in compliance with the Declaration of Helsinki. Data were derived from the Canadian Study of Health and Aging (CSHA)—a longitudinal, multicenter, population-based assessment of elderly Canadians. A representative sample of 10,263 community-dwelling and institutionalized persons, aged 65 and older, was randomly selected (CSHA Working Group, 1994, 2000; McDowell et al., 2004). Participants were invited to undergo a clinical evaluation consisting of a nurse's examination, a neuropsychological evaluation, a physical exam, and bloodwork, if they met one of the following inclusion criteria: (1) Community-dwelling participants scoring <78 on the Modified Mini-Mental State Exam (3MS; Teng & Chui, 1987); (2) were unable to complete the screening evaluation due to physical or other limitations; or (3) resided in an institution. An additional random sample of participants with 3MS scores >78 underwent a clinical evaluation (CSHA Working Group, 1994; McDowell et al., 2004). All patients provided written informed consent. For individuals with dementia, assent to participate was required.

Specific to each time point, all available clinical information was used to assign a consensus classification of NCI, CIND, or dementia (according to the DSM-III-R criteria; American Psychiatric Association [APA], 1987). A classification of CIND was assigned to nondemented participants estimated to exhibit cognitive impairment (Tuokko et al., 2003). Repeated clinical evaluations, with minor modifications, were conducted at 5- and 10-year intervals (CSHA-2 and CSHA-3, respectively; McDowell et al., 2004). For deceased participants, a diagnosis of dementia was assigned in the presence of one of the following: (1) Proxy report of premortem memory difficulties, (2) death certificate identifying dementia as the underlying cause of death, or (3) >.95 probability of dementia according to a predictive algorithm (see Stewart et al., 2001, for further information).

Participants

The current study uses data from the second and the third waves of the CSHA (CSHA-2 and CSHA-3, respectively). Overall inclusion criteria required: (1) The completion of the neuropsychological component of the clinical evaluation at CSHA-2; (2) a CSHA-2 consensus diagnosis of CIND; and (3) available information regarding participants’ CSHA-3 cognitive status (i.e., not demented [NCI or CIND] and demented). To ensure a maximum and sufficient number of participants, subjects with and without missing neuropsychological data were included in the sample. The total number of participants meeting inclusion criteria was 289.

Predictor Variables

The CSHA consists of over 2000 clinical variables for each time point (Lindsay et al., 2004). Categories of predictor variables were formulated based on Monastero et al.’s (2007) frailty, vascular, neuropsychiatric, and social hypotheses. Three additional categories of predictors (demographic, cognitive, and family history) were also created. Note that, given the exploratory nature of the current study, raw data were used for continuous predictor variables. Table 1 lists the categories and associated predictor variables.

Table 1.

Variable selection by category, description, and rationale

Domain Measure Description/rationale Criterion Percent missing
 
    Unknown Disability 
Demographic Age on October 1, 1990 Participant age recorded at the start of CSHA-1, irrespective of inclusion in the clinical component of CSHA-1 Years 
 Sex Participant gender Male, Female 
 Education Participant total years of education Years 0.3 
Cognitive CAMDEX–Section H (Roth et al., 1988Structured interview with the participant's significant other (i.e., informant) addressing the subject's history Yes, No   
  1. Memory changes  2.1 
  2. Changes in general mental function  2.1 
 Wilson–Barona Index Formula Measure of premorbid intelligence Estimated IQ; Max score = 122 7.6 
 WMS Information (Wechsler, 1974Measure of long-term recall Total score; Max score = 6 0.3 
 Buschke Cued Memory Paradigm (Buschke, 1984; Tuokko & Crockett, 1989Measure of short-term memory; free and cued recall conditions Total score; Max scores: Free = 12; Total cued = 36; Total free = 36   
   1. Free recall: Trial 1 0.3 4.8 
   2. Total cued recall: Sum of cued recall trials 1–3 7.3 
  3. Total free recall: Sum of free recall trials 1–3  6.9 
 WAIS-R Block Design (Wechsler, 1981Measure of visuospatial ability, construction, and motor function (short version [items 1–9]; odd items only) Total score; Max score = 30 0.6 7.9 
 WAIS-R Similarities (Wechsler, 1981Measure of abstract thinking and verbal problem-solving (short version [items 1–13]; odd questions only) Total score; Max score = 14 0.3 
 WAIS-R Comprehension (Wechsler, 1981Verbal measure of judgment (short version [items 1–15]; odd questions only) Total score; Max score = 15 0.3 
 Rey Auditory-Verbal Learning Test (Rey, 1964Measure of short-term verbal memory Total score; Max scores: Trials 1 = 15; Trial 6 = 15; Total = 75   
 1. List A: Trial 1 3.5 2.1 
 2. List A: Trial 6 7.6 3.1 
  3. Total recall list A: Sum of trials 1–5  11.1 5.2 
 Controlled Oral Word Association Test (Spreen & Benton, 1977Measure of verbal fluency and cognitive flexibility (total number of words generated for words beginning with the letters F, A, and S). Total score 3.5 
 Animal Naming (Rosen, 1980Measure of categorical verbal fluency Total score 1.7 0.3 
 Benton Visual Retention Test (MC; Benton, 1974Measure of nonverbal memory Number correct; Max score = 16 3.0 8.0 
 WAIS-R Digit Symbol Test (Wechsler, 1981Measure of attention, problem-solving, and processing speed Total score; Max score = 93 5.2 13.1 
Frailty/Morbidity Hearing Screening for impaired hearing (“adequate” = intact; “borderline” & “inadequate” = compromised) Intact/ Compromised 1.4 N/A 
 Vision Screening for impaired vision (“adequate” = intact; “borderline” & “inadequate” = compromised) Intact, Compromised 8.7 N/A 
 Polypharmacy Total number of medications Total Max score = 24 3.1 N/A 
 CAMDEX – Section H (Roth et al., 1988Mobility Impairment Yes, No, Questionable 3.1 
Social/Physical Marital Status Marital status of participant at CSHA-2 Never married, married, common-law, divorced, separated, widowed 
Social/ Physical Place of Residence Participant's place of residence at CSHA-2 Community, Institution 
 ADLs Composite variable identifying intact (eating, dressing, self-care, toileting, and bathing activities all done without help) or compromised ADLs Intact, compromised 
 IADLs Composite variable identifying intact (able to use telephone, get to places outside of walking distance, shop, prepare meals, do housework, take medicine, and handle money without help) or compromised IADLs Intact, compromised 1.0 
Neuropsychiatric Alcohol Abuse As determined by the clinician's history evaluation Yes. No, Questionable 
 CAMDEX–Section H (Roth et al., 19881. Depressed? Yes, No 2.8 N/A 
 2. Substance Dependence—composite variable for the detection of substance dependence (if answered yes to tranquilizers, hypnotics, stimulants, barbiturates)  4.5 N/A 
  3. Changes in personality  2.1 N/A 
  4. Changes in mood  2.4 N/A 
  5. More or less irritable or angry  2.4 N/A 
  6. Complained of persecution (i.e., delusions)  2.4 N/A 
  7. Troubled by voices or visions (i.e., hallucinations)  2.8 N/A 
Vascular History Arterial Hypertension As determined by the clinician's history evaluation Yes. No, Questionable 0.3 N/A 
 Hachinski Ischemic Score Total score 0–6 or 7+ N/A 
 Heart Disease Composite score of presence or absence of heart disease (myocardial infarction, arrhythmia, angina, congestive heart failure), as determined by the clinician's history evaluation Yes, No N/A 
 CAMDEX–Section H (Roth et al., 1988Heavy smoker? Yes, No 3.5 N/A 
Family History Family History As determined by the nurse's evaluation of family history for: Yes, No  N/A 
  1. Dementia (combined AD & dementia)  6.6  
  2. Stroke with memory loss  6.6  
  3. Parkinson's disease with memory loss  7.6  
  4. Psychiatric illness (combined depression and bipolar disease)  8.0  
  5. Other psychiatric or neurological disease  8.3  
Domain Measure Description/rationale Criterion Percent missing
 
    Unknown Disability 
Demographic Age on October 1, 1990 Participant age recorded at the start of CSHA-1, irrespective of inclusion in the clinical component of CSHA-1 Years 
 Sex Participant gender Male, Female 
 Education Participant total years of education Years 0.3 
Cognitive CAMDEX–Section H (Roth et al., 1988Structured interview with the participant's significant other (i.e., informant) addressing the subject's history Yes, No   
  1. Memory changes  2.1 
  2. Changes in general mental function  2.1 
 Wilson–Barona Index Formula Measure of premorbid intelligence Estimated IQ; Max score = 122 7.6 
 WMS Information (Wechsler, 1974Measure of long-term recall Total score; Max score = 6 0.3 
 Buschke Cued Memory Paradigm (Buschke, 1984; Tuokko & Crockett, 1989Measure of short-term memory; free and cued recall conditions Total score; Max scores: Free = 12; Total cued = 36; Total free = 36   
   1. Free recall: Trial 1 0.3 4.8 
   2. Total cued recall: Sum of cued recall trials 1–3 7.3 
  3. Total free recall: Sum of free recall trials 1–3  6.9 
 WAIS-R Block Design (Wechsler, 1981Measure of visuospatial ability, construction, and motor function (short version [items 1–9]; odd items only) Total score; Max score = 30 0.6 7.9 
 WAIS-R Similarities (Wechsler, 1981Measure of abstract thinking and verbal problem-solving (short version [items 1–13]; odd questions only) Total score; Max score = 14 0.3 
 WAIS-R Comprehension (Wechsler, 1981Verbal measure of judgment (short version [items 1–15]; odd questions only) Total score; Max score = 15 0.3 
 Rey Auditory-Verbal Learning Test (Rey, 1964Measure of short-term verbal memory Total score; Max scores: Trials 1 = 15; Trial 6 = 15; Total = 75   
 1. List A: Trial 1 3.5 2.1 
 2. List A: Trial 6 7.6 3.1 
  3. Total recall list A: Sum of trials 1–5  11.1 5.2 
 Controlled Oral Word Association Test (Spreen & Benton, 1977Measure of verbal fluency and cognitive flexibility (total number of words generated for words beginning with the letters F, A, and S). Total score 3.5 
 Animal Naming (Rosen, 1980Measure of categorical verbal fluency Total score 1.7 0.3 
 Benton Visual Retention Test (MC; Benton, 1974Measure of nonverbal memory Number correct; Max score = 16 3.0 8.0 
 WAIS-R Digit Symbol Test (Wechsler, 1981Measure of attention, problem-solving, and processing speed Total score; Max score = 93 5.2 13.1 
Frailty/Morbidity Hearing Screening for impaired hearing (“adequate” = intact; “borderline” & “inadequate” = compromised) Intact/ Compromised 1.4 N/A 
 Vision Screening for impaired vision (“adequate” = intact; “borderline” & “inadequate” = compromised) Intact, Compromised 8.7 N/A 
 Polypharmacy Total number of medications Total Max score = 24 3.1 N/A 
 CAMDEX – Section H (Roth et al., 1988Mobility Impairment Yes, No, Questionable 3.1 
Social/Physical Marital Status Marital status of participant at CSHA-2 Never married, married, common-law, divorced, separated, widowed 
Social/ Physical Place of Residence Participant's place of residence at CSHA-2 Community, Institution 
 ADLs Composite variable identifying intact (eating, dressing, self-care, toileting, and bathing activities all done without help) or compromised ADLs Intact, compromised 
 IADLs Composite variable identifying intact (able to use telephone, get to places outside of walking distance, shop, prepare meals, do housework, take medicine, and handle money without help) or compromised IADLs Intact, compromised 1.0 
Neuropsychiatric Alcohol Abuse As determined by the clinician's history evaluation Yes. No, Questionable 
 CAMDEX–Section H (Roth et al., 19881. Depressed? Yes, No 2.8 N/A 
 2. Substance Dependence—composite variable for the detection of substance dependence (if answered yes to tranquilizers, hypnotics, stimulants, barbiturates)  4.5 N/A 
  3. Changes in personality  2.1 N/A 
  4. Changes in mood  2.4 N/A 
  5. More or less irritable or angry  2.4 N/A 
  6. Complained of persecution (i.e., delusions)  2.4 N/A 
  7. Troubled by voices or visions (i.e., hallucinations)  2.8 N/A 
Vascular History Arterial Hypertension As determined by the clinician's history evaluation Yes. No, Questionable 0.3 N/A 
 Hachinski Ischemic Score Total score 0–6 or 7+ N/A 
 Heart Disease Composite score of presence or absence of heart disease (myocardial infarction, arrhythmia, angina, congestive heart failure), as determined by the clinician's history evaluation Yes, No N/A 
 CAMDEX–Section H (Roth et al., 1988Heavy smoker? Yes, No 3.5 N/A 
Family History Family History As determined by the nurse's evaluation of family history for: Yes, No  N/A 
  1. Dementia (combined AD & dementia)  6.6  
  2. Stroke with memory loss  6.6  
  3. Parkinson's disease with memory loss  7.6  
  4. Psychiatric illness (combined depression and bipolar disease)  8.0  
  5. Other psychiatric or neurological disease  8.3  

Notes: ADLs = Activities of Daily Living; IADLs = instrumental ADLs; CSHA-1 = Canadian Study of Health and Aging Time 1; CSHA-2; Canadian Study of Health and Aging Time 2; CAMDEX = Cambridge Mental Disorders of the Elderly Examination; WMS = Wechsler Memory Scale; WAIS-R = Wechsler Adult Intelligence Scale – Revised; MC = multiple choice; N/A = not applicable; AD = Alzheimer's disease

Data Analyses

Sample characteristics were derived using SPSS-16 (SPSS, Inc., Chicago, IL, USA). T-tests for continuous variables and χ2 tests for nominal variables were used to identify differences in CSHA-2 sample demographic characteristics. A data-mining machine learning algorithm (Quick, Unbiased, and Efficient Statistical Tree [QUEST]; Loh & Shih, 1997; SPSS, 1999) was used to identify the symptoms that best discriminated converting (i.e., progression from CIND at CSHA-2 to dementia at CSHA-3) and nonconverting (i.e., maintained a consensus diagnosis of CIND from CSHA-2 to CSHA-3 or reverted to NCI) participants. The evaluation of 45 potential risk factors was made possible with the selection of the QUEST program. Unlike previous examinations of risk factors for cognitive impairment and decline that employ logistic regression analyses, QUEST performs exploratory statistical analyses by recursively partitioning variables to form a binary decision tree (SPSS, 1999). Recursive partitioning performs as well as logistic regression in the prediction of cognitive impairment and the decision trees are practical for clinical settings (James, White, & Kraemer, 2005). Like logistic regression, the QUEST algorithm does not assume normality, linearity, or homogeneity of variance. Unlike regression analyses, interactions between predictors are automatically calculated by QUEST. Additionally, the QUEST program is an unbiased algorithm in that, unlike other exhaustive search algorithms (e.g., CART), it does not select variables for splitting simply because they enable more splits. Thus, the resulting binary tree is limited by fewer nodes, but is superior given its unbiased node selection (Loh & Shih, 1997). Moreover, QUEST easily and rapidly manages large data sets, such as the CSHA, consisting of nominal, ordinal, and continuous variables. Another strength of this program is its handling of missing data. Using the QUEST algorithm, missing data are not included in the growth of classification trees, but are subsequently classified using surrogate predictors (SPSS, 1999). This process maximizes the number of participants included in the sample.

Binary Decision Trees

As described in the Answer Tree Algorithm Summary (SPSS, 1999), for each split in the tree, analysis of variance F-tests or Levene's test, or the Pearson χ2 test, determines the relation between the dependent variable (i.e., cognitive classifications) and continuous/nominal and ordinal variables, respectively. Splitting occurs with the predictor variable with the highest association with the dependent variable. Quadratic discriminant analysis selects the cut-point for the predictor variable. This process is repeated until either no further splitting is possible or ad hoc stopping rules are reached (SPSS, 1999). We employed the following standard ad hoc user-defined stopping rules: Recursive partitioning continued until (1) p > .05; (2) the minimum number of cases in the parent node (n = 10) or the child node (n = 5) were met; and/or (3) the tree reached a maximum depth of five levels below the root (i.e., starting) node. Pruning (i.e., the removal of unnecessary splits from the tree to avoid over-fitting) proceeded using the default standard error rule (SE = 1.0), where the subtree with the smallest risk (i.e., smallest number of misclassified participants) is grown (SPSS, 1999). Independent classification trees were developed for each category of predictor variables and all predictor variables together. The root node (i.e., Node 0) describes the summary statistics for the sample. Subsequent nodes illustrate the number of participants belonging to each classification and describe predictor split information. Dementia status at CSHA-3 served as the dependent variable for the analyses. Figures illustrating the results of the pruned tree are reported for each category of predictors, as are the associated sensitivity, specificity, positive predictive power, and negative predictive power.

Results

With the exception of the outcome variable (i.e., cognitive status at CSHA-3), all of the demographic and predictor results are based on data collected at CSHA-2.

Sample Characteristics

Of the 289 participants, 174 individuals progressed to dementia at CSHA-3. CSHA-2 demographic characteristics for converters and nonconverters are listed in Table 2. Overall, converters were significantly older, more educated, and produced significantly lower scores on the 3MS.

Table 2.

CSHA-2 sample demographics

 Nonconverters (n = 115) Converters (n = 174) t or χ2 P-value 
Mean Age on October 1, 1990 (SD75.3 (6.4) 78.7 (7.1) −4.15 <.001 
Years of Education, mean (SD8.4 (3.8) 9.9 (4.0) −3.33 .001 
3MS Score, Mean (SD81.1 (8.2) 77.1 (10.8) 3.36 .001 
Gender (% Female) 66.1 67.8  0.09 .759 
Institutionalized (%) 15.7 22.4  2.00 .157 
Married (%) 34.8 25.3  3.03 .082 
 Nonconverters (n = 115) Converters (n = 174) t or χ2 P-value 
Mean Age on October 1, 1990 (SD75.3 (6.4) 78.7 (7.1) −4.15 <.001 
Years of Education, mean (SD8.4 (3.8) 9.9 (4.0) −3.33 .001 
3MS Score, Mean (SD81.1 (8.2) 77.1 (10.8) 3.36 .001 
Gender (% Female) 66.1 67.8  0.09 .759 
Institutionalized (%) 15.7 22.4  2.00 .157 
Married (%) 34.8 25.3  3.03 .082 

Notes: SD= standard deviation; Married = married and common-law; CSHA-2; Canadian Study of Health and Aging Time 2.

Classification Trees

Cognitive predictors

The cognitive classification tree (Fig. 1) correctly identified 70.2% of participants (Table 3). The sum of Trials 1–3 on the Buschke Free Recall measure (i.e., Total Buschke Free Recall) was the best predictor of conversion. Approximately 71.3% of persons who converted to dementia at CSHA-3 had a total score of <21.9 on the Buschke Free Recall test of short-term memory (i.e., 124 of 174 participants). Approximately 21.3% of converters scored between 21.9 and 25.3 on this measure (Node 5: 37 of 174 converters scored between 21.9 and 25.3). Overall, the cognitive model more accurately identified participants who converted to dementia (Node 1 + Node 5: Demented = 161 participants = 92.5%), compared with those who did not convert (Node 6: Not Demented = 42 participants = 36.5%).

Fig. 1.

Cognitive and overall classification tree differentiating converters from nonconverters.

Notes: Revised Dem T3=Dementia status at time 3 (Canadian Study of Health and Aging Time 3 follow-up); T2=Canadian Study of Health and Aging Time 2.

Fig. 1.

Cognitive and overall classification tree differentiating converters from nonconverters.

Notes: Revised Dem T3=Dementia status at time 3 (Canadian Study of Health and Aging Time 3 follow-up); T2=Canadian Study of Health and Aging Time 2.

Table 3.

Statistical outcomes of predictive algorithms

Algorithm Sensitivity Specificity PPV NPV Correct Classification 
All Predictors 92.5 36.5 68.8 76.4 70.2 
Cognitive 92.5 36.5 68.8 76.4 70.2 
Social/Physical 76.4 50.4 70.0 50.4 66.1 
Frailty/Morbidity 100.0 100.0 60.2 
Vascular 100.0 100.0 60.2 
Neuropsychiatric 100.0 100.0 60.2 
Demographic 100.0 100.0 60.2 
Family History 100.0 100.0 59.4 
Algorithm Sensitivity Specificity PPV NPV Correct Classification 
All Predictors 92.5 36.5 68.8 76.4 70.2 
Cognitive 92.5 36.5 68.8 76.4 70.2 
Social/Physical 76.4 50.4 70.0 50.4 66.1 
Frailty/Morbidity 100.0 100.0 60.2 
Vascular 100.0 100.0 60.2 
Neuropsychiatric 100.0 100.0 60.2 
Demographic 100.0 100.0 60.2 
Family History 100.0 100.0 59.4 

Notes: PPV = positive predictive power; NPV = negative predictive power.

Social/physical predictors

The social/physical classification tree (Fig. 2) correctly identified 66.1% of participants (Table 3). Compromised instrumental activities of daily living (IADLs) significantly discriminated between persons who progressed to dementia and those that did not. Specifically, 76.4% of those who progressed to dementia had compromised IADLs.

Fig. 2.

Social/physical classification tree differentiating converters v. nonconverters.

Notes: Revised Dem T3=Dementia status at time 3 (Canadian Study of Health and Aging Time 3 follow-up); IADL=Instrumental activities of daily living at time 2.

Fig. 2.

Social/physical classification tree differentiating converters v. nonconverters.

Notes: Revised Dem T3=Dementia status at time 3 (Canadian Study of Health and Aging Time 3 follow-up); IADL=Instrumental activities of daily living at time 2.

Demographic, frailty/morbidity, neuropsychiatric, vascular, and family history predictors

None of the Demographic, Frailty/Morbidity, Neuropsychiatric, Vascular, or Family History classification trees extended beyond the root node. Each tree classified all participants as converters, thus misclassifying ∼40% of the sample.

All predictors

The classification tree using all predictors (Fig. 1) correctly classified 70.2% of participants. These results are an exact replication of the cognitive classification tree, indicating that, in this sample, performance on a measure of short-term memory is the best discriminator between stable and progressive cognitive impairment. As with the cognitive classification tree, the model generated using all predictors more accurately identified participants who converted to dementia (92.5%), compared with those who did not convert (36.5%).

Discussion

The 60.2% conversion rate from CIND to dementia over 5 years exceeds previous reports using data from the CSHA (i.e., 47%; Tuokko et al., 2003). This disparity likely reflects different sample parameters. Tuokko and colleagues examined conversion from CSHA-1 to CSHA-2, compared with CSHA-2 to CSHA-3 in the current study. Moreover, inclusion in the current study required knowledge of cognitive status at follow-up (i.e., CSHA-3). Thus, our sample may represent participants with greater baseline cognitive impairment at CSHA-2.

To our knowledge, this is the first study to predict conversion from CIND to dementia through the unique application of a machine learning algorithm to clinical variables addressing multiple clinical domains in a population-based sample. Our results confirm reports supporting the early identification of persons at-risk for dementia by means of neuropsychological (e.g., Albert, Moss, Tanzi, & Jones, 2001; Devanand, Folz, Gorlyn, Moeller, & Stern, 1997; Fleisher et al., 2007; Griffith et al., 2006; Kluger, Ferris, Golomb, Mittelman, & Reisberg, 1999; Perri, Serra, Carlesimo, Caltagirone, & the Early Diagnosis Group of the Italian Interdisciplinary Network on AD, 2007; Tabert et al., 2006; Tierney, Yao, Kiss, & McDowell, 2005; Tuokko et al., 2003) and functional (Rozzini et al., 2007) assessment. Similar to Tabert and colleagues (2006), total immediate recall using a selective reminding test (SRT) at baseline (i.e., CSHA-2) was the best predictor of conversion to dementia at follow-up. Poorer performance on the same enhanced cued recall measure (Buschke, 1984; Tuokko & Crockett, 1989) differentiated between converters and nonconverters 12–18 months after baseline assessment (Tuokko, Vernon-Wilkinson, Weir, & Beattie, 1991). The predictive utility of an SRT is reportedly attributable to the promotion of learning while controlling for attention and cognitive processing, thereby differentiating memory impairment indicative of incipient dementia from age-associated memory impairment (Grober, Lipton, Hall, & Crystal, 2000). Our results are consistent with reports of poorer episodic memory among cognitively impaired persons who convert to dementia (Perri et al., 2007; Tuokko et al., 1991) and suggest that, in the presence of other significant risk factors, poor episodic memory (specifically retrieval) supersedes all other predictors in the early identification of incipient dementia within 5 years.

The cognitive classification tree partially confirms reports of the neuropsychological prediction of incident dementia in the CSHA. In addition to an episodic memory test (short delayed recall on the RAVLT), Tierney and colleagues (2005) identified the Wechsler Memory Scale Information subtest and the Animal Naming test as significant predictors of conversion to dementia over 5 years. The discrepant findings may be attributable to Tierney and colleagues' prediction of conversion to AD, rather than overall dementia, and the exclusion of persons with neurological conditions and sensory deficits.

The absence of other significant neuropsychological predictors may reflect the population-based sample, the CIND case definition, or the 5-year follow-up interval. First, compared with clinical-based studies, population-based studies tend to include participants in the earlier stages of pathological cognitive decline (Spaan, Raaijmakers, & Jonker, 2005). Second, the heterogeneity of the CIND sample may mask less robust differences observed between converters and nonconverters in studies employing more restrictive exclusion criteria (e.g., Fleisher et al., 2007; Rozzini et al., 2007; Tabert et al., 2006; Tierney et al., 2005). Third, increasingly impaired episodic memory, executive function, and verbal intelligence are noted 7 years, 2–3 years, and just prior to a diagnosis of dementia, respectively (Grober et al., 2008). Thus, the 5-year follow-up interval may preclude cognitive impairment extending beyond the memory domain.

Similar to Rozzini and colleagues (2007), impairment of IADL function was predictive of progression to dementia, with up to 76% of CIND participants with dementia at follow-up demonstrating impaired IADLs. The absence of IADLs in the overall algorithm is attributed to its association with impaired cognitive function. For example, impaired psychomotor and memory functions are associated with impaired IADL functioning, showering, and walking among MCI/CIND persons (Tuokko, Morris, & Ebert, 2005). Thus, the absence of functional impairment in the overall model may be attributable to its association with memory function (which is represented by the total score on the Buschke Free Recall measure in the overall predictive algorithm). Importantly, the results of the social/physical algorithm reveal that functional ability at CSHA-2 had the highest specificity for conversion to dementia at follow-up and highlight the predictive utility of functional impairment in the early identification of persons at-risk for dementia.

None of the models fully differentiated converters from nonconverters. Additionally, although the sensitivity of both the cognitive and overall classification trees was 92.5%, the specificities and predictive values were less than optimal, confirming reports of limited precision associated with the neuropsychological prediction of conversion (Tian, Bucks, Haworth, & Wilcock, 2003). The low specificities and predictive values may corroborate previous reports that performance on a single neuropsychological measure may be predictive of conversion for some CIND persons but, given the heterogeneity resulting from the CIND case definition and reports of substantial intra-individual variability in neuropsychological performance, several persons who will develop dementia may not be captured using a single neuropsychological measure (Peters et al., 2004). As such, it is recommended that the identified risk factors serve as markers for ongoing monitoring and assessment of functional and cognitive decline, rather than diagnostic markers for incipient dementia.

These results should be interpreted within the context of the study's strengths and weaknesses. First, inclusion in the current study was dependent on CSHA-2 cognitive status determined by clinical judgment at the CSHA-2 consensus conference following review of all available clinical data from that time point. This feature is believed to mimic clinical diagnoses in practice. Second, it is important to note that, the outcome measure (i.e., dementia status at CSHA-3) is independent from the CSHA-2 diagnostic predictors; thus, presumably controlling for circularity within the study. Circularity occurs when an outcome measure is a product of the variables that, in turn, serve as diagnostic predictors (Tuokko & Frerichs, 2000). However, given that CSHA-3 diagnostic classifications were based on similar data obtained during the third wave of the CSHA, the current absence of gold standard diagnostic criteria for cognitive decline and dementia, and the presence of circularity when different cognitive measures are used to create and predict cognitive outcomes (given the correlations between instruments measuring similar cognitive constructs; Tuokko & Frerichs, 2000), circularity will continue to be an intrinsic feature of this and other studies aiming to predict cognitive decline and dementia. Third, it is important to note the lack of biological markers of dementia, including the apolipoprotein E (ApoE) gene. However, poor predictive values for the ApoE ε4 allele do not support its use as a diagnostic marker of conversion from CIND to dementia within 5 years (Hsiung, Sadovnik, & Feldman, 2004). Moreover, the ApoE ε4 allele accounts for only an additional 1% of the predictive accuracy in the identification of conversion from amnestic MCI to dementia (Fleisher et al., 2007) and confirms reports of its limited prediction of cognitive decline or conversion to dementia (Albert et al., 2001; Amieva et al., 2004; Devanand et al., 2005). As such, the absence of ApoE ε4 allele in the current study is not believed to limit the results or their implications. Nevertheless, inclusion of biological markers in future clinical decision trees is recommended. Finally, the number and nature (i.e., categorical versus continuous) of the selected variables may have impacted the results of the study. For example, there were more variables in the cognitive domain compared with the other domains (excepting the overall domain) and most of the cognitive predictors were measured on a continuous scale. However, intrinsically, cognitive measures are continuous, and only after comparison to standardized norms is a specific score translated into an interpretation. Therefore, the nature of the predictors is estimated to reflect the current clinical practice.

Conclusions

Poor performance on Buschke's (1984) enhanced cued recall paradigm was a significant predictor of conversion to dementia. In fact, retrieval was the sole significant predictor of conversion to dementia over 5 years, supporting earlier reports that impaired memory performance is the earliest marker of progression to dementia (Albert et al., 2001; Alexopoulos, Gimmer, Perneczky, Domes & Kurz, 2006; Grober et al., 2008). The results suggest that impairment in memory (especially retrieval) is a key feature in the architecture of pathological cognitive decline. Future research should include biological and neuroimaging markers, shorter follow-up intervals, and cost/benefit analyses of misclassification. Additionally, as the use of the QUEST algorithm is a novel approach to the field of cognitive impairment, comparison with other data-mining approaches is recommended. Using identified risk factors as markers for ongoing monitoring and assessment of functional and cognitive decline, rather than as diagnostic markers for incipient dementia, is recommended. Overall, the results of the current study, including the classification tree process itself, are believed to mimic the current diagnostic process employed by clinical practitioners.

Funding

Phases 1 and 2 of the Canadian Study of Health and Aging core study were funded by the Seniors’ Independence Research Program, through Health Canada's National Health Research and Development Program (NHRDP project #6606-3954-MC(S)); supplementary funding for analysis of the caregiver component was provided by the Medical Research Council. Additional funding was provided by Pfizer Canada Incorporated, through the Medical Research Council/Pharmaceutical Manufacturers Association of Canada Health Activity Program, NHRDP (project #6603-1417-302(R)), by Bayer Incorporated, and by the British Columbia Health Research Foundation (projects #38 (93-2) and #34 (96-1)). Core funding for Phase 3 was obtained from the Canadian Institutes for Health Research (CIHR grant # MOP-42530); supplementary funding for the caregiver component was obtained from CIHR (#MOP-43945). Additional funding was provided by Merck-Frosst and by Janssen-Ortho Inc. The study was coordinated through the University of Ottawa and Health Canada.

Conflict of interest

None declared.

Acknowledgements

A doctoral award from the Alzheimer Society of Canada Research Program provided support for the first author in preparing this manuscript. A research personnel award from the Canadian Institutes of Health Research, Institute of Aging provided support for the second author in the preparation of this manuscript as did a Michael Smith Foundation for Health Research award for research unit infrastructure held by the Centre on Aging at the University of Victoria.

References

Albert
M. S.
Moss
M. B.
Tanzi
R.
Jones
K.
Preclinical prediction of AD using neuropsychological tests
Journal of the International Neuropsychological Society
 , 
2001
, vol. 
7
 (pg. 
631
-
639
)
Alexopoulos
P.
Gimmer
T.
Perneczky
R.
Domes
G.
Kurz
A.
Progression to dementia in clinical subtypes of mild cognitive impairment
Dementia and Geriatric Cognitive Disorders
 , 
2006
, vol. 
22
 (pg. 
27
-
34
)
American Psychiatric Association
Diagnostic and statistical manual of mental disorders
 , 
1987
3rd—revised ed.
Washington, DC
Author
Amieva
H.
Letenneur
L.
Dartigues
J.-F.
Rouch-Leroyer
I.
Sourgen
C.
D'Alchée- Birée
F.
, et al.  . 
Annual rate and predictors of conversion to dementia in subjects presenting mild cognitive impairment criteria according to a population-based study
Dementia and Geriatric Cognitive Disorders
 , 
2004
, vol. 
18
 (pg. 
87
-
93
)
Benton
A. L.
Revised Visual Retention Test: Clinical and experimental applications
 , 
1974
New York
Psychological Corporation
Buschke
H.
Cued recall in amnesia
Journal of Clinical Neuropsychology
 , 
1984
, vol. 
6
 (pg. 
433
-
440
)
Canadian Study of Health and Aging Working Group
Canadian Study of Health and Aging: Study methods and prevalence of dementia
Canadian Medical Association Journal
 , 
1994
, vol. 
150
 
6
(pg. 
899
-
913
)
Canadian Study of Health and Aging Working Group
The incidence of dementia in Canada
Neurology
 , 
2000
, vol. 
55
 (pg. 
66
-
73
)
Devanand
D. P.
Folz
M.
Gorlyn
M.
Moeller
J. R.
Stern
Y.
Questionable dementia: Clinical course and predictors of outcome
Journal of the American Geriatrics Society
 , 
1997
, vol. 
45
 (pg. 
321
-
328
)
Devanand
D. P.
Pelton
G. H.
Zamora
D.
Liu
X.
Tabert
M. H.
Goodkind
M.
Predictive utility of apolipoprotein E genotype for Alzheimer disease in outpatients with mild cognitive impairment
Archives of Neurology
 , 
2005
, vol. 
62
 (pg. 
975
-
980
)
Fleisher
A. S.
Sowell
B. B.
Taylor
C.
Gamst
A. C.
Petersen
R. C.
Thal
L. J.
Clinical predictors of progression to Alzheimer disease in amnestic mild cognitive impairment
Neurology
 , 
2007
, vol. 
68
 (pg. 
1588
-
1595
)
Graham
J. E.
Rockwood
K.
Beattie
B. L.
Eastwood
R.
Gauthier
S.
Tuokko
H.
, et al.  . 
Prevalence and severity of cognitive impairment with and without dementia in an elderly population
Lancet
 , 
1997
, vol. 
349
 (pg. 
1793
-
1796
)
Griffith
H. R.
Netson
K. I.
Harrell
L. E.
Zamrini
E.
Brockington
J. C.
Marson
D.C.
Amnestic mild cognitive impairment: Diagnostic outcomes and clinical prediction over a two-year time period
Journal of the International Neuropsychological Society
 , 
2006
, vol. 
12
 (pg. 
166
-
175
)
Grober
E.
Hall
C. B.
Lipton
R. B.
Zonderman
A. B.
Resnick
S. M.
Kawas
C.
Memory impairment, executive dysfunction, and intellectual decline in preclinical Alzheimer's disease
Journal of the International Neuropsychological Society
 , 
2008
, vol. 
14
 (pg. 
266
-
278
)
Grober
E.
Lipton
R.
Hall
C.
Crystal
H.
Memory impairment on free and cued selective reminding predicts dementia
Neurology
 , 
2000
, vol. 
54
 (pg. 
827
-
832
)
Hsiung
G. R.
Sadonick
A. D.
Feldman
H.
Apolipoprotein E 84 genotype as a risk factor for cognitive decline and dementia: Data from the Canadian Study of Health and Aging
Canadian Medical Association Journal
 , 
2004
, vol. 
171
 (pg. 
863
-
867
)
James
K. E.
White
R. F.
Kraemer
H. C.
Repeated split sample validation to assess logistic regression and recursive partitioning: An application to the prediction of cognitive impairment
Statistics in Medicine
 , 
2005
, vol. 
24
 (pg. 
3019
-
3035
)
Kluger
A.
Ferris
S. H.
Golomb
J.
Mittelman
M. S.
Reisberg
B.
Neuropsychological prediction of decline to dementia in nondemented elderly
Journal of Geriatric Psychiatry and Neurology
 , 
1999
, vol. 
12
 (pg. 
168
-
179
)
Levy
R.
Aging-associated cognitive decline
International Psychogeriatrics
 , 
1994
, vol. 
6
 (pg. 
63
-
68
)
Lindsay
J.
Sykes
E.
McDowell
I.
Verreault
R.
Laurin
D.
More than the epidemiology of Alzheimer's disease: Contributions of the Canadian Study of Health and Aging
Canadian Journal of Psychiatry
 , 
2004
, vol. 
49
 (pg. 
83
-
91
)
Loh
W.-Y.
Shih
Y.-S.
Split selection methods for classification trees
Statistica Sinica
 , 
1997
, vol. 
7
 (pg. 
815
-
840
)
McDowell
I.
Xi
G.
Lindsay
J.
Tuokko
H.
Canadian Study of Health and Aging: Study description and patterns of early cognitive decline
Aging, Neuropsychology, and Cognition
 , 
2004
, vol. 
11
 (pg. 
149
-
168
)
Monastero
R.
Palmer
K.
Qiu
C.
Winblad
B.
Fratiglioni
L.
Heterogeneity in risk factors for cognitive impairment, no dementia: Population-based longitudinal study from the Kungsholmen Project
American Journal of Geriatric Psychiatry
 , 
2007
, vol. 
15
 (pg. 
60
-
69
)
Perri
R.
Serra
L.
Carlesimo
G. A.
Caltagirone
C.
the Early Diagnosis Group of the Italian Interdisciplinary Network on Alzheimer's Disease
Preclinical dementia: An Italian multicentre study on amnestic mild cognitive impairment
Dementia and Geriatric Cognitive Disorders
 , 
2007
, vol. 
23
 (pg. 
289
-
300
)
Peters
K. R.
Graf
P.
Hayden
S.
Feldman
H.
Neuropsychological characterization of cognitive-impaired-not-demented (CIND) individuals: Clinical comparison data
The Clinical Neuropsychologist
 , 
2004
, vol. 
18
 (pg. 
208
-
228
)
Rey
A.
L'examen clinique en psychologie
 , 
1964
Paris
Presses Universitaires de France
Ritchie
L. J.
Identifying cognitive impairment in older adults
 , 
2008
Victoria, British Columbia, Canada
Unpublished doctoral dissertation, University of Victoria
Rosen
W. G.
Verbal fluency in aging and dementia
Journal of Clinical Neuropsychology
 , 
1980
, vol. 
2
 (pg. 
135
-
146
)
Roth
M.
Huppert
F. A.
Tym
E.
Mountjoy
C. Q.
CAMDEX: The Cambridge Examination for Mental Disorders of the Elderly
 , 
1988
Cambridge, England
Cambridge University Press
Rozzini
L.
Chilovi
B. V.
Conti
M.
Bertoletti
E.
Delrio
I.
Trabucchi
M.
, et al.  . 
Conversion of amnestic mild cognitive impairment to dementia of Alzheimer type is independent to memory deterioration
International Journal of Geriatric Psychiatry
 , 
2007
, vol. 
22
 (pg. 
1217
-
1222
)
Spaan
P. E. J.
Raaijmakers
J. G. W.
Jonker
C.
Early assessment of dementia: The contribution of different memory components
Neuropsychology
 , 
2005
, vol. 
19
 (pg. 
629
-
640
)
Spreen
O.
Benton
A. L.
Neurosensory Center Comprehensive Examination for Aphasia: Manual of instructions
 , 
1977
Victoria, BC
University of Victoria
SPSS
AnswerTree algorithm summary
1999
 
Stewart
M.
McDowell
I.
Hill
G.
Aylesworth
R.
Estimating antemortem cognitive status of deceased subjects in a longitudinal study of dementia
International Psychogeriatrics
 , 
2001
, vol. 
13
 (pg. 
99
-
106
)
Tabert
M. H.
Manly
J. J.
Liu
X.
Pelton
G. H.
Rosenblum
S.
Jacobs
M.
, et al.  . 
Neuropsychological prediction of conversion to Alzheimer disease in patients with mild cognitive impairment
Archives of General Psychiatry
 , 
2006
, vol. 
63
 (pg. 
916
-
924
)
Teng
E. L.
Chui
H. C.
The Modified Mini Mental State (3MS) examination
Journal of Clinical Psychiatry
 , 
1987
, vol. 
48
 (pg. 
314
-
318
)
Tian
J.
Bucks
R. S.
Haworth
J.
Wilcock
G.
Neuropsychological prediction of conversion to dementia from questionable dementia: Statistically significant but not yet clinically useful
Journal of Neurology, Neurosurgery, and Psychiatry
 , 
2003
, vol. 
74
 (pg. 
433
-
438
)
Tierney
M. C.
Yao
C.
Kiss
A.
McDowell
I.
Neuropsychological tests accurately predict incident Alzheimer disease after 5 and 10years
Neurology
 , 
2005
, vol. 
64
 (pg. 
1853
-
1859
)
Tuokko
H.
Crockett
D.
Cued recall and memory disorders in dementia
Journal of Clinical and Experimental Neuropsychology
 , 
1989
, vol. 
11
 (pg. 
278
-
294
)
Tuokko
H.
Frerichs
R. J.
Cognitive impairment with no dementia (CIND): Longitudinal studies, the findings and issues
The Clinical Neuropsychologist
 , 
2000
, vol. 
14
 (pg. 
1
-
21
)
Tuokko
H.
Frerichs
R.
Graham
J.
Rockwood
K.
Kristjansson
B.
Fisk
J.
, et al.  . 
Five-year follow-up of cognitive impairment with no dementia
Archives of Neurology
 , 
2003
, vol. 
60
 (pg. 
577
-
582
)
Tuokko
H. A.
Frerichs
R. J.
Kristjansson
B.
Cognitive impairment, no dementia: Concepts and issues
International Psychogeriatrics
 , 
2001
, vol. 
13
 
Suppl. 1
(pg. 
183
-
202
)
Tuokko
H.
Morris
C.
Ebert
P.
Mild cognitive impairment and everyday functioning in older adults
Neurocase
 , 
2005
, vol. 
11
 (pg. 
40
-
47
)
Tuokko
H.
Vernon-Wilkinson
R.
Weir
J.
Beattie
B. L.
Cued recall and early identification of dementia
Journal of Clinical and Experimental Neuropsychology
 , 
1991
, vol. 
13
 (pg. 
871
-
879
)
Wechsler
D.
Wechsler Memory Scale
 , 
1974
New York:
Psychological Corporation.
Wechsler
D.
Wechsler Adult Intelligence Scale – Revised
 , 
1981
New York
Psychological Corporation
Winblad
B.
Palmer
K.
Kivipelto
M.
Kivipelto
M.
Jelic
V.
Fratiglioni
L.
, et al.  . 
Mild cognitive impairment—beyond controversies, towards a consensus: Report of the International Working Group on Mild Cognitive Impairment
Journal of Internal Medicine
 , 
2004
, vol. 
256
 (pg. 
240
-
246
)