Abstract

This study examined the relationship between phonemic and semantic (category) verbal fluency and cognitive status in the Wisconsin Registry for Alzheimer’s Prevention (WRAP), a longitudinal cohort enriched for family history of Alzheimer’s disease. Participants were 283 WRAP subjects (age 53.1[6.5] years at baseline); who had completed three waves of assessment, over ∼6 years and met psychometric criteria either for “cognitively healthy” (CH) or for psychometric amnestic mild cognitive impairment (aMCI) using an approach that did not consider fluency scores. CH and aMCI groups differed significantly on phonemic total scores, category total scores, phonemic switching, and category mean cluster size. These results suggest that measures of both phonemic and semantic fluency yield lower scores in persons with evidence of psychometric aMCI compared with those who are CH. Differences have not previously been reported in a group this young, and provide evidence for the importance of including multiple verbal fluency tests targeting preclinical Alzheimer’s disease.

Introduction

The amnestic form of mild cognitive impairment (aMCI) is often considered a possible transitional state between healthy aging and Alzheimer's disease (AD) (Albert et al., 2011; Morris & Price, 2001; Petersen, 2004). While impairment in episodic memory is seen most commonly in MCI patients who subsequently progress to AD, the nature of semantic memory deficits in preclinical AD is less clear. Semantic memory refers to the permanent store of representational knowledge including facts, concepts, as well as words and their meaning (Chertkow & Bub, 1990; Hodges et al., 1992) and normal performance on semantic memory tests relies on both intact memory stores and unencumbered access to the information.

Among the neuropsychological measures most frequently used to assess semantic memory are verbal fluency tests, or speeded word-list generation. In addition to assessing knowledge and memory, verbal fluency tests may also tap other cognitive processes including components of executive function such as divergent reasoning to generate category exemplars, maintaining cognitive set of the overarching category while flexibly searching subcategories, working memory (WM) for prior responses, and inhibition of non-category items. Verbal fluency tests are of two types: phonemic (letter) fluency tests, where words are generated that begin with a given letter such as “C”; and category fluency tests, where words are generated that are exemplars of a given category, such as “animals,” both within a 60-s time limit. Typically, the primary outcome variable is the total number of words produced. While both phonemic and category fluency total word performances reveal declines in persons diagnosed with probable AD (Clark et al., 2009; Gomez & White, 2006), category fluency has been found to be disproportionately impaired compared with phonemic fluency (Barr & Brandt, 1996; Troyer et al., 1998), although the literature on this discrepancy is conflicting. For example, Clark and colleagues (2009) compared three groups longitudinally (cognitively normal, preclinical AD, and prevalent AD) and reported that category fluency scores declined at a faster rate than phonemic fluency in all three groups, with steeper declines for the preclinical AD group than the cognitively normal group. Phonemic fluency remained intact for the cognitively normal and preclinical AD groups, but showed concurrent decline with category fluency in the prevalent AD group. The effect of disparate letter and category performance may be heightened in aMCI, in that some studies show that phonemic fluency remains relatively intact, while category fluency measures show impairment (Adlam, Bozeat, Arnold, Watson, & Hodges, 2006; Murphy, Rich, and Troyer, 2006). Contrarily, other studies show concurrent declines in both tasks for people with MCI. For example, Nutter-Upham and colleagues (2008) found declines in both phonemic and semantic fluency in a group with MCI versus healthy controls, and Rinehardt and colleagues (2014) found that individuals with MCI performed worse on phonemic fluency than category fluency. Interestingly, Brandt and Manning (2009) found an effect of type of letter or category chosen, suggesting that generalizations about fluency performance across stimuli may not always be accurate.

Fluency performance involves multiple brain processes and brain regions across the two types of fluency tasks. Impaired fluency has been observed in the context of lesions in the frontal (Coslett, Bowers, Verfaelli, & Heilman, 1991; Miller, 1984) and temporal (Corcoran & Upton, 1993; Newcombe, 1969) lobes. Functional magnetic resonance imaging and positron emission tomography studies have also implicated multiple brain regions during fluency tasks (see Troyer, Moscovitch, & Winocur, 1997 for review). Whereas phonemic fluency tasks are proposed to rely mainly upon executive functioning and pre-frontal lobe processes (Miceli, Caltagirone, Gainotti, Masullo, & Silveri, 1981; Newman, Trivedi, Bendlin, Ries, & Johnson, 2007), category fluency tasks are thought to rely on the additional processes of successful search within the semantic store and efficient temporal-lobe functioning in order to retrieve from that store (Newcombe, 1969).

It has been suggested that the total word measures of verbal fluency tasks do not adequately address the underlying cognitive processes of fluency tasks. Therefore, Troyer and colleagues (1997) described a two-component model of verbal fluency which attempted to address these processes: “Clustering,” which is the production of words within phonemic or semantic categories and “switching,” which is the ability to shift efficiently to a new subcategory. Clustering is theorized to rely upon temporal-lobe processes such as semantic storage, and switching is purported to be a frontal lobe process that includes executive abilities such as strategic search, cognitive flexibility, and set-shifting (Troyer et al., 1997).

Over the past decade, studies of the utility of clustering and switching measures have generated conflicting results. Some have found that a decrease in number of switches during category fluency tasks is a better reflection of decline in early AD and Parkinson's dementia (PD) than is the total number of words (Tröster et al., 1998; Troyer, Moscovitch, Winocur, Alexander, & Stuss, 1998; Troyer, Moscovitch, Winocur, Leach, & Freedman, 1998). Raoux and colleagues (2008) reported similar findings, examining subjects from the Paquid longitudinal study, analyzing 51 cases of possible and probable AD at onset, and examining test performance 2 and 5 years before onset. They found a significantly lower switching index in future AD subjects than in elderly controls at all intervals, including 5 years pre-onset. Mean cluster size did not discriminate the two groups. This phenomenon was attributed to impaired shifting abilities (frontal lobe processes) as opposed to a degradation of the semantic store (temporal lobe). More recently, Clark and colleagues (2012) used a specific category/switching condition taken from the Delis–Kaplan Executive Function System (Delis et al., 2001) in a prospective study of 71 individuals (mean age 69), and found that participants performed significantly worse on the verbal fluency category switching condition 1 year before evidenced decline on the Dementia Rating Scale (Mattis, 1988). The authors concluded that switching between semantic categories may rely upon both semantic store integrity and executive function, therefore serving as a uniquely sensitive measure predictive of decline.

Conversely, a study by Price and colleagues (2012) looked specifically at category fluency tasks in aMCI versus healthy older adults, and found that total number of words was reduced in aMCI, and that the aMCI group produced smaller cluster sizes. The investigators described this phenomenon as a difficulty in not only isolating semantic categories but also a loss in the ability to associate additional items within that category. No significant differences in switching abilities were found. This effect of smaller cluster size and intact switching abilities was also found by Haugrud, Lanting, and Crossley (2010) comparing early AD participants with healthy older adults. Additionally, Fagundo and colleagues (2008) compared three groups (subjects with memory complaints, subjects with probable AD, and healthy controls) at baseline and 2-year follow-up, and found that AD development was better predicted by reduced cluster size than by total number of words generated or by switching. Other studies report no differences in semantic cluster size or number of switches among elderly controls, AD patients, and PD patients with and without dementia (Epker, Lacritz, & Munro Cullum, 1999).

An under-investigated group is that of persons at familial/genetic risk of developing AD. Rosen and colleagues (2005) examined verbal fluency scores in healthy older adults (aged 50–75) at risk for developing AD based on Apolipoprotein E (APOE) genotype from the Biomarkers for Older Controls at Risk for Dementia study. They found significantly fewer words generated and longer times to access clusters on a 10-min category fluency task among those with the APOE epsilon 4 allele (APOE4) versus controls who were APOE4 negative. However, on the traditional 1-min phonemic and category fluency tasks, there were no between-group differences in total number of words produced, cluster size, or switching. While some initial work has investigated “psychometric MCI” (a construct in which longitudinal methods reveal early, preclinical decline but psychometric criteria is not met for clinical MCI based on published norms (also referred to in the literature as “pre-MCI”), the literature in this area with respect to verbal fluency is lacking. Loewenstein and colleagues (2012) found that declines in category fluency at the pre-MCI stage were predictive of progression to clinical MCI and dementia; however, phonemic fluency was not one of the examined variables.

There have been methodological limitations to some of these prior investigations. Studies examining the predictive value of clustering and switching for aMCI or early AD have had small sample sizes, and/or have been limited to groups with mean ages >70 years (Clark et al., 2009; Fagundo et al., 2008; Price et al., 2012).

The current study examined the relationship between phonemic and category verbal fluency and aMCI status in the Wisconsin Registry for Alzheimer's Prevention (WRAP), a unique sample of late middle-aged participants at increased risk for sporadic AD due to parental history. We hypothesized that participants who were CH would differ on verbal fluency outcomes from a subset with psychometric evidence of aMCI (i.e., the subset did not necessarily have a clinical diagnosis of aMCI but did meet statistical criteria for early decline as described in Methods). While some initial work has investigated psychometric MCI (also referred to in the literature as “pre-MCI”), the literature in this area with respect to verbal fluency is lacking. Loewenstein and colleagues (2012) found that declines in category fluency at the pre-MCI stage were predictive of progression to clinical MCI and dementia. We also hypothesized that effects of aMCI status on verbal fluency outcomes would be most pronounced among the aMCI subgroup that also had evidence of declines in executive function. We hypothesized further that measures of clustering and switching might prove more sensitive to early aMCI than traditional total word scores. However, given the conflicting outcome of prior studies, we did not have a specific prediction regarding the comparative sensitivity of clustering versus switching in early aMCI.

Methods

Sample

WRAP is a longitudinal cohort (n > 1500) enriched for positive parental family history of AD. WRAP's study design and assessment protocols are described in detail elsewhere (La Rue et al., 2008; Sager, Hermann, & La Rue, 2005). In brief, participants in the family history group have at least one biological parent with either autopsy-confirmed or probable AD as defined by National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer's Disease and Related Disorders Association research criteria (McKhann et al., 1984). Participants were English speaking and cognitively intact at baseline; most were between the ages of 40 and 65 (M = 54) at the start of the study. At each visit, an extensive neuropsychological battery was given which included a phonemic fluency measure acquired at all visits and a semantic fluency measure added at the third wave of data collection. Participants included in the present study were 283 WRAP subjects who had completed three waves of assessment and had data for both phonemic and category verbal fluency measures.

Verbal Fluency Outcomes

Primary outcomes were based on phonemic and category fluency measures obtained at Wave 3 of testing. Phonemic fluency was assessed by instructing the participants to name as many words as possible that begin with the letters C, F, and L (Benton et al., 1994) within 60 s for each letter. Participants were instructed to name as many animals as possible in 60 s to assess category fluency. Each verbal fluency test yielded three summary scores as shown in Table 1. The cluster and switching scores were obtained using the guidelines described by Troyer and colleagues (1997). Briefly, clusters were identified by two or more successively generated words belonging to the same semantic or phonemic subcategory, and switches were calculated as the number of times a subject changed from one cluster to another. Repetitions and intrusions were not included in the total raw score or in the clustering/switching scores (Gomez & White, 2006; Raoux et al., 2008). Inter-rater reliability for this coding procedure was assessed by comparing the scores given by three independent raters for at least 25% of total cases; the intraclass coefficients ranged from 0.74 to 0.86 for these ratings and suggest good to excellent inter-rater reliability (Cicchetti & Sparrow, 1981).

Table 1.

Verbal fluency variables, cognitive factors, and corresponding cognitive tests three cognitive domains identified in the Wisconsin Registry for Alzheimer's Prevention battery

DOMAIN: variables Cognitive test 
Verbal fluency 
Phonemic fluency, switching, and cluster size Controlled Oral Word Association Test (Benton, Hamsher, & Sivan, 1994
Category fluency, switching, and cluster size Animal Naming task 
Episodic memory 
Immediate memory Rey Auditory Verbal Learning test—Trials 1 and 2 (Lezak, Howieson, & Loring, 2004
Verbal learning and memory Rey Auditory Verbal Learning test—Trials 3 through 5 and delayed recall 
Executive function 
Working memory Digit Span Forward, Digit Span Backward, and Letter-number Sequencing (Wechsler Adult Intelligence Scale-III) (Wechsler, 1999
Speed and flexibility Stroop Color-Word test—interference trial (Trenerry, Crosson, DeBoe, & Leber, 1989
 Trail-Making test—Parts A and B (Reitan & Wolfson, 1993
DOMAIN: variables Cognitive test 
Verbal fluency 
Phonemic fluency, switching, and cluster size Controlled Oral Word Association Test (Benton, Hamsher, & Sivan, 1994
Category fluency, switching, and cluster size Animal Naming task 
Episodic memory 
Immediate memory Rey Auditory Verbal Learning test—Trials 1 and 2 (Lezak, Howieson, & Loring, 2004
Verbal learning and memory Rey Auditory Verbal Learning test—Trials 3 through 5 and delayed recall 
Executive function 
Working memory Digit Span Forward, Digit Span Backward, and Letter-number Sequencing (Wechsler Adult Intelligence Scale-III) (Wechsler, 1999
Speed and flexibility Stroop Color-Word test—interference trial (Trenerry, Crosson, DeBoe, & Leber, 1989
 Trail-Making test—Parts A and B (Reitan & Wolfson, 1993

Predictors, Covariates, and Other Variables

The primary predictor in our analysis was the aMCI status group at Wave 3 assessment (i.e., CH versus aMCI). The details of group assignment using cognitive factors representing memory and executive function domains are reported elsewhere (Dowling, Hermann, La Rue, & Sager, 2010; Koscik et al., 2014; cognitive factors are shown by domain in Table 1). Briefly, aMCI status was determined by examining individual performance at each wave relative to robust internal norms that adjusted for age, gender, and literacy level. Individuals with neurological diagnoses including meningitis, stroke, epilepsy, multiple sclerosis, and Parkinson's disease were excluded from the analyses. The CH group consisted of those who had no memory or executive function scores falling >1.5 SD below robust norms at any of the three assessments, while the aMCI group consisted of participants who had episodic memory factor scores >1.5 standard deviation below the robust norms on at least 2 of 3 assessment waves but concurrent normal independent activities of daily living (Koscik et al., 2014). To test the second hypothesis, the aMCI group was divided into those who only showed evidence of declines in memory versus those with additional evidence of decline in executive function. The latter group included those with one or more WM or speed and flexibility (SF) scores >1.5 SD below the sample-based robust norms on at least 2 of 3 assessment waves. Covariates included age, gender, and literacy level as indicated by the Wide Range Achievement Test (Wilkinson, 1993) standard reading score. Other variables included demographic information (e.g., age, gender, and education level), depressive symptoms as measured by the Centers for Epidemiologic Studies Depression scale (CES-D, Radloff (1977)), parental family history of AD (family history positive) or presence of one or more APOE4 allele, obtained as described elsewhere (e.g., Johnson et al., 2011).

Statistical Analysis

The baseline characteristics of the CH and aMCI groups were compared using t-tests and χ2 tests. To facilitate comparisons across verbal fluency variables, each verbal fluency outcome was standardized to be distributed approximately normally with mean = 0 and SD = 1 (i.e., ∼N(0,1), “z-scores”). For descriptive purposes, partial correlation coefficients were obtained for correlations among all verbal fluency outcomes, the two memory factor scores (immediate memory [IM] and verbal learning and memory [VLM]), and the two previously mentioned executive function factor scores (SF and WM), after adjusting for age, gender, and literacy level. To test our first hypothesis, we used analysis of covariance (ANCOVA) to examine differences between the CH and aMCI groups in Wave 3 verbal fluency measures after adjusting for covariates. For our second hypothesis, the additional effects of executive function on the relationship between aMCI status and verbal fluency outcomes were examined by creating a three-level cognitive status predictor with levels CH, aMCI only, and aMCI plus executive function deficit. When the predictor was significant, all pairwise contrasts were examined. Secondary analyses included: testing for differences between CH and aMCI groups on baseline phonemic verbal fluency measures, repeating the primary analyses examining the influence of depressive symptoms and performance IQ (PIQ) on verbal fluency outcomes, and reviewing consistency of results of primary analyses after including a random effect for family. All analyses were performed in SAS v9.3 and statistical significance was α = 0.05 unless otherwise noted.

Results

Sample characteristics are displayed by cognitive status in Table 2. The two groups did not differ in terms of age at baseline or Wave 3, education, family history of AD, or APOE4 status. The aMCI group had more males, lower full-scale intelligence quotient (FSIQ), higher literacy scores, and higher depression scores on the CES-D. The aMCI group also had lower AVLT total and delayed recall scores. The mean CES-D score was well below the screening cut-off value of 16 for both groups.

Table 2.

Sample characteristics, by amnestic mild cognitive impairment (aMCI) group

 Cognitively healthy (n = 237) aMCI (n = 46) p-value 
Baseline age, mean (SD) 52.9 (6.6) 54.3 (5.9) .20 
Wave 3 age, mean (SD) 59.3 (6.5) 60.7 (5.7) .17 
Follow-up interval, mean (SD) years 6.4 (0.85) 6.5 (0.72) .63 
Gender, n (%) female 173 (73.0) 26 (56.5) .03 
Baseline immediate memory 0.47 (0.93) −0.80 (0.79) <.0001 
Baseline verbal learning and memory 0.49 (0.75) −1.21 (0.91) <.0001 
Baseline speed and flexibility 0.35 (0.86) −0.28 (0.89) <.0001 
Baseline working memory 0.30 (0.96) −0.32 (0.90) <.0001 
Baseline FSIQ 114.8 (9.2) 111.8 (9.5) .04 
Baseline PIQ 113.3 (10.3) 110.0 (10.1) .05 
Baseline WRAT standard score 105.3 (9.5) 109.6 (9.0) .004 
Baseline AVLT total 55.1 (6.4) 41.6 (6.2) <.0001 
Baseline AVLT delayed recall 7.3 (2.8) 11.5 (2.3) <.0001 
Education, n (%) with ≥BA 146 (61.6) 29 (63.0) .85 
Baseline CES-D, mean (SD) 5.1 (5.4) 7.2 (6.5) .02 
Family history of AD, n (%) 163 (68.8) 29 (63.0) .45 
APOE ϵ4, n (%) 96 (40.5) 13 (28.3) .14 
Baseline self-reported memory problem    
No, n (%) 142 (60.2) 24 (52.2) .49 
Yes, n (%) 49 (20.8) 13 (28.3)  
Do not know, n (%) 45 (19.1) 9 (19.6)  
Wave 3 self-reported memory problem    
No, n (%) 147 (62.3) 28 (60.9) .98 
Yes, n (%) 45 (19.1) 9 (19.6)  
Do not know, n (%) 44 (18.6) 9 (19.6)  
 Cognitively healthy (n = 237) aMCI (n = 46) p-value 
Baseline age, mean (SD) 52.9 (6.6) 54.3 (5.9) .20 
Wave 3 age, mean (SD) 59.3 (6.5) 60.7 (5.7) .17 
Follow-up interval, mean (SD) years 6.4 (0.85) 6.5 (0.72) .63 
Gender, n (%) female 173 (73.0) 26 (56.5) .03 
Baseline immediate memory 0.47 (0.93) −0.80 (0.79) <.0001 
Baseline verbal learning and memory 0.49 (0.75) −1.21 (0.91) <.0001 
Baseline speed and flexibility 0.35 (0.86) −0.28 (0.89) <.0001 
Baseline working memory 0.30 (0.96) −0.32 (0.90) <.0001 
Baseline FSIQ 114.8 (9.2) 111.8 (9.5) .04 
Baseline PIQ 113.3 (10.3) 110.0 (10.1) .05 
Baseline WRAT standard score 105.3 (9.5) 109.6 (9.0) .004 
Baseline AVLT total 55.1 (6.4) 41.6 (6.2) <.0001 
Baseline AVLT delayed recall 7.3 (2.8) 11.5 (2.3) <.0001 
Education, n (%) with ≥BA 146 (61.6) 29 (63.0) .85 
Baseline CES-D, mean (SD) 5.1 (5.4) 7.2 (6.5) .02 
Family history of AD, n (%) 163 (68.8) 29 (63.0) .45 
APOE ϵ4, n (%) 96 (40.5) 13 (28.3) .14 
Baseline self-reported memory problem    
No, n (%) 142 (60.2) 24 (52.2) .49 
Yes, n (%) 49 (20.8) 13 (28.3)  
Do not know, n (%) 45 (19.1) 9 (19.6)  
Wave 3 self-reported memory problem    
No, n (%) 147 (62.3) 28 (60.9) .98 
Yes, n (%) 45 (19.1) 9 (19.6)  
Do not know, n (%) 44 (18.6) 9 (19.6)  

Note: p-values are based on t-tests for continuous data and χ2-tests or Fisher's exact tests for categorical data.

Partial correlations among the verbal fluency outcomes and cognitive factor scores representing memory and executive function are depicted in Table 3 after adjusting for age, gender, and literacy. Among the phonemic fluency variables, the total and switching variables were correlated with all episodic memory and executive function factors, with correlations ranging from 0.19 to 0.32 (p < .0001 to p = .0012), while the mean cluster size variable did not significantly correlate with any of the four factors. The category fluency total score also correlated with all episodic memory and executive function factors, with correlations ranging from 0.17 to 0.30 (p < .0001 to p = .0037). In contrast, the category fluency cluster variable had non-zero correlation only with VLM (r = .15, p = .011) and the category fluency switching variable correlated significantly with WM (r = .14, p = .016).

Table 3.

Partial correlations among verbal fluency, episodic memory, and executive function variables

 Phonem
 
Semantic
 
Episodic Mem
 
Exec Fnc
 
Clust Switch Total Clust Switch IM VLM SF WM 
Phonem 
 Total 0.16 0.78 0.35 0.13 0.15 0.29 0.23 0.29 0.32 
 Clust  −0.47 0.05 0.07 0.06 0.10 0.01 0.03 0.01 
 Switch   0.32 0.14 0.13 0.20 0.19 0.24 0.28 
Semantic 
 Total    0.28 0.44 0.24 0.27 0.30 0.17 
 Clust     −0.70 0.06 0.15 0.10 0.03 
 Switch      0.11 0.05 0.11 0.14 
Episodic Mem 
 IM       0.67 0.31 0.26 
 VLM        0.34 0.23 
Exec Fnc 
 SF         0.37 
 Phonem
 
Semantic
 
Episodic Mem
 
Exec Fnc
 
Clust Switch Total Clust Switch IM VLM SF WM 
Phonem 
 Total 0.16 0.78 0.35 0.13 0.15 0.29 0.23 0.29 0.32 
 Clust  −0.47 0.05 0.07 0.06 0.10 0.01 0.03 0.01 
 Switch   0.32 0.14 0.13 0.20 0.19 0.24 0.28 
Semantic 
 Total    0.28 0.44 0.24 0.27 0.30 0.17 
 Clust     −0.70 0.06 0.15 0.10 0.03 
 Switch      0.11 0.05 0.11 0.14 
Episodic Mem 
 IM       0.67 0.31 0.26 
 VLM        0.34 0.23 
Exec Fnc 
 SF         0.37 

Notes: Phonem = phonemic fluency set of variables based on CFL; Semantic = category fluency set of variables based on Animal Naming test; Episodic Mem = episodic memory; Exec Fnc = executive function; Clust = mean cluster size; Switch = category switching; IM = immediate memory; VLM = verbal learning and memory; SF = speed and flexibility; WM = working memory. Values that are italicized and in gray scale did not differ significantly from 0 (i.e., p > .05)

The Effect of aMCI on Verbal Fluency

Figure 1a depicts the adjusted means and p-values from ANCOVA's of aMCI status versus CH status on each verbal fluency z-score. The CH and aMCI groups differed significantly on four of six verbal fluency measures: CFL total, Animal Total, CFL Switching, and Animal Cluster Size, after adjusting for covariates (phonemic fluency [CFL total]: F(1, 278) = 23.6, p < .0001, ηp2=0.0784; category fluency [Animal Naming total]: F(1, 278) = 11.1, p = .001, ηp2=0.0385). The adjusted mean (s.e.) for the phonemic fluency total scores corresponding to the depicted z-scores were 44.8 (0.79) words for the CH group compared with 38.30 (0.77) words for the aMCI group. Similarly, the category fluency total scores corresponding to the z-scores in Fig. 1a were 23.6 (0.36) for the CH group compared with 21.0 (0.77) in the aMCI group. The adjusted mean (s.e.) scores for the CFL Switching scores corresponding to the depicted z-scores were 30.83(0.54) and 25.69(1.13) in the CH and aMCI groups, and the adjusted mean (s.e.) scores for the Animal Naming Cluster Size were 1.58(0.06) and 1.27(0.12).

Fig. 1.

The significance of cognitive status as a predictor for each of the six measures of verbal fluency. All figures depict means and standard errors after adjusting for age, gender, and literacy. (c) Also adjusts for baseline CFL total score. In (a), cognitive status is cognitively healthy (CH) versus amnestic mild cognitive impairment (aMCI). In (b), the aMCI group is further divided into subsets with and without deficits in executive function. (c) compares CH and aMCI after also adjusting for baseline CFL total score. (a) Effects of memory deficits on verbal fluency. (b) Effects of memory and executive function deficits on verbal fluency. (c) Effects of memory deficits on verbal fluency, adjusting for baseline verbal fluency.

Fig. 1.

The significance of cognitive status as a predictor for each of the six measures of verbal fluency. All figures depict means and standard errors after adjusting for age, gender, and literacy. (c) Also adjusts for baseline CFL total score. In (a), cognitive status is cognitively healthy (CH) versus amnestic mild cognitive impairment (aMCI). In (b), the aMCI group is further divided into subsets with and without deficits in executive function. (c) compares CH and aMCI after also adjusting for baseline CFL total score. (a) Effects of memory deficits on verbal fluency. (b) Effects of memory and executive function deficits on verbal fluency. (c) Effects of memory deficits on verbal fluency, adjusting for baseline verbal fluency.

AMCI Effects, with or Without Deficits in Executive Function, on Verbal Fluency

Of the 46 participants who met criteria for psychometric aMCI, 25(54.3%) also had evidence of deficits in executive function (i.e., WM or SF, as described in the methods). Figure 1b depicts the adjusted means and p-values from ANCOVA's of the three-level cognitive status variable versus each verbal fluency z-score. The three-level cognitive status predictor was significant for phonemic fluency total (F(2, 277) = 13.0, p < .0001, ηp2=0.0858), phonemic fluency switching (F(2, 277) = 9.71, p < .0001, ηp2=0.0655), and category fluency total (F(2, 277) = 5.83, p = .0033, ηp2=0.0404). Follow-up pairwise contrasts showed that all pairs differed significantly on the two phonemic fluency measures while the CH and aMCI + executive dysfunction group also differed on the category fluency total score.

In secondary analyses, we used ANCOVA to examine whether the phonemic verbal fluency measures obtained at baseline differed across aMCI status at Wave 3. After adjusting for covariates, the aMCI group differed significantly from the CH group on baseline CFL total words and switching but not mean cluster size. The adjusted means (s.e.) for the CH versus aMCI groups were 45.6 (0.78) versus 39.2 (1.63) for CFL total and 30.0 (0.88) versus 25.3 (1.44) for CFL switching. Since the CH and aMCI groups differed in baseline phonemic fluency, we reran the primary analyses using baseline CFL total score as a covariate. The pattern of significance remained largely unchanged, although the differences between the adjusted mean were reduced between the CH and aMCI groups after adjusting for baseline CFL scores. The significance of cognitive status and the adjusted means are shown for each verbal fluency outcome in Fig. 1c. Similarly, we reran the analyses depicted in Fig. 1 including PIQ as a covariate and all conclusions remained the same. In additional secondary analyses, we added the Wave 3 CES-D scores as a covariate and reran the models; the pattern of significance in the cognitive status variable was unchanged. Finally, we reran all primary models including a variable for family effects; again, the effects of the aMCI variable remained the same. Both APOE4 and Family History status were examined, and neither variable was a significant predictor of verbal fluency status.

Discussion

Our analyses indicate that measures of both phonemic and category fluency show lower scores in persons with psychometric evidence of aMCI compared with those who are CH, confirming the initial hypothesis. The CH and aMCI groups differed significantly on four of six verbal fluency measures, including both phonemic (CFL) total and category (animal) total scores, switching on the phonemic task, and mean cluster size on the category task. Further, consistent with our second hypothesis, decreased total fluency and switching were most pronounced in an aMCI subgroup with executive function deficits in addition to memory impairment.

Our study showed no significant association with either APOE or family history status and depressed fluency scores on the traditional 1-min fluency tasks. Similarly, while Rosen and colleagues (2005) did find reduced cluster sizes in APOE4 carriers (mean age = 62) in a 10-min category fluency task, no differences were found in the traditional 1-min fluency tasks between carriers and non-carriers.

Traditional total scores differed between the groups for both phonemic and category fluency, and these differences held after adjusting for baseline fluency scores. Differences have not previously been reported in a group this young (Amieva et al., 2005; Clark et al., 2009—mean ages at baseline = 74 and 71, respectively); further, phonemic fluency measures have yielded conflicting results in the literature for early memory impairment (Adlam et al., 2006; Clark et al., 2009; Murphy et al., 2006; Nutter-Upham et al., 2008; Rinehardt et al., 2014).

The psychometric MCI group in our study performed better on both fluency tasks when compared with aMCI groups cited previously. For example, Murphy and colleagues (2006) reported mean category fluency of 15.4(4.6) and letter fluency of 37.1 (9.5) for an aMCI group, while the psychometric MCI group in our sample had a mean category fluency score of 21.0 (0.77) and letter fluency of 38.30 (0.77). Similarly, from the Alzheimer's disease neuroimaging initiative ADNI, Johnson and colleagues (2012) report a mean category fluency score of 15.9 (4.9) in an aMCI group (letter fluency not examined). The differences in verbal fluency scores between our psychometric MCI group and previously published studies of aMCI groups indicate a possibility of detecting verbal fluency deficits in an earlier, preclinical phase of decline.

That a decreased switching effect was seen for the phonemic fluency task—and was more pronounced in the aMCI subgroup with executive dysfunction in addition to memory impairment—may concur with the concept of “switching deficit” described by Troyer and Moscovitch (2006). The deficit is described as a degradation in the processing of strategic searching, disengaging from an activated cluster, and/or producing a new cluster. While the effect of a switching deficit has been previously noted in a preclinical population on a category fluency task (Raoux et al., 2008), significantly reduced phonemic switching has not been reported in a younger, preclinical group such as WRAP. However, the phonemic total words and phonemic switching scores were highly correlated (r = .78), possibly suggesting that decreased switching is simply a function of decreased total words. Conversely, the mean cluster size and switching variables on the category fluency task were not strongly correlated with the total scores. It is therefore possible that the evidence for a two-component model exists more for semantic tasks than phonemic. Further, greater impairments on category fluency than phonemic have been more commonly reported for people with aMCI or AD (Crossley et al., 1997; Henry, Crawford, & Phillips, 2004; Murphy et al., 2006). The differences in both category and phonemic fluency we observed could reflect deficits that occur at an earlier point in the continuum of preclinical AD, or alternatively, may stem from other subtle brain changes unrelated to AD.

In this younger and more cognitively intact population, the other verbal fluency variables (mean cluster size for phonemic fluency, switching for category fluency) did not provide added utility. Reductions in mean cluster sizes in both types of verbal fluency tests have been evidenced in patients diagnosed with AD (Gomez & White, 2006; Tröster et al., 1998; Troyer et al., 1997), and have been explained by the disorganization or degradation of the semantic store (Henry et al., 2004; Hodges et al., 1992). Possibly, these variables will provide additional benefit when deficits are larger, or when they can be further examined longitudinally.

Although verbal IQ was slightly lower in the aMCI group compared with the CH group, WRAT reading was slightly higher in the aMCI group; therefore, we would not expect the aMCI group to be at a disadvantage on fluency tasks due to generally lower verbal ability. Also, although the difference in depression symptoms on the CESD was statistically significant, mean scores for both groups were well below published cut scores for clinical depression.

Limitations of our study include the inherent subjectivity of the clustering/switching scoring system, particularly for the category fluency condition. Some clusters were apparent upon consensus scoring among three raters but were not included in the Troyer et al. (1997) original scoring system. While these additions were documented and followed throughout the scoring process, the myriad possible word combinations may be difficult to capture. The use of a specific semantic category switching condition (e.g., instructing participants to switch between different semantic categories [fruits versus furniture]) as did Clark et al. (2012), may be a useful measure to examine in this group in future analyses. Further, in light of some studies showing differential performance across types of fluency tasks (e.g., Brandt & Manning, 2009), future analyses may also include analyzing each letter (in this case, C, F, and L) to test for differences among stimuli.

The current study may be viewed as an extension of Koscik and colleagues (2014)'s description of robust internally based norms (which were based on immediate memory and verbal learning and memory scores) to verbal fluency. While such continuity has advantages, there is also the possibility of a cohort effect or circularity even though fluency was not considered in making the psychometric MCI distinction. The fact that fluency measures are only modestly correlated with memory (0.29 for letter fluency, 0.24 for category fluency) supports the notion that this domain explains unique variance and adds clinical relevance and is not merely a circular diagnostic effect.

While individuals with neurological conditions were originally excluded from analyses, the effect of other emerging comorbid conditions (i.e., hypertension, metabolic syndrome) on cognition was not examined. Although such comorbidities may be unlikely to show a focal effect on verbal fluency, understanding such relationships in an aging population would be an important area of future study.

An additional limitation may include the nature of the sample: the WRAP cohort, which draws many of its subjects from the college-town of Madison, Wisconsin, has a higher than average mean baseline IQ, and many of the participants self-selected based on family history of AD. Replication studies using a cohort with a mean IQ more typical of the general population, as well as a more randomized sample of middle-aged adults, may be of added benefit to the verbal fluency literature.

Strengths of our study include a larger sample compared with previous studies of verbal fluency performance in aMCI/AD, as well as a group that is enriched with people at risk for AD. Further, our use of relatively conservative, robust internal norms to define aMCI across multiple testing visits provides a unique insight into subtle cognitive changes. While the psychometric aMCI subgroup in this study did evidence memory with or without executive function deficits (which were non-fluency based) according to internally developed robust norms, these participants are far less impaired as a group than clinically referred MCI group means (Weiner et al., 2013), suggesting that verbal fluency tests may yield useful signal in the preclinical stages of AD. In summary, the findings of depressed total word scores for both phonemic and semantic measures in this younger cohort with evidence of psychometric aMCI versus subjects who are CH, suggest that there may be added utility in obtaining both semantic and phonemic fluency scores as markers of early cognitive dysfunction. The tests are inexpensive and quick to administer, further adding to their clinical utility. Future study of the qualitative fluency measures (clustering and switching) in a longitudinal analysis may provide additional information about the brain processes involved in these tasks. Publication of additional normative data for both cross-sectional and longitudinal samples would enhance both the research and clinical utility of these added verbal fluency measures. An additional area of future research may be to replicate these verbal fluency results based on a more generalizable population-based sample (not enriched for family history of AD), which would not only add to the understanding of verbal fluency processes and deficits but also to the understanding of preclinical stages of MCI and dementia. Finally, a better understanding of executive function deficits in early stages of memory loss will lead to improved interventions for this group.

Funding

This research was supported by NIA grant R01AG27161 (Wisconsin Registry for Alzheimer Prevention: Biomarkers of Preclinical AD), the Helen Bader Foundation, Northwestern Mutual Foundation, and Extendicare Foundation and the Clinical and Translational Science Award (CTSA) program, through the NIH National Center for Advancing Translational Sciences (NCATS), grant UL1TR000427.

Conflict of Interest

None declared.

Acknowledgements

The authors gratefully acknowledge the assistance of Shawn Bolin, Nia Norris, Diane Wilkinson, Lisa Bluder, Joanne Nordeen, Danielle Schuld, Amy Hawley, Janet Rowley, Susan Schroeder, and Carolyn Flock for WRAP data collection. We especially thank the WRAP participants.

References

Adlam
A. L.
Bozeat
S.
Arnold
R.
Watson
P.
Hodges
J. R.
(
2006
).
Semantic knowledge in mild cognitive impairment and mild Alzheimer's disease
.
Cortex
 ,
42
(5)
,
675
684
.
Albert
M. S.
DeKosky
S. T.
Dickson
D.
Dubois
B.
Feldman
H. H.
Fox
N. C.
et al
(
2011
).
The diagnosis of mild cognitive impairment due to Alzheimer's disease: Recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease
.
Alzheimers Dementia
 ,
7
(3)
,
270
279
.
Amieva
H.
Jacqmin-Gadda
H.
Orgogozo
J. M.
Le Carret
N.
Helmer
C.
Letenneur
L.
et al
(
2005
).
The 9 year cognitive decline before dementia of the Alzheimer type: a prospective population-based study
.
Brain
 ,
128
(5)
,
1093
1101
.
Barr
A.
Brandt
J.
(
1996
).
Word-list generation deficits in dementia
.
Journal of Clinical and Experimental Neuropsychology
 ,
18
(6)
,
810
822
.
Benton
A. L.
Hamsher
K. S.
Sivan
A. B.
(
1994
).
Multilingual aphasia examination
 .
Lutz, FL
:
Psychological Assessment
Resources
.
Brandt
J.
Manning
K. J.
(
2009
).
Patterns of word-list generation in mild cognitive impairment and Alzheimer's disease
.
The Clinical Neuropsychologist
 ,
23
(5)
,
870
879
.
Chertkow
H.
Bub
D.
(
1990
).
Semantic memory loss in dementia of Alzheimer's type: What do various measures measure?
Brain
 ,
113
(2)
,
397
417
.
Cicchetti
D. V.
Sparrow
S. A.
(
1981
).
Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior
.
American Journal of Mental Deficiency
 ,
86
(2)
,
127
137
.
Clark
L. J.
Gatz
M.
Zheng
L.
Chen
Y. L.
McCleary
C.
Mack
W. J.
(
2009
).
Longitudinal verbal fluency in normal aging, preclinical, and prevalent Alzheimer's disease
.
American Journal of Alzheimer's Disease & Other Dementias
 ,
24
(6)
,
461
468
.
Clark
L. R.
Schiehser
D. M.
Weissberger
G. H.
Salmon
D. P.
Delis
D. C.
Bondi
M. W.
(
2012
).
Specific measures of executive function predict cognitive decline in older adults
.
Journal of the International Neuropsychological Society
 ,
18
(1)
,
118
127
.
Corcoran
R.
Upton
D.
(
1993
).
A role for the hippocampus in card sorting?
Cortex
 ,
29
(2)
,
293
304
.
Coslett
H. B.
Bowers
D.
Verfaellie
M.
Heilman
K. M.
(
1991
).
Frontal verbal amnesia. Phonological amnesia
.
Archives of Neurology
 ,
48
(9)
,
949
955
.
Crossley
M.
D'arcy
C.
Rawson
N. S.
(
1997
).
Letter and category fluency in community-dwelling Canadian seniors: A comparison of normal participants to those with dementia of the Alzheimer or vascular type
.
Journal of clinical and experimental neuropsychology
 ,
19
(1)
,
52
62
.
Delis
D. C.
Kaplan
E.
Kramer
J. H.
(
2001
).
Delis-Kaplan executive function system (D-KEFS)
 .
San Antonio, TX
:
The Psychological Corporation
.
Dowling
N. M.
Hermann
B.
La Rue
A.
Sager
M. A.
(
2010
).
Latent structure and factorial invariance of a neuropsychological test battery for the study of preclinical Alzheimer's disease
.
Neuropsychology
 ,
24
(6)
,
742
756
.
Epker
M. O.
Lacritz
L. H.
Munro Cullum
C.
(
1999
).
Comparative analysis of qualitative verbal fluency performance in normal elderly and demented populations
.
Journal of Clinical and Experimental Neuropsychology
 ,
21
(4)
,
425
434
.
Fagundo
A. B.
López
S.
Romero
M.
Guarch
J.
Marcos
T.
Salamero
M.
(
2008
).
Clustering and switching in semantic fluency: Predictors of the development of Alzheimer's disease
.
International Journal of Geriatric Psychiatry
 ,
23
(10)
,
1007
1013
.
Gomez
R. G.
White
D. A.
(
2006
).
Using verbal fluency to detect very mild dementia of the Alzheimer type
.
Archives of Clinical Neuropsychology
 ,
21
(8)
,
771
775
.
Haugrud
N.
Lanting
S.
Crossley
M.
(
2010
).
The effects of age, sex and Alzheimer's disease on strategy use during verbal fluency tasks
.
Aging, Neuropsychology, and Cognition
 ,
17
(2)
,
220
239
.
Henry
J. D.
Crawford
J. R.
Phillips
L. H.
(
2004
).
Verbal fluency performance in dementia of the Alzheimer's type: A meta-analysis
.
Neuropsychologia
 ,
42
(9)
,
1212
1222
.
Hodges
J. R.
Salmon
D. P.
Butters
N.
(
1992
).
Semantic memory impairment in Alzheimer's disease: Failure of access or degraded knowledge?
Neuropsychologia
 ,
30
(4)
,
301
314
.
Johnson
J. K.
Gross
A. L.
Pa
J.
McLaren
D. G.
Park
L. Q.
Manly
J. J.
,
for the Alzheimer's Disease Neuroimaging Initiative
(
2012
).
Longitudinal change in neuropsychological performance using latent growth models: A study of mild cognitive impairment
.
Brain Imaging Behav
 ,
6
,
540
550
.
Johnson
S. C.
La Rue
A.
Hermann
B. P.
Xu
G.
Koscik
R. L.
Jonaitis
E. M.
et al
(
2011
).
The effect of TOMM40 poly-T length on gray matter volume and cognition in middle-aged persons with APOE ϵ3/ϵ3 genotype
.
Alzheimers Dement
 ,
7
(4)
,
456
465
.
Koscik
R. L.
La Rue
A.
Jonaitis
E. M.
Okonkwo
O. C.
Johnson
S. C.
Bendlin
B. B.
et al
(
2014
).
Emergence of mild cognitive impairment in late middle-aged adults in the Wisconsin registry for Alzheimer's prevention
.
Dementia and Geriatric Cognitive Disorders
 ,
38
(1–2)
,
16
30
.
La Rue
A.
Hermann
B.
Jones
J. E.
Johnson
S.
Asthana
S.
Sager
M. A.
(
2008
).
Effect of parental family history of Alzheimer's disease on serial position profiles
.
Alzheimer's & Dementia
 ,
4
(4)
,
285
290
.
Lezak
M. D.
Howieson
D. B.
Loring
D. W.
(
2004
).
Neuropsychological assessment
  (
4th ed.
).
New York, NY
:
Oxford University Press
.
Loewenstein
D. A.
Greig
M. T.
Schinka
J. A.
Barker
W.
Shen
Q.
Potter
E.
et al
(
2012
).
An investigation of PreMCI: Subtypes and longitudinal outcomes
.
Alzheimers & Dementia: The Journal of the Alzheimers Association
 ,
8
(3)
,
172
179
.
Mattis
S.
, (
1988
).
Dementia rating scale: Professional manual
 .
Odessa, FL
:
Psychological Assessment Resources
.
McKhann
G.
Drachman
D.
Folstein
M.
Katzman
R.
Price
D.
Stadlan
E. M.
(
1984
).
Clinical diagnosis of Alzheimer's disease: Report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's disease
.
Neurology
 ,
34
(7)
,
939
944
.
Miceli
G.
Caltagirone
C.
Gainotti
G.
Masullo
C.
Silveri
M. C.
(
1981
).
Neuropsychological correlates of localized cerebral lesions in non-aphasic brain-damaged patients
.
Journal of Clinical Neurophysiology
 ,
3
(1)
,
53
63
.
Miller
E.
(
1984
).
Verbal fluency as a function of a measure of verbal intelligence and in relation to different types of cerebral pathology
.
British Journal of Clinical Psychology
 ,
23
(Pt 1)
,
53
57
.
Morris
J. C.
Price
J. L.
(
2001
).
Pathologic correlates of nondemented aging, mild cognitive impairment, and early-stage Alzheimer's disease
.
Journal of Molecular Neuroscience
 ,
17
(2)
,
101
118
.
Murphy
K. J.
Rich
J. B.
Troyer
A. K.
(
2006
).
Verbal fluency patterns in amnestic mild cognitive impairment are characteristic of Alzheimer's type dementia
.
Journal of the International Neuropsychological Society
 ,
12
(4)
,
570
574
.
Newcombe
F.
(
1969
).
Missile wounds of the brain: A study of psychological deficits
 .
Oxford University Press: London
.
Newman
L. M.
Trivedi
M. A.
Bendlin
B. B.
Ries
M. L.
Johnson
S. C.
(
2007
).
The relationship between gray matter morphometry and neuropsychological performance in a large sample of cognitively healthy adults
.
Brain Imaging and Behavior
 ,
1
(1–2)
,
3
10
.
Nutter-Upham
K. E.
Saykin
A. J.
Rabin
L. A.
Roth
R. M.
Wishart
H. A.
Pare
N.
et al
(
2008
).
Verbal fluency performance in amnestic MCI and older adults with cognitive complaints
.
Archives of Clinical Neuropsychology
 ,
23
(3)
,
229
241
.
Petersen
R. C.
(
2004
).
Mild cognitive impairment as a diagnostic entity
.
Journal of Internal Medicine
 ,
256
(3)
,
183
194
.
Price
S. E.
Kinsella
G. J.
Ong
B.
Storey
E.
Mullaly
E.
Phillips
M.
et al
(
2012
).
Semantic verbal fluency strategies in amnestic mild cognitive impairment
.
Neuropsychology
 ,
26
(4)
,
490
497
.
Radloff
L. S.
(
1977
).
The CES-D scale: A self report depression scale for research in the general population
.
Applied Psychological Measurement
 ,
1
,
385
401
.
Raoux
N.
Amieva
H.
Le Goff
M.
Auriacombe
S.
Carcaillon
L.
Letenneur
L.
et al
(
2008
).
Clustering and switching processes in semantic verbal fluency in the course of Alzheimer's disease subjects: Results from the PAQUID longitudinal study
.
Cortex: A Journal Devoted to the Study of the Nervous System and Behavior
 ,
44
(9)
,
1188
1196
.
Reitan
R. M.
Wolfson
D.
(
1993
).
The Halstead-Reitan neuropsychological test battery: Theory and clinical interpretation
  (
2nd ed.
).
Tuscon, AZ
:
Neuropsychology Press
.
Rinehardt
E.
Eichstaedt
K.
Schinka
J. A.
Loewenstein
D. A.
Mattingly
M.
Fils
J.
et al
(
2014
).
Verbal fluency patterns in mild cognitive impairment and Alzheimer's disease
.
Dementia and Geriatric Cognitive Disorders
 ,
38
(1–2)
,
1
9
.
Rosen
V. M.
Sunderland
T.
Levy
J.
Harwell
A.
McGee
L.
Hammond
C.
et al
(
2005
).
Apolipoprotein E and category fluency: Evidence for reduced semantic access in healthy normal controls at risk for developing Alzheimer's disease
.
Neuropsychologia
 ,
43
(4)
,
647
658
.
Sager
M. A.
Hermann
B.
La Rue
A.
(
2005
).
Middle-aged children of persons with Alzheimer's disease: APOE genotypes and cognitive function in the Wisconsin Registry for Alzheimer's Prevention
.
Journal of Geriatric Psychiatry and Neurology
 ,
18
(4)
,
245
249
.
Trenerry
M.
Crosson
B.
DeBoe
J.
Leber
L.
(
1989
).
Stroop neuropsychological screening test
 .
Odessa, FL
:
Psychological Assessment Resources
.
Troyer
A. K.
Moscovitch
M.
(
2006
).
Cognitive processes of verbal fluency tasks
. In
Poreh
A. M.
(Ed.),
The quantified process approach to neuropsychological assessment
  (pp.
143
160
).
Philadelphia
:
Taylor & Francis
.
Troyer
A. K.
Moscovitch
M.
Winocur
G.
(
1997
).
Clustering and switching as two components of verbal fluency: Evidence from younger and older healthy adults
.
Neuropsychology
 ,
11
(1)
,
138
146
.
Troyer
A. K.
Moscovitch
M.
Winocur
G.
Alexander
M. P.
Stuss
D.
(
1998
).
Clustering and switching on verbal fluency: The effects of focal frontal- and temporal-lobe lesions
.
Neuropsychologia
 ,
36
(6)
,
499
504
.
Troyer
A. K.
Moscovitch
M.
Winocur
G.
Leach
L.
Freedman
M.
(
1998
).
Clustering and switching on verbal fluency tests in Alzheimer's and Parkinson's disease
.
Journal of the International Neuropsychological Society
 ,
4
(2)
,
137
143
.
Tröster
A. I.
Fields
J. A.
Testa
J. A.
Paul
R. H.
Blanco
C. R.
Hames
K. A.
et al
(
1998
).
Cortical and subcortical influences on clustering and switching in the performance of verbal fluency tasks
.
Neuropsychologia
 ,
36
(4)
,
295
304
.
Wechsler
D.
(
1999
).
Wechsler abbreviated intelligence scale
 .
San Antonio, TX
:
Psychological Corporation
.
Weiner
M. W.
Veitch
D. P.
Aisen
P. S.
Beckett
L. A.
Cairns
N. J.
Green
R. C.
et al
(
2013
).
The Alzheimer's disease neuroimaging initiative: A review of papers published since its inception
.
Alzheimers Dementia
 ,
9
(5)
,
e111
e194
.
Wilkinson
G. S.
(
1993
).
The wide range achievement test: Manual
  (
3rd ed.
).
Wilmington, DE
:
Wide Range
.