Abstract

The main goal of this study was to produce adjusted normative data for the Portuguese population on two verbal fluency measures: the semantic fluency test (animals category) and the phonemic fluency test (letters M, R, and P). The study included 950 community-dwelling individuals (624 women and 326 men) aged between 18 and 98 (mean = 57.8, SD = 19.0), who had educational backgrounds ranging from 0 to 20 years (mean = 8.8, SD = 5.2). The results showed that age and education were significantly associated with semantic fluency and phonemic fluency performance. These demographic characteristics accounted for 42% of the semantic fluency and between 23% and 31% of the phonemic fluency performance variance. No significant sex effects were found. The normative data are presented as regression-based algorithms to adjust test scores for age and education, with subsequent correspondence between adjusted scores and percentile distribution.

Introduction

The semantic and phonemic fluency tests are two of the most widely used instruments in clinical and experimental neuropsychology (Strauss, Sherman, & Spreen, 2006). Verbal fluency tests are brief assessment tools with relatively simple administration and scoring procedures. Semantic and phonemic fluency are measures of non-motor processing speed, language production, and executive functions (Greenaway, Smith, Tangalos, Geda, & Ivnik, 2009).

Data from lesion patients (Baldo, Schwartz, Wilkins, & Dronkers, 2006; Gouveia, Brucki, Malheiros, & Bueno, 2007; Szatkowska, Grabowska, & Szymanska, 2000) and functional neuroimaging of healthy individuals (Alvarez & Emory, 2006; Baciu, Juphard, Cousin, & Bas, 2005; Billingsley et al., 2004; Birn et al., 2010; Gauthier, Duyme, Zanca, & Capron, 2009; Kinkingnehun et al., 2007; Ravnkilde, Videbech, Rosenberg, Gjedde, & Gade, 2002; Tupak et al., 2012) provide strong evidence for the involvement of the left frontal (particularly the dorsolateral prefrontal cortex) and temporal lobes on both semantic fluency and phonemic fluency. However, some studies (e.g., Baldo et al., 2006; Billingsley et al., 2004; Tupak et al., 2012) suggest that semantic fluency and phonemic fluency rely on partially different neural networks. Frontal lobe lesions can disproportionately impair phonemic fluency, whereas temporal lobe damage impairs semantic fluency to a greater extent (Baldo et al., 2006). There are indications of a greater left hemispheric involvement in phonemic fluency than in semantic fluency (Billingsley et al., 2004). However, the inverse trend has also been reported (Tupak et al., 2012).

Semantic fluency and phonemic fluency have been used to detect individuals at risk of developing dementia (Jacobs, Marder, Sano, Stern, & Mayeux, 1995; Levy et al., 2002; Palmer, Backman, Winblad, & Fratiglioni, 2003; Santangelo et al., 2007), to identify patients with mild cognitive impairment (Green et al., 2002) or dementia (Cerhan et al., 2002; Crossley, D'arcy, & Rawson, 1997; Henry, Crawford, & Phillips, 2004), to differentiate different types of dementia (Libon et al., 2009; Rascovsky et al., 2002; Rogers, Ivanoiu, Patterson, & Hodges, 2006), and to monitor the progression of disease (Dujardin et al., 2004; Santangelo et al., 2007).

Even though Alzheimer's disease (Clark et al., 2010; Murphy, Rich, & Troyer, 2006) and semantic dementia (Libon et al., 2009; Rogers et al., 2006) affect both verbal fluency measures, the decline is more pronounced on semantic fluency than phonemic fluency. Neither semantic fluency nor phonemic fluency appears to be disproportionately affected in patients with progressive non-fluent aphasia, frontal variant frontotemporal dementia, or posterior cortical atrophy (Libon et al., 2009; Rogers et al., 2006). Verbal fluency impairments are common manifestations of diseases with predominant frontostriatal degeneration (Henry, Crawford, & Phillips, 2005; Williams-Gray et al., 2009). Deficits in phonemic fluency can even be found in preclinical stages of Huntington's disease (Larsson, Almkvist, Luszcz, & Wahlin, 2008). Poor performance on verbal fluency tests, particularly semantic fluency, is one of the best predictors of dementia in Parkinson's disease (Jacobs et al., 1995; Williams-Gray et al., 2009).

Multiple studies from different countries have demonstrated that verbal fluency tests are highly influenced by demographic characteristics, in particular age and educational level (e.g., Ivnik, Malec, Smith, & Tangalos, 1996; Kosmidis, Vlahou, Panagiotaki, & Kiosseoglou, 2004; Lucas et al., 2005; Ratcliff et al., 1998; Ryu et al., 2012; Tombaugh, Kovak, & Rees, 1999). The presence of illiterates and the wide variety of educational backgrounds pose a major challenge to clinical neuropsychology in Portugal both researchers and clinicians. The need for adequate normative data is paramount.

The present study proposes a regression-based approach to establish Portuguese normative data for the semantic fluency test and the phonemic fluency test. Some psychometric properties of the tests are also examined.

Methods

Subjects

Normative sample

Participants in this study included 950 community-dwelling Portuguese individuals (624 women and 326 men) between 18 and 98 years of age (mean = 57.8, SD = 19.0) and between 0 and 20 years of education (i.e., formal schooling completed with success; mean = 8.8, SD = 5.2). Equivalences from the “New Opportunities program” initiative (i.e., a governmental program designed to enhance school certification and qualification levels of the Portuguese adult population) were not credited. The semantic fluency test was administered to 949 subjects (624 women and 325 men; mean age = 57.8, SD = 19.0; mean education = 8.83, SD = 5.2), whereas the phonemic fluency test was administered to 821 subjects (529 women and 292 men; mean age = 55.8, SD = 18.7; mean education = 9.8, SD = 4.8). The inter-rater reliability was explored in a subgroup of 91 subjects of the normative sample (gender: 57 women and 34 men; age: 18–67, mean = 31.8, SD = 13.6; education: 4–18 years, mean = 13.8, SD = 3.6).

The inclusion criteria were ≥18 years of age, Portuguese as the first language, have lived in Portugal in the last 5 years, did ≥50% of formal schooling in Portugal or in a territory with Portuguese administration (for participants with ≥3 years of education), and do not have significant auditory deficits after correction. Individuals with history of developmental disorders (e.g., learning disability), neurological disease (e.g., traumatic brain injury), or moderate to severe psychopathology (e.g., major depression, psychosis, alcoholism) were not included. The phonemic fluency test was not applied to participants with less than 4 years of education. Due to logistic issues, one participant did not perform the semantic fluency test and 18 participants with ≥4 years of education did not complete the phonemic fluency test. All participants provided their written informed consent in accordance with the Helsinki Declaration.

Procedures

Based on findings from previous studies (Petersson, Reis, Askelof, Castro-Caldas, & Ingvar, 2000; Reis & Castro-Caldas, 1997; Silva, Petersson, Faísca, Ingvar, & Reis, 2004), the application of the phonemic fluency test was restricted to individuals with ≥4 years of education, because phonemic fluency requires specific knowledge (i.e., about the letters and their relationship with word formation) that is traditionally acquired in formal education. Basic semantic knowledge about animals is usually acquired in early childhood and outside the classroom. So, we considered it appropriate to apply the semantic fluency test to illiterates, even though the effect of education on test performance is well recognized.

Based on a preliminary study with 59 healthy subjects (mean age = 32.7, SD = 13.3; mean education = 13.9, SD = 3.4), the chosen letters for standardization of the phonemic fluency test were M, R, and P, because these letters had different levels of difficulty. These letters have been used in Portugal for clinical practice and neuropsychological research (Cavaco et al., 2012; Reis & Castro-Caldas, 1997).

Verbal fluencytTests

Semantic fluency

The subjects were asked to generate the name of as many species of animals as possible within 1min. The instructions in Portuguese are presented in the Supplementary material. Regional designations of animals were accepted. Any repetition of the same animal species (including name variations according to the animal gender or age) was not credited. Credit was given to superordinate categories only if specific items within that category were not given during the trial. The total test score corresponds to the number of animal species named within 60 s. Higher scores correspond to better performance.

Phonemic fluency

The subjects were asked to produce orally as many words as possible beginning with a specific letter. The test consists of three trials, of 1min each. The instructions in Portuguese are presented in the Supplementary material. Whenever the subject provided multiple responses with the same root (e.g., variations of gender, number) referring to the same object, action, or concept, only the first response was credited. When the subject named numbers as responses, only the first response was credited. The same word with two different meanings was admissible if the subject indicated the different meanings. Slang terms and foreign words were admissible if they were of general use in Portugal. The total trial score corresponds to the number of words correctly produced within 60 s. The total test score corresponds to the sum of the three trials. Higher scores correspond to better performance.

Reliability studies

The internal reliability of the phonemic fluency test was analyzed using the total normative sample for the test. To explore the inter-rater reliability, three raters examined the responses and quantified independently a series of semantic fluency and phonemic fluency performances. These reliability studies focused on raw scores.

Statistical Analyses

Descriptive statistics (i.e., frequency, percentage, mean, and SD) were used for demographic characterization and for presentation of raw test scores. Pearson's correlations (r) and shared variances (r2) were used to explore the effects of demographic characteristics (i.e., sex, age, and education) on test performances. Scatter plots were used to visualize the associations between demographic variables and test results. The Mann–Whitney test was used to compare subgroups of subjects (i.e., gender and education group) on test performance.

Multiple regression analyses, without variable selection, were conducted with the raw scores as dependent variables and age and education (i.e., number of years or education group) as covariates. We considered the possibility of a quadratic effect for age and education. The assumptions of homoscedasticity and normal distribution of the residuals were verified. The adjustment of test scores for demographic characteristics was based on regression coefficients. The cumulative percent distribution of the standardized regression residuals (standardized residuals = residuals/SD) was used to identify the associated percentiles. The residuals represent the difference between the score of an individual and the mean score of individuals with the same age and education. Higher adjusted scores correspond to better performance.

Cronbach's α was used to measure internal consistency of the phonemic fluency test. Two-way random single-measure intraclass correlation coefficients (ICCs) were computed to assess the absolute agreement between phonemic fluency trials and between different raters (inter-rater reliability) for both tests.

Results

Demographic Effects

Tables 1 and 2 present the normative raw scores grouped by sex, age, and education groups. The effects of the variables gender, age, and education were investigated for each test. No statistically significant differences were found between women and men on semantic fluency (mean = 16.4, SD = 5.2 vs. mean = 16.6, SD = 5.8; p = .734) and phonemic fluency (letter M: mean = 10.2, SD = 4.4 vs. mean = 10.1, SD = 4.3, p = .770; letter R: mean = 9.8, SD = 4.1 vs. mean = 10, SD = 4.3, p = .693; letter P: mean = 11, SD = 4.5 vs. mean = 10.7, SD = 4.5, p = .400; Total: mean = 31, SD = 11.6 vs. mean = 30.8, SD = 11.9, p = .732). Significant (p < .001) linear associations were found for age and number of years of education (Table 3). Scatter plots revealed quadratic relations between these demographic variables and the semantic fluency and phonemic fluency raw scores.

Table 1.

Semantic fluency raw scores per sex, age, and education groups

 Semantic fluency (n = 949)
 
n Percent Animals 
Sex 
 Women 624 65.8 16.4 (5.2) 
 Men 325 34.2 16.6 (5.8) 
Age 
 18–29 110 11.6 21.1 (5.1) 
 30–39 76 8.0 18.3 (4.1) 
 40–49 98 10.3 17.6 (4.4) 
 50–59 161 17.0 17.5 (4.9) 
 60–69 205 21.6 16.9 (5.0) 
 70–79 179 18.9 13.8 (4.6) 
 80–89 103 10.9 12.1 (4.2) 
 ≥90 17 1.8 8.8 (3.6) 
Education 
 <3 53 5.6 10.9 (3.5) 
 3 58 6.1 11.4 (3.9) 
 4 199 21.0 13.0 (4.5) 
 5–9 219 23.1 15.9 (4.1) 
 10–12 153 16.1 18.8 (4.6) 
 >12 267 28.1 20.2 (4.5) 
 Semantic fluency (n = 949)
 
n Percent Animals 
Sex 
 Women 624 65.8 16.4 (5.2) 
 Men 325 34.2 16.6 (5.8) 
Age 
 18–29 110 11.6 21.1 (5.1) 
 30–39 76 8.0 18.3 (4.1) 
 40–49 98 10.3 17.6 (4.4) 
 50–59 161 17.0 17.5 (4.9) 
 60–69 205 21.6 16.9 (5.0) 
 70–79 179 18.9 13.8 (4.6) 
 80–89 103 10.9 12.1 (4.2) 
 ≥90 17 1.8 8.8 (3.6) 
Education 
 <3 53 5.6 10.9 (3.5) 
 3 58 6.1 11.4 (3.9) 
 4 199 21.0 13.0 (4.5) 
 5–9 219 23.1 15.9 (4.1) 
 10–12 153 16.1 18.8 (4.6) 
 >12 267 28.1 20.2 (4.5) 
Table 2.

Phonemic fluency raw scores per sex, age, and education groups

 Phonemic fluency (n = 821)
 
n Percent Total score 
Sex 
 Women 529 64.4 10.2 (4.4) 9.8 (4.1) 11.0 (4.5) 31.0 (11.6) 
 Men 292 35.6 10.1 (4.3) 10.0 (4.3) 10.7 (4.5) 30.8 (11.9) 
Age 
 18–29 104 12.7 12.2 (4.7) 11.4 (4.0) 12.8 (4.8) 36.4 (11.9) 
 30–39 75 9.1 11.0 (4.6) 10.4 (4.2) 10.3 (4.5) 31.6 (12.1) 
 40–49 93 11.3 10.9 (4.4) 10.4 (4.6) 10.5 (4.4) 31.8 (12.1) 
 50–59 152 18.5 10.1 (4.3) 9.7 (4.3) 10.9 (4.5) 30.7 (11.6) 
 60–69 191 23.3 10.5 (3.9) 10.1 (3.8) 11.5 (4.2) 32.1 (10.7) 
 70–79 126 15.3 8.6 (3.8) 9.2 (3.9) 10.5 (4.0) 28.3 (10.4) 
 80–89 69 8.4 7.7 (3.6) 7.8 (4.2) 9.1 (4.4) 24.6 (11.4) 
 ≥90 11 1.3 6.6 (3.1) 6.5 (3.4) 7.9 (5.0) 21.0 (11.0) 
Education 
 4 191 23.3 6.8 (3.2) 6.8 (3.1) 7.9 (3.54) 21.4 (8.7) 
 5–9 218 26.6 9.5 (3.8) 9.3 (3.9) 9.9 (4.17) 28.7 (10.5) 
 10–12 149 18.1 10.4 (3.8) 10.5 (3.6) 12.3 (3.94) 33.2 (9.6) 
 >12 263 32.0 13.0 (3.9) 12.3 (3.8) 13.2 (4.04) 38.5 (10.2) 
 Phonemic fluency (n = 821)
 
n Percent Total score 
Sex 
 Women 529 64.4 10.2 (4.4) 9.8 (4.1) 11.0 (4.5) 31.0 (11.6) 
 Men 292 35.6 10.1 (4.3) 10.0 (4.3) 10.7 (4.5) 30.8 (11.9) 
Age 
 18–29 104 12.7 12.2 (4.7) 11.4 (4.0) 12.8 (4.8) 36.4 (11.9) 
 30–39 75 9.1 11.0 (4.6) 10.4 (4.2) 10.3 (4.5) 31.6 (12.1) 
 40–49 93 11.3 10.9 (4.4) 10.4 (4.6) 10.5 (4.4) 31.8 (12.1) 
 50–59 152 18.5 10.1 (4.3) 9.7 (4.3) 10.9 (4.5) 30.7 (11.6) 
 60–69 191 23.3 10.5 (3.9) 10.1 (3.8) 11.5 (4.2) 32.1 (10.7) 
 70–79 126 15.3 8.6 (3.8) 9.2 (3.9) 10.5 (4.0) 28.3 (10.4) 
 80–89 69 8.4 7.7 (3.6) 7.8 (4.2) 9.1 (4.4) 24.6 (11.4) 
 ≥90 11 1.3 6.6 (3.1) 6.5 (3.4) 7.9 (5.0) 21.0 (11.0) 
Education 
 4 191 23.3 6.8 (3.2) 6.8 (3.1) 7.9 (3.54) 21.4 (8.7) 
 5–9 218 26.6 9.5 (3.8) 9.3 (3.9) 9.9 (4.17) 28.7 (10.5) 
 10–12 149 18.1 10.4 (3.8) 10.5 (3.6) 12.3 (3.94) 33.2 (9.6) 
 >12 263 32.0 13.0 (3.9) 12.3 (3.8) 13.2 (4.04) 38.5 (10.2) 
Table 3.

Correlations and shared variances between test raw scores and demographic variables

Test Age
 
Education
 
r r2 r r2 
Semantic fluency 
 Animals −.489* .240 .598* .358 
Phonemic fluency 
 M −.281* .079 .534* .285 
 R −.208* .043 .492* .242 
 P −.154 .024 .462* .214 
 Total score −.237* .056 .550* .302 
Test Age
 
Education
 
r r2 r r2 
Semantic fluency 
 Animals −.489* .240 .598* .358 
Phonemic fluency 
 M −.281* .079 .534* .285 
 R −.208* .043 .492* .242 
 P −.154 .024 .462* .214 
 Total score −.237* .056 .550* .302 

*p < .0001

Literacy and Semantic Fluency

No significant differences were found between subjects with 0 and 1 year of education (n = 35 vs. n = 7; mean = 10.8, SD = 4 vs. mean = 10.7, SD = 1.8; p = .672), between 0 and 2 years of education (n = 35 vs. n = 11; mean = 10.8, SD = 4 vs. mean = 11.3, SD = 2.7; p = .477), and between 0 and 3 years of education (n = 35 vs. n = 58; mean = 10.8, SD = 4 vs. mean = 11.4, SD = 3.9; p = .364). Subjects with 0 years of education had lower semantic fluency scores than subjects with 4 years of education (n = 35 vs. n = 199; mean = 10.8, SD = 4 vs. mean = 13, SD = 4.5; p = .006). This group difference (p = .074) was no longer statistically significant when it was controlled for age.

The use of a broader criterion for illiteracy (i.e., 0–2 years of education; Cavaco & Teixeira-Pinto, 2011) revealed that illiterate subjects had poorer performance on the semantic fluency test than subjects with 4 years of education (n = 53 vs. n = 199; mean = 10.9, SD = 3.5 vs. mean = 13, SD = 4.5; p = .002). This group difference (p = .025) remained statistically significant when it was controlled for age.

Normative Data

The multiple regression analyses were conducted with age, age2, number of years of education, and number of years of education2 as the regression model. Each of these variables was found to be independently associated (p < .06) with performance on the semantic fluency test and the phonemic fluency test—letter M, R, and Total score. Both the linear and the quadratic of effect of age were not significantly associated (p > .1) with performance on the P trial of the phonemic fluency test (Table 4). Age, age2, number of years of education, and number of years of education2 partly explained (r2) the variance of both semantic fluency (42%) and phonemic fluency—letter M (30%), letter R (25%), letter P (23%), and Total score (31%).

Table 4.

Partial β for the multiple regression analyses

 Semantic fluency: Animals Phonemic fluency
 
Total 
Age 0.079* 0.086* 0.082* 0.041 0.208* 
Age2 −0.001*** −0.001** −0.001* <0.001 −0.002* 
Education 0.801*** 0.749*** 0.756*** 1.041*** 2.546*** 
Education2 −0.016** −0.013 −0.015* −0.028*** −0.056** 
 Semantic fluency: Animals Phonemic fluency
 
Total 
Age 0.079* 0.086* 0.082* 0.041 0.208* 
Age2 −0.001*** −0.001** −0.001* <0.001 −0.002* 
Education 0.801*** 0.749*** 0.756*** 1.041*** 2.546*** 
Education2 −0.016** −0.013 −0.015* −0.028*** −0.056** 

*p < .05.

**p < .01.

***p < .001.

Table 5 presents the regression-based algorithms to adjust test scores for age and number of years of education. In other words, the algorithms convert raw scores of an individual into standardized Z scores. The percentiles associated with the age and education adjusted scores are shown in Table 6. A user-friendly program is available online (http://neuropsi.up.pt/) to adjust scores (Fig. 1). The clinician only needs to introduce the subject's age, education, and number of words correctly produced. For instance, if an individual with 45 years of age and 12 years of education generates 13 animal names and a total of 19 words on the three phonemic fluency trials, the adjusted scores are −1.5 for the semantic fluency test and −1.7 for the phonemic fluency test. These adjusted scores fall within percentile range 3–5. Thus, 3–5% of the “normal” population with 45 years of age and 12 years of education generates ≤13 words on the semantic fluency test and ≤19 words on the phonemic fluency test.

Table 5.

Algorithms to calculate adjusted scores

Semantic fluency (raw score − 11.3984 − (age * 0.0795) + (age2 * 0.0014) − (education * 0.8008) + (education2 * 0.0165))/4.1399 
Phonemic fluency  
 M (raw score − 2.5878 − (age × 0.0858) + (age2 × 0.0009) − (education × 0.7495) + (education2 × 0.0133))/3.6498 
 R (raw score − 2.0379 − (age × 0.0817) + (age2 × 0.0007) − (education × 0.7555) + (education2 × 0.0150))/3.6089 
 P (raw score − 2.2708 − (age × 0.0409) + (age2 × 0.0002) − (education × 1.0408) + (education2 × 0.0275))/3.9190 
 Total score (raw score − 6.8965 − (age × 0.2084) + (age2 × 0.0019) − (education × 2.5458) + (education2 × 0.0558))/9.7035 
Semantic fluency (raw score − 11.3984 − (age * 0.0795) + (age2 * 0.0014) − (education * 0.8008) + (education2 * 0.0165))/4.1399 
Phonemic fluency  
 M (raw score − 2.5878 − (age × 0.0858) + (age2 × 0.0009) − (education × 0.7495) + (education2 × 0.0133))/3.6498 
 R (raw score − 2.0379 − (age × 0.0817) + (age2 × 0.0007) − (education × 0.7555) + (education2 × 0.0150))/3.6089 
 P (raw score − 2.2708 − (age × 0.0409) + (age2 × 0.0002) − (education × 1.0408) + (education2 × 0.0275))/3.9190 
 Total score (raw score − 6.8965 − (age × 0.2084) + (age2 × 0.0019) − (education × 2.5458) + (education2 × 0.0558))/9.7035 
Table 6.

Empirical percentile ranks associated with age and education adjusted scores

Percentile ranks Scaled scores Adjusted scores
 
Semantic fluency: Animals Phonemic fluency
 
Total score 
−1.9 −2.2 −2.2 −2.2 −2.2 
−1.8 −2.0 −1.9 −2.0 −1.9 
3–5 −1.6 to −1.5 −1.8 to −1.5 −1.8 to −1.6 −1.8 to −1.6 −1.7 to −1.6 
6–10 −1.4 to −1.3 −1.5 to −1.2 −1.6 to −1.3 −1.5 to −1.3 −1.5 to −1.2 
11–18 −1.2 to −0.9 −1.2 to −0.9 −1.2 to −1.0 −1.2 to −1.0 −1.2 to −0.9 
19–28 −0.9 to −0.6 −0.9 to −0.6 −0.9 to −0.6 −1.0 to −0.6 −0.9 to −0.6 
29–40 −0.6 to −0.3 −0.6 to −0.3 −0.6 to −0.3 −0.6 to −0.3 −0.6 to −0.3 
41–59 10 −0.2 to 0.2 −0.3 to 0.1 −0.3 to 0.2 −0.3 to 0.2 −0.3 to 0.2 
60–71 11 0.2–0.5 0.2–0.5 0.2–0.5 0.2–0.5 −0.2–0.6 
72–81 12 0.6–0.8 0.5–0.9 0.6–0.9 0.5–0.9 0.6–0.9 
82–89 13 0.9–1.2 1.0–1.3 0.9–1.2 0.9–1.3 0.9–1.3 
90–94 14 1.3–1.7 1.3–1.6 1.3–1.5 1.3–1.6 1.3–1.6 
95–97 15 1.8–2.0 1.7–2.0 1.6–1.9 1.7–1.9 1.7–1.9 
98 16 2.3 2.1 2.1 2.1 2.2 
99 17 2.8 2.4 2.4 2.4 2.3 
Minimum −2.3 −2.9 −2.7 −2.7 −29 
Maximum 4.2 3.5 3.1 3.1 3.8 
Percentile ranks Scaled scores Adjusted scores
 
Semantic fluency: Animals Phonemic fluency
 
Total score 
−1.9 −2.2 −2.2 −2.2 −2.2 
−1.8 −2.0 −1.9 −2.0 −1.9 
3–5 −1.6 to −1.5 −1.8 to −1.5 −1.8 to −1.6 −1.8 to −1.6 −1.7 to −1.6 
6–10 −1.4 to −1.3 −1.5 to −1.2 −1.6 to −1.3 −1.5 to −1.3 −1.5 to −1.2 
11–18 −1.2 to −0.9 −1.2 to −0.9 −1.2 to −1.0 −1.2 to −1.0 −1.2 to −0.9 
19–28 −0.9 to −0.6 −0.9 to −0.6 −0.9 to −0.6 −1.0 to −0.6 −0.9 to −0.6 
29–40 −0.6 to −0.3 −0.6 to −0.3 −0.6 to −0.3 −0.6 to −0.3 −0.6 to −0.3 
41–59 10 −0.2 to 0.2 −0.3 to 0.1 −0.3 to 0.2 −0.3 to 0.2 −0.3 to 0.2 
60–71 11 0.2–0.5 0.2–0.5 0.2–0.5 0.2–0.5 −0.2–0.6 
72–81 12 0.6–0.8 0.5–0.9 0.6–0.9 0.5–0.9 0.6–0.9 
82–89 13 0.9–1.2 1.0–1.3 0.9–1.2 0.9–1.3 0.9–1.3 
90–94 14 1.3–1.7 1.3–1.6 1.3–1.5 1.3–1.6 1.3–1.6 
95–97 15 1.8–2.0 1.7–2.0 1.6–1.9 1.7–1.9 1.7–1.9 
98 16 2.3 2.1 2.1 2.1 2.2 
99 17 2.8 2.4 2.4 2.4 2.3 
Minimum −2.3 −2.9 −2.7 −2.7 −29 
Maximum 4.2 3.5 3.1 3.1 3.8 
Fig. 1.

Screenshot of the online tool to compute semantic and phonemic fluency adjusted scores (http://neuropsi.up.pt).

Fig. 1.

Screenshot of the online tool to compute semantic and phonemic fluency adjusted scores (http://neuropsi.up.pt).

Internal Reliability

The absolute agreement among M, R, and P of the phonemic fluency test was ICC = 0.71 (95% CI = 0.67–0.74), whereas the internal consistency was Cronbach's α = 0.89. The number of correct responses was significantly higher for letter M than for letter R (t = 2.52, 95% CI = 0.06–0.50). Both letter M (t = 6.50, 95% CI = 0.53–0.99) and letter R (t = 9.22, 95% CI = 0.82–1.26) elicited fewer responses within 1min than letter P.

Inter-rater Reliability

The inter-rater reliability for category animals was ICC = 0.996 (95% CI: 0.995–0.998), for letter M was ICC = 0.988 (95% CI: 0.982–0.992), for letter R was ICC = 0.977 (95% CI: 0.960–0.986), and for letter P was ICC = 0.979 (95% CI: 0.960–0.996).

Discussion

In agreement with most normative studies, the performance on the semantic fluency test declined with age (Benito-Cuadrado, Esteba-Castillo, Bohm, Cejudo-Bolivar, & Pena-Casanova, 2002; Crossley et al., 1997; Gladsjo et al., 1999; Herrmann, Walter, Ehlis, & Fallgatter, 2006; Kosmidis et al., 2004; Tomer & Levin, 1993; Van der Elst, Boxtel, Breukelen, & Jolles, 2006) and increased with education (Benito-Cuadrado et al., 2002; Kosmidis et al., 2004; Ratcliff et al., 1998; Tombaugh et al., 1999). Similar to other reports (Steinberg, Bieliauskas, Smith, & Ivnik, 2005; Tallberg, Ivachova, Tinchag, & Ostberg, 2008; Tombaugh et al., 1999), the effects of age on phonemic fluency were less pronounced than the effects of education. In our study, no significant associations were found between verbal fluency tests and sex. Most studies in the literature are consistent with these negative findings (Benito-Cuadrado et al., 2002; Gladsjo et al., 1999; Harrison, Buxton, Husain, & Wise, 2000; Lucas et al., 1998; Tombaugh et al., 1999). Nonetheless, there are reports (Capitani, Laiacona, & Barbarotto, 1999; Kosmidis et al., 2004; Van der Elst et al., 2006) of significant sex effects on semantic fluency when other semantic categories (e.g., fruits, professions and tools) were used.

The Portuguese population provides a unique opportunity to explore the effects of schooling on verbal fluency tests. Consistent with Manly and colleagues' (1999) report, no significant differences on semantic fluency were found between subjects with 0, 1, 2, and 3 years of education. The effects of education on semantic fluency were only significant after the third grade. These results support the importance of literacy in semantic fluency (Reis & Castro-Caldas, 1997; Silva et al., 2004). However, the reported pattern of association is specific to category “animals”. Other semantic categories (e.g., “supermarket”) may be more or less affected by literacy, depending on their ecological or cultural relevance (Silva et al., 2004).

The wide range of educational backgrounds in the Portuguese population poses an important challenge for any normative or clinical neuropsychology study. Illiteracy, elementary level education, high school level education, and college level education coexist in most adult age groups. To collect enough data to split a normative or a clinical sample into subgroups of age and education is problematic. The option in this study was to treat age and education as continuous variables and to use multiple regression analysis to maximize the number of normative individuals contributing for each specific age and education. Similar approach to establish normative data has been used in recent studies (Cavaco & Teixeira-Pinto, 2011; Van Breukelen & Vlaeyen, 2005; Van der Elst et al., 2006; Van der Elst, Hurks, Wassenberg, Meijs, & Jolles, 2011). This regression-based normative method computes the expected test scores for each possible combination of age and education. This highly individualized approach to normative test references facilitates the study of groups of patients with a wide range of demographic characteristics.

The normative data are presented as algorithms to adjust test scores for age and education, with subsequent correspondence between adjusted scores and percentile distribution. The adjusted scores can be interpreted as Z scores, because they have normal distribution with mean ∼0 and standard deviation ∼1. The percentile ranks can be converted into scaled scores (i.e., mean = 10 and SD = 3), as used by the Mayo's Older Americans Normative Studies (Ivnik et al., 1996; Lucas et al., 1998, 2005). These standardization procedures have significant advantages for both clinical and research practices. The adjustment to the individual's demographic characteristics and the use of common metrics facilitate the comparisons between tests and between individuals.

In this normative study, education was operationalized as the number of years of formal regular schooling completed with success. This approach is vulnerable to the numerous changes in the educational system that have occurred throughout the last decades in Portugal. For instance, the curricula of regular school have suffered significant modifications. Since 2005, a large portion of the Portuguese adult population with low education has enrolled in the “New Opportunities” initiative to enhance their qualifications and to acquire primary (9th grade) or secondary (12th grade) level education certificates. However, the actual equivalence between regular school and “New Opportunities” education programs is still unknown. So, for the purpose of this study, only regular schooling was credited to the participants' education background. Considering these continuing changes in education, future studies ought to assess the reliability of the normative algorithms and update the norms if necessary.

The normative sample was almost exclusively white and was derived mostly from north and central regions of Portugal. Other races and other regions were underrepresented in this study. Regional effects in verbal fluency performance (i.e., number of words generated), beyond the effects of age and education, are not to be expected in a small country with significant internal migration. However, the generalizability of the norms to immigrants from other Portuguese speaking countries (i.e., immigrants from Brazil or the ex-colonies - Angola, Cape Verde, China—Macau, Guinea Bissau, India—Goa, Daman and Diu, Mozambique, East-Timor) may be limited.

The study sample had an unbalanced ratio of women and men. This limitation of the study was minimized by the large sample size and the absence of significant differences between women and men regarding age, education, and test results. It is unlikely that the overrepresentation of women has produced a significant bias on the study results.

Letters M, R, and P are among the easiest for word production both in Portuguese (Senhorini, Amaro Junior, Mello Ayres, Simone, & Busatto, 2006) and in English (Borkowski, Benton, & Spreen, 1967). This set of letters has been chosen for the Spanish norms (Pena-Casanova et al., 2009). Other commonly used letters for phonemic fluency are: F, A, and S (Bolla, Lindgren, Bonaccorsy, & Bleecker, 1990; Tallberg et al., 2008; Tombaugh et al., 1999; Tomer & Levin, 1993); C, F, and L (Barry, Bates, & Labouvie, 2008; Benton & Hamsher, 1976; Ross, Furr, Carter, & Weinberg, 2006; Steinberg et al., 2005); and P, R, and W (Benton and Hamsher, 1976; Ross et al., 2006). Consistent with a Brazilian study (Senhorini et al., 2006), in our Portuguese normative sample, P was the letter that elicited more words within 1min and letter M generated more words than letter R. Regarding the internal reliability of the phonemic fluency test, using letters M, R, and P, the consistency was high, but the absolute agreement between trials was modest. The internal consistency results are slightly higher than other reports (Ruff, Light, Parker, & Levin, 1996; Tombaugh et al., 1999).

Both semantic fluency and phonemic fluency tests revealed excellent inter-rater reliability. Ross (2003) also reported high inter-rater reliability in a phonemic fluency test. Future studies ought to explore other psychometric properties of the tests, such as practice effects. Determining reliable change indices would be useful for the clinicians in retest situations. There are indications in the literature that verbal fluency tests have good test–retest reliability (Harrison et al., 2000; Lemay, Bedard, Rouleau, & Tremblay, 2004; Ross et al., 2007; Ruff et al., 1996). However, the repeated exposure is known to produce significant practice effects (Cooper, Lacritz, Weiner, Rosenberg, & Cullum, 2004; Ross et al., 2007; Wilson, Watson, Baddeley, Emslie, & Evans, 2000), particularly in healthy individuals (Wilson et al., 2000).

Semantic fluency and phonemic fluency were standardized as part of a larger normalization project. Data from a series of widely used neuropsychological instruments are being collected in the same (or substantially overlapping) study sample. This conorming approach is believed to facilitate the comparison between results from different neuropsychological measures. The collection of normative data from multiple tests is also beneficial because it provides a closer parallel to the clinical setting.

Supplementary Material

Supplementary material is available at Archives of Clinical Neuropsychology online.

Funding

This research received no specific grant from any funding agency, commercial or not-for-profit sectors. Armando Teixeira-Pinto was supported by the Australian National Health and Medical Research Council Grant 402764 to the Screening and Test Evaluation Program.

Conflit of interest

None declared.

Acknowledgments

Psychologists Alexandra Pereira, Alexandre Abreu, Ana Cristina Nunes, Ana Sampaio, Andreia Gonçalves Azevedo, Cátia Alves, Cláudia Dias, Estelle Lima, Felícia Silva, Filipa Júlio, Liliana Noronha, Maria do Céu Seabra, Mariana Oliveira, Marta Pinto, Patrícia Matos, Rui Quintas, Sandra Freitas, Sara Cabaço, Susana Pinto, Susana Vieira da Silva, and Vânia Santos collaborated in the collection of data.

References

Alvarez
J. A.
Emory
E.
Executive function and the frontal lobes: A meta-analytic review
Neuropsychology Review
 , 
2006
, vol. 
16
 (pg. 
17
-
42
)
Baciu
M.
Juphard
A.
Cousin
E.
Bas
J. F.
Evaluating fMRI methods for assessing hemispheric language dominance in healthy subjects
European Journal of Radiology
 , 
2005
, vol. 
55
 (pg. 
209
-
218
)
Baldo
J. V.
Schwartz
S.
Wilkins
D.
Dronkers
N. F.
Role of frontal versus temporal cortex in verbal fluency as revealed by voxel-based lesion symptom mapping
Journal of the International Neuropsychological Society
 , 
2006
, vol. 
12
 (pg. 
896
-
900
)
Barry
D.
Bates
M. E.
Labouvie
E.
FAS and CFL forms of verbal fluency differ in difficulty: A meta-analytic study
Applied Neuropsychology
 , 
2008
, vol. 
15
 (pg. 
97
-
106
)
Benito-Cuadrado
M. M.
Esteba-Castillo
S.
Bohm
P.
Cejudo-Bolivar
J.
Pena-Casanova
J.
Semantic verbal fluency of animals: A normative and predictive study in a Spanish population
Journal of Clinical and Experimental Neuropsychology
 , 
2002
, vol. 
24
 (pg. 
1117
-
1122
)
Benton
A. L.
Hamsher
K.
Multilingual Aphasia Examination manual
 , 
1976
Iowa City, IA
University of Iowa
Billingsley
R. L.
Simos
P. G.
Castillo
E. M.
Sarkari
S.
Breier
J. I.
Pataraia
E.
, et al.  . 
Spatio-temporal cortical dynamics of phonemic and semantic fluency
Journal of Clinical & Experimental Neuropsychology
 , 
2004
, vol. 
26
 (pg. 
1031
-
1043
)
Birn
R. M.
Kenworthy
L.
Case
L.
Caravella
R.
Jones
T. B.
Bandettini
P. A.
, et al.  . 
Neural systems supporting lexical search guided by letter and semantic category cues: A self-paced overt response fMRI study of verbal fluency
Neuroimage
 , 
2010
, vol. 
49
 (pg. 
1099
-
1107
)
Bolla
K. I.
Lindgren
K. N.
Bonaccorsy
C.
Bleecker
M. L.
Predictors of verbal fluency (FAS) in the healthy elderly
Journal of Clinical Psychology
 , 
1990
, vol. 
46
 (pg. 
623
-
628
)
Borkowski
J. G.
Benton
A. L.
Spreen
O.
Word fluency and brain damage
Neuropsychologia
 , 
1967
, vol. 
5
 (pg. 
135
-
140
)
Capitani
E.
Laiacona
M.
Barbarotto
R.
Gender affects word retrieval of certain categories in semantic fluency tasks
Cortex
 , 
1999
, vol. 
35
 (pg. 
273
-
278
)
Cavaco
S.
Martins da Silva
A.
Coutinho
E.
Marinho
A.
Moreira
I.
Gonçalves
A.
, et al.  . 
Are cognitive and olfactory dysfunctions in neuropsychiatric lupus erythematosus dependent on anxiety or depression?
The Journal of Rheumatology
 , 
2012
, vol. 
39
 (pg. 
770
-
776
)
Cavaco
S.
Teixeira-Pinto
A.
Dementia Rating Scale-2: Manual Técnico (Versão Portuguesa)
 , 
2011
Lisboa
Cegoc-Tea
Cerhan
J. H.
Ivnik
R. J.
Smith
G. E.
Tangalos
E. C.
Petersen
R. C.
Boeve
B. F.
Diagnostic utility of letter fluency, category fluency, and fluency difference scores in Alzheimer's disease
The Clinical Neuropsychologist
 , 
2002
, vol. 
16
 (pg. 
35
-
42
)
Clark
L. J.
Gatz
M.
Zheng
L.
Chen
Y. L.
McCleary
C.
Mack
W. J.
Longitudinal verbal fluency in normal aging, preclinical, and prevalent Alzheimer's disease
American Journal of Alzheimer's Disease and Other Dementias
 , 
2010
, vol. 
24
 (pg. 
461
-
8
)
Cooper
D. B.
Lacritz
L. H.
Weiner
M. F.
Rosenberg
R. N.
Cullum
C. M.
Category fluency in Mild Cognitive Impairment: Reduced effect of practice in test-retest conditions
Alzheimer Disease and Associated Disorders
 , 
2004
, vol. 
18
 (pg. 
120
-
2
)
Crossley
M.
D'arcy
C.
Rawson
N. S. B.
Letter and category fluency in community-dwelling canadian seniors: A comparison of normal participants to those with dementia of the Alzheimer or vascular type
Journal of Clinical and Experimental Neuropsychology
 , 
1997
, vol. 
19
 (pg. 
52
-
62
)
Dujardin
K.
Defebvre
L.
Duhamel
A.
Lecouffe
P.
Rogelet
P.
Steinling
M.
, et al.  . 
Cognitive and SPECT characteristics predict progression of Parkinson's disease in newly diagnosed patients
Journal of Neurology
 , 
2004
, vol. 
251
 (pg. 
1383
-
1392
)
Gauthier
C. T.
Duyme
M.
Zanca
M.
Capron
C.
Sex and performance level effects on brain activation during a verbal fluency task: A functional magnetic resonance imaging study
Cortex
 , 
2009
, vol. 
45
 (pg. 
164
-
176
)
Gladsjo
J. A.
Schuman
C. C.
Evans
J. D.
Peavy
G. M.
Miller
S. W.
Heaton
R. K.
Norms for letter and category fluency: Demographic corrections for age, education, and ethnicity
Assessment
 , 
1999
, vol. 
6
 (pg. 
147
-
178
)
Gouveia
P. A.
Brucki
S. M.
Malheiros
S. M.
Bueno
O. F.
Disorders in planning and strategy application in frontal lobe lesion patients
Brain and Cognition
 , 
2007
, vol. 
63
 (pg. 
240
-
246
)
Green
J.
McDonald
W. M.
Vitek
J. L.
Evatt
M.
Freeman
A.
Haber
M.
, et al.  . 
Cognitive impairments in advanced PD without dementia
Neurology
 , 
2002
, vol. 
59
 (pg. 
1320
-
1324
)
Greenaway
M. C.
Smith
G. E.
Tangalos
E. G.
Geda
Y. E.
Ivnik
R. J.
Mayo older Americans normative studies: Factor analysis of an expanded neuropsychological battery
The Clinical Neuropsychologist
 , 
2009
, vol. 
23
 (pg. 
7
-
20
)
Harrison
J. E.
Buxton
P.
Husain
M.
Wise
R.
Short test of semantic and phonological fluency: Normal performance, validity and test-retest reliability
British Journal of Clinical Psychology
 , 
2000
, vol. 
39
 (pg. 
181
-
191
)
Henry
J. D.
Crawford
J. R.
Phillips
L. H.
Verbal fluency performance in dementia of the Alzheimer's type: A meta-analysis
Neuropsychologia
 , 
2004
, vol. 
42
 (pg. 
1212
-
1222
)
Henry
J. D.
Crawford
J. R.
Phillips
L. H.
A meta-analytic review of verbal fluency deficits in Huntington's disease
Neuropsychology
 , 
2005
, vol. 
19
 (pg. 
243
-
252
)
Herrmann
M. J.
Walter
A.
Ehlis
A.-C.
Fallgatter
A. J.
Cerebral oxygenation changes in the prefrontal cortex: Effects of age and gender
Neurobiology of Aging
 , 
2006
, vol. 
27
 (pg. 
888
-
894
)
Ivnik
R. J.
Malec
J. F.
Smith
G. E.
Tangalos
E. G.
Neuropsychological tests' norms above age 55: COWAT, BNT, MAE Token, WRAT-R Reading, AMNART, STROOP, TMT, and JLO
The Clinical Neuropsychologist
 , 
1996
, vol. 
10
 (pg. 
262
-
278
)
Jacobs
D. M.
Marder
K.
Sano
M.
Stern
Y.
Mayeux
R.
Neuropsychological characteristics of preclinical dementia in Parkinson's disease
Neurology
 , 
1995
, vol. 
45
 (pg. 
1691
-
1696
)
Kinkingnehun
S.
Volle
E.
Pelegrini-Issac
M.
Golmard
J.-L.
Lehericy
S.
Boisgueheneuc
F.
, et al.  . 
A novel approach to clinical–radiological correlations: Anatomo-Clinical Overlapping Maps (AnaCOM): Method and validation
Neuroimage
 , 
2007
, vol. 
37
 (pg. 
1237
-
1249
)
Kosmidis
M. H.
Vlahou
C. H.
Panagiotaki
P.
Kiosseoglou
G.
The verbal fluency task in the Greek population: Normative data, and clustering and switching strategies
Journal of the International Neuropsychological Society
 , 
2004
, vol. 
10
 (pg. 
164
-
172
)
Larsson
M. U.
Almkvist
O.
Luszcz
M. A.
Wahlin
T. B.
Phonemic fluency deficits in asymptomatic gene carriers for Huntington's disease
Neuropsychology
 , 
2008
, vol. 
22
 (pg. 
596
-
605
)
Lemay
S.
Bedard
M. A.
Rouleau
I.
Tremblay
P. L.
Practice effect and test-retest reliability of attentional and executive tests in middle-aged to elderly subjects
The Clinical Neuropsychologist
 , 
2004
, vol. 
18
 (pg. 
284
-
302
)
Levy
G.
Jacobs
D. M.
Tang
M. X.
Cote
L. J.
Louis
E. D.
Alfaro
B.
, et al.  . 
Memory and executive function impairment predict dementia in Parkinson's disease
Movement Disorders
 , 
2002
, vol. 
17
 (pg. 
1221
-
1226
)
Libon
D. J.
McMillan
C.
Gunawardena
D.
Powers
C.
Massimo
I.
Khan
A.
, et al.  . 
Neurocognitive contributions to verbal fluency deficits in frontotemporal lobar degeneration
Neurology
 , 
2009
, vol. 
73
 (pg. 
535
-
542
)
Lucas
J. A.
Ivnik
R. J.
Smith
G. E.
Bohac
D. L.
Tangalos
E. G.
Graff-Radford
N. R.
, et al.  . 
Mayo's older Americans normative studies: Category fluency norms
Journal of Clinical and  Experimental Neuropsychology
 , 
1998
, vol. 
20
 (pg. 
194
-
200
)
Lucas
J. A.
Ivnik
R. J.
Smith
G. E.
Ferman
T. J.
Willis
F. B.
Petersen
R. C.
, et al.  . 
Mayo's Oder African Americans Normative Studies: Norms for Boston Naming Test, Controlled Oral Word Association, Category Fluency, Animal Naming, Token Test, Wrat-3 Reading, Trail Making Test, Stroop Test, and Judgment of Line Orientation
The Clinical Neuropsychologist
 , 
2005
, vol. 
19
 (pg. 
243
-
269
)
Manly
J. J.
Jacobs
D. M.
Sano
M.
Bell
K.
Merchant
C. A.
Small
S. A.
, et al.  . 
Effect of literacy on neuropsychological test performance in nondemented, education-matched elders
Journal of the International Neuropsychological Society
 , 
1999
, vol. 
5
 (pg. 
191
-
202
)
Murphy
K. J.
Rich
J. B.
Troyer
A. K.
Verbal fluency patterns in amnestic mild cognitive impairment are characteristic of Alzheimer's type dementia
Journal of the International Neuropsychological Society
 , 
2006
, vol. 
12
 (pg. 
570
-
574
)
Palmer
K.
Backman
L.
Winblad
B.
Fratiglioni
L.
Detection of Alzheimer's disease and dementia in the preclinical phase: Population based cohort study
British Medical Journal
 , 
2003
, vol. 
326
 (pg. 
245
-
249
)
Pena-Casanova
J.
Quinones-Ubeda
S.
Gramunt-Fombuena
N.
Quintana-Aparicio
M.
Aguilar
M.
Badenes
D.
, et al.  . 
Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for Verbal Fluency Tests
Archives of Clinical Neuropsychology
 , 
2009
, vol. 
24
 (pg. 
395
-
411
)
Petersson
K. M.
Reis
A.
Askelof
S.
Castro-Caldas
A.
Ingvar
M.
Language processing modulated by literacy: A network analysis of verbal repetition in literate and illiterate subjects
Journal of Cognitive Neuroscience
 , 
2000
, vol. 
12
 (pg. 
364
-
382
)
Rascovsky
R.
Salmon
D. P.
Ho
G. J.
Galasko
D.
Peavy
G. M.
Hansen
L. A.
, et al.  . 
Cognitive profiles differ in autopsy-confirmed frontotemporal dementia and AD
Neurology
 , 
2002
, vol. 
58
 (pg. 
1801
-
1808
)
Ratcliff
G.
Ganguli
M.
Chandra
V.
Sharma
S.
Belle
S.
Seaberg
E.
, et al.  . 
Effects of literacy and education on measures of word fluency
Brain and Language
 , 
1998
, vol. 
61
 (pg. 
115
-
122
)
Ravnkilde
B.
Videbech
P.
Rosenberg
R.
Gjedde
A.
Gade
A.
Putative tests of frontal lobe function: A PET-study of brain activation during Stroop's test and verbal fluency
Journal of Clinical and Experimental Neuropsychology
 , 
2002
, vol. 
24
 (pg. 
534
-
547
)
Reis
A.
Castro-Caldas
A.
Illiteracy: A cause for biased cognitive development
Journal of the International Neuropsychological Society
 , 
1997
, vol. 
3
 (pg. 
444
-
450
)
Rogers
T. T.
Ivanoiu
A.
Patterson
K.
Hodges
J. R.
Semantic memory in Alzheimer's disease and the frontotemporal dementias: A longitudinal study of 236 patients
Neuropsychology
 , 
2006
, vol. 
20
 (pg. 
319
-
335
)
Ross
T. P.
The reliability of cluster and switch scores for the Controlled Oral Word Association Test
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 (pg. 
153
-
164
)
Ross
T. P.
Calhoun
E.
Cox
T.
Wenner
C.
Kono
W.
Pleasant
M.
The reliability and validity of qualitative scores for the Controlled Oral Word Association Test
Archives of Clinical Neuropsychology
 , 
2007
, vol. 
22
 (pg. 
475
-
488
)
Ross
T. P.
Furr
A. E.
Carter
S. E.
Weinberg
M.
The psychometric equivalence of two alternate forms of the controlled oral word association test
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
414
-
431
)
Ruff
R. M.
Light
R. H.
Parker
S. B.
Levin
H. S.
Benton Controlled Oral Word Association Test: Reliability and updated norms
Archives of Clinical Neuropsychology
 , 
1996
, vol. 
11
 (pg. 
329
-
338
)
Ryu
S. H.
Kim
K. W.
Kim
S.
Park
J. H.
Kim
T. H.
Jeong
H. G.
, et al.  . 
Normative study of the category fluency test (CFT) from nationwide data on community-dwelling elderly in Korea
Archives of Gerontology and Geriatrics
 , 
2012
, vol. 
54
 (pg. 
305
-
309
)
Santangelo
G.
Trojano
L.
Vitale
C.
Ianniciello
M.
Amboni
M.
Grossi
D.
, et al.  . 
A neuropsychological longitudinal study in Parkinson's patients with and without hallucinations
Movement Disorders
 , 
2007
, vol. 
22
 (pg. 
2418
-
2425
)
Senhorini
M. C. T.
Amaro Junior
E.
Mello Ayres
A.
Simone
A.
Busatto
G. F.
Phonemic fluency in Portuguese-speaking subjects in Brazil: Ranking of letters
Journal of Clinical and Experimental Neuropsychology
 , 
2006
, vol. 
28
 (pg. 
1191
-
1200
)
Silva
C. G.
Petersson
K. M.
Faísca
L.
Ingvar
M.
Reis
A.
The effects of literacy and education on the quantitative and qualitative aspects of semantic verbal fluency
Journal of Clinical and Experimental Neuropsychology
 , 
2004
, vol. 
26
 (pg. 
266
-
277
)
Steinberg
B. A.
Bieliauskas
L. A.
Smith
G. E.
Ivnik
R. J.
Mayo's older Americans normative studies: Age- and IQ-adjusted norms for the trail making test, the stroop test, and MAE controlled oral word association test
The Clinical Neuropsychologist
 , 
2005
, vol. 
19
 (pg. 
329
-
377
)
Strauss
E.
Sherman
E. M. S.
Spreen
O.
A compendium of neuropsychological tests: Administration, norms, and commentary
 , 
2006
3rd ed.
New York
Oxford University Press
Szatkowska
I.
Grabowska
A.
Szymanska
O.
Phonological and semantic fluencies are mediated by different regions of the prefrontal cortex
Acta Neurobiologiae Experimentalis
 , 
2000
, vol. 
60
 (pg. 
503
-
508
)
Tallberg
I. M.
Ivachova
E.
Tinchag
K. J.
Ostberg
P.
Swedish norms for word fluency tests: FAS, animals and verbs
Scandinavian Journal of Psychology
 , 
2008
, vol. 
49
 (pg. 
479
-
485
)
Tombaugh
T. N.
Kovak
J.
Rees
L.
Normative data stratified by age and education for two measures of verbal fluency: FAS and animal naming
Archives of Clinical Neuropsychology
 , 
1999
, vol. 
14
 (pg. 
167
-
177
)
Tomer
R.
Levin
B. E.
Differential effects of aging on two verbal fluency tasks
Perceptual and Motor Skills
 , 
1993
, vol. 
76
 (pg. 
465
-
466
)
Tupak
S. V.
Badewien
M.
Dresler
T.
Hahn
T.
Ernst
L. H.
Herrmann
M. J.
, et al.  . 
Differential prefrontal and frontotemporal oxygenation patterns during phonemic and semantic verbal fluency
Neuropsychologia
 , 
2012
, vol. 
50
 (pg. 
1565
-
1569
)
Williams-Gray
C. H.
Evans
J. R.
Goris
A.
Foltynie
T.
Ban
M.
Robbins
T. W.
, et al.  . 
The distinct cognitive syndromes of Parkinson's disease: 5 year follow-up of the CamPalGN cohort
Brain
 , 
2009
, vol. 
132
 (pg. 
2958
-
2969
)
Wilson
B. A.
Watson
P. C.
Baddeley
A. D.
Emslie
H.
Evans
J. J.
Improvement or simply practice? The effects of twenty repeated assessments on people with and without brain injury
Journal of the International Neuropsychological Society
 , 
2000
, vol. 
6
 (pg. 
469
-
479
)
Van Breukelen
G. J. P.
Vlaeyen
J. W. S.
Norming clinical questionnaires with multiple regression: The pain cognition list
Psychological Assessment
 , 
2005
, vol. 
17
 (pg. 
336
-
344
)
Van der Elst
W.
Boxtel
M.
Breukelen
G.
Jolles
J.
Normative data for the animal, profession and letter M naming verbal fluency tests for Dutch speaking participants and the effects of age, education, and sex
Journal of the International Neuropsychological Society
 , 
2006
, vol. 
12
 (pg. 
80
-
89
)
Van der Elst
W.
Hurks
P.
Wassenberg
R.
Meijs
C.
Jolles
J.
Animal verbal fluency and design fluency in school-aged children: Effects of age, sex, and mean level of parental education, and regression-based normative data
Journal of Clinical and Experimental Neuropsychology
 , 
2011
, vol. 
33
 (pg. 
1005
-
1015
)