Abstract

The Letter Digit Substitution Test (LDST) was administered to a sample of N = 296 healthy children (aged between 8.03 and 15.87). The aim of the present study was to evaluate the impact of age, gender, and parental educational level on LDST performance and to establish demographically corrected normative data. The results showed that the relationship between age and LDST performance was curvilinear (i.e., improvements in test performance were more pronounced for younger children than for older children) and was moderated by gender (i.e., the gender differences were small at younger ages but increased as a function of age, with girls outperforming boys). Moreover, children who had parents with a higher level of education outperformed their counterparts who had parents with a lower level of education. Regression-based normative LDST data were established, and an automatic scoring program was provided.

Introduction

Substitution tests are speed-dependent cognitive tasks that require the participants to match particular signs (e.g., symbols, digits, or letters) to other signs (Lezak, Howieson, & Loring, 2004). Substitution tests are mainly used to assess information processing speed, a cognitive ability that reflects the speed by which elementary cognitive operations can be performed. Information processing speed is a fundamental cognitive ability in neuropsychological theories on “normal” cognitive development and “normal” cognitive aging (Fry & Hale, 1996, 2000; Kail, 2000, 2007; Salthouse, 1996; Sheppard & Vernon, 2008). For example, developmental studies have shown that age-related improvements in processing speed result in an increased working memory span, better reasoning abilities, and more efficient long-term memory processes (Kail, 2000). Similarly, an age-related reduction in information processing speed has been associated with a deterioration in the working memory and long-term episodic memory processes of older people (Salthouse, 1996). Information processing speed is also affected in a number of clinical conditions in children and adults, such as attention deficit hyperactivity disorder, depression, pediatric traumatic brain injury, autism, Alzheimer's disease, multiple sclerosis, and schizophrenia (Calhoun & Mayes, 2005; DeLuca & Kalmar, 2008; Donders, 2006; Lezak et al., 2004; Spreen & Strauss, 1998; Weiler, Bernstein, Bellinger, & Waber, 2000).

In view of the central place of information processing speed in different cognitive theories and its sensitivity for a variety of clinical conditions, substitution tests have become popular assessment tools in both research and clinical settings (Lezak et al., 2004). The present study focuses on the Letter Digit Substitution Test (LDST; Jolles, Houx, Van Boxtel, & Ponds, 1995), which is an adaptation of earlier substitution tests—such as the Digit Symbol Substitution Test (DSST; Wechsler, 1955, 1981) and the Symbol Digit Modalities Test (SDMT; Smith, 1982). In the DSST, digits are presented and the test participant has to respond by writing down a symbol. This relation is reversed in the SDMT, that is, symbols are presented and the test participant has to respond by writing down a digit. The LDST differs from the DSST and the SDMT in that the key consists of well-known signs, that is, letters and digits (Jolles et al., 1995). The SDMT and the LDST have the advantage over the DSST that responses can be given in both writing and verbally. In this way, the SDMT and the LDST can also be administered to people who have motor problems that hamper writing (which is not possible with the DSST).

Normative LDST data for adults have been established earlier (Van der Elst, Van Boxtel, Van Breukelen, & Jolles, 2006a), but norms for children were lacking until now. In the present study, we administered the LDST to a sample of N = 296 children (aged between 8.03 and 15.87). Previous research in adults has suggested that age, gender, and educational level affected LDST performance (Van Boxtel et al., 1997; Van der Elst et al., 2006a). Thus, we also evaluated the impact of demographic variables on the LDST performance of the children in the normative sample and derived demographically corrected norms by means of a regression-based procedure.

Method

Participants

The data were derived from the COOS (Cognitief Ontwikkelings Onderzoek bij Schoolgaande kinderen, in English: Cognitive developmental study in school-aged children), a large scale study into “normal” cognitive development. All children came from public schools. Children who (a) had not repeated or skipped a grade, (b) did not use medication known to affect cognitive performance (such as Ritalin), and (c) did not suffer from severe medical conditions known to affect cognitive performance (such as epilepsy) were eligible for study inclusion. Medication use and health status were assessed by means of a parental-report questionnaire.

The LDST was individually administered to a sample of 296 children (at school). None of the children had significant motor or sensorial impairments that hampered administration of the LDST. Basic demographic data for the participants were provided in Table 1. The children were aged between 8.03 and 15.87. Age was used as a continuous variable in the normative analyses (see below), but it was categorized into bands of 2 years in Table 1 (for descriptive purposes). The educational level of the children's parents (or caregivers) was measured with a commonly used Dutch educational 8-point rating scale which ranges from primary school to university degree (De Bie, 1987). Mean Level of Parental Education (MLPE) was dichotomized into Low and High groups (after a median split) for parents who had MLPE values that were <5 and ≥5 on the 8-point scale, respectively (with 5 = at most junior vocational education). These two levels of education corresponded with a mean (SD) of about 9.88 (2.59) and 14.68 (3.30) years of full-time education, respectively. All children were native Dutch speakers, and all parents (or caregivers) of the children gave consent for their child to participate in the research. The Ethics Committee of the Faculty of Psychology and Neuroscience of Maastricht University approved the study protocol.

Table 1.

Basic demographic data for the normative sample (N = 296)

Age range (years) N Age (M [SD]) Gender
 
Mean level of parental education
 
   Women Men Low High 
≥8.03 and <10 60 8.67 (0.57) 24 36 31 29 
≥10 and <12 70 10.66 (0.53) 45 25 39 31 
≥12 and <14 109 12.97 (0.54) 52 57 50 59 
≥14 and ≤15.87 57 14.52 (0.35) 27 30 38 19 
Total 296 11.85 (2.12) 148 148 158 138 
Age range (years) N Age (M [SD]) Gender
 
Mean level of parental education
 
   Women Men Low High 
≥8.03 and <10 60 8.67 (0.57) 24 36 31 29 
≥10 and <12 70 10.66 (0.53) 45 25 39 31 
≥12 and <14 109 12.97 (0.54) 52 57 50 59 
≥14 and ≤15.87 57 14.52 (0.35) 27 30 38 19 
Total 296 11.85 (2.12) 148 148 158 138 

Procedure and Instruments

The LDST sheet presents the substitution key on the top of the page. The key shows the numbers 1–9, which are all paired with a different letter. The test items are printed beneath the key. The children were instructed to write down the appropriate digits in the blank spaces under the letters. The first 10 items were used as practice items (to ensure that the children understood the test instructions). After completion of the practice items, the children were instructed to fill in the remaining items as quickly and as accurately as possible. It was not allowed to skip items. After 60 s, the test was stopped. The number of correct substitutions that were made in 60 s was the outcome variable.

Statistical Analyses

The LDST data were analyzed by means of a multiple linear regression model. Age, Age2, Sex, MLPE, and all two-way interactions between these variables were used as predictors in the full regression model. Age was centered (=calendar age − mean age in the sample [=12 years]) before computing the quadratic age term (to avoid multicollinearity; Aiken & West, 1991). Both Age and Age2 were added as predictors in the model to allow for a curvilinear relationship between age and the LDST scores. When only a linear age term is included in the model, the relation between age and the outcome is assumed to be linear over the entire age range. This is a strong assumption that may not be fulfilled, for example, it is possible that the age-related increase in LDST performance is larger for younger children than for older children. MLPE was dummy coded as 1 = high and 0 = low. Sex was dummy coded as 1 = men and 0 = women. The full regression model was reduced by means of a hierarchical step-down procedure in which the non-significant predictors were removed from the model (starting with the higher-order terms). The assumptions of regression analysis were tested by conducting a Kolmogorov–Smirnov test (to evaluate the normality of the residuals), by grouping the participants into quartiles of the predicted scores and applying the Levene test (to evaluate homoscedasticity), by calculating Variance Inflation Factors (VIFs) and condition indices (to identify multicollinearity), and by computing Cook's distances (to identify influential cases). The Kolmogorov–Smirnov and the Levene tests should not be significant. The VIF and the condition index of a predictor should not exceed 10 and 15, respectively (Belsley, Kuh, & Welsch, 1980; Kutner, Nachtsheim, Neter, & Li, 2005). The maximum Cook's distance value was related to an F(p, np) distribution (with p = the number of predictors in the regression model and n = the total number of observations). When the corresponding percentile value is about 50 or higher, the observation is influential (Kutner et al., 2005).

The normative data were established by means of a four-step procedure (Testa, Winicki, Pearlson, Gordon, & Schretlen, 2009; Van Breukelen & Vlaeyen, 2005; Van der Elst et al., 2006a, in press; Van der Elst, Van Boxtel, Van Breukelen, & Jolles, 2006b). First, the expected LDST score of a test participant was computed as based on the final multiple regression model that was established in the normative sample, that is, Expected test scorei = B0 + B1X1i + ··· + BkXki (with B0 = the intercept, Bk = the regression weights for the demographic variables, and Xki = the values of the demographic variables for test participant i). Second, the residual was calculated, that is, ei = observed test scorei − expected test scorei. Third, the residual was standardized, that is, Zi = ei/SD(residual). If heteroscedasticity occurred, the SD(residual) values were estimated by regressing the squared residuals (forumla) on the predicted values, and taking the square root of the obtained value (Kutner et al., 2005). Fourth, the standardized residuals were converted into percentile values via the standard normal cumulative distribution function.

The analyses were conducted with the “R” software package (version 2.11.1 for Linux), using a α level of 0.05.

Results

The LDST Regression Model

The final regression model for the LDST score is presented in Table 2. The Levene test suggested that the homoscedasticity assumption was violated (Levene statistic = 3.45, p = .02). The estimated SD(residual) function equaled √(−6.09 + (0.962 × predicted scorei)). The assumption of the normal distribution of the standardized residuals was not violated (Kolmogorov–Smirnov Z = 0.789, p = .56), multicollinearity was not a problem (the VIFs and the condition indices of the predictors had a maximum value of 2.41 and 4.12, respectively), and there were no influential cases (the maximum Cook's distance value was 0.07; referring to an F6,290 distribution, this value corresponds to percentile 0.13).

Table 2.

Final multiple linear regression model for the LDST score that resulted from a step-down hierarchical procedure

Variable B SE B Std. B T R2 
Constant 32.52 0.55  58.55*  
Age 2.92 0.21 0.79 14.21*  
Age2 −0.25 0.07 −0.14 −3.47*  
Gender −3.01 0.57 −0.19 −5.31*  
MLPE 1.24 0.57 0.08 2.19*  
Age × Gender −0.62 0.27 −0.12 −2.34* .63 
Variable B SE B Std. B T R2 
Constant 32.52 0.55  58.55*  
Age 2.92 0.21 0.79 14.21*  
Age2 −0.25 0.07 −0.14 −3.47*  
Gender −3.01 0.57 −0.19 −5.31*  
MLPE 1.24 0.57 0.08 2.19*  
Age × Gender −0.62 0.27 −0.12 −2.34* .63 

Notes: Encoding of the predictors: Age = calendar age − 12; Age2 = (calendar age − 12)2. Gender: Women = 0, men = 1. Mean Level of Parental Education: Low = 0, high = 1. R2 is adjusted for the number of predictors in the regression equation.

*p < .05.

As shown in Table 2, there was a significant interaction between age and gender on the LDST score. Fig. 1 shows the effects of age and gender on the expected LDST score graphically (for children who have parents with a low MLPE; the shape of these curves is identical for children who have parents with a high MLPE, up to a constant; see Table 2). As shown, the gender differences in the LDST scores were small for young children but increased as a function of age. For example, the test scores of 8-year-old boys and girls were approximately equal, but by the age of 15.5, girls scored on average 5.2 points higher (i.e., 15% higher) than boys.

Fig. 1.

Expected LDST scores as a function of age and gender (for children who had parents with a low MLPE; the shape of these curves is identical for children who have parents with a high MLPE, up to a constant).

Fig. 1.

Expected LDST scores as a function of age and gender (for children who had parents with a low MLPE; the shape of these curves is identical for children who have parents with a high MLPE, up to a constant).

In addition to age and sex, MLPE was found to significantly affect the LDST performance. Children who had parents with a high MLPE scored on average 1.24 points higher than children who had parents with a low MLPE. The regression model explained 63% of the variance in the LDST scores.

The Four-Step Normative Procedure

The LDST score of a tested child is normed by means of the four-step procedure described above. For example, suppose that an 8-year-old boy who had parents with a high MLPE made 15 correct substitutions in 60 s. The predictors to be used in the regression model are thus coded as follows: Age = −4 (=8 − 12), Age2 = 16 (=[8 − 12]2), Gender = 1, MLPE = 1, and Age × Gender = −4 (=[8 − 12] × 1). In the first step of the normative procedure, the expected LDST score for the tested child is computed, that is, expected test score= 32.52 + (−4 × 2.92) + (16 × −0.25) + (1 × −3.01) + (1 × 1.24) + (−4 × −0.62) = 17.55. Secondly, the residual is calculated, that is, −2.55 (=15 − 17.55). Thirdly, the residual is standardized. As the predicted LDST score for this child equaled 17.55, the SD(residual) value to be used equals 3.29 (=√(−6.09 + (0.962 × 17.55))). Thus, the standardized residual is −0.78 (=−2.55/3.29). Fourthly, the standardized residual is converted into a percentile value by means of the standard normal distribution. A standardized residual that equals −0.78 corresponds to a percentile value of 22. Thus, 22% of the “normal” population of 8-year-old boys who have parents with a high MLPE obtain an LDST score that is lower than 15. It can, thus, be concluded that the information processing ability of this child (as assessed with the LDST) is within normal limits.

User-Friendly Normative Data

The four-step normative procedure that was described in the previous paragraph lacks user-friendliness, because the user of the norms has to actively make the required computations. To increase the user-friendliness of the normative data, we provided a normative table that requires no computations at all (seeSupplementary material online, Table S1). For example, Supplementary material online, Table S1 immediately shows that the LDST score equal to 15 that was obtained by the 8-year-old boy from the previous example corresponds to a percentile value between 20 and 30.

The simplified normative table is easy to use but lacks some accuracy because the child's age has to be rounded off (if he or she is not exactly 8, 8.5, … , or 15.5 years old), and because only a limited number of percentile values can be presented in this table (because of space limitations). To maximize both the user-friendliness and the accuracy of the norms, the normative conversion procedure was implemented into an Excel worksheet (which can be downloaded at http://home.deds.nl/~wimvde/). The use of this worksheet is straightforward: The user simply types in the age, gender, and MLPE of the tested child together with his or her obtained LDST score, and the algorithm automatically computes the corresponding percentile value.

Discussion

The main aim of the present study was to establish the normal range of performance on the LDST for school-aged children. As a first step in the normative analyses, the influence of demographic variables on the LDST performance was evaluated. The results showed that the relationship between age and LDST performance was curvilinear (i.e., the relative increase in test performance was more pronounced for younger children than for older children; see Fig. 1). Moreover, the relation between age and LDST performance was moderated by gender (i.e., the age-related increase in test performance was more pronounced for girls than for boys). For example, the average relative increase in the LDST score of a 9-year-old child when compared with the test score of an 8-year-old child was 27.5% for girls and 24.5% for boys, while the average relative increase in the LDST score of a 15.5-year-old child when compared with the test score of a 14.5-year-old child was only 3.6% for girls and 2.3% for boys. These results are in line with previous studies (Camarata & Woodcock, 2006). The developmental increase in processing speed has been attributed to a variety of neuropsychological processes, such as an increase in the speed of stimulus identification and encoding, faster response selection, and a reduction in decision-making time (Miller & Vernon, 1997). At the neurological level, the age-related increase in information processing speed has been attributed to myelination processes and increases in the number of connections in the nervous system during childhood and early adolescence (Bjorklund & Harnishfeger, 1990; Kail, 2000; Kail & Salthouse, 1994). The advantage of girls in substitution test performance has been attributed to their superior verbal abilities (Fry & Hale, 2000; Joy, Kaplan, & Fein, 2004), which is at the neurological level reflected by a more bilateral pattern of activation of the frontal and temporal regions in girls (Burman, Bitan, & Booth, 2008). In addition to age and gender, MLPE was found to affect the LDST performance significantly. As expected, children who had parents with a high MLPE had higher LDST scores than children who had parents with a low MLPE. The total variance in the LDST score that was explained by all predictors together (i.e., age, age2, gender, MLPE, and the age × gender interaction) equaled 63% and was thus high.

The LDST and its normative data provide a useful tool in applied and research settings when information processing speed needs to be assessed, but there are some limitations that warrant further discussion. First, all children who participated in the study were native Dutch speakers. The question rises whether the established normative data can also be applied to children who have a different native language or a different cultural background. An earlier study has shown that the LDST performance of adults who lived in eight European countries and the USA was comparable (Møller et al., 1998). This finding suggests that the LDST performance of children from different Western countries may also be similar, but empirical research is needed to substantiate this claim. In non-Western countries, the use of the normative LDST data is not recommended because the letters and/or digits may not be familiar symbols for these children and because of general cross-cultural differences which are known to affect performance on cognitive tests (such as degree of familiarity with formal testing; Ardila, 1995).

Second, the normative data that were established in the present study were based on a sample of children who were aged between 8.03 and 15.87. The question rises whether the regression model and its SD(residual) values that were established in the present study can also be used to norm the LDST scores of children whose age falls outside this age range. Extrapolating a regression model requires several assumptions. For example, if we want to estimate the expected test score of a 17-year-old child (as based on the regression model shown in Table 2), it has to be assumed that the relation between age and LDST performance in the age range between 15.87 and 17 is identical (or at least very similar) to the relation between age and LDST performance in the age range between 8.03 and 15.87. If this assumption is not fulfilled, the extrapolation of the regression model may result in biased conclusions (e.g., a child with a “normal” LDST score may be classified as being “impaired,” or vice versa). Since this assumption is not testable (as based on the present data set), it is not recommended to extrapolate the model (except perhaps when the value to be extrapolated is very near to the age limits that were considered in the normative sample, e.g., for a person aged 16 years).

On a related note, it is not recommended to administer the LDST to children who are younger than 8 years. Apart from the extrapolation problem that was described in the previous paragraph, a prerequisite of valid LDST administration is that the letters and digits are well-known symbols for the testee. By the age of 8 years, most children are sufficiently familiar with letters and numbers (as they had at least 1 year of practice in reading and writing at school), but this may not be the case for younger children (see also the SDMT manual, which also uses 8 years as the lower age limit for test administration; Smith, 1982).

Third, the LDST responses can be given both in writing and verbally. In the present study, the written LDST version was administered to all children. Previous research in adults has shown that the test scores that were obtained with the oral LDST version were on average higher than the scores that were obtained with the written LDST version (Van der Elst et al., 2006a). It is conceivable that similar effects of response modality also occur in children, and consequently, the normative data that were established in the present study should not be used when the oral version of the LDST is administered. Future studies should establish the normal range of performance for the oral LDST version in children, which may be especially useful in clinical settings (e.g., to assess the information processing speed of children with paresis or other motor problems that hamper writing).

Fourth, the sample was not balanced with respect to gender and MLPE in each of the age groups (e.g., there were n = 45 girls and n = 25 boys in the 10–12 years age group; see Table 1). This is, however, not a problem when a regression-based normative approach is used, because regression weights are estimated in an unbiased way no matter what the distribution of the predictors in the sample is (as long as the regression model is correctly specified, i.e., as long as no predictors or interaction terms are overlooked; Fox, 1997). Moreover, regression models use the entire sample for estimating and testing effects of the predictors (i.e., the analyses are not based on age subgroups, but treat age as a continuous variable). There is only some loss of statistical power when an unbalanced sample is analyzed, because the standard error of a regression weight is proportional to the √VIF of the predictor at hand (Kleinbaum, Kupper, Muller, & Nizam, 1998)—but this was not a problem in the present study because both gender and MLPE were significant.

Supplementary Material

Supplementary material is available at Archives of Clinical Neuropsychology online.

Conflict of Interest

None declared.

References

Aiken
L. S.
West
S. G.
Multiple regression: Testing and interpreting interactions
 , 
1991
Newbury Park, CA
Sage
Ardila
A.
Directions of research in cross-cultural neuropsychology
Journal of Clinical and Experimental Neuropsychology
 , 
1995
, vol. 
17
 (pg. 
143
-
150
)
Belsley
D. A.
Kuh
E.
Welsch
R. E.
Regression diagnostics: Identifying influential data and sources of collinearity
 , 
1980
New York
Wiley
Bjorklund
D. F.
Harnishfeger
K. K.
Cowan
N.
Children's strategies: Their definition and origins
Children's strategies: Contemporary views of cognitive development
 , 
1990
Hillsdale, NJ
Erlbaum
(pg. 
309
-
323
)
Burman
D. D.
Bitan
T.
Booth
J. R.
Sex differences in neural processing of language among children
Neuropsychologia
 , 
2008
, vol. 
46
 (pg. 
1349
-
1362
)
Calhoun
S. L.
Mayes
S. D.
Processing speed in children with clinical disorders
Psychology in the Schools
 , 
2005
, vol. 
42
 (pg. 
333
-
343
)
Camarata
S.
Woodcock
R.
Sex differences in processing speed: Developmental effects in males and females
Intelligence
 , 
2006
, vol. 
34
 (pg. 
231
-
252
)
De Bie
S. E.
Standaardvragen 1987: Voorstellen voor uniformering van vraagstellingen naar achtergrondkenmerken en interviews [Standard questions 1987: Proposal for uniformization of questions regarding background variables and interviews]
 , 
1987
Leiden, the Netherlands
Leiden University Press
DeLuca
J.
Kalmar
J. H.
Information processing speed in clinical populations
 , 
2008
New York
Taylor & Francis
Donders
J.
Farmer
J. E.
Donders
J.
Warschausky
S.
Traumatic brain injury
Treating neurodevelopmental disabilities
 , 
2006
New York
The Guilford Press
(pg. 
23
-
41
)
Fox
J.
Applied regression analysis, linear models, and related methods
 , 
1997
Thousand Oaks, CA
Sage
Fry
A. F.
Hale
S.
Processing speed, working memory, and fluid intelligence: Evidence for a developmental cascade
Psychological Science
 , 
1996
, vol. 
7
 (pg. 
237
-
241
)
Fry
A. F.
Hale
S.
Relationships among processing speed, working memory, and fluid intelligence in children
Biological Psychology
 , 
2000
, vol. 
54
 (pg. 
1
-
34
)
Jolles
J.
Houx
P. J.
Van Boxtel
M. P. J.
Ponds
R. W. H. M.
Maastricht Aging Study: Determinants of cognitive aging
 , 
1995
Maastricht, the Netherlands
Neuropsych
Joy
S.
Kaplan
E.
Fein
D.
Speed and memory in the WAIS-III Digit Symbol-Coding subtest across the adult lifespan
Archives of Clinical Neuropsychology
 , 
2004
, vol. 
19
 (pg. 
759
-
767
)
Kail
R. V.
Speed of information processing: Developmental change and links to intelligence
Journal of School Psychology
 , 
2000
, vol. 
38
 (pg. 
51
-
61
)
Kail
R. V.
Longitudinal evidence that increases in processing speed and working memory enhance children's reasoning
Psychological Science
 , 
2007
, vol. 
18
 (pg. 
312
-
313
)
Kail
R. V.
Salthouse
T. A.
Processing speed as a mental capacity
Acta Psychologica
 , 
1994
, vol. 
86
 (pg. 
199
-
225
)
Kleinbaum
D. G.
Kupper
L. L.
Muller
K. E.
Nizam
A.
Applied regression analysis and other multivariate methods
 , 
1998
3rd ed.
New York
Duxbury Press
Kutner
M. H.
Nachtsheim
C. J.
Neter
J.
Li
W.
Applied linear statistical models
 , 
2005
5th ed.
New York
McGraw Hill
Lezak
M. D.
Howieson
D. B.
Loring
D. W.
Neuropsychological assessment
 , 
2004
4th ed.
New York
Oxford University Press
Miller
L. T.
Vernon
P. A.
Developmental changes in speed of information processing in young children
Developmental Psychology
 , 
1997
, vol. 
33
 (pg. 
549
-
554
)
Møller
J. T.
Cluitmans
P.
Rasmussen
L. S.
Houx
P. J.
Rasmussen
H.
Canet
J.
, et al.  . 
Long-term postoperative cognitive dysfunction in the elderly: ISPOCD1 study
Lancet
 , 
1998
, vol. 
351
 (pg. 
857
-
861
)
Salthouse
T. A.
The processing-speed theory of adult age differences in cognition
Psychological Review
 , 
1996
, vol. 
103
 (pg. 
403
-
428
)
Sheppard
L. D.
Vernon
P. A.
Intelligence and speed of information-processing: A review of 50 years of research
Personality and Individual Differences
 , 
2008
, vol. 
44
 (pg. 
535
-
551
)
Smith
A.
Symbol Digit Modalities Test, manual
 , 
1982
Los Angeles
Western Psychological Services
Spreen
O.
Strauss
E.
A compendium of Neuropsychological Tests
 , 
1998
New York
Oxford University Press
Testa
S. M.
Winicki
J. M.
Pearlson
G. D.
Gordon
B.
Schretlen
D. J.
Accounting for estimated IQ in neuropsychological test performance with regression-based techniques
Journal of the International Neuropsychological Society
 , 
2009
, vol. 
15
 (pg. 
1012
-
1022
)
Van Boxtel
P. J.
Gaillard
C.
Houx
P. J.
Buntinx
F.
de Leeuw
P. W.
Jolles
J.
Can blood pressure predict cognitive task performance in a healthy population sample?
Journal of Hypertension
 , 
1997
, vol. 
15
 (pg. 
1069
-
1076
)
Van Breukelen
G. J. P.
Vlaeyen
J. W. S.
Norming clinical questionnaires with multiple regression: The Pain Cognition List
Psychological Assessment
 , 
2005
, vol. 
17
 (pg. 
336
-
344
)
Van der Elst
W.
Ouwehand
C.
van Rijn
P.
Lee
N.
Van Boxtel
M. P. J.
Jolles
J.
The shortened Raven Standard Progressive Matrices: Item Response Theory-based psychometric analyses and normative data
Assessment
 
Van der Elst
W.
Van Boxtel
M. P. J.
Van Breukelen
G. J. P.
Jolles
J.
The Letter Digit Substitution Test: Normative data for 1,858 healthy participants aged 24–81 from the Maastricht Aging Study (MAAS): Influence of age, education, and sex
Journal of Clinical and Experimental Neuropsychology
 , 
2006
, vol. 
28
 (pg. 
998
-
1009
)
Van der Elst
W.
Van Boxtel
M. P. J.
Van Breukelen
G. J. P.
Jolles
J.
The Stroop Color-Word Test: Influence of age, sex, and education; and normative data for a large sample across the adult age range
Assessment
 , 
2006
, vol. 
13
 (pg. 
62
-
79
)
Wechsler
D.
Manual for the Wechsler Adult Intelligence Scale
 , 
1955
New York
Psychological Corporation
Wechsler
D.
Manual for the Wechsler Adult Intelligence Scale-Revised
 , 
1981
New York
Psychological Corporation
Weiler
M. D.
Bernstein
J. H.
Bellinger
D. C.
Waber
D. P.
Processing speed in children with Attention Deficit/Hyperactivity Disorder, attentive type
Child Neuropsychology
 , 
2000
, vol. 
6
 (pg. 
218
-
234
)