Abstract

The latent factor structure of the Dutch version of the Wechsler Memory Scale-Fourth Edition (WMS-IV-NL) was examined with a series of confirmatory factor analyses. As part of the Dutch standardization, 1,188 healthy participants completed the WMS-IV-NL. Four models were tested for the Adult Battery (16–69 years; N = 699), and two models were tested for the Older Adult Battery (65–90 years; N = 489). Results corroborated the presence of three WMS-IV-NL factors in the Adult Battery consisting of Auditory Memory, Visual Memory, and Visual Working Memory. A two-factor model (consisting of Auditory Memory and Visual Memory) provided the best fit for the data of the Older Adult Battery. These findings provide evidence for the structural validity of the WMS-IV-NL, and further support the psychometric integrity of the WMS-IV.

Introduction

Assessment of memory function is crucial, as memory problems are present in a large variety of neurological or psychiatric disorders (Groth-Marnat, 2009; Lezak, Howieson, Bigler, & Tranel, 2012). To fully capture the multidimensional nature of memory, test batteries that assess the different memory processes, including working memory, verbal recall, and visuospatial memory, have been developed. For decades, the Wechsler Memory Scale (WMS) has been a widely used memory test battery (Rabin, Barr, & Burton, 2005). Since its original release in 1945 (Wechsler & Stone, 1945), the index structure for its interpretation has moved from one General Memory Index to five memory domain-related indices which is also in line with theories fractionating memory function (Wechsler, 1997a).

Several studies have examined the factor structure of the WMS, the Wechsler Memory Scale Revised (WMS-R; Wechsler, 1987) and the Wechsler Memory Scale-Third Edition (WMS-III; Wechsler, 1997a) in healthy controls and various clinical groups. These studies have yielded inconsistent results for the underlying latent factor structure (for an overview, see Supplementary material online, Table S1). For example, the original confirmatory factor analysis (CFA) of the WMS-III, as reported in its technical manual, showed that a model of five factors (consisting of Auditory Immediate, Auditory Delayed, Visual Immediate, Visual Delayed, and Working Memory) best fit the data from the normative sample. However, Millis, Malina, Bowers, and Ricker (1999) and Price, Tulsky, Millis, and Weiss (2002) further evaluated the factor structure of the WMS-III using CFA in both the normative sample and an independent sample of healthy participants. They reported that a model of three factors (consisting of Auditory Memory, Visual Memory, and Working Memory) fitted the data better. Also, a joint factor analysis of the Wechsler Adult Intelligence Scale-Third Edition (WAIS-III; Wechsler, 1997b) and WMS-III provided evidence for Visual Memory, Auditory Memory, and Working Memory (Verbal and Visual) as separate factors (Tulsky & Price, 2003).

Importantly, Millis and colleagues (1999) and Tulsky and Price (2003) argue that a five-factor model is inadequate due to inadmissible parameter estimates (i.e., high correlations between immediate and delayed memory measures). Moreover, Millis and colleagues (1999) and Tulsky and Price (2003) emphasized the importance of conducting CFA of the WMS in clinical samples, as different latent factor structures may emerge in different populations. That is, in healthy adults, immediate and delayed memory may function optimally and may, therefore, show a similar performance. In contrast, dissociations between immediate and delayed memory functioning have been frequently reported in brain-injured patients (Squire, 2009). To overcome this problem, Wilde and colleagues (2003) conducted a CFA in a clinical sample with left and right temporal lobe epilepsy. They found a two-factor model including General Memory and Working Memory and unexpectedly, gained minimal support for visual and auditory memory dimensions. In turn, Burton, Ryan, Axelrod, Schellenberger, and Richards (2003) reported a four-factor model (consisting of Auditory Memory, Visual Memory, Working Memory, and Learning) in a mixed clinical and control sample. Clearly, there is little support for separate immediate and delayed memory indices in the WMS-III, as the factor structure reported in the WMS-III manual has neither been replicated in healthy adults nor in clinical samples.

The latest revision of the WMS, the Wechsler Memory Scale-Fourth Edition (WMS-IV), was published in 2009 (Wechsler, 2009). This thorough revision consists of an Adult Battery for participants aged 16–69 years old and an Older Adult Battery for participants aged 65–90 years old. Furthermore, three existing subtests were adapted and four new subtests were introduced. The Adult Battery includes all subtests which results in five index scores: Auditory Memory, Visual Memory, Visual Working Memory, Immediate Memory, and Delayed Memory. The Older Adult Battery consists of a selection of four primary subtests and four index scores: Auditory Memory, Visual Memory, Immediate Memory, and Delayed Memory.

So far, three studies have examined the factor structure of the WMS-IV in samples of healthy controls (Wechsler, 2009; Holdnack, Zhou, Larrabee, Millis, & Salthouse, 2011; Miller, Davidson, Schindler, & Messier, 2013). One study has examined the factor structure of the WMS-IV in a clinical sample consisting of German patients diagnosed with depression (Pauls, Petermann, & Lepach, 2013). As presented in the technical manual of the original U.S. version of the WMS-IV (Wechsler, 2009), a three-factor model best fits the data from the normative sample in the WMS-IV Adult Battery (age group: 16–69 years). These factors were labelled as Auditory Memory, Visual Memory, and Visual Working Memory. In addition, a joint factor structure of the Wechsler Adult Intelligence Scale—Fourth Edition (WAIS-IV; Wechsler, 2008) and WMS-IV Adult Battery also supported evidence for Visual Memory, Auditory Memory, and Working Memory (Verbal and Visual) as separate factors (Holdnack et al., 2011).

Since previous findings with the WMS-III revealed that factor analysis did not support immediate and delayed memory as separate factors in healthy controls (Millis et al., 1999; Tulsky & Price, 2003), the above-mentioned models only used the delayed memory subtests and the two visual working memory subtests in their factor analyses. The study by Pauls and colleagues (2013) using clinical depressed patients and healthy controls, examined six different models, including immediate memory subtests. They revealed the same three-factor solution, and found no support for immediate and delayed memory as separate factors.

To date, only one study has examined the factor structure of the WMS-IV Older Adult Battery (Miller et al., 2013), which was evaluated in a joint manner with the WAIS-IV in an independent sample of older adults (65–92 years old). They examined the factor structure of a second-order model with a first-order general ability factor and second-order factors corresponding to domain-specific intellectual abilities (consisting of Verbal Comprehension, Perceptual Reasoning, and Processing Speed) and memory abilities (consisting of Delayed Memory and Working Memory). Their results provided support for the structural validity of the WMS-IV Older Adult Battery with Delayed Memory (Logical Memory II, Visual Reproduction II, and Verbal Paired Associates II) and Working Memory (Visual and Verbal: Digit Span and Arithmetic from the WAIS-IV and Symbol Span from the WMS-IV) as separate factors. However, the models tested in the joint CFA did not include auditory and visual memory as separate factors. Therefore, further investigation of the factor structure of the WMS-IV Older Adult Battery, separate from the WAIS-IV, is needed.

The present study aims to examine and directly compare different factor models in an independent sample using the Dutch equivalent of the Wechsler Memory Scale-Fourth Edition (WMS-IV-NL; Hendriks, Bouman, Kessels, & Aldenkamp, 2014). In particular, we conducted CFA on the WMS-IV-NL Adult and Older Adult Batteries in an independent Dutch standardization sample. In addition, we tested for measurement invariance between the Dutch standardization sample and the original U.S. standardization sample. Based on the findings on the U.S. WMS-IV, we expect to find a three-factor structure (consisting of Auditory Memory, Visual Memory, and Visual Working Memory) for the WMS-IV Adult Battery and a two-factor structure (consisting of Auditory Memory and Visual Memory) for the WMS-IV Older Adult Battery.

Method

Participants

Participants were 1,188 healthy persons, between 16 and 90 years of age (mean age = 55.9, SD = 22.9; 550 males), from the WMS-IV-NL standardization sample (Wechsler, 2009; Hendriks et al., 2014). Participants from different age groups and with different educational levels were recruited by trained assessors through their network, via advertisement, and via a database of Pearson Assessment (from May 2012 to July 2013). The sample-selection was based on the Dutch population according to census results from the Central Office for Statistics of the Netherlands (CBS, 2011). The sample was stratified according to age, sex, education level, ethnicity, and geographic region, and the participants were only included if they met the inclusion criteria: ability to speak/understand the Dutch language; no significant hearing or visual impairment; no psychiatric or neurologic disorder; no substance abuse affecting cognitive functioning; and no use of medicines affecting cognitive functioning. The sample was divided into 12 age groups (Adult Battery: 16–19, 20–29, 30–34, 35–44, 45–54, 55–64, 65–69; Older Adult Battery: 65–69, 70–74, 75–79, 80–84, 85–90). Of these participants, 699 were assessed with the WMS-IV-NL Adult Battery (16–69 years old; mean age = 40.9, SD = 17.3; 348 males) and 489 with the WMS-IV-NL Older Adult Battery (65–90 years old; mean age = 77.4, SD = 7.3; 202 males). The WMS-IV-NL standardization study was approved by the Institutional Review Board of Radboud University Nijmegen and written informed consent was obtained.

Measures

All participants completed the WMS-IV-NL, a memory battery designed to evaluate several episodic memory and visual working memory abilities. As mentioned above, the WMS-IV-NL consists of an Adult Battery and an Older Adult Battery. The WMS-IV-NL Adult Battery consists of one optional subtest, the Brief Cognitive Status Exam, and six primary subtests: Logical Memory (LM), Verbal Paired Associates (VPA), Designs (DE), Visual Reproduction (VR), Spatial Addition (SA), and Symbol Span (SP). Of these, four subtests (LM, VPA, DE, and VR) have immediate and delayed recall conditions. The primary subtests contribute to five index scores: Auditory Memory Index (AMI), Visual Memory Index (VMI), Visual Working Memory Index, Immediate Memory Index (IMI), and Delayed Memory Index (DMI). The WMS-IV-NL Older Adult Battery consists of a selection of four primary subtests (LM, VPA, VR, and SP) and four index scores (AMI, VMI, IMI, and DMI). The age-adjusted scaled scores of the WMS-IV-NL subtests were used in all analyses.

The Dutch version of the WMS-IV was developed to be equivalent to the original published U.S. version. The nonverbal visual stimuli were identical to those in the U.S. WMS-IV. Instructions, auditory stimuli, and scoring criteria were translated and adapted to the Dutch language. Pilot studies (first pilot study N = 60; second pilot study N = 120) were performed to check and improve the Dutch language adaptation of the WMS-IV. Moreover, an expert group consisting of clinical neuropsychologists from the Netherlands and Belgium checked the Dutch adaptation after both pilot studies. This process has resulted in an authorized Dutch version of the WMS-IV (Wechsler, 2009; Hendriks et al., 2014).

Models

CFA was conducted using LISREL 8.54 (Jöreskog & Sörbom, 2003). The maximum likelihood estimator was used to estimate the parameters. This estimator is robust against violations of normality (Satorra, 1992).

All models were designed according to the theoretical hypotheses about the factor structure and previous factor analytic research on the WMS-R, WMS-III, and WMS-IV (see Jöreskog & Sörbom, 1996 for a description of CFA). Moreover, only the delayed memory subtests and the visual working memory subtests were included in the reported models because CFA does not reveal separate immediate and delayed memory factors within one measurement model in healthy controls. Including immediate memory subtests would result in model specification errors. We examined two models including immediate memory subtests (a three-factor model including Immediate Memory, Delayed Memory, and Visual Working Memory and a five-factor model including Immediate Auditory Memory, Delayed Auditory Memory, Immediate Visual Memory, Delayed Visual Memory, and Visual Working Memory). The simple structure factor models, i.e., no modifications, produced inadmissible parameter estimates in healthy controls. The correlations Immediate and Delayed factors exceed 1.0. These results are in line with previous studies (Millis et al., 1999; Tulsky & Price, 2003), indicating that separate immediate and delayed factors cannot be obtained within one model. Therefore, the immediate memory subtests were omitted from the further analyses. Table 1 provides the CFA models for the WMS-IV-NL Adult and Older Adult Batteries.

Table 1.

Model specifications for confirmatory factor analysis (CFA) for the WMS-IV-NL

Model  Factors Variables 
WMS-IV-NL Adult Battery 
Model I One-factor General Memory LM II, VPA II, VR II, DE II, SA, SSP 
Model II Two-factor General Memory LM II, VPA II, VR II, DE II 
  Visual Working Memory SA, SSP 
Model III Two-factor Auditory Memory LM II, VPA II 
  Visual Memory VR II, DE II, SA, SSP 
Model IV Three-factor Auditory Memory LM II, VPA II 
  Visual Memory VR II, DE II 
  Visual Working Memory SA, SSP 
WMS-IV-NL Older Adult Battery 
Model V One-factor General Memory LM II, VPA II, VR II, SSP 
Model VI Two-factor Auditory Memory LM II, VPA II 
  Visual Memory VR II, SSP 
Model  Factors Variables 
WMS-IV-NL Adult Battery 
Model I One-factor General Memory LM II, VPA II, VR II, DE II, SA, SSP 
Model II Two-factor General Memory LM II, VPA II, VR II, DE II 
  Visual Working Memory SA, SSP 
Model III Two-factor Auditory Memory LM II, VPA II 
  Visual Memory VR II, DE II, SA, SSP 
Model IV Three-factor Auditory Memory LM II, VPA II 
  Visual Memory VR II, DE II 
  Visual Working Memory SA, SSP 
WMS-IV-NL Older Adult Battery 
Model V One-factor General Memory LM II, VPA II, VR II, SSP 
Model VI Two-factor Auditory Memory LM II, VPA II 
  Visual Memory VR II, SSP 

Notes: WMS-IV-NL = Dutch version of the Wechsler Memory Scale-Fourth Edition; subtest abbreviations: LM = Logical Memory; VPA = Verbal Paired Associates; VR = Visual Reproduction; DE = Designs; SA = Spatial Addition; SSP = Symbol Span.

First, we conducted CFA on the WMS-IV-NL Adult Battery. For the Adult Battery, four CFA models were estimated. Model I (one factor: General Memory) and Model II (two factors: General Memory and Visual Working Memory) were designed according to the models tested in the technical manuals of the U.S. WMS-R and WMS-III and a recently published study by Pauls and colleagues (2013). Model III (two factors: Auditory Memory and Visual Memory) and Model IV (three factors: Auditory Memory, Visual Memory, and Visual Working Memory) were designed according to models tested in the technical and interpretative manual of the U.S. WMS-IV (Wechsler, 2009).

Second, we conducted CFA on the WMS-IV-NL Older Adult Battery to assess the fit of two factor models for the WMS-IV-NL Older Adult Battery alone (i.e., no joint factor structure with another test). In correspondence with the Adult Battery, Model V (one factor: General Memory) and Model VI (two factors: Auditory Memory and Visual Memory) were designed. Because a factor should be measured by at least two subtest scores, the models which contain a separate Visual Working Memory factor could not be examined for the Older Adult Battery due to the absence of the subtests Spatial Addition (i.e., Symbol Span is the only visual working memory subtest). All models are oblique and have a simple structure (i.e., variables load on only one factor).

After identifying which models best fitted our data in the Adult and Older Adult Batteries, we computed simultaneous multigroup confirmatory factor analyses (MGCFA) on the covariance matrices of the Dutch standardization sample and the original U.S. standardization sample to test for measurement invariance between both groups (Steenkamp & Baumgartner, 1998). For the U.S. standardization sample, we obtained the subtest intercorrelation matrices and standard deviations reported in the U.S. WMS-IV technical manual (Wechsler, 2009). Firstly, we tested configural invariance, that is, whether the factor structures are the same in both groups. Secondly, we tested metric invariance, that is, whether the factor loadings are the same in both groups.

Following the recommendations of Hu and Bentler (1999), we included a set of indexes to evaluate the goodness of fit of each model: the likelihood ratio chi-square statistic (χ2); the χ2 degrees of freedom ratio (df); the root-mean-square error of approximation (RMSEA; Steiger, 1990); the standardized root-mean-square residual (SRMR; Jöreskog & Sörbom, 1996); Akaike's information criterion (AIC; Akaike, 1987); the non-normed fit index (NNFI; Bentler & Bonett, 1980); and the comparative fit index (CFI; Bentler, 1990). The χ2 statistic is very sensitive to sample size and may therefore lead to the rejection of plausible models. A common strategy for addressing this issue is to report additional fit statistics (Hu & Bentler, 1999; Thompson, 2000).

Results

Table 2 provides the goodness-of-fit model analyses of the CFA. In the Adult Battery, the poorest values were observed for Model I (General Memory factor) and Model II (General Memory and Visual Working Memory), with the RMSEA and SRMR values of 0.05 and above. Model III (two-factors: Auditory Memory and Visual Memory) and Model IV (three-factors: Auditory Memory, Visual Memory, and Visual Working Memory) have very similar results. Both models had reasonable fit overall with CFI and NNFI values of 0.97 and higher, and the RMSEA and SRMR of 0.06 and lower. Moreover, both models fit the data well, and there is no statistically significant difference between both models (χ2 (2) = 5.22, p = .074). The standardized factor loadings for the three-factor model are presented in Fig. 1. Furthermore, inspection of the factor correlations revealed that all factors are highly correlated, and not unexpectedly, the Visual Memory and Visual Working Memory factors revealed a very high correlation of 0.90.

Table 2.

Goodness-of-fit indexes for the confirmatory factor analyses

Model χ2 df SRMR NNFI CFI AIC RMSEA 90% CI Δχ2 Δdf p 
WMS-IV-NL Adult Battery 
Model I 87.03** 0.06 0.88 0.93 110.03 0.11 0.09–0.13 — — — 
Model II 76.88** 0.05 0.88 0.94 102.88 0.11 0.09–0.13 10.15 <.001 
Model III 23.95* 0.03 0.97 0.98 49.95 0.05 0.03–0.08 52.93 <.001 
Model IV 18.73* 0.02 0.97 0.99 48.73 0.06 0.03–0.08 5.22 0.07 
Model V 42.95** 0.06 0.72 0.91 58.95 0.21 0.16–0.26 — — — 
Model VI 0.92 0.01 1.00 1.00 18.92 0.00 0–0.12 42.03 <.001 
Model χ2 df SRMR NNFI CFI AIC RMSEA 90% CI Δχ2 Δdf p 
WMS-IV-NL Adult Battery 
Model I 87.03** 0.06 0.88 0.93 110.03 0.11 0.09–0.13 — — — 
Model II 76.88** 0.05 0.88 0.94 102.88 0.11 0.09–0.13 10.15 <.001 
Model III 23.95* 0.03 0.97 0.98 49.95 0.05 0.03–0.08 52.93 <.001 
Model IV 18.73* 0.02 0.97 0.99 48.73 0.06 0.03–0.08 5.22 0.07 
Model V 42.95** 0.06 0.72 0.91 58.95 0.21 0.16–0.26 — — — 
Model VI 0.92 0.01 1.00 1.00 18.92 0.00 0–0.12 42.03 <.001 

Notes: WMS-IV-NL = Dutch version of the Wechsler Memory Scale-Fourth Edition; SRMR = standardized root-mean residual: values ≤0.05 indicate good model fit; AIC = Akaike information criterion: smaller values indicate the model with the better fit; NNFI = non-normed fit index: values ≥0.90 indicate good model fit; CFI = comparative fit index: values ≥0.90 indicate good model fit; RMSEA = root-mean-square error of approximation: values ≤0.08 indicate an acceptable model fit, and values ≤0.05 indicate good model fit; 90% CI = 90% confidence interval for RMSEA (Bentler, 1990; Hu & Bentler, 1999; Thompson, 2000).

*p < .01, **p < .001.

Fig. 1.

The three-factor model of the Adult Battery of the Dutch version of the Wechsler Memory Scale-Fourth Edition. Single-headed arrows represent standardized factor loadings and double-headed arrows represent correlations between factors.

Fig. 1.

The three-factor model of the Adult Battery of the Dutch version of the Wechsler Memory Scale-Fourth Edition. Single-headed arrows represent standardized factor loadings and double-headed arrows represent correlations between factors.

In the Older Adult Battery, the fit indexes revealed that Model V (General Memory factor) fit the data poorly, with the RMSEA and SRMR values >0.05. The best fit statistics were observed for Model VI (Auditory Memory and Visual Memory). A χ2 test confirms that Model VI fits significantly better than Model V (χ2 (1) = 42.03, p < .001). The standardized factor loadings for the two-factor model are presented in Fig. 2.

Fig. 2.

The two-factor model of the Older Adult Battery of the Dutch version of the Wechsler Memory Scale-Fourth Edition. Single-headed arrows represent standardized factor loadings and double-headed arrows represent correlations between factors.

Fig. 2.

The two-factor model of the Older Adult Battery of the Dutch version of the Wechsler Memory Scale-Fourth Edition. Single-headed arrows represent standardized factor loadings and double-headed arrows represent correlations between factors.

By performing an MGCFA, it is possible to test whether the correlations among factors and the individual factor loadings are invariant between the Dutch and U.S. standardization samples. The results of the MGCFA are presented in Table 3. For both the Adult and Older Adult Batteries, there was no statistically significant difference between the configural variance and the metric variance which indicates that the factor structures and factor loadings are equal across the Dutch and U.S. standardization samples.

Table 3.

Multigroup CFA and measurement invariance testing between the Dutch standardization sample and the original U.S. standardization sample

Invariance model χ2 df SRMR NNFI CFI AIC RMSEA 90% CI Δχ2 Δdf p Decision 
WMS-IV Adult Battery 
Configural 38.08** 15 0.02 0.98 0.99 92.08 0.04 0.03–0.06 — — — — 
Metric 38.91* 18 0.02 0.99 0.99 86.91 0.03 0.02–0.06 0.83 0.84 Accepted 
WMS-IV Older Adult Battery 
Configural 2.33 0.02 1.01 1.00 34.33 0.00 0–0.05 — — — — 
Metric 3.40 0.02 1.01 1.00 31.40 0.00 0–0.04 1.07 0.59 Accepted 
Invariance model χ2 df SRMR NNFI CFI AIC RMSEA 90% CI Δχ2 Δdf p Decision 
WMS-IV Adult Battery 
Configural 38.08** 15 0.02 0.98 0.99 92.08 0.04 0.03–0.06 — — — — 
Metric 38.91* 18 0.02 0.99 0.99 86.91 0.03 0.02–0.06 0.83 0.84 Accepted 
WMS-IV Older Adult Battery 
Configural 2.33 0.02 1.01 1.00 34.33 0.00 0–0.05 — — — — 
Metric 3.40 0.02 1.01 1.00 31.40 0.00 0–0.04 1.07 0.59 Accepted 

Notes: WMS-IV-NL = Dutch version of the Wechsler Memory Scale-Fourth Edition; SRMR = standardized root-mean residual; AIC = Akaike information criterion; NNFI = non-normed fit index; CFI = comparative fit index; RMSEA = root-mean-square error of approximation; 90% CI = 90% confidence interval for RMSEA.

*p < .01, **p < .001.

Discussion

The current study examined the latent factor structure of the WMS-IV-NL Adult and Older Adult Batteries in the Dutch standardization sample. Of the four models evaluated for the Adult Battery, Model III (two factors consisting of Auditory Memory and Visual Memory) and Model IV (three factors consisting of Auditory Memory, Visual Memory, and Visual Working Memory) revealed good fit. Both these models fit the data equally well and no statistically significant difference between both models is demonstrated. These results are in agreement with the results reported in the U.S. WMS-IV technical manual (Wechsler, 2009). The U.S. WMS-IV test publishers tested two models (identical to models III and IV) for the overall standardization sample and for three separate age groups (ages 16–24, 25–44, and 45–69). Their results also revealed very similar fit statistics for both models. Based on these findings, one could argue that the most parsimonious model should be accepted, that is, the three-factor model should be rejected in favor of the two-factor model. However, there are theoretical grounds to opt for the three-factor model, also taking into account the hypothesized memory processes that are measured by the WMS-IV (Jöreskog & Sörbom, 1996; Wechsler, 2009). Specifically, visual working memory and visual long-term memory have been consistently demonstrated to rely on distinct memory systems (Squire, 2009). The three-factor model is consistent with this distinction and with previous studies that also revealed three separate memory factors (i.e., Auditory Memory, Visual Memory, and (Visual) Working Memory) in the WMS-IV Adult Battery (Holdnack et al., 2011; Pauls et al., 2013) and WMS-III (Millis et al., 1999; Price et al., 2002; Tulsky & Price, 2003). As our current findings are similar to the results on the U.S. WMS-IV, we argue that the three-factor model is the most appropriate one to represent the core WMS-IV index structure.

Moreover, simultaneous MGCFA revealed that the factor structures and factor loadings of the Dutch and U.S. standardization samples are invariant for the Adult Battery. These results strengthen the case for equivalence of the WMS-IV in general. The findings of the current study are, therefore, not only providing evidence for the structural validity of the Dutch version of the WMS-IV but also providing evidence for the psychometric integrity of the original published U.S. version of the WMS-IV.

Our findings extend the results of Miller and colleagues (2013), as we are the first to examine the factor structure of the WMS-IV Older Adult Battery alone and not in a joint factor structure with a second test. Of the two models evaluated, Model VI (two factors consisting of Auditory Memory and Visual Memory) resulted in the best fit. Notably, because visual working memory is measured by only one subtest, a separate Visual Working Memory factor could not be determined. Moreover, simultaneous MGCFA revealed that the factor structures and factor loadings of the Dutch and U.S. standardization samples are invariant for the Older Adult Battery.

In line with previous studies (Millis et al., 1999; Tulsky & Price, 2003), we did not find support for immediate and delayed memory as separate factors. Although the use of these indices has been questioned because they have not yet been validated in factor analysis (Kent, 2013), separate immediate, and delayed memory indices are included in the WMS-IV for their “clinical usefulness” (Wechsler, 2009). Because the WMS-IV is often used in patients with neurological impairment, Pauls and colleagues (2013) already emphasized the need for conducting CFA of the WMS-IV in these patients as this could result in different model solutions. In future research, it is therefore advised to conduct CFA on the WMS-IV in various clinical samples with known memory impairment.

In model IV (three factors consisting of Auditory Memory, Visual Memory, and Visual Working Memory), a high correlation exists between visual memory and visual working memory. This indicates poor discriminant validity between these factors in this sample of healthy controls. This finding is consistent with previous studies on the WMS-IV (Wechsler, 2009; Holdnack et al., 2011; Pauls et al., 2013). This is not unexpected, because the visual memory and visual working memory subtests rely on common abilities such as recollection of visual stimuli and visuospatial information processing, and share the same materials such as the use of the memory grid for a visual memory subtest (Designs) and visual working memory subtest (Spatial Addition), there remains an overlap between these factors. Moreover, it is suggested that various clinical populations are more likely to feature dissociations between the visual memory and visual working memory indexes than healthy controls. Future research should clarify the usability of these indexes in various clinical samples. For now, it should be stressed that these indexes must be interpreted with caution.

Some limitations of our study should be mentioned here. We only examined the factor structure of the WMS-IV-NL Adult and Older Adult Batteries, but could not include additional tests such as the WAIS-IV. The results are, therefore, not directly comparable with the previous joint factor structures of the WAIS-IV and WMS-IV (Holdnack et al., 2011; Miller et al., 2013). Moreover, including other tests in the analysis may result in other factors. Future research could examine more extensive test batteries in one CFA. Also, it would be interesting to examine whether the factor structures of the WMS-IV-NL Adult and Older Adult Batteries remain the same across the same across different age groups and education levels. In the current study, we used the entire standardization sample to develop a baseline model of the WMS-IV-NL (Tulsky & Price, 2003).

Overall, findings from the present study corroborate and add to previous results, providing evidence for the structural validity of the Dutch version of the WMS-IV. The replication of the three-factor structure, in turn, may increase confidence in the use of the WMS-IV factor indices in diagnostic testing and assessment.

Supplementary Material

Supplementary material is available at Archives of Clinical Neuropsychology online.

Funding

This work was supported by Pearson Assessment and Information; Kempenhaeghe, Academic Centre for Epileptology; and Donders Institute for Brain, Cognition and Behaviour of Radboud University Nijmegen.

Conflict of Interest

The authors declared a potential conflict of interest as follows: a financial relationship with the commercial organization and product discussed in this article. MK is working as a psychometrician at Pearson, which is the publisher of the WMS-IV-NL.

Acknowledgements

The authors thank William van der Veld for his help with the statistical analyses and Daniel Sheppard for the final grammar check of our revised manuscript.

References

Akaike
H.
(
1987
).
Factor analysis and AIC
.
Psychometrika
 ,
52
,
317
332
.
Bentler
P. M.
(
1990
).
Comparative fit indexes in structural models
.
Psychological Bulletin
 ,
107
,
238
246
.
Bentler
P. M.
Bonett
D. G.
(
1980
).
Significance tests and goodness in the analysis of covariance structures
.
Psychological Bulletin
 ,
88
,
588
606
.
Burton
B. D.
Ryan
J. J.
Axelrod
B. N.
Schellenberger
T.
Richards
H. M.
(
2003
).
A confirmatory factor analysis of the WMS-III in a clinical sample with crossvalidation in the standardization sample
.
Archives of Clinical Neuropsychology
 ,
18
,
629
641
.
Centraal Bureau voor Statistiek (CBS)—Statline
(
2011
,
July 2011
).
Datalevering Enquête Beroepsbevolking. Retrieved July 07, 2011, from http://www.cbs.nl
.
Groth-Marnat
G.
(
2009
).
Handbook of psychological assessment
  (
5th ed
.)
Hoboken, NJ
:
John Wiley & Sons
.
Hendriks
M. P. H.
Bouman
Z.
Kessels
R. P. C.
Aldenkamp
A. P.
(
2014
).
Wechsler Memory Scale-Fourth Edition, Dutch Edition (WMS-IV-NL).
 
Amsterdam
:
Pearson Assessment
.
Holdnack
J. A.
Zhou
X.
Larrabee
G. J.
Millis
S. R.
Salthouse
T. A.
(
2011
).
Confirmatory factor analysis of the WAIS-IV/WMS-IV
.
Assessment
 ,
18
,
178
191
.
Hu
L.
Bentler
P. M.
(
1999
).
Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives
.
Structural Equation Modelling
 ,
6
,
1
55
.
Jöreskog
K. G.
Sörbom
D.
(
1996
).
LISREL 8: User’s reference guide.
 
Chicago
:
Scientific Software International
.
Jöreskog
K. G.
Sörbom
D.
(
2003
).
LISREL 8.54 [Computer software]
 .
Chicago
:
Scientific Software International
.
Kent
P.
(
2013
).
The Evolution of the Wechsler Memory Scale: A selective review
.
Applied Neuropsychology Adult
 ,
20
,
277
291
.
Lezak
M. D.
Howieson
D. B.
Bigler
E. D.
Tranel
D.
(
2012
).
Neuropsychological assessment
  (
5th ed
.).
New York
:
Oxford University Press
.
Miller
D. L.
Davidson
P. S. R.
Schindler
D.
Messier
C.
(
2013
).
Confirmatory factor analysis of the WAIS-IV and WMS-IV in older adults
.
Journal of Psychoeducational Assessment
 ,
31
,
375
390
.
Millis
S. R.
Malina
A. C.
Bowers
D. A.
Ricker
J. H.
(
1999
).
Confirmatory factor analysis of the Wechsler Memory Scale-III
.
Journal of Clinical and Experimental Neuropsychology
 ,
21
,
87
93
.
Pauls
F.
Petermann
F.
Lepach
A. C.
(
2013
).
Memory assessment and depression: Testing for factor structure and measurement invariance of the Wechsler Memory Scale-Fourth Edition across a clinical and matched control sample
.
Journal of Clinical and Experimental Neuropsychology
 ,
35
,
702
717
.
Price
L.
Tulsky
D.
Millis
S.
Weiss
L.
(
2002
).
Redefining the factor structure of the Wechsler Memory Scale-III: Confirmatory factor analysis with cross-validation
.
Journal of Clinical Experimental Neuropsychology
 ,
24
,
574
585
.
Rabin
L. A.
Barr
W. B.
Burton
L. A.
(
2005
).
Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA Division 40 members
.
Archives of Clinical Neuropsychology
 ,
20
,
33
65
.
Satorra
A.
(
1992
).
Asymptotic robust inferences in the analysis of mean and covariance structures
.
Sociological Methodology
 ,
22
,
249
278
.
Steenkamp
J. E. M.
Baumgartner
H.
(
1998
).
Assessing measurement invariance in cross-national consumer research
.
Journal of Consumer Research
 ,
25
,
78
90
.
Steiger
J. H.
(
1990
).
Structural model evaluation and modification: An interval estimation approach
.
Multivariate Behavioral Research
 ,
25
,
173
180
.
Squire
L. R.
(
2009
).
Memory and brain systems: 1969–2009
.
The Journal of Neuroscience
 ,
29
,
12711
12716
.
Thompson
B.
(
2000
).
Ten commandments of structural equation modelling
. In
Grimm
L. G.
Yarnold
P. R.
(Eds.),
Reading and understanding more multivariate statistics
  (pp.
261
284
).
Washington, DC
:
American Psychological Association
.
Tulsky
D. S.
Price
L. R.
(
2003
).
The joint WAIS-III and WMS-III factor structure: Development and cross-validation of a six-factor model of cognitive functioning
.
Psychological Assessment
 ,
15
,
149
162
.
Wechsler
D.
(
1987
).
Wechsler Memory Scale-Revised (WMS-R).
 
San Antonio, TX
:
The Psychological Corporation
.
Wechsler
D.
(
1997a
).
Wechsler Memory Scale-Third Edition (WMS-III).
 
San Antonio, TX
:
The Psychological Corporation
.
Wechsler
D.
(
1997b
).
Wechsler Adult Intelligence Scale-Third Edition (WAIS-III).
 
San Antonio, TX
:
The Psychological Corporation
.
Wechsler
D.
(
2008
).
Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV).
 
San Antonio, TX
:
Pearson Assessment
.
Wechsler
D.
(
2009
).
Wechsler Memory Scale-Fourth Edition (WMS-IV).
 
San Antonio, TX
:
Pearson Assessment
.
Wechsler
D.
Stone
C. P.
(
1945
).
A standardized memory scale for clinical use
.
The Journal of Psychology
 ,
19
,
87
95
.
Wilde
N. J.
Strauss
E.
Chelune
G. J.
Hermann
B. P.
Hunter
M.
Loring
D. W.
et al
(
2003
).
Confirmatory factor analysis of the WMS-III in patients with temporal lobe epilepsy
.
Psychological Assessment
 ,
15
,
56
63
.

Supplementary data