Abstract

The present study, utilizing a sample of inpatients with schizophrenia or schizoaffective disorder (n = 167), examined the factor structure of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). Principal axis exploratory factor analysis, multiple factor extraction criteria, and higher-order factor analysis were used. Results were inconsistent with the five-factor structure of the RBANS purported in the test manual. Factor extraction criteria recommended extraction of one or two factors. Extraction of two factors resulted in a memory dimension and a less homogeneous visual perception and processing speed dimension. Higher-order analysis found that a second-order factor, representing general neurocognitive functioning, accounted for over three times the total and common variance than the two first-order factors combined. It was concluded that although the RBANS appears to be a useful measure of general neurocognitive functioning for inpatients with schizophrenia or schizoaffective disorder, clinical interpretation beyond a general factor (i.e., Total Scale score) should be done with caution in this population. Limitations of the present study and directions for future research are discussed.

Introduction

The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998) is an individually administered test that purports to assess the neuropsychological status of adults. The RBANS consists of 12 subtests that are used to construct five age-corrected Indexes. The Indexes and their respective subtests are as follows: Immediate Memory Index (List Learning and Story Memory subtests), Visuospatial/Constructional Index (Figure Copy and Line Orientation subtests), Language Index (Picture Naming and Semantic Fluency subtests), Attention Index (Digit Span and Coding subtests), and Delayed Memory Index (List Recall, List Recognition, Story Recall and Figure Recall subtests). A Total score is also provided by the RBANS which is a composite of the five Index scores. Although the RBANS was initially developed for measuring neurocognitive functioning of older adults, it has been contended that the RBANS is useful for measuring neurocognitive functioning of psychiatric patients. As such, normative data on this measure have been collected using samples of psychiatric patients for clinical purposes (Iverson, Brooks, & Haley, 2009; Wilk et al., 2003).

There appears to be evidence for the utility of the RBANS Total Scale score when assessing psychiatric patients, although reliability and validity estimates of the Index scores have been variable. Wilk and colleagues (2002) examined the reliability of the RBANS in a sample of patients with schizophrenia or schizoaffective disorder. Although these authors found moderate reliability for the Total Scale score (Intraclass Coefficient [ICC] = 0.84), reliabilities for the Index scores ranged from low to moderate (ICCs from 0.51 to 0.81). Gold, Queern, Iannone, and Buchanan (1999) also studied the utility of the RBANS as a neurocognitive measure for patients with schizophrenia. Utilizing a sample of outpatients, these authors found moderate reliability for the Total Scale score (ICC = 0.83), but reliabilities for the Index scores varied from low to high (ICCs from 0.51 to 0.91). Additionally, these authors found the Total Scale score to be moderately associated, but Index scores to be weakly to moderately associated, with measures of general cognitive ability and memory. Similarly, in a follow-up study, Hobart, Goldberg, Bartko, and Gold (1999) found the Total Scale score to be moderately related to measures of general cognitive ability, attention, vigilance, and memory, but the Index scores to be weakly to moderately related to these measures. Taken together, prior studies examining the psychometric properties of the various Index scores obtained from the RBANS have shed some light on the relative utility of these scores in psychiatric populations. The findings of stronger reliability and validity for the Total Scale score than for the Index scores may be due, at least partially, to the Total Scale consisting of more items than each of the Indexes. Another possible explanation for these findings, however, concerns heterogeneity among subtests. In this view, the tenability of an Index would be threatened if its subtests are no more strongly related to each other than to subtests not belonging to that Index. Thus, it would be important to determine if the factor structure of the RBANS, as outlined in the test manual, best captures how psychiatric patients perform on this measure.

The existing factor structure of the RBANS was theoretically derived, and no factor analytic studies of the RBANS are cited in its test manual (seeRandolph, 1998). Of relevance to the present study, there are no extant factor analytic studies of the RBANS utilizing a sample of psychiatric patients. To date, there have been five-factor analytic studies of the RBANS using non-psychiatric samples. Notably, none of these studies have found a factor structure consistent with the five-Index structure suggested by the RBANS test manual. For example, Duff and colleagues (2006) conducted a factor analysis of the RBANS utilizing a sample of healthy older adults. These authors found a two-factor solution, one factor consisted of memory subtests (List Learning, Story Memory, List Recall, List Recognition, and Story Recall) and the other factor consisted of visual perceptual subtests (Figure Copy, Line Orientation, Coding, and Figure Recall). In a sample of ischemic stroke patients, Wilde (2006) also found a two-factor solution. In addition to subtests measuring attention and memory, subtests measuring language skills comprised the first factor (List Learning, Story Memory, Story Recall, List Recall, List Recognition, Digit Span, Semantic Fluency, and Picture Naming). Subtests measuring visual perception comprised the second factor (Figure Copy, Line Orientation, Figure Recall, and Coding). Similarly, Carlozzi, Horner, Yang, and Tilley (2008) found a two-factor solution in a sample of veterans referred to a memory disorders clinic. One factor included memory subtests (List Learning, Story Memory, List Recall, List Recognition, and Story Recall) and the other factor included subtests with visual perceptual demands (Figure Copy, Line Orientation, and Picture Naming). Utilizing a sample of older adults referred for a dementia evaluation, Schmitt and colleagues (2010) also found a two-factor solution. The first factor was comprised of memory subtests (Story Recall, List Recall, Story Memory, List Learning, List Recognition, and Figure Recall). The second factor was comprised of subtests requiring visual perception, but it also included an attention subtest (Coding, Line Orientation, Figure Copy, Picture Naming, and Digit Span). In contrast to findings of other RBANS factor analytic studies, Garcia, Leahy, Corradi, and Forchetti (2008), utilizing a sample of older adults referred to a memory disorders clinic, found a three-factor solution. These authors found a memory dimension (Story Recall, List Recall, Figure Recall, List Recognition, and Story Memory), a visuomotor processing dimension (Figure Copy, Line Orientation, and Coding), and a verbal processing dimension (Digit Span, Picture Naming, List Learning, and Semantic Fluency). In sum, the majority of factor analytic studies of the RBANS have found a two-factor solution, with one study finding a three-factor solution. A commonality among these studies is that subtests involving memory have generally loaded on a single factor. Subtests loading on other factors have been more variable but have generally involved visual perceptual processing.

A potential limitation of extant factor analytic studies of the RBANS is that they have relied upon traditional factor extraction criteria (e.g., eigenvalues >1.0, visual scree test). One possible reason for this is that although several objective criteria have been developed for determining explicitly the number of factors to extract and retain, such as Horn's Parallel Analysis (HPA; Horn, 1965), Standard Error of Scree (SEScree; Zoski & Jurs, 1996), and the Minimum Average Partials (MAP) test (Velicer, 1976), they are unavailable in most statistical software packages (Costello & Osborne, 2005). It has been noted that the default in most statistical software packages is to retain factors with eigenvalues >1.0, a procedure that has been generally described as “among the least accurate methods” for selecting the number of factors to retain (Costello & Osborne, 2005, p. 2). The visual scree test, an alternative to relying on eigenvalues >1.0, is available in most statistical software packages, but results of this method can be “unclear if there are data points clustered together” (Costello & Osborne, 2005, p. 3). In addition to utilizing traditional factor extraction criteria, it has been recommended that researchers use supplementary methods, including HPA, SEScree, and MAP (Fabrigar, Wegener, MacCallum, & Strahan, 1999). All told, it is important that studies examining the factor structure of the RBANS use additional factor extraction criteria.

Another potential limitation of extant factor analytic studies of the RBANS is that their findings have not accounted for a second-order “general” factor. This is of particular relevance because a general factor is theoretically supported given the RBANS was developed to include an overall score. Higher-order factor analysis of the RBANS would be important, not only to assess the tenability of a second-order factor (i.e., Total Scale score), but also to assess whether first-order factors warrant attention beyond that of a second-order factor. At present, users of the RBANS do not have available the necessary information to judge the relative importance of first-order factors (i.e., Index scores) relative to a second-order factor (i.e., Total Scale score). The extent to which Index scores capture meaningful proportions of variance beyond that captured by the Total Scale score should be considered when interpreting RBANS results and making subsequent clinical decisions.

The present study used RBANS data collected on psychiatric inpatients at a large state hospital. The goal of the present study was to utilize a psychiatric sample for examining the factor structure of the RBANS. To our knowledge, no prior factor analytic investigations of the RBANS have examined its factor structure for psychiatric inpatients, nor have they incorporated multiple objective factor extraction criteria, nor have they included techniques for examining a higher-order dimension. The present study provides unique contributions to the literature on the utility of the RBANS for measuring neurocognitive functioning of psychiatric inpatients by (a) using several criteria for recommending the number of factors to be extracted and retained from RBANS subtest scores; (b) determining the factor structure of psychiatric inpatients' RBANS performances; and (c) determining what proportions of variance are attributable to a second-order factor and first-order factors.

Method

Participants and Procedure

Archival data were drawn following a hospital-wide initiative to incorporate the RBANS as part of patients' routine clinical care. Participants were administered the RBANS and a brief semi-structured interview by either a licensed psychologist or a pre-doctoral trainee being supervised by a licensed psychologist. All 12 subtests of the RBANS were administered and scored according to standard instructions in the test manual. The semi-structured interview inquired about demographic topics, including age, gender, ethnicity, and years of completed education. This interview also inquired about neuropsychological risk factors, such as history of seizures, cerebrovascular incidents, and previous head injuries. DSM-IV-TR diagnoses were independently made by an interdisciplinary treatment team with knowledge of patients' clinical history and symptom profile at the time of the evaluation. The present study was approved by the State of California's Committee for the Protection of Human Subjects (CPHS).

Participants (n = 167) were forensically committed psychiatric patients receiving treatment at a large state hospital in Southern California who had either a primary diagnosis of schizophrenia (n = 112, 67.1%) or schizoaffective disorder (n = 55, 32.9%). A large proportion of participants had either a secondary substance abuse (n = 90, 53%) or secondary substance dependence (n = 35, 21%) diagnosis. Nearly all participants (162; 97.0%) were taking psychotropic medication, with 146 (87.4%) taking an atypical antipsychotic (e.g., olanzapine), 82 (49.1%) taking a mood stabilizer (e.g., lamotrigine), 36 (21.6%) taking an antidepressant (e.g., sertraline), and 32 (19.2%) taking a traditional antipsychotic (e.g., haloperidol). The great majority of participants, 148 (88.6%), were men and 19 (11.4%) were women. The mean age was 42.76 (SD = 9.73), and mean years of education was 11.18 (SD = 2.49). The sample was ethnically diverse, with 73 (43.7%) participants who identified as Caucasian, 45 (26.9%) who identified as African American, 23 (13.8%) who identified as Hispanic or Latino, 11 (6.6%) who identified as Multiethnic, 8 (4.8%) who identified as Asian or Pacific Islander, and 7 (4.2%) who identified as “Other.”

A total of 269 participants were initially considered for the study. Of these, 69 (25.7%) were excluded because they reported a history of head injury, 23 (8.6%) were excluded because they reported a history of seizures, 5 (1.9%) were excluded because they reported a history of cerebrovascular incident(s), and 5 (1.9%) were excluded because they had a diagnosis of dementia. Participants with a substance abuse or dependence diagnosis were not excluded because such diagnoses are common among patients with psychotic-spectrum disorders (Saddichha, Sur, Sinha, & Khess, 2010). The final sample comprised 62.1% of participants considered for the present study.

Data Analyses

Principal axis exploratory factor analysis (PFA) was used to analyze reliable common variance from the 12 RBANS subtests using IBM SPSS version 19.0. PFA was used because of its ability to identify latent constructs underlying measured variables, take systematic and error variance into account, and its non-dependence on the assumption of multivariate normality (Costello & Osborne, 2005; Fabrigar et al., 1999). Bartlett's Test of Sphericity (χ2 = 853.52, p < .001) and the Kaiser–Meyer–Olkin measure of sampling adequacy (KMO = 0.86) provided evidence that the correlation matrix was adequate for factor analysis (Tabachnick & Fidell, 2001). Because it has been recommended that relying upon traditional criteria for determining the number of factors to extract can be misleading (Costello & Osborne, 2005; Stevens, 2002), the present study utilized several extraction criteria. One of these criteria was the retention of factors with eigenvalues >1.0 (Kaiser, 1960). Another extraction criterion was HPA (Horn, 1965). HPA determines whether obtained eigenvalues are larger than that resulting from random data containing the same number of factors and cases. The visual scree was also considered, wherein factors with eigenvalues in the sharp descent of the line in the scree plot are retained (Cattell, 1966). Additionally, the Standard Error of Scree (SEScree) was used, which is an objective counterpart to the visual scree based on the “tightness of fit about the scree line of the points which represent the smaller eigenvalues” (Zoski & Jurs, 1996, p. 445). Last, Velicer's MAP test was considered (Velicer, 1976). MAP examines remaining variance in the correlation matrix following incremental extractions of components, with components no longer being retained when the correlation matrix contains a greater proportion of unsystematic variance than systematic variance (O'Connor, 2000). HPA, with 100 randomly generated data sets, and MAPs were conducted using syntax developed by O'Connor (2000). The program provided by Watkins (2007) was used to perform SEScree.

For higher-order factor analysis, iterations in the first-order PFA were limited to two in estimating final communality estimates (Gorsuch, 2003). PFA was followed by promax (oblique) rotation (k = 4). Oblique rotation was considered appropriate because correlations among factors were expected (Fabrigar et al., 1999). The resulting first-order factors were orthogonalized by removing variance associated with a second-order dimension using the Schmid and Leiman Solution (SLS; Schmid & Leiman, 1957). SLS is a “procedure for transforming an oblique factor analysis solution containing a hierarchy of higher-order factors into an orthogonal solution which not only preserves the desired interpretation characteristics of the oblique solution, but also discloses the hierarchical structuring of the variables” (Schmid & Leiman, 1957, p. 53). SLS was conducted using syntax provided by Wolff and Preising (2005).

Results

Comparison of Factor Extraction Criteria

As seen in Fig. 1, inspection of Cattell's (1966) visual scree plot for the last substantial drop in reduced eigenvalues suggested a one-factor solution. Also displayed in Fig. 1, HPA (Horn, 1965) suggested a one-factor solution, as only the eigenvalue corresponding to a single factor was larger than eigenvalues generated from random data (eigenvalue factor 1 observed data = 5.39, random data = 1.46; eigenvalue factor 2 observed data = 1.06, random data = 1.33). Similarly, Zoski and Jurs' (1996) SEScree and Velicer's (1976) MAP suggested a one-factor solution. In contrast, Kaiser's (1960) rule (eigenvalues >1.0) indicated a two-factor solution. Table 1 summarizes results from the five extraction criteria considered in the present study.

Table 1.

Number of factors suggested for extraction across five criteria

Extraction criteria Number of factors 
Eigenvalue >1 
Visual Scree 
Standard Error of Scree (SEScree
Horn's Parallel Analysis 
Minimum Average Partials 
Extraction criteria Number of factors 
Eigenvalue >1 
Visual Scree 
Standard Error of Scree (SEScree
Horn's Parallel Analysis 
Minimum Average Partials 
Fig. 1.

Scree plot of eigenvalues associated with exploratory factor analysis of the RBANS (Randolph, 1998) subtests and HPA (Horn, 1965).

Fig. 1.

Scree plot of eigenvalues associated with exploratory factor analysis of the RBANS (Randolph, 1998) subtests and HPA (Horn, 1965).

First-Order Factor Analysis

Based upon the five RBANS Indexes purported in the test manual (Randolph, 1998), theoretical consideration was given to the possibility of five first-order factors. However, extraction of five factors was problematic because iterations failed to converge. Initially, the maximum number of iterations for convergence was set to 25. Extraction of five factors failed to converge even when increasing maximum iterations to 1,000, and it led to communality estimates greater than 1.0, suggesting a misspecified model (Fabrigar et al., 1999). (Confirmatory factor analysis was also conducted using Amos software [Arbuckle, 2006] to examine the tenability of the five-factor model purported in the RBANS test manual. However, the goodness-of-fit of the resulting model was poor [χ2 = 474.03, p < .001; CFI = 0.84; RMSEA = 0.14].) Similarly, the extraction of three and four factors resulted in iterations failing to converge. In contrast, the extraction of two factors resulted in convergence following seven iterations. Therefore, a maximum of two first-order factors was considered plausible. Two first-order factors were extracted to explore the tenability of the largest number of factors suggested by the present study's extraction criteria and to compare results to prior RBANS studies that have found a two-factor solution (Carlozzi et al., 2008; Duff et al., 2006; Schmitt et al., 2010; Wilde, 2006).

Table 2 displays pattern coefficients (partial correlations between subtests and factors) and structure coefficients (simple correlations between subtests and factors) for the first-order two-factor solution. (Two factors were also extracted with communalities set to 1.0 [i.e., principal components analysis], the results of which were similar to those presented in Table 2.) Factor 1 was relatively homogeneous and representative of memory. According to Comrey and Lee's (1992) guidelines, the magnitudes of the List Learning, List Recall, List Recognition, and Story Recall subtests' loadings on this factor were “excellent,” and the magnitude of the Story Memory subtest's loading on this factor was “very good.” Factor 2 was less homogeneous in its content than Factor 1. The Line Orientation and Picture Naming subtests had “good” loadings, and the Figure Copy and Figure Recall had “fair” loadings, on Factor 2, representative of visual perception. Additionally, the Semantic Fluency and Coding subtests had “fair” loadings on Factor 2, representative of speed of information processing. The Digit Span subtest's loadings were “poor,” non-significant, and considerably smaller than loadings of the other subtests (seeStevens, 2002). Considering the magnitudes of their respective subtests' loadings, Factor 2 was viewed as being less stable than Factor 1 (Guadagnoli & Velicer, 1988). Disparate pattern and structure coefficients suggested that the RBANS subtests were loading on a single, higher-order factor (Thompson, 2004). A strong correlation between Factor 1 and Factor 2 (r = .76) also suggested the presence of a higher-order factor (Gorsuch, 2003).

Table 2.

First-order pattern and structure coefficients for RBANS subtests

RBANS subtest Factor 1
 
Factor 2
 
Pattern coefficient Structure coefficient Pattern coefficient Structure coefficient 
List Learning 0.849 0.812 −0.049 0.596 
List Recall 0.855 0.775 −0.105 0.545 
List Recognition 0.738 0.720 −0.024 0.537 
Story Recall 0.709 0.859 0.197 0.736 
Story Memory 0.646 0.769 0.162 0.653 
Line Orientation −0.008 0.457 0.591 0.597 
Picture Naming −0.073 0.363 0.574 0.519 
Figure Copy 0.015 0.420 0.528 0.517 
Semantic Fluency −0.005 0.396 0.527 0.524 
Coding 0.206 0.553 0.457 0.613 
Figure Recall 0.275 0.609 0.441 0.649 
Digit Span 0.160 0.391 0.304 0.425 
RBANS subtest Factor 1
 
Factor 2
 
Pattern coefficient Structure coefficient Pattern coefficient Structure coefficient 
List Learning 0.849 0.812 −0.049 0.596 
List Recall 0.855 0.775 −0.105 0.545 
List Recognition 0.738 0.720 −0.024 0.537 
Story Recall 0.709 0.859 0.197 0.736 
Story Memory 0.646 0.769 0.162 0.653 
Line Orientation −0.008 0.457 0.591 0.597 
Picture Naming −0.073 0.363 0.574 0.519 
Figure Copy 0.015 0.420 0.528 0.517 
Semantic Fluency −0.005 0.396 0.527 0.524 
Coding 0.206 0.553 0.457 0.613 
Figure Recall 0.275 0.609 0.441 0.649 
Digit Span 0.160 0.391 0.304 0.425 

Notes: Pattern coefficients >|0.40| were considered significant and are in bold.

RBANS = Repeatable Battery for the Assessment of Neuropsychological Status (Randolph, 1998).

Higher-Order Factor Analysis

Higher-order factor analysis was conducted to evaluate the extent to which a second-order factor, along with the present study's two first-order factors, account for meaningful proportions of variance in RBANS performances. The Digit Span subtest did not load substantially on a first-order factor, but it was included in the higher-order analysis to assess the ability of a second-order factor to account for variance in all 12 RBANS subtests. As shown in Table 3, the second-order general factor accounted for 77.8% of the common variance and 34.8% of the total variance in RBANS subtest scores. The general factor also accounted for 16%–62% (Mdn = 36%) of individual subtest variability. Factor 1 accounted for 13.9% of the common variance, the largest proportion of variance accounted for by a first-order factor. Regarding total variance, Factor 1 accounted for 6.2%. Factor 2 accounted for 8.2% of the common variance and 3.7% of the total variance. The first- and second-order factors together explained 44.7% of the total variance in subtest scores, although the first-order factors accounted for relatively small proportions of variance in their respective subtests when compared with the general factor. Whereas Factor 1 accounted for 10%–18% (Mdn = 14%) of variance in its subtest scores, the general factor accounted for 38%–62% (Mdn = 49%) of the variance in Factor 1 subtest scores. Likewise, whereas Factor 2 accounted for 5%–9% (Mdn = 7%) of the variance in its subtest scores, the general factor accounted for 19%–38% (Mdn = 25%) of the variance in Factor 2 subtest scores. The six subtests that loaded highest on the general factor were Story Recall (0.79), Story Memory (0.70), List Learning (0.70), List Recall (0.65), List Recognition (0.62), and Figure Recall (0.62), all of which are subtests designed to assess memory. The remaining subtests (Digit Span, Picture Naming, Semantic Fluency, Line Orientation, Figure Copy, and Coding) loaded less heavily on the general factor and had relatively low communality estimates, suggesting the presence of unexplained variance at the subtest level.

Table 3.

Orthogonalized higher-order factor analysis of RBANS subtests

RBANS subtest General
 
Factor 1
 
Factor 2
 
h2 u2 
b %S2 b %S2 b %S2 
List Learning 0.70 49 0.42 18 −0.02 0.64 0.36 
List Recall 0.65 42 0.42 18 −0.05 0.60 0.40 
List Recognition 0.62 38 0.37 14 −0.01 0.48 0.52 
Story Recall 0.79 62 0.35 12 0.10 0.79 0.21 
Story Memory 0.70 49 0.32 10 0.08 0.72 0.28 
Line Orientation 0.52 27 0.00 0.30 9 0.32 0.68 
Picture Naming 0.44 19 −0.04 0.28 8 0.24 0.76 
Figure Copy 0.47 22 0.01 0.26 7 0.35 0.65 
Semantic Fluency 0.45 20 0.00 0.26 7 0.31 0.69 
Coding 0.58 33 0.10 0.23 5 0.37 0.63 
Figure Recall 0.62 38 0.14 0.22 5 0.51 0.49 
Digit Span 0.40 16 0.08 0.15 0.20 0.80 
% Total S2  34.8  6.2  3.7 44.7 55.3 
% Common S2  77.8  13.9  8.2   
RBANS subtest General
 
Factor 1
 
Factor 2
 
h2 u2 
b %S2 b %S2 b %S2 
List Learning 0.70 49 0.42 18 −0.02 0.64 0.36 
List Recall 0.65 42 0.42 18 −0.05 0.60 0.40 
List Recognition 0.62 38 0.37 14 −0.01 0.48 0.52 
Story Recall 0.79 62 0.35 12 0.10 0.79 0.21 
Story Memory 0.70 49 0.32 10 0.08 0.72 0.28 
Line Orientation 0.52 27 0.00 0.30 9 0.32 0.68 
Picture Naming 0.44 19 −0.04 0.28 8 0.24 0.76 
Figure Copy 0.47 22 0.01 0.26 7 0.35 0.65 
Semantic Fluency 0.45 20 0.00 0.26 7 0.31 0.69 
Coding 0.58 33 0.10 0.23 5 0.37 0.63 
Figure Recall 0.62 38 0.14 0.22 5 0.51 0.49 
Digit Span 0.40 16 0.08 0.15 0.20 0.80 
% Total S2  34.8  6.2  3.7 44.7 55.3 
% Common S2  77.8  13.9  8.2   

Notes: b = loading of subtest on factor; S2 = variance explained; h2 = communality; u2 = uniqueness.

RBANS = Repeatable Battery for the Assessment of Neuropsychological Status (Randolph, 1998). Bold indicates coefficients and variance estimates bearing on the first-order factor analysis.

Discussion

The present study provided several unique contributions to the research literature on the RBANS by (a) examining the factor structure of the RBANS in a sample of inpatients with schizophrenia or schizoaffective disorder; (b) using several criteria for determining the number of factors to extract and retain (eigenvalues >1, HPA, visual scree, SEScree, and MAP); and (c) conducting higher-order factor analysis to determine the tenability of a second-order factor and the extent to which first-order factors warrant attention beyond that of a general factor. In comparison with prior studies, the present study investigated the RBANS factor structure with a distinct methodology, by the way of multiple extraction criteria and higher-order factor analysis, and distinct sample.

According to its test manual, the 12 subtests of the RBANS make up five age-corrected Index scores that together comprise a Total Scale score (Randolph, 1998). Validation of the RBANS factor structure is critical for making judgments regarding the credibility of interpretative statements based upon its various scores. Five first-order factors, as suggested by the five RBANS Indexes purported in the test manual, were not supported by the present study. This is not surprising given that no extant factor analytic studies of the RBANS using non-psychiatric samples have found a five-factor solution (Carlozzi et al., 2008; Duff et al., 2006; Garcia et al., 2008; Schmitt et al., 2010; Wilde, 2006). In fact, attempting to extract five factors was statistically problematic, as was attempting to extract three and four factors. This was likely due to attempting to extract more factors than justified by the present data. It is likely that the RBANS is overfactored, at least for the present sample; that is, there are probably an insufficient number of subtests to define the RBANS Indexes (Fabrigar et al., 1999). For patients with schizophrenia or schizoaffective disorder, associations among subtests that comprise the RBANS Indexes do not appear to be strong or distinct enough for them to be considered unique dimensions.

A two-factor solution was consistent with the majority of RBANS factor analytic studies to date (Carlozzi et al., 2008; Duff et al., 2006; Schmitt et al., 2010; Wilde, 2006). Subtests comprising a dimension of memory had “very good” to “excellent” factor loadings (List Learning, List Recall, List Recognition, Story Recall, and Story Memory). Indeed, the first factor in all prior factor analytic studies of the RBANS has been comprised largely of subtests assessing memory and, as in the present study, has accounted for a larger proportion of variance than other first-order factors (Carlozzi et al., 2008; Duff et al., 2006; Garcia et al., 2008; Schmitt et al., 2010; Wilde, 2006). A memory dimension accounting for a relatively large proportion of variance in RBANS scores is not unexpected given that 6 of the 12 RBANS subtests were specifically designed to assess memory. In comparison with the memory dimension, the second first-order factor was heterogeneous in its content and likely a less stable dimension. Subtests comprising the second first-order factor had “fair” to “good” factor loadings, making up a dimension entailing visual perception and speed of information processing (Line Orientation, Figure Copy, Figure Recall, Picture Naming, Coding and Semantic Fluency). Digit Span, an attention subtest, did not produce significant loadings.

A substantial correlation between the two first-order factors (r = .76) suggested the presence of a higher-order factor (Thompson, 2004). A higher-order factor was also supported theoretically because the RBANS was developed to include an overall score. Higher-order factor analysis was conducted to examine the utility of a general factor and the present study's two-factor solution to account for meaningful proportions of variance in RBANS scores. The Schmid and Leiman (1957) procedure allowed for the removal of variance accounted for by a second-order factor. The second-order, general factor was the predominate source of variation among RBANS subtests, accounting for 77.8% of the common variance and 34.8% of the total variance. In fact, the general factor explained over three times the variance than the two first-order factors combined. Likewise, the general factor accounted for larger proportions of variance in every subtest than did each subtest's respective first-order factor. Additionally, results of the higher-order analysis were consistent with the present study's factor extraction criteria. Although Kaiser's (1960) rule (eigenvalues >1) suggested the presence of two factors,  HPA (Horn, 1965), Cattell's (1966) visual scree, Zoski and Jurs’ (1996) SEScree, and Velicer's (1976) MAP, all suggested the presence of a single factor. In other words, four of the five factor extraction criteria suggested a one-factor solution. It seems the law of parsimony applies in that a simpler explanation of the RBANS structure is preferred given the small proportions of variance accounted for by first-order factors once the second-order factor is considered. Overall, based upon the present results, clinicians using the RBANS for patients with schizophrenia or schizoaffective disorder should place greatest interpretive weight on the general factor (i.e., Total Scale score).

The present study's finding that a substantial proportion of common variance was accounted for by a general factor is consistent with the idea that much of what is measured by traditional neuropsychological assessment in samples of people with psychotic-spectrum disorders reflects broad neurocognitive ability rather than genuine domain-specific performance (Dickinson & Harvey, 2009). However, controversy remains in the literature on the nature and extent of neurocognitive deficits associated with schizophrenia and schizoaffective disorder (seeHarvey & Keefe, 2009), and studies that have used factor analytic methods are no exception (for a comparison, seeGladsjo et al., 2004; Keefe et al., 2007). Nonetheless, Dickinson and Gold (2008), based upon a review of the literature, contended that “cognitive dimensions assayed by typical neuropsychological test batteries, and the underlying measures, share substantial common variance” (p. 432) and that there are “underappreciated constraints on the amount of reliable cognitive performance variance in traditional neuropsychological test batteries that is free to vary independently” (p. 423) when assessing people with schizophrenia.

Limitations and Directions for Future Research

When drawing inferences based upon the present study's findings, several aspects of its method should be considered. A strength of the present research is that participants with a reported history of neuropsychological risk factor(s), other than substance use, were dropped from analyses. This, in combination with all participants having a diagnosis of schizophrenia or schizoaffective disorder, allowed for factor analytic procedures to be conducted using a relatively homogeneous clinical sample. It has been demonstrated that the use of clinically mixed samples, as well as the use of healthy samples, can obscure the emergence of factors that are particularly informative for assessing certain clinical groups (Delis, Jacobson, Bondi, Hamilton, & Salmon, 2003). A potential drawback of the present study's exclusion criteria, however, is that it resulted in reduction in sample size. Although the present study's data appeared suitable for exploratory factor analysis, particularly given results of the Kaiser–Meyer–Olkin measure of sampling adequacy and Bartlett's test of sphericity, future research is needed to determine whether the present study's findings are replicable across similar samples. Another potential drawback is that overly homogeneous samples can lead to a restriction of range in scores when compared with the population of interest, possibly attenuating meaningful relationships among variables (Fabrigar et al., 1999). Additionally, because of the relative homogeneity of the current sample, the present results may not generalize to populations with neuropsychological risk factors other than, or in addition to, schizophrenia or schizoaffective disorder. It is also important to point out that the majority of participants had a comorbid substance use disorder, possibly making the present study's sample less homogeneous than might be desirable.

Another consideration regarding the present study's method is that all participants were psychiatric inpatients committed to a state hospital in Southern California, limiting the confidence with which conclusions based upon the present research can be generalized to other psychiatric populations. For example, it is possible that the factor structure of the RBANS varies across psychiatric diagnoses, levels of symptom severity, types and dosages of prescribed psychotropic medication, inpatient–outpatient status, types of hospital, and/or geographical region. The present sample was also limited in its representation of older adult psychiatric populations and psychiatric populations with higher levels of education. Future research could directly test for factor structure invariance across different levels of these variables. Additionally, although the present study's sample was ethnically diverse, it was comprised largely of men. As such, the present conclusions may be more applicable to men than women. On the other hand, the relationship between gender and RBANS test performance among psychiatric patients has been found to be minimal (Gogos, Joshua, & Rossell, 2010). Nonetheless, future research utilizing samples with a larger proportion of women is needed.

Last, despite a general factor accounting for a substantial proportion of variance in RBANS test scores, nontrivial proportions of variance in scores at the subtest level remained unexplained. It is unclear to what extent this unexplained variance is reflective of systematic variance or random error variance. In any case, future research should examine whether lower-order factors or “scatter” at the subtest level can be of utility, beyond a general score, in differentiating between RBANS performances attributable to psychiatric illness and RBANS performances attributable to other neuropsychological risk factors. For example, discrepancies between RBANS subtest scores have been shown to be useful for differentiating people at risk for dementia from healthy controls (Clark, Hobson, & O'Bryant, 2010; McDermott & DeFilippis, 2010; Schoenberg et al., 2008) and for tracking neurocognitive change over time in patients with certain movement disorders (Beglinger et al., 2010; Rinehardt et al., 2010). However, the extent to which lower-order scores are useful for such purposes in psychiatric populations remains uncertain. Future research could also determine if lower-order scores, or variability among lower-order scores, predict external criteria (e.g., functional outcomes such as employment status) beyond that predicted by the RBANS Total score.

Conclusions

The present results did not support the RBANS five-Index structure for use with patients with schizophrenia or schizoaffective disorder. Although a two-factor solution similar to that of prior studies was found, higher-order factor analysis indicated that it would likely be misleading to interpret these two dimensions as constructs independent of a second-order factor—a factor representing general neurocognitive functioning. To our knowledge, the present research is the first study to examine the factor structure of the RBANS in a psychiatric population. Insofar as the present results are replicable, when using the RBANS to assess patients with schizophrenia or schizoaffective disorder, consideration should be given to the likelihood that the RBANS Total Scale score represents a homogeneous dimension, a dimension that is more reliable, and presumably more valid, than other scores provided by the RBANS. Although the general factor accounted for a substantial proportion of variance in RBANS scores, nontrivial proportions of variance remained unexplained at the subtest level. To the extent this unexplained variance is systematic, it remains possible that at least some of the RBANS subtests are tapping clinically relevant “narrow” aspects of neuropsychological functioning not captured by the general factor. Because examination of scores at the subtest level has been shown to be useful in other clinical populations, future research should explore the utility of such scores for informing test interpretation and clinical decision making in psychiatric populations. Such research is needed so well-informed judgments can be made regarding interpretations beyond the Total Scale score when assessing people with schizophrenia or schizoaffective disorder.

Conflict of Interest

None declared.

Acknowledgements

The authors wish to thank Kasmira J. Sobkow, MS, and Maribel P. Najera, BA, for their invaluable help in extracting archival data. The authors also wish to thank Kevin Duff, PhD, ABPP, for his helpful comments and suggestions during the preparation of this manuscript.

References

Arbuckle
J. L.
Amos (Version 7.0) [Computer Program]
 , 
2006
Chicago
SPSS
Beglinger
L. J.
Duff
K.
Allison
J.
Theriault
D.
O'Rourke
J. J. F.
Leserman
A.
, et al.  . 
Cognitive change in patients with Huntington disease on the Repeatable Battery for the Assessment of Neuropsychological Status
Journal of Clinical and Experimental Neuropsychology
 , 
2010
, vol. 
32
 (pg. 
573
-
578
)
Carlozzi
N. E.
Horner
M. D.
Yang
C.
Tilley
B. C.
Factor analysis of the Repeatable Battery for the Assessment of Neuropsychological Status
Applied Neuropsychology
 , 
2008
, vol. 
15
 (pg. 
274
-
279
)
Cattell
R. B.
Cattell
R. B.
The meaning and strategic use of factor analysis
Handbook of multivariate experimental psychology
 , 
1966
Chicago
Rand McNally
(pg. 
174
-
243
)
Clark
J. L.
Hobson
V. L.
O'Bryant
S. E.
Diagnostic accuracy of percent retention scores on RBANS verbal memory subtests for the diagnosis of Alzheimer's disease and mild cognitive impairment
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 (pg. 
318
-
326
)
Comrey
A. L.
Lee
H. B.
A first course in factor analysis
 , 
1992
2nd ed.
Hillsdale, NJ
Lawrence Erlbaum
Costello
A. B.
Osborne
J. W.
Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis
Practical Assessment, Research and Evaluation
 , 
2005
, vol. 
10
 
7
(pg. 
1
-
9
)
Delis
D. C.
Jacobson
M.
Bondi
M. W.
Hamilton
J. M.
Salmon
D. P.
The myth of testing construct validity using factor analysis or correlations with normal or mixed clinical populations: Lessons from memory assessment
Journal of the International Neuropsychological Society
 , 
2003
, vol. 
9
 (pg. 
936
-
946
)
Dickinson
D.
Gold
J. M.
Less unique variance than meets the eye: Overlap among traditional neuropsychological dimensions in schizophrenia
Schizophrenia Bulletin
 , 
2008
, vol. 
34
 
3
(pg. 
423
-
434
)
Dickinson
D.
Harvey
P. D.
Systemic hypotheses for generalized cognitive deficits in schizophrenia: A new take on an old problem
Schizophrenia Bulletin
 , 
2009
, vol. 
35
 
2
(pg. 
403
-
414
)
Duff
K.
Langbehn
D. R.
Schoenberg
M. R.
Moser
D. J.
Baade
L. E.
Mold
J.
, et al.  . 
Examining the Repeatable Battery for the Assessment of Neuropsychological Status: Factor analytic studies in an elderly sample
American Journal of Geriatric Psychiatry
 , 
2006
, vol. 
14
 (pg. 
1
-
4
)
Fabrigar
L. R.
Wegener
D. T.
MacCallum
R. C.
Strahan
E. J.
Evaluating the use of exploratory factor analysis in psychological research
Psychological Methods
 , 
1999
, vol. 
4
 
3
(pg. 
272
-
299
)
Garcia
C.
Leahy
B.
Corradi
K.
Forchetti
C.
Component structure of the Repeatable Battery for the Assessment of Neuropsychological Status in dementia
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
63
-
72
)
Gladsjo
J. A.
McAdams
L. A.
Palmer
B. A.
Moore
D. J.
Jeste
D. V.
Heaton
R. K.
A six-factor model of cognition in schizophrenia and related psychotic disorders: Relationships with clinical symptoms and functional capacity
Schizophrenia Bulletin
 , 
2004
, vol. 
30
 (pg. 
739
-
754
)
Gogos
A.
Joshua
N.
Rossell
S. L.
Use of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) to investigate group and gender differences in schizophrenia and bipolar disorder
Australian and New Zealand Journal of Psychiatry
 , 
2010
, vol. 
44
 (pg. 
220
-
229
)
Gold
J. M.
Queern
C.
Iannone
V. N.
Buchanan
R. W.
Repeatable Battery for the Assessment of Neuropsychological Status as a screening test in schizophrenia I: Sensitivity, reliability, and validity
American Journal of Psychiatry
 , 
1999
, vol. 
156
 
12
(pg. 
1944
-
1950
)
Gorsuch
R. L.
Schinka
J. A.
Velicer
W. F.
Factor analysis
Handbook of psychology: Vol. 2. Research methods in psychology
 , 
2003
New Jersey
Wiley
(pg. 
143
-
164
)
Guadagnoli
E.
Velicer
W. F.
Relation to sample size to the stability of component patterns
Psychological Bulletin
 , 
1988
, vol. 
103
 
2
(pg. 
265
-
275
)
Harvey
P. D.
Keefe
R. S. E.
Grant
I.
Adams
K. M.
Clinical neuropsychology of schizophrenia
Neuropsychological assessment of neuropsychiatric and neuromedical disorders
 , 
2009
3rd ed.
Oxford
University Press
(pg. 
507
-
522
)
Hobart
M. P.
Goldberg
R.
Bartko
J. J.
Gold
J. M.
Repeatable Battery for the Assessment of Neuropsychological Status as a screening test in schizophrenia II: Convergent/discriminant validity and diagnostic group comparisons
American Journal of Psychiatry
 , 
1999
, vol. 
156
 
12
(pg. 
1951
-
1957
)
Horn
J. L.
A rationale and test for the number of factors in factor analysis
Psychometrika
 , 
1965
, vol. 
19
 (pg. 
149
-
161
)
Iverson
G. L.
Brooks
B. L.
Haley
G. M. T.
Interpretation of the RBANS in inpatient psychiatry: Clinical normative data and prevalence of low scores for patients with schizophrenia
Applied Neuropsychology
 , 
2009
, vol. 
16
 (pg. 
31
-
41
)
Kaiser
H. F.
The application of electronic computers to factor analysis
Educational and Psychological Measurement
 , 
1960
, vol. 
20
 (pg. 
141
-
151
)
Keefe
R. S. E.
Bilder
R. M.
Harvey
P. D.
Davis
S. M.
Palmer
B. W.
Gold
J. M.
, et al.  . 
Baseline neurocognitive deficits in the CATIE schizophrenia trial
Neuropsychopharmacology
 , 
2007
, vol. 
31
 (pg. 
2033
-
2046
)
McDermott
A. T.
DeFilippis
N. A.
Are the indices of the RBANS sufficient for differentiating Alzheimer's disease and subcortical vascular dementia
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 (pg. 
327
-
334
)
O'Connor
B. P.
SPSS and SAS programs for determine the number of components using parallel analysis and Velicer's MAP test
Behavior Research Methods, Instruments, and Computers
 , 
2000
, vol. 
32
 (pg. 
396
-
402
)
Randolph
C.
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Manual
 , 
1998
San Antonio, TX
The Psychological Corporation
Rinehardt
E.
Duff
K.
Schoenberg
S.
Mattingly
M.
Bharucha
K.
Scott
J.
Cognitive change on the Repeatable Battery of Neuropsychological Status (RBANS) in Parkinson's disease with and without bilateral subthalamic nucleus deep brain stimulation surgery
The Clinical Neuropsychologist
 , 
2010
, vol. 
24
 (pg. 
1339
-
1354
)
Saddichha
S.
Sur
S.
Sinha
B. N. P.
Khess
C. R. J.
How is substance use linked to psychosis? A study of the course and patterns of substance dependence in psychosis
Substance Abuse
 , 
2010
, vol. 
31
 (pg. 
58
-
67
)
Schmid
J.
Leiman
J. M.
The development of hierarchical factor solutions
Psychometrika
 , 
1957
, vol. 
22
 
1
(pg. 
53
-
61
)
Schmitt
A. L.
Livingston
R. B.
Smernoff
E. N.
Reese
E. M.
Hafer
D. G.
Harris
J. B.
Factor analysis of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) in a large sample of patients suspected of dementia
Applied Neuropsychology
 , 
2010
, vol. 
17
 
1
(pg. 
8
-
17
)
Schoenberg
M. R.
Duff
D.
Beglinger
L. J.
Moser
D. J.
Bayless
J. D.
Mold
J.
, et al.  . 
Retention rates on RBANS memory subtests in elderly adults
Journal of Geriatric Psychiatry and Neurology
 , 
2008
, vol. 
21
 (pg. 
26
-
33
)
Stevens
J. P.
Applied multivariate statistics for the social sciences
 , 
2002
4th Ed.
New Jersey
Lawrence Erlbaum
Tabachnick
B. G.
Fidell
L. S.
Using multivariate statistics (4th ed.)
 , 
2001
Boston
Allyn & Bacon
Thompson
B.
Exploratory and confirmatory factor analysis: Understanding concepts and applications
 , 
2004
Washington, DC
American Psychological Association
Velicer
W. F.
Determining the number of components from the matrix of partial correlations
Psychometrika
 , 
1976
, vol. 
41
 (pg. 
321
-
327
)
Watkins
M. W.
SEscree [Computer software]
 , 
2007
State College, PA
Ed & Psych Associates
Wilde
M. C.
The validity of the Repeatable Battery of Neuropsychological Status in acute stroke
The Clinical Neuropsychologist
 , 
2006
, vol. 
20
 (pg. 
702
-
715
)
Wilk
C. M.
Gold
J. M.
Bartko
J. J.
Dickerson
F.
Fenton
W. S.
Knable
M.
, et al.  . 
Test-retest stability of the Repeatable Battery for the Assessment of Neuropsychological Status in schizophrenia
American Journal of Psychiatry
 , 
2002
, vol. 
159
 
5
(pg. 
838
-
844
)
Wilk
C. M.
Gold
J. M.
Humber
K.
Dickerson
F.
Fenton
W. S.
Buchanan
R. W.
Brief cognitive assessment in schizophrenia: Normative data for the Repeatable Battery for the Assessment of Neuropsychological Status
Schizophrenia Research
 , 
2003
, vol. 
70
 (pg. 
175
-
186
)
Wolff
H.
Preising
K.
Exploring item and higher order factor structure with the Schmid-Leiman solution: Syntax codes for SPSS and SAS
Behavior Research Methods
 , 
2005
, vol. 
37
 
1
(pg. 
48
-
58
)
Zoski
K. W.
Jurs
S.
An objective counterpart to the visual scree test for factor analysis: The standard error of scree
Educational and Psychological Measurement
 , 
1996
, vol. 
56
 (pg. 
443
-
451
)

Author notes

Present address Defense and Veterans Brain Injury Center, Navy Medical Center, San Diego, CA, USA.