Abstract

Little is known about the relation between measures of visuospatial function and daily functioning in community-dwelling older adults. The current study addresses this gap in the literature. Forty individuals with mean (SD) age and education of 78.4 (7.5) and 11.9 (2.6) years, respectively, completed a battery of neuropsychological measures including traditional tests of visuospatial function, a test of visuospatial function with verisimilitude, and performance-based measures of global daily functioning and visuospatial daily functioning. Unlike previous studies, statistical analyses directly evaluated the magnitude of the correlations between cognitive tests and daily functioning. Results indicated that all visuospatial measures significantly correlated with both measures of daily functioning (rs = .34–.59). Although the measure designed with verisimilitude was not significantly better than the traditional visuospatial measures at predicting daily functioning, it did account for significant variance beyond that accounted for by the other tests, supporting its incremental validity.

Introduction

The “functional and predictive relation between the patient's performance on a set of neuropsychological tests and the patient's behavior in a variety of real-world settings” is referred to as ecological validity (Sbordone, 1996, p. 16). Ecological validity questions how results obtained in controlled settings relate to functioning in “real-world” environments and is a clinical example of the broader concept of external validity (Tupper & Cicerone, 1990). One cannot assume the ecological validity of neuropsychological tests for a number of reasons. First, the controlled assessment setting involves a quiet environment with minimal distractions, communication with only the examiner, praise and encouragement, frequent breaks, and simple, direct instructions for behavior (Sbordone, 1996). The rationale for this approach is to maximize test reliability (by way of task standardization) and to obtain the patient's best performance. “Real-world” environments are typically less supportive, involving significant distraction, unhelpful interactions, vague goals, and complicated requirements for behavior. Second, neuropsychological tests often attempt to divide cognitive function into specific cognitive domains that are not equivalent to the skills required to complete daily tasks (Heinrichs, 1990). Real-world tasks involve multiple cognitive domains, and it can be difficult to determine the exact functions required to perform a specific task. Third, neuropsychological measures do not give us the detail required for predicting individuals' everyday problems (Wilson, 1993). Fourth, other factors such as emotional problems, health status, compensatory strategies, and overall intellectual function impact real-world behavior (Chaytor & Schmitter-Edgecombe, 2003). Fifth, measuring real-world functioning is difficult. Both questionnaires and performance-based measures have been used to assess basic activities of daily living (ADLs), such as bathing, and complex “instrumental” ADLs (IADLs), such as managing finances.

Although a growing body of literature has addressed the ecological validity of neuropsychological measures of memory and executive function (seeChaytor & Schmitter-Edgecombe, 2003; Marcotte & Grant, 2010, for reviews), less attention has been paid to measures of visuospatial function. This is surprising because visuoperception is an integral part of most daily activities and because changes in visuospatial function are common in neurological trauma, degenerative disease (Cubic & Gouvier, 1996), and normal aging (Royall, Palmer, Chiodo, & Polk, 2004; Salthouse, 2010; Woodruff-Pak, 1997). With older adults experiencing visuospatial difficulties due to both normal and pathological changes in brain function, concern is increasing about the impact of these difficulties on ADLs and IADLs in this population.

A number of empirical studies support a relation between visuospatial test performance and daily functioning in older adults experiencing neurologic and/or psychiatric difficulties. Richardson, Nadler, and Malloy (1995) reported that the Hooper Visual Organization Test (HVOT; Hooper, 1983) was the best predictor of older adult psychiatric inpatients' everyday functioning (rs = .25–.61). The performance of individuals with dementia on an overlapping figures task significantly contributed to the prediction of ADLs (Hill, Backman, & Fratiglioni, 1995). Benedict, Goldstein, Dobraski, and Tannenhaus (1997) reported that a measure of visuospatial function contributed to the prediction of a kitchen task in a mixed sample composed mostly of geriatric psychiatry inpatients. In a sample of individuals early in the course of Alzheimer's disease (AD), visuospatial function was “the sole cognitive predictor” of functional ability (Perry & Hodges, 2000). Glosser and colleagues (2002) found that a measure of object form discrimination significantly correlated with ADLs (r = .60). Farias, Harrell, Neumann, and Houtz (2003) reported a significant relation between a measure of daily functioning and ability to copy the Rey–Osterrieth complex figure (r = .65) in a sample of individuals with AD. Freilich and Hyer (2007) found a significant linear relation between the Visuospatial/Constructional Index from the Repeatable Battery for Assessment of Neuropsychological Status and ADLs (r = .30) and IADLs (r = .25) in a sample of individuals with dementia. It was noted that the “ … Attention and Visuospatial/Constructional indices generally had the strongest correlations with the daily activity measures” compared with the other indices, Immediate Memory, Delayed Memory, and Language (p. 124).

Despite this rather convincing evidence that measures of visuospatial function are among the best neuropsychological predictors of the daily functioning of older adults experiencing neurologic and psychiatric disorders, the literature has not considered this relation in community-dwelling older adults. There are only two exceptions to this and they provide inconsistent results. First, Cahn-Weiner, Malloy, Boyle, Marran, and Salloway (2000) did not find a significant relation between the Judgment of Line Orientation (JOLO) Test (Benton, Varney, & Hamsher, 1978) and IADLs in a sample of community-dwelling older adults. Second, Royall and colleagues (2004) reported a close relation between the rate of decline in visuospatial function and the rate of decline in ADLs but not IADLs in a sample of older adults receiving minimal daily assistance.

In addition to this sparse and contradictory evidence, very few studies directly compare neuropsychological tests developed with ecological validity in mind to traditional neuropsychological tests in their ability to predict everyday functioning (Marcotte & Grant, 2010). In the sole study considering measures of visuospatial function, Nadolne and Stringer (2001) found a significant relation between the Slide Route Recall test, a neuropsychological measure of visuospatial function designed to maximize ecological validity, and wayfinding, the ability to navigate through familiar and novel environments, that was not observed with more traditional visuospatial measures in a unilateral stroke sample. This is consistent with Chaytor and Schmitter-Edgecombe (2003) who state that tests high in verisimilitude (i.e., tests that are similar to everyday tasks; Franzen & Wilhelm, 1996) may be better predictors of everyday functioning.

Studies comparing the ecological validity of specific measures have typically used a regression approach. Although able to determine which measure is the best predictor of everyday functioning, this method does not determine whether the predictive ability of one measure is significantly better in a statistical sense than that of another measure. That is, in a stepwise regression, the predictor removed on the first step may not be a significantly better predictor of the criterion than another measure, even if no other predictors enter the equation. In a standard (simultaneous) regression, the beta weights reflect the relative importance of each predictor in the equation; however, these indicate the “independent” variability in the criterion accounted for by each predictor rather than the total variability. This is problematic in ecological validity studies because the predictors account for overlapping variance in the criterion. By arranging the order in which the variables enter the equation, the incremental validity of each predictor (i.e., whether a variable explains additional variance) can be determined.

The purpose of the current study was to compare the ecological validity of specific neuropsychological measures of visuospatial function in community-dwelling older adults. Both traditional measures of visuospatial function as well as a newer measure designed to maximize verisimilitude were included. The statistical analyses directly compare the predictive ability of the measures. To address the importance of visuospatial function, measures including additional elements of cognition and two measures of everyday functioning, a global measure and a measure focusing on visuospatial function, were completed. We predicted that a neuropsychological measure of visuospatial function with verisimilitude would be a better predictor of everyday functioning than traditional measures of visuospatial function and that neuropsychological measures with predominant visuospatial components would be better predictors of everyday functioning than measures including other elements of cognition.

Method

Participants

The sample consisted of 40 (35 women and 5 men) older adults, with mean (SD) age and education of 78.4 (7.5) and 11.9 (2.6) years, respectively. Age ranged from 65 to 91 years and education ranged from 6 to 19 years. Thirty-six participants identified themselves as Caucasian and four participants identified themselves as Black/African American. Ninety percent (36 of 40) of the sample was right-handed. Participants were recruited from three (n = 15, 15, and 10) US Department of Housing and Urban Development (HUD) subsidized, independent-living retirement communities in the Baltimore, MD, area. Participants were excluded if they scored <24 on the Mini-Mental State Exam (MMSE; Folstein, Folstein, & McHugh, 1975) or had been diagnosed with any form of dementia, made more than two omissions on the Test of Visual Neglect (Albert, 1973), were too physically frail to undergo neuropsychological testing, had motor, visual, or auditory deficits that would significantly interfere with cognitive testing, or were taking medication that was sedating or significantly interfered with cognitive function. Using these criteria, two potential participants were excluded, one due to impaired vision and another due to dementia. Regarding health conditions that could potentially impact cognition, 16 participants had hypertension, 9 had diabetes, 8 had coronary artery disease, 4 had hypothyroidism, and 2 had congestive heart failure. One participant was not able to complete testing for medical reasons and was removed from the study. Approval for the project was obtained from the Loyola University Maryland Institutional Review Board (IRB). All participants provided informed consent, and the treatment of participants was in accordance with the ethical standards of the American Psychological Association (APA) and the state of Maryland.

Materials and Procedure

Neuropsychological measures

The neuropsychological battery included tests commonly conceptualized as measures of visuospatial function, a newer measure of visuospatial function with verisimilitude, and measures including other elements of cognition. Traditional measures of visuospatial function were JOLO (Benton et al., 1978); Wechsler Adult Intelligence Scale, Third Edition (WAIS-III), Block Design (BD; Wechsler, 1997a), HVOT (Hooper, 1983), Wechsler Memory Scale, Third Edition (WMS-III), and Visual Reproduction copy score (VR Copy; Wechsler, 1997b). The Neuropsychological Assessment Battery (NAB; White & Stern, 2003) Map Reading subtest was the measure of visuospatial function with verisimilitude. In this test, examinees view a city map with avenues and boulevards traversing north–south and streets and roads traversing east–west. Participants use the map to answer 12 questions regarding the mileage legend, the compass rose, and right–left orientation. Questions are scored correct (1 point) or incorrect (0 point). Although there is evidence to support the reliability and validity of NAB Map Reading subtest scores (White & Stern, 2003), its veridicality has not been established.

Other measures in the battery included WMS-III Spatial Span (SpSp; Wechsler, 1997b) as a measure of visual attention and working memory, WAIS-III Matrix Reasoning (MR; Wechsler, 1997a) as a measure of nonverbal abstract problem-solving and visuospatial reasoning, and WMS-III VRII (Wechsler, 1997b) as a measure of long-term visual memory. These tests were chosen because they assess elements of cognition other than visuospatial function (i.e., attention, reasoning, and long-term memory) but they also include a visuospatial component, which exemplifies the focus of the study. Their inclusion also provides for a more stringent test of the importance of visuospatial function since competing measures will share variance.

Two additional measures were included in the battery. The MMSE was used to estimate overall cognitive functioning. The Geriatric Depression Scale (GDS) was used to assess for symptoms of depression (Yesavage et al., 1983).

The neuropsychological measures were grouped to create a Visuospatial Tests Group and an Other Cognitive Tests Group. The Visuospatial Tests Group was composed of JOLO, BD, HVOT, VR Copy, and Map Reading. The Other Cognitive Tests Group was composed of SpSp, MR, and VRII. Total scores for each test group were created by converting raw scores to z-scores for each test using the sample's mean and standard deviation then averaging the z-scores. We recognize that categorizing tests into groups based on the cognitive domains that they assess is arbitrary due to their inherently multifactorial nature. However, this procedure is common in the literature and a useful heuristic. We attempted to follow conventions commonly used in the literature.

Measures of everyday function

The Revised Observed Tasks of Daily Living (OTDL-R; Diehl, Marsiske, & Horgas, 2005) is a performance-based measure of IADLs designed for use with older adults. Participants use real-life stimuli to complete three tasks in each of three domains, medication use, telephone use, and financial management, for a total of nine tasks. Medication use involves answering questions about medication labels and a physician's health questionnaire. Telephone use involves looking up and dialing telephone numbers and determining a long distance charge rate. Financial management involves making change from a restaurant check, balancing a checkbook, and paying a bill. Tasks are divided into discrete, observable actions and one point is given for each action performed correctly without a prompt. The total score is a sum of the nine task scores and can range from 0 to 28.

A measure of real-world visuospatial function was developed for use in this study because one appropriate for the sample could not be identified in the literature. This “Environmental Visuospatial Task” (EVT) required participants to answer 16 questions about their apartment building, including estimating distances and determining directions. In establishing the reliability of the measure, 3 of the 16 items were found to be exceptionally easy and one was exceptionally difficult. These items were removed resulting in a 12-item scale composed of six direction questions and six distance questions (see Appendix). To minimize demands on long-term memory, each question was administered in the location that it addresses. Three parallel versions of the measure were developed, one for each building where participants lived. The measure was scored by coding questions as correct (1 point) or incorrect (0 point). Distance questions were scored correct if estimates were within ±33% of the exact value. Total scores range from 0 to 12 with higher scores reflecting better performance.

Regarding the measure's reliability, the internal consistency as indicated by Chronbach's α was 0.75. This value is above the commonly used benchmark of 0.70 and suggests that the measure is assessing a single construct (Netemeyer, Bearden, & Sharma, 2003). Performance was not significantly different between the three retirement communities—F(2, 37) = 0.60, p = .55; however, performance was better on direction questions compared with distance questions—t(39) = 8.89, p< .001. This latter result is not surprising since direction questions have a limited number of options to choose from (i.e., straight, left, right, turn around) which makes guessing easier. Evidence of validity is provided below.

Participants completed testing in two sessions at least 2 days apart to reduce the impact of fatigue on performance. Participants' were closely monitored for fatigue and breaks were taken when necessary. In addition, to assess for the presence of fatigue and test order effects, two administration orders were used with participants randomly assigned to each order.

Statistical analyses

The data analytic strategy involved the comparison of correlation coefficients to evaluate the ability of the various neuropsychological (independent/predictor) variables to predict the two measures of daily functioning. Correlation coefficients were compared using a z-test following a Fisher's r to z transformation. This transformation improves normality substantially when using small sample sizes (Meng, Rosenthal, & Rubin, 1992) and equates the ability to detect differences between two r values across the range of r (Cohen, 1988). The z-test indicates whether the magnitude of a correlation between two variables, X and Y, is significantly larger in a statistical sense than a second correlation between two variables from the same sample, X and Z. Because this test is sensitive to power limitations, effect sizes were also computed using Cohen's q (Cohen, 1988). Cohen's q represents the difference between the two z-transformed correlation coefficients. Values of .10, .30, and .50 are considered small, medium, and large effects, respectively (Cohen, 1988). To mirror the typical analytic strategy of previous studies and determine whether select predictor variables could account for significant additional independent variance in daily functioning beyond that accounted for by the variable with the highest correlation with the criterion (i.e., incremental validity), stepwise regression analyses were also carried out (probability-of-F-to-enter ≤ .05; probability-of-F-to-remove ≥ .10). Statistical assumptions were examined by reviewing variable tolerances, standardized residuals, and a normal probability plot (Pallant, 2005). Analyses were performed on raw scores rather than demographically adjusted scores because our interest was in absolute level of performance rather than performance relative to peers. This approach has been recommended in studies evaluating ecological validity (Silverberg & Millis, 2009), and there is evidence that raw scores are better predictors of daily functioning than demographically adjusted scores (Barrash et al., 2010).

Statistical analyses were performed via the personal computer version of the Statistical Package for the Social Sciences (SPSS 16) as well as a Microsoft Excel program developed to convert Pearson's r to z and perform z-tests (DeCoster, 2007; DeCoster & Leistico, 2005). Statistical tests were one-tailed due to directional hypotheses and evaluated with α equal to 0.05.

Results

Means and standard deviations for demographic, neuropsychological, and daily functioning variables are listed in Table 1. The mean performance of participants administered test order A was not significantly different from that of participants administered test order B on any of the neuropsychological variables. Consistent with exclusion criteria, the mean score on the MMSE was well above the cutoff, indicating that the sample's cognitive function was within normal limits. Similarly, the mean depression score on the GDS was in the normal range, with 33 participants scoring in the normal range, seven scoring in the “mild depressive” range and no participants scoring in the “severe depressive” range. Variability appeared adequate on all measures with no sign of ceiling or floor effects.

Table 1.

Means and standard deviations

 Mean SD 
Demographic variables 
 Age (years) 78.40 7.52 
 Education (years) 11.88 2.56 
 Geriatric Depression Scale 5.72 4.65 
Mini-Mental State Exam 27.20 1.60 
Visuospatial Tests Group 
 Map Reading 5.00 2.61 
 Judgment of Line Orientation 14.42 8.52 
 Block Design 19.58 8.58 
 Hooper Visual Organization Task 20.55 4.93 
 Visual Reproduction Copy 86.53 15.50 
Other Cognitive Tests Group 
 Spatial Span 9.62 2.97 
 Matrix Reasoning 6.68 3.04 
 Visual Reproduction II 16.62 14.43 
Daily Functioning Measures 
 Environmental Visuospatial Task 7.00 2.80 
 Revised Observed Tasks of Daily Living 16.25 5.56 
 Mean SD 
Demographic variables 
 Age (years) 78.40 7.52 
 Education (years) 11.88 2.56 
 Geriatric Depression Scale 5.72 4.65 
Mini-Mental State Exam 27.20 1.60 
Visuospatial Tests Group 
 Map Reading 5.00 2.61 
 Judgment of Line Orientation 14.42 8.52 
 Block Design 19.58 8.58 
 Hooper Visual Organization Task 20.55 4.93 
 Visual Reproduction Copy 86.53 15.50 
Other Cognitive Tests Group 
 Spatial Span 9.62 2.97 
 Matrix Reasoning 6.68 3.04 
 Visual Reproduction II 16.62 14.43 
Daily Functioning Measures 
 Environmental Visuospatial Task 7.00 2.80 
 Revised Observed Tasks of Daily Living 16.25 5.56 

Revised Observed Tasks of Daily Living

Pearson's correlation coefficients between predictor variables and the OTDL-R are listed in Table 2. Performance on the OTDL-R was significantly correlated with most of the demographic and cognitive variables, excluding MMSE and GDS scores. The size of the effects generally ranged from medium to large (Cohen, 1992). Z-tests indicated that the magnitude of the correlation between Map Reading and the OTDL-R was not significantly different from that between the four other visuospatial tasks and the OTDL-R (JOLO z = 1.03, p = .15, q = 0.19; BD z = −0.37, p = .36, q = −0.07; HVOT z = −0.15, p = .44, q = −0.03; and VR Copy z = 0.00, p = 1, q = 0). Values for Cohen's q indicate that these z values represent small effects at most.

Table 2.

Correlations with measures of daily functioning

 EVT OTDL-R 
Demographic variables 
 Age (years) .11 −.27* 
 Education (years) .09 .55** 
 Geriatric Depression Scale .10 −.07 
Mini-Mental State Exam .18 .04 
Visuospatial Measures 
 Map Reading .46** .54** 
 Benton's Judgment of Line Orientation .47** .39** 
 Block Design .40* .59** 
 Hooper Visual Organization Test .34* .56** 
 Visual Reproduction Copy .39* .54** 
 Visuospatial Tests Group .53** .67** 
Measures Including Other Elements of Cognition 
 Spatial Span .18 .59** 
 Matrix Reasoning .39* .48** 
 Visual Reproduction II .21 .55** 
 Other Cognitive Tests Group .34* .71** 
 EVT OTDL-R 
Demographic variables 
 Age (years) .11 −.27* 
 Education (years) .09 .55** 
 Geriatric Depression Scale .10 −.07 
Mini-Mental State Exam .18 .04 
Visuospatial Measures 
 Map Reading .46** .54** 
 Benton's Judgment of Line Orientation .47** .39** 
 Block Design .40* .59** 
 Hooper Visual Organization Test .34* .56** 
 Visual Reproduction Copy .39* .54** 
 Visuospatial Tests Group .53** .67** 
Measures Including Other Elements of Cognition 
 Spatial Span .18 .59** 
 Matrix Reasoning .39* .48** 
 Visual Reproduction II .21 .55** 
 Other Cognitive Tests Group .34* .71** 

Notes: OTDL-R = Revised Observed Tasks of Daily Living; EVT = Environmental Visuospatial Task.

*p < .05.

**p < .01.

In a stepwise regression analysis including all visuospatial measures in the prediction of OTDL-R scores, BD entered on the first step, predicting 35% of the variance, F(1, 38) = 20.45, p< .001. On the second step, Map Reading entered the regression model, predicting 10% of the remaining variance, F(1, 37) = 6.37, p = .016. This was the final step and no other variables entered the regression model. Results of the analysis are listed in Table 3.

Table 3.

Regression analysis for the prediction of Revised Observed Tasks of Daily Living performance

Variable B SE B β 
Step 1 
 Block Design 0.38 0.09 0.59*** 
Step 2 
 Block Design 0.29 0.09 0.44** 
 Map Reading 0.73 0.29 0.34* 
Variable B SE B β 
Step 1 
 Block Design 0.38 0.09 0.59*** 
Step 2 
 Block Design 0.29 0.09 0.44** 
 Map Reading 0.73 0.29 0.34* 

*p< .05.

**p< .01.

***p< .001.

Regarding the composite measures of cognitive function, both the Visuospatial Tests Group and the Other Cognitive Tests Group were significantly correlated with the OTDL-R with large effect sizes (Cohen, 1992); however, the magnitude of the correlations was not significantly different (z = −0.43, p = .33, q = −0.08) and the magnitude of the difference was small. In a stepwise regression, the Other Cognitive Tests Group entered on the first step, predicting 51% of the variance in the OTDL-R, F(1, 38) = 39.10, p < .001. On the second step, the Visuospatial Tests Group entered the regression model, predicting an additional 8% of the remaining variance, F(1, 37) = 6.99, p = .012. Results of the analysis are listed in Table 4.

Table 4.

Regression analysis for composite variables predicting OTDL-R performance

Variable OTDL-R
 
B SE B β 
Step 1 
 Other Cognitive Tests Group 5.22 0.84 0.71*** 
Step 2 
 Other Cognitive Tests Group 3.54 1.00 0.48** 
 Visuospatial Tests Group 2.58 0.98 0.36* 
Variable OTDL-R
 
B SE B β 
Step 1 
 Other Cognitive Tests Group 5.22 0.84 0.71*** 
Step 2 
 Other Cognitive Tests Group 3.54 1.00 0.48** 
 Visuospatial Tests Group 2.58 0.98 0.36* 

Note: OTDL-R = Revised Observed Tasks of Daily Living.

*p< .05.

**p< .01.

***p< .001.

Environmental Visuospatial Task

Pearson's correlation coefficients between predictor variables and the EVT are listed in Table 2. Performance on the EVT correlated with fewer variables. It was not significantly correlated with any demographic variables or global cognitive function measures but it was correlated with all five of the visuospatial measures with effect sizes in the medium range (Cohen, 1992). No other cognitive measure significantly correlated with the EVT; however, the magnitude of the correlation with MR had a medium effect size (Cohen, 1992). Z-tests indicated that the magnitude of the correlation between Map Reading and the EVT was not significantly different from that between the four other visuospatial tasks and the EVT (JOLO z = −0.07, p = .95, q = −0.01; BD z = 0.39, p = .70, q = 0.07; HVOT z = 0.78, p = .44, q = 0.14; and VR Copy z = 0.50, p = .62, q = 0.09). The effect sizes for these z-tests were small. In a stepwise regression analysis including all visuospatial measures in the prediction of EVT scores, JOLO entered on the first step, predicting 22% of the variance, F(1, 38) = 10.49, p = .002. On the second step, Map Reading entered the regression model, predicting an additional 8% of the remaining variance, F(1, 37) = 4.19, p = .048. Results of this analysis are listed in Table 5.

Table 5.

Regression analysis for the prediction of Environmental Visuospatial Task performance

Variable B SE B β 
Step 1 
 Judgment of Line Orientation 1.53 0.05 0.47** 
Step 2 
 Judgment of Line Orientation 1.06 0.05 0.32* 
 Map Reading 0.34 0.17 0.32* 
Variable B SE B β 
Step 1 
 Judgment of Line Orientation 1.53 0.05 0.47** 
Step 2 
 Judgment of Line Orientation 1.06 0.05 0.32* 
 Map Reading 0.34 0.17 0.32* 

*p< .05.

**p< .01.

Regarding the composite measures of cognitive function, both the Visuospatial Tests Group and the Other Cognitive Tests Group were significantly correlated with the EVT with large and medium effect sizes, respectively (Cohen, 1992); the magnitude of the difference between the correlations approached significance (z = 1.56, p = .06, q = 0.24), representing a small effect. In a stepwise regression analysis, the Visuospatial Tests Group entered on the first step and the Other Cognitive Tests Group did not enter the regression model.

Discussion

The purpose of the current study was to compare the ecological validity of measures of visuospatial function in community-dwelling older adults. It was hypothesized that a measure of visuospatial function with verisimilitude would be a better predictor of everyday functioning than traditional measures of visuospatial function and that neuropsychological measures focusing on visuospatial function would be better predictors of everyday functioning than measures including other elements of cognition. The results partially supported the hypotheses, suggesting that measures designed with ecological validity in mind make an important contribution to the prediction of daily functioning and that visuospatial function is an essential element of daily functioning in relatively independent older adults.

In general, correlations between the neuropsychological measures and the global measure of daily functioning, the OTDL-R, are consistent with previous studies indicating that neuropsychological measures are good predictors of daily function in older adults (Benedict et al., 1997; Cahn-Weiner et al., 2000; Farias et al., 2003; Freilich & Hyer, 2007; Glosser et al., 2002; Hill et al., 1995; Perry & Hodges, 2000; Richardson et al., 1995; Royall et al., 2004). The size of the correlations ranged from .39 to .59, representing medium to large effects (Cohen, 1992). Although the visuospatial measure designed to maximize its verisimilitude had a strong linear association with global daily functioning, contrary to hypotheses, the magnitude of the relation was not significantly greater than that between the other visuospatial measures and daily functioning. That is, Map Reading was not a significantly better predictor of global daily functioning than traditional measures of visuospatial function. Importantly, effect size estimates indicated that the differences between the correlations were small. Thus, there were no moderate or large effects that were undetected.

Results from a stepwise regression analysis predicting global daily functioning indicated that Map Reading entered the model on the second step, predicting a statistically significant additional 10% of the remaining variance beyond that accounted for by BD. It is not entirely surprising that BD entered the regression equation on the first step since previous research has noted a relation between BD and daily functioning in healthy young adults (Groth-Marnat & Teal, 2000). These results suggest that Map Reading may have incremental validity by virtue of its ability to predict a different component of daily functioning than traditional neuropsychological measures of visuospatial function. The ability of measures with verisimilitude to predict variance in daily functioning that is independent from that predicted by traditional measures has previously been demonstrated by Higginson, Arnett, and Voss (2000).

In order to provide evidence of the importance of measures of visuospatial function in the prediction of daily functioning, measures commonly conceptualized as visuospatial tests and measures of other elements of cognition that include a visuospatial component were grouped into two composite variables. Both composites produced large correlations with global daily functioning (.67 and .71, respectively). This is fairly consistent with Chaytor, Temkin, Machamber, and Dikmen (2007) who found that as a group, neuropsychological tests had “moderate” ecological validity and accounted for 21%–30% of the variance in daily functioning. Our somewhat larger correlations may be due to our use of a performance-based measure of daily functioning rather than the questionnaire measures used by Chaytor and colleagues.

Contrary to hypotheses, the magnitude of the correlations between each composite and daily functioning was not significantly different. Again, the size of this effect was small. Interestingly, a stepwise regression indicated that measures of visuospatial functioning predicted significant additional, independent variance in global daily functioning beyond that predicted by the other measures. This is an especially remarkable finding since the measures assessing other elements of cognition all included a visuospatial component. This component would have been removed on the first step of the regression, leaving less variance to be predicted by the visuospatial composite. Therefore, daily functioning in older adults appears to be multidimensional but has a significant, specifically visuospatial component.

In addition to the global measure of daily functioning, a second measure that focused on visuospatial daily functioning (i.e., the EVT) was included. Because a measure appropriate for the sample could not be found in the literature, one was developed for this study. The measure is not well-suited for widespread use, due to its specificity to the buildings in which the participants reside, but its psychometric properties appear adequate. Chronbach's α indicated that the internal consistency was adequate and the measure was reliably assessing a single construct in this sample (Netemeyer et al., 2003). This was an important question since the measure includes two different types of questions, involving distances or directions. In fact, the value of α was impressive given the small number of items and conservative estimate of reliability that Chronbach's α provides. The correlations with the neuropsychological measures provide evidence of the EVT's convergent and discriminant validity in this sample. All five visuospatial measures significantly correlated with the EVT (with medium effect sizes), whereas none of the measures including other elements of cognition significantly correlated with the EVT. The moderate correlation with the measure including a significant problem-solving component (i.e., MR) suggests that the EVT has a reasoning component. It could be argued that the estimation of distances within the EVT is better conceptualized as a measure of reasoning (Lezak, Howieson, Loring, Hannay, & Fischer, 2004) given its similarity to other estimation tasks such as the Cognitive Estimation Test (CET; Axelrod & Mills, 1994). However, the estimations made in the EVT differ from those in the CET. In the CET, individuals answer questions that require significant reasoning based on stimuli that are not present and with which they have little experience (e.g., “How tall is the Empire State Building?”). In the EVT, individuals directly observe the distance to be estimated in areas that they frequent; therefore, this task has a significantly smaller reasoning component. In addition, recent evidence suggests that the CET is not best conceptualized as a measure of executive function (Spencer & Johnson-Greene, 2008). Finally, the solid internal consistency of the EVT suggests that both types of questions within the measure are assessing a single construct.

Results regarding the EVT mirror those with the global measure of cognitive function. Contrary to hypotheses, the magnitude of the correlation between the visuospatial test with verisimilitude and the EVT was not significantly greater than that between the other visuospatial measures and the EVT. Once again, the differences between these correlations were small. In the regression, JOLO entered on the first step, predicting 22% of the variance in EVT and Map Reading entered on the second step, predicting an additional 8% of the variance. This result is surprising given the measure's simplicity and apparent lack of verisimilitude. It is also contrary to Cahn-Weiner and colleagues (2000) who did not find a significant relation between JOLO and IADLs in a sample of community-dwelling older adults. Thus, even when considering a more focused element of daily functioning, the test designed to maximize ecological validity was not a significantly better predictor than traditional measures. However, the test with verisimilitude had incremental validity, predicting significant, independent variance in everyday visuospatial functioning beyond that explained by the more traditional neuropsychological measure.

Our results can be compared with those of Nadolne and Stringer (2001). Using a sample of individuals who had suffered a stroke, these authors found that a visuospatial test designed with ecological validity in mind was the only significant predictor of everyday visuospatial function among a number of other more traditional visuospatial measures. The regression results reported here are similar to those reported by Nadolne and Stringer except that those authors did not find significant correlations between the traditional visuospatial measures and visuospatial daily functioning. This discrepancy could be explained by differences between study samples and tests administered.

Results regarding the composite indices indicated that both measures with a predominant visuospatial component as well as measures including other elements of cognition were related to visuospatial daily functioning. However, as a group, the visuospatial measures were marginally better predictors. In addition, the measures including other elements of cognition were not able to predict significant additional variance in visuospatial daily functioning beyond that accounted for by the visuospatial tasks. It appears that, as a group, the visuospatial measures were reasonably good at predicting visuospatial functioning in the real world.

Correlations between demographic and daily function variables warrant brief discussion. Both age and education were significantly correlated with global daily functioning but not visuospatial daily functioning. The finding that age and education were not related to visuospatial daily function was not anticipated considering that such functioning is expected to decline in healthy aging (Woodruff-Pak, 1997). It may be that the specific aspects of daily functioning involved in the task are stable across age and education, but the lack of significant relationship may also be the result of the limited age range of the sample. The failure of the MMSE to significantly correlate with either measure of daily functioning may suggest that the measure is not a good predictor of daily functioning in community-dwelling older adults; however, limited variability on the MMSE secondary to our exclusion criteria could also explain this finding. Similarly, the relation between depression and daily function reported in the literature (e.g., Chaytor et al., 2007; Sbordone, 1997; Sbordone & Guilmette, 1999) but not found here is likely due to range restriction on the GDS since no participants reported severe depression and only 18% reported mild depressive symptoms.

To our knowledge, this is the first published study of the ecological validity of neuropsychological measures that has directly compared correlations in order to determine whether one measure is significantly better than a second measure in terms of predicting functional status. None of these statistical tests produced significant results, indicating that the independent predictive ability of the various measures were not significantly different. The sizes of the differences between correlations were all small. This suggests that regression studies addressing ecological validity should be interpreted with caution.

A number of study limitations indicate a need for additional research. First, it is important to note that results may not generalize beyond the sample, which consisted of older adults who were mostly women living independently. The greater proportion of female participants is only partially accounted for by the gender ratio for this age group in the population. Second, results also may not generalize to other measures of cognitive and daily functioning. Third, our sample size limited our ability to detect small effects. The clinical significance of such effects remains to be determined. Fourth, although the neuropsychological tests predicted significant variance in daily functioning, it must be recognized that in these correlation based analyses a large amount of variance was unexplained. This is a common finding in the literature and indicates that other factors are contributing to daily functioning (Chaytor & Schmitter-Edgecombe, 2003; Loewenstein & Acevedo, 2010). Another weakness of regression is that it addresses prediction in groups rather than individuals, the latter of which is more clinically interesting (Loewenstein & Acevedo, 2010).

In summary, visuospatial function was an important, independent predictor of daily functioning in this sample of community-dwelling older adults. Although a test of visuospatial function developed with ecological validity in mind was not significantly better than a number of other commonly-used clinical measures of visuospatial function in predicting daily functioning, it appeared to predict an element of daily functioning that was not accounted for by the traditional measures. These results support the use of both traditional measures and measures with verisimilitude in clinical practice for the consideration of functional ability.

Conflict of Interest

None declared.

Appendix: Environmental Visuospatial Task

Administration Instructions 
1. Begin in medical suite for question 1 
2. Leave medical suite and stand in hallway for questions 2–4 
3. Walk to the lobby and stand to the left of the mailboxes for questions 5–11 
4. Walk to the elevators and stand with the participant's back facing the elevator doors for question 12 
5. Questions may be repeated as many times as necessary 
6. If participant is unsure of a response, prompt with “just give your best guess” 
Task Items 
Location: Medical Suite (stand at door of testing room) 
1. How far is it, in feet, from the door of the testing room to the door of the medical suite? 
Location: Hallway (halfway between medical suite and stairs) 
2. You are leaving the medical suite, do you turn right, left, or stay straight to go to the mailboxes? 
3. You are moving down the hallway from the medical suite, do you turn right, left, or stay straight to go to the wheelchair ramp? 
4. How far is it, in feet, from the door of the medical suite to the stairs (point)? 
Location: Lobby (stand left of mailboxes) 
5. You have gone down the stairs (point), do you turn right, left, turn around, or stay straight to get to the mailboxes? 
6. You are facing the mailboxes, do you turn right, left, turn around, or stay straight if you want to go to the library? 
7. What is the distance, in feet, between the mailboxes and the library? 
8. How far is it, in feet, from the door of the library (point) to the lounge in the lobby (point)? 
9. You are entering the front door of the building (point), do you turn right, left, turn around, or stay straight to go to the medical suite? 
10. You are facing the front desk (point), do you turn right, left, turn around, or stay straight to go to the elevators? 
11. How far is it, in feet, from the front desk (point) to the elevators (point)? 
Location: Elevators (∼5 feet front of elevators, back towards the elevator doors) 
12. What is the distance, in feet, from the door of the library to the cafeteria? 
Administration Instructions 
1. Begin in medical suite for question 1 
2. Leave medical suite and stand in hallway for questions 2–4 
3. Walk to the lobby and stand to the left of the mailboxes for questions 5–11 
4. Walk to the elevators and stand with the participant's back facing the elevator doors for question 12 
5. Questions may be repeated as many times as necessary 
6. If participant is unsure of a response, prompt with “just give your best guess” 
Task Items 
Location: Medical Suite (stand at door of testing room) 
1. How far is it, in feet, from the door of the testing room to the door of the medical suite? 
Location: Hallway (halfway between medical suite and stairs) 
2. You are leaving the medical suite, do you turn right, left, or stay straight to go to the mailboxes? 
3. You are moving down the hallway from the medical suite, do you turn right, left, or stay straight to go to the wheelchair ramp? 
4. How far is it, in feet, from the door of the medical suite to the stairs (point)? 
Location: Lobby (stand left of mailboxes) 
5. You have gone down the stairs (point), do you turn right, left, turn around, or stay straight to get to the mailboxes? 
6. You are facing the mailboxes, do you turn right, left, turn around, or stay straight if you want to go to the library? 
7. What is the distance, in feet, between the mailboxes and the library? 
8. How far is it, in feet, from the door of the library (point) to the lounge in the lobby (point)? 
9. You are entering the front door of the building (point), do you turn right, left, turn around, or stay straight to go to the medical suite? 
10. You are facing the front desk (point), do you turn right, left, turn around, or stay straight to go to the elevators? 
11. How far is it, in feet, from the front desk (point) to the elevators (point)? 
Location: Elevators (∼5 feet front of elevators, back towards the elevator doors) 
12. What is the distance, in feet, from the door of the library to the cafeteria? 

References

Albert
M. L.
A simple test of visual neglect
Neurology
 , 
1973
, vol. 
23
 (pg. 
658
-
664
)
Axelrod
B. N.
Mills
S. R.
Preliminary standardization of the cognitive estimation test
Assessment
 , 
1994
, vol. 
1
 
3
(pg. 
269
-
274
)
Barrash
J.
Stillman
A.
Anderson
S. W.
Uc
E. Y.
Dawson
J. D.
Rizzo
M.
Prediction of driving ability with neuropsychological tests: Demographic adjustments diminish accuracy
Journal of the International Neuropsychological Society
 , 
2010
, vol. 
16
 (pg. 
679
-
686
)
Benedict
R. H.
Goldstein
M. Z.
Dobraski
M.
Tannenhaus
J.
Neuropsychological predictors of adaptive kitchen behavior in geriatric psychiatry inpatients
Journal of Geriatric Psychiatry and Neurology
 , 
1997
, vol. 
10
 (pg. 
146
-
153
)
Benton
A. L.
Varney
N. R.
Hamsher
K.
Visuospatial judgment: A clinical test
Archives of Neurology
 , 
1978
, vol. 
35
 (pg. 
364
-
367
)
Cahn-Weiner
D. A.
Malloy
P. F.
Boyle
P. A.
Marran
M.
Salloway
S.
Prediction of functional status from neuropsychological tests in community-dwelling elderly individuals
Clinical Neuropsychologist
 , 
2000
, vol. 
14
 (pg. 
187
-
195
)
Chaytor
N.
Schmitter-Edgecombe
M.
The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills
Neuropsychological Review
 , 
2003
, vol. 
13
 (pg. 
181
-
197
)
Chaytor
N.
Temkin
N.
Machamber
J.
Dikmen
S.
The ecological validity of neuropsychological assessment and the role of depressive symptoms in moderate to severe traumatic brain injury
Journal of the International Society of Neuropsychology
 , 
2007
, vol. 
13
 (pg. 
377
-
385
)
Cohen
J.
Statistical power analyses for the behavioral sciences
 , 
1988
2nd ed.
Hillsdale, NJ
Lawrence Erlbaum Associates
Cohen
J.
A power primer
Psychological Bulletin
 , 
1992
, vol. 
112
 (pg. 
155
-
159
)
Cubic
B. A.
Gouvier
W. D.
Sbordone
R. J.
Long
C. L.
The ecological validity of perceptual tests
Ecological validity of neuropsychological testing
 , 
1996
Delray Beach, FL
GR Press/St. Lucie Press
(pg. 
15
-
41
)
DeCoster
J.
Applied linear regression notes set 1
2007
 
. Retrieved, 2008, from http://www.stat-help.com/notes.html
DeCoster
J.
Leistico
A. M.
Comparing correlation coefficients [Excel spreadsheet]
2005
 
Diehl
M.
Marsiske
M.
Horgas
A. L.
The revised observed tasks of daily living: A performance-based assessment of everyday problem solving in older adults
Journal of Applied Gerontology
 , 
2005
, vol. 
24
 (pg. 
211
-
230
)
Farias
S. T.
Harrell
E.
Neumann
C.
Houtz
A.
The relationship between neuropsychological performance and daily functioning in individuals with Alzheimer's disease: Ecological validity of neuropsychological tests
Archives of Clinical Neuropsychology
 , 
2003
, vol. 
18
 (pg. 
655
-
672
)
Folstein
M. F.
Folstein
S. E.
McHugh
P. R.
“Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician
Journal of Psychiatric Research
 , 
1975
, vol. 
12
 (pg. 
189
-
198
)
Franzen
M. D.
Wilhelm
K. L.
Sbordone
R. J.
Long
C. L.
Conceptual foundations of ecological validity in neuropsychology
Ecological validity of neuropsychological testing
 , 
1996
Delray Beach, FL
GR Press/St. Lucie Press
(pg. 
91
-
112
)
Freilich
B. M.
Hyer
L. A.
Relation of the repeatable battery for assessment of neuropsychological status to measures of daily functioning in dementia
Psychological Reports
 , 
2007
, vol. 
101
 (pg. 
119
-
129
)
Glosser
G.
Gallo
J.
Duda
N.
de Vries
J. J.
Clark
C. M.
Grossman
M.
Visual perceptual functions predict instrumental activities of daily living in patients with dementia
Neuropsychiatry Neuropsychology and Behavioral Neurology
 , 
2002
, vol. 
15
 (pg. 
198
-
206
)
Groth-Marnat
G.
Teal
M.
Block design as a measure of everyday spatial ability: A study of ecological validity
Perceptual and Motor Skills
 , 
2000
, vol. 
90
 (pg. 
522
-
526
)
Heinrichs
R. W.
Current and emergent applications of neuropsychological assessment: Problems of validity and utility
Professional Psychology: Research and Practice
 , 
1990
, vol. 
21
 (pg. 
171
-
176
)
Higginson
C. I.
Arnett
P. A.
Voss
W. D.
The ecological validity of clinical tests of memory and attention in multiple sclerosis
Archives of Clinical Neuropsychology
 , 
2000
, vol. 
15
 (pg. 
185
-
204
)
Hill
R. D.
Backman
L.
Fratiglioni
L.
Determinants of functional abilities in dementia
Journal of the American Geriatrics Society
 , 
1995
, vol. 
43
 (pg. 
1092
-
1097
)
Hooper
H. E.
The Hooper Visual Organization Test manual
 , 
1983
Los Angeles
Western Psychological Services
Lezak
M. D.
Howieson
D. B.
Loring
D. W.
Hannay
H. J.
Fischer
J. S.
Neuropsychological Assessment
 , 
2004
4th ed.
New York
Oxford University Press
Loewenstein
D.
Acevedo
A.
Marcotte
T. D.
Grant
I.
The relationship between instrumental activities of daily living and neuropsychological performance
Neuropsychology of everyday functioning
 , 
2010
New York
Guilford Press
(pg. 
93
-
112
)
T. D.
Marcotte
I.
Grant
Neuropsychology of everyday functioning
 , 
2010
New York
Guilford Press
Meng
X. L.
Rosenthal
R.
Rubin
D. B.
Comparing correlated correlation coefficients
Psychological Bulletin
 , 
1992
, vol. 
111
 (pg. 
172
-
175
)
Nadolne
M. J.
Stringer
A. Y.
Ecological validity in neuropsychological assessment: Prediction of wayfinding
Journal of the International Neuropsychological Society
 , 
2001
, vol. 
7
 (pg. 
675
-
682
.
Netemeyer
R. G.
Bearden
W. O.
Sharma
S.
Scaling procedures: Issues and applications
 , 
2003
Thousand Oaks, CA
Sage
Pallant
J.
SPSS survival manual: A step by step guide to data analysis using SPSS version 12
 , 
2005
2nd ed.
New York
Open University Press
Perry
R. J.
Hodges
J. R.
Relationship between functional and neuropsychological performance in early Alzheimer disease
Alzheimer's Disease and Associated Disorders
 , 
2000
, vol. 
14
 (pg. 
1
-
10
)
Richardson
E. D.
Nadler
J. D.
Malloy
P. F.
Neuropsychologic prediction of performance measures of daily living skills in geriatric patients
Neuropsychology
 , 
1995
, vol. 
9
 (pg. 
565
-
572
)
Royall
D. R.
Palmer
R.
Chiodo
L. K.
Polk
M. J.
Declining executive control in normal aging predicts change in functional status: The Freedom House Study
Journal of the American Geriatrics Society
 , 
2004
, vol. 
52
 (pg. 
346
-
352
)
Salthouse
T. A.
Selective review of cognitive aging
Journal of the International Neuropsychological Society
 , 
2010
, vol. 
16
 (pg. 
754
-
760
)
Sbordone
R. J.
Sbordone
R. J.
Long
C. L.
Ecological validity: Some critical issues for the neuropsychologist
Ecological validity of neuropsychological testing
 , 
1996
Delray Beach, FL
GR Press/St. Lucie Press
(pg. 
15
-
41
)
Sbordone
R. J.
Horton
A.
Wedding
D.
Webster
J.
The ecological validity of neuropsychological testing
The neuropsychology handbook, Vol. 1, foundations and assessment
 , 
1997
2nd ed.
New York
Springer
Sbordone
R. J.
Guilmette
T. J.
Sweet
J. J.
Ecological validity: Prediction of everyday and vocational functioning from neuropsychological test data
Forensic neuropsychology: Fundamentals and practice
 , 
1999
Lisse, The Netherlands
Swets and Zeitlinger
(pg. 
227
-
254
)
Silverberg
N. D.
Millis
S. R.
Impairment versus deficiency in neuropsychological assessment: Implications for ecological validity
Journal of the International Neuropsychological Society
 , 
2009
, vol. 
15
 (pg. 
94
-
102
)
Spencer
R. J.
Johnson-Greene
D.
The cognitive estimation test (CET): Psychometric limitations in neurorehabilitation populations
Journal of Clinical and Experimental Neuropsychology
 , 
2008
, vol. 
31
 (pg. 
373
-
377
)
Tupper
D.
Cicerone
K.
Tupper
D.
Cicerone
K.
Introduction to the neuropsychology of everyday life
The neuropsychology of everyday life: Assessment and basic competencies
 , 
1990
Boston
Kluwer Academic
(pg. 
3
-
18
)
Wechsler
D.
WAIS III administration and scoring manual
 , 
1997
San Antonio, TX
The Psychology Corporation
Wechsler
D.
WMS III administration and scoring manual
 , 
1997
San Antonio, TX
The Psychology Corporation
White
T.
Stern
R.
Neuropsychological Assessment Battery: Psychometric and technical manual
 , 
2003
Lutz, FL
Psychological Assessment Resources
 
(Original work published 2001)
Wilson
B. A.
Ecological validity of neuropsychological assessment: Do neuropsychological indexes predict performance in everyday activities
Applied and Preventative Psychology
 , 
1993
, vol. 
2
 (pg. 
209
-
215
)
Woodruff-Pak
D. S.
The neuropsychology of aging
 , 
1997
Oxford, UK
Blackwell
Yesavage
J. A.
Brink
T. L.
Rose
T. L.
Lum
O.
Huang
V.
Adey
M. B.
, et al.  . 
Development and validation of a geriatric depression scale: A preliminary report
Journal of Psychiatric Research
 , 
1983
, vol. 
17
 (pg. 
37
-
49
)