- Split View
-
Views
-
Cite
Cite
Eeva-Leena Kataja, Linnea Karlsson, Mimmi Tolvanen, Christine Parsons, Adrian Schembri, Hanna Kiiski-Mäki, Hasse Karlsson, Correlation between the Cogstate computerized measure and WAIS-IV among birth cohort mothers, Archives of Clinical Neuropsychology, Volume 32, Issue 2, March 2017, Pages 252–258, https://doi.org/10.1093/arclin/acw099
- Share Icon Share
Abstract
Large studies, with limited resources call for cost-effective cognitive assessment methods. Computerized tests offer viable alternatives but more data are needed on their functioning. Our aim was to evaluate the overlap between a computerized neuropsychological test battery and a traditional test of general intelligence (IQ).
Cognitive functioning was assessed in birth cohort mothers (n = 80) with two widely used methods: Cogstate, computerized test battery, and WAIS-IV, a traditional IQ test. Correlational analyses were conducted.
We found weak-to-moderate correlations between the measures, except for verbal comprehension. The indices of overall performance showed more consistent correlations than Subtests.
The overall correlations were in accordance with earlier studies. Cogstate is relatively independent of verbal comprehension abilities. The choice of the cognitive assessment method should be strongly guided by the research question. More studies are needed to evaluate the applicability of the Cogstate Composite Score in cognitive screening.
Introduction
Computerized cognitive testing offers one viable, cost-effective way to evaluate cognitive functioning in large-scale studies. While evaluation by trained clinicians using standardized neuropsychological tests is the gold-standard, computerized testing methods hold several advantages. These include rapid administration via the automatization of the test procedure, and the fewer training demands on test administrators (Crook, Kay & Larrabee, 2009). Given the potential of these methods, a greater understanding of how different test batteries relate to traditional tests of cognitive functioning, for example, general intelligence, is needed.
Many computerized test batteries with good psychometric properties exist in the field (e.g. Crook et al., 2009; Wild, Howieson, Webbe, Seelye & Kaye, 2008; Witt, Alpherts & Helmstaedter, 2013). One promising approach, “Cogstate” is a computer-administered cognitive screening test battery designed specifically for use in clinical trials and academic research studies (www.cogstate.com; e.g. Collie, Maruff, Darby & McStephen, 2003). The test battery comprises a customizable range of tasks that can provide indices of different cognitive domains such as processing speed, working memory, learning and attention, as well as composite measures (Maruff et al., 2009). The advantages of the Cogstate test battery are for instance, its repeatability (Falleti, Maruff, Collie & Darby, 2006; Pietrzak et al., 2009) and high test-retest reliability (ranging from 0.67 to 0.89 for memory and psychomotor functioning in different studies; e.g. Collie et al., 2003; Falleti, Maruff, Collie, Darby, & McStephen, 2003; Lim et al., 2013), though also lower correlation coefficients have been reported among children (Bangirana, Sikorskii, Giordani, Nakasujja & Boivin, 2015).
Like Cogstate, most computerized tasks use adaptations of standard psychological paradigms, which make the validation of the tasks feasible and the interpretation of the test results easier. Nevertheless, the variation across computerized test batteries hampers comparability, and judgments about their validity and usability usually have to be done on a case-by-case basis (Wild et al., 2008; Witt et al., 2013). Most Cogstate tasks have been validated against different traditional neuropsychological tests such as The Trail Making Test and Wisconsin Card Sorting Test (e.g. Benoit et al., 2015; Hammers et al., 2012; Maruff et al., 2009; Pietrzak et al., 2009) among different populations (e.g. Cysique, Maruff, Darby & Brew, 2006; Lim et al., 2013; Overton et al., 2011).
To our knowledge, no studies have evaluated how performance on Cogstate is related to the performance on a traditional, established test of general intelligence, such as WAIS-IV (Wechsler, 2008; 2012). We therefore undertook this comparison using a sub-sample from a Finnish birth cohort study. In our cohort study, we are interested in parental mood and cognition, and moreover fluid cognitive functions (e.g. working memory) as keys to adaptive parenting (review: Bridgett, Burt, Edwards & Deater-Deckard, 2015). In this study, we wanted to explore, how performance in a computerized cognitive measure relates to the performance in a widely used test of general cognitive functioning.
The WAIS-IV is comprised of four indices: verbal comprehension, visual perception, working memory, and processing speed. In addition, an intelligence quotient can be calculated. Cogstate tasks reportedly assess psychomotor speed, memory, attention, and executive functions. A total cognitive score can also be calculated. Due to the overlap in cognitive domains assessed with both test batteries, we wanted to explore these associations between all Cogstate and WAIS-IV tasks administered in our study. Furthermore, it is interesting to know whether the total cognitive score obtained from Cogstate can be used as an estimate of general intelligence and to whether different Cogstate tasks overlap with general intelligence test performance commonly used in both research and clinical settings. We expected at least a moderate positive correlation between the test batteries, since different cognitive tests typically correlate moderately with each other (Plomin & Deary, 2015). We also expected the working memory measures to be most consistently correlated, as has been reported in some earlier studies evaluating the overlap between different type of cognitive tests (Duan, Wei, Wang & Shi, 2010; Friedman et al., 2006).
Materials and Methods
Participants
Participants were drawn from the larger FinnBrain Birth Cohort Study population (www.finnbrain.fi). Recruitment to the cohort took place at the gestational week (gwk) 12 and relied on a personal contact by a research nurse. Participants were considered eligible to the FinnBrain Birth Cohort Study if they had a verified pregnancy and sufficient knowledge of Finnish or Swedish (the official languages of Finland) to fill in the study questionnaires. The Joint Ethics Committee of Turku University Hospital and University of Turku gave ethical approval for this study. Written informed consent was obtained from participants before the study sessions.
Participants for this study were randomly selected from the cohort, over May 2012–May 2013. The exclusion criteria were (1) self-reported insufficient Finnish/Swedish language skills, (2) self-reported neurologic or psychiatric illness.
Of 240 eligible participants, (1) 109 (46%) wanted to participate, (2) 92 (38%) did not want to participate in this study, and (3) 39 (16%) were not reached. Those agreeing to participate did not differ from those not participating or not reached in terms of age or years of formal education (F(2,236) = 0.41, p = .67; χ²(6) = 11.25, p = .08, respectively).
Finally, after 14 drop-outs, a total of 95 women completed the WAIS-IV during pregnancy (gestational weeks 22–35, mean 28.1). Of these, 80 women also completed the Cogstate either during pregnancy (n = 35, mean age 31.2, gestational weeks 21–34, mean 28.5) or after delivery (n = 45, mean age 30.4, 14–25 weeks postpartum, mean 18.0). The distributions of key background characteristics (age, formal education, parity) were equal between the prenatal and postnatal assessment groups (p > .05). In this study, the study population comprised all women participating both in the WAIS-IV and Cogstate measurements (n = 80).
Measures
The Wechsler Adult Intelligence Scale, Fourth Edition
The Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV, Wechsler, 2008; 2012) is one of the most widely used intelligence test for adults. The test battery is comprised of 10 core subtests, providing four index scores. These are Verbal Comprehension Index (VCI; derived from three subtests: Similarities, Information, and Vocabulary), Perceptual Reasoning Index (PRI; derived from three subtests: Block design, Matrix Reasoning, and Visual Puzzles), Working Memory Index (WMI; derived from two subtests: Digit Span and Arithmetic) and Processing Speed Index (PSI; derived from two subtests: Symbol Search, and Coding). In addition, an overall Full Scale IQ Index score (FSIQ) is obtained from the 10 core subtests converted to a standard score (M = 100, SD = 15). The test is a traditional paper-and-pencil test.
The Cogstate test battery
Nine tasks were administered to measure different aspects of cognitive functioning ( Appendix): verbal learning and memory (International Shopping List Task + recall; ISL, ISLR, respectively. Number of correct responses was used as the unit of measurement), processing speed/psychomotor function (Detection Task; DET, speed of performance), visual attention/vigilance (Identification Task; IDN, speed of performance), visual working memory/attention (One Back Task; OBK, speed of performance), visual recognition memory/attention (Ones Card Learning Task; OCL, accuracy of performance), spatial working memory (Continuous Paired Associate Learning Task; CPAL, accuracy of performance), reasoning and problem solving (Groton Maze Learning Test; GML, number of total errors), and social cognition (Social Emotional Cognition Task; SECT, accuracy of performance). A Total Cognitive Score (a composite score) was calculated from all the tasks.
Procedure
The WAIS-IV assessments were administered by two graduate students trained and supervised by a senior researcher from the Department of Psychology, University of Turku. The test took approximately 90 min to complete.
The Cogstate assessments were administered by a trained doctoral student, according to guidelines and presented on a laptop computer, under supervision of the experimenter. The test session took 45 min to complete, with a short practice before every task.
The whole testing procedure was divided into two different sessions (random order), which took place at the university, in quiet examination rooms.
Data analysis
First, means with standard deviations were calculated for the WAIS-IV subtests and indices and the Cogstate tasks (N = 80). We further calculated the parameters for the Cogstate measures separately for the groups tested in the prenatal (n = 35) and postnatal (n = 45) period and compared the scores between the two groups (Mann–Whitney U test).
For WAIS-IV, the subtest scaled scores and index standard scores were calculated (Finnish norms, Hogrefe Psykologien Kustannus, 2014). For the Cogstate analysis, the completion and integrity pass rates were calculated according to the manual. The primary outcome measures recommended by the Cogstate research manual were used.
Cogstate scores remained non-normally distributed after logarithmic and arcsine transformations. Therefore, non-parametric Spearman correlations were used to analyze the relationships between the WAIS-IV subtests and indices and the Cogstate measures.
Results
Descriptive statistics for the WAIS-IV subtests and the Cogstate tasks are presented in Table 1. The sample appeared to be representative of the general Finnish population showing a normally distributed WAIS-IV scores in subtests (M = 10, SD = 3) and indices (normative M = 100, SD = 15). The completion pass rate was 100% in every Cogstate task, and the integrity pass rate exceeded 97.6% in every task.
Cognitive measures . | Mean scores (SD) N = 80 . |
---|---|
WAIS-IV subtests | |
Block design | 10.5 (3.0) |
Similarities | 10.7 (2.8) |
Digit span | 9.9 (2.6) |
Matrix reasoning | 10.9 (2.3) |
Vocabulary | 9.3 (3.2) |
Arithmetic | 10.0 (2.7) |
Symbol search | 10.9 (3.2) |
Visual puzzles | 9.7 (2.6) |
Information | 9.7 (3.7) |
Coding | 12.1 (3.1) |
WAIS-IV indices | |
VCI | 99.1 (16.38) |
PRI | 101.9 (12.90) |
WMI | 99.86 (14.05) |
PSI | 108.52 (14.70) |
FSIQ | 102.69 (14.55) |
Cogstate measures | |
CPALa | 8.19 (10.62) |
DETa | 2.50 (0.06) |
GMLa | 40.18 (10.68) |
OBKa | 2.88 (0.08) |
IDNa | 2.68 (0.05) |
ISLb | 28.30 (2.91) |
ISLRb | 10.52 (1.21) |
OCLb | 1.10 (0.10) |
SECTb | 1.19 (0.09) |
Total cognitive scoreb | 0.02 (0.46) |
Cognitive measures . | Mean scores (SD) N = 80 . |
---|---|
WAIS-IV subtests | |
Block design | 10.5 (3.0) |
Similarities | 10.7 (2.8) |
Digit span | 9.9 (2.6) |
Matrix reasoning | 10.9 (2.3) |
Vocabulary | 9.3 (3.2) |
Arithmetic | 10.0 (2.7) |
Symbol search | 10.9 (3.2) |
Visual puzzles | 9.7 (2.6) |
Information | 9.7 (3.7) |
Coding | 12.1 (3.1) |
WAIS-IV indices | |
VCI | 99.1 (16.38) |
PRI | 101.9 (12.90) |
WMI | 99.86 (14.05) |
PSI | 108.52 (14.70) |
FSIQ | 102.69 (14.55) |
Cogstate measures | |
CPALa | 8.19 (10.62) |
DETa | 2.50 (0.06) |
GMLa | 40.18 (10.68) |
OBKa | 2.88 (0.08) |
IDNa | 2.68 (0.05) |
ISLb | 28.30 (2.91) |
ISLRb | 10.52 (1.21) |
OCLb | 1.10 (0.10) |
SECTb | 1.19 (0.09) |
Total cognitive scoreb | 0.02 (0.46) |
aLower score = better performance.
bHigher score = better performance.
Cognitive measures . | Mean scores (SD) N = 80 . |
---|---|
WAIS-IV subtests | |
Block design | 10.5 (3.0) |
Similarities | 10.7 (2.8) |
Digit span | 9.9 (2.6) |
Matrix reasoning | 10.9 (2.3) |
Vocabulary | 9.3 (3.2) |
Arithmetic | 10.0 (2.7) |
Symbol search | 10.9 (3.2) |
Visual puzzles | 9.7 (2.6) |
Information | 9.7 (3.7) |
Coding | 12.1 (3.1) |
WAIS-IV indices | |
VCI | 99.1 (16.38) |
PRI | 101.9 (12.90) |
WMI | 99.86 (14.05) |
PSI | 108.52 (14.70) |
FSIQ | 102.69 (14.55) |
Cogstate measures | |
CPALa | 8.19 (10.62) |
DETa | 2.50 (0.06) |
GMLa | 40.18 (10.68) |
OBKa | 2.88 (0.08) |
IDNa | 2.68 (0.05) |
ISLb | 28.30 (2.91) |
ISLRb | 10.52 (1.21) |
OCLb | 1.10 (0.10) |
SECTb | 1.19 (0.09) |
Total cognitive scoreb | 0.02 (0.46) |
Cognitive measures . | Mean scores (SD) N = 80 . |
---|---|
WAIS-IV subtests | |
Block design | 10.5 (3.0) |
Similarities | 10.7 (2.8) |
Digit span | 9.9 (2.6) |
Matrix reasoning | 10.9 (2.3) |
Vocabulary | 9.3 (3.2) |
Arithmetic | 10.0 (2.7) |
Symbol search | 10.9 (3.2) |
Visual puzzles | 9.7 (2.6) |
Information | 9.7 (3.7) |
Coding | 12.1 (3.1) |
WAIS-IV indices | |
VCI | 99.1 (16.38) |
PRI | 101.9 (12.90) |
WMI | 99.86 (14.05) |
PSI | 108.52 (14.70) |
FSIQ | 102.69 (14.55) |
Cogstate measures | |
CPALa | 8.19 (10.62) |
DETa | 2.50 (0.06) |
GMLa | 40.18 (10.68) |
OBKa | 2.88 (0.08) |
IDNa | 2.68 (0.05) |
ISLb | 28.30 (2.91) |
ISLRb | 10.52 (1.21) |
OCLb | 1.10 (0.10) |
SECTb | 1.19 (0.09) |
Total cognitive scoreb | 0.02 (0.46) |
aLower score = better performance.
bHigher score = better performance.
To control for the potential confounding effect of measurement time of the Cogstate, we compared Cogstate mean scores by the assessment time (prenatal and postnatal test groups). The performance was similar across different Cogstate tasks during the pre- and postnatal period (U-tests, p-values .199–.965), with the exception of the ISLR (U = 579.00, p = 036, d = .50). Hence, we decided to treat these groups as one, except for the ISLR task, where partial correlation was used to control for the effects of timing of the test.
Correlations between Cogstate tasks and the WAIS-IV Subtests and Indices
To account for multiple comparisons, we applied an alpha level of (p < .01). Intercorrelations of the Cogstate tasks and WAIS-IV are presented in Table 2.
WAIS-IV subtests indices . | Cogstate Tests . | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
CPAL erra . | DET lmna . | GML tera . | IDN lmna . | ISL corb . | ISLR corb,c . | OCL accb . | OBK lmna . | SECT acca . | COGN scorea . | |
Block design | −0.07 | −0.06 | −0.10 | −0.01 | 0.05 | −0.04 | 0.10 | −0.28* | 0.22 | 0.17 |
Similarities | −0.12 | −0.07 | −0.04 | −0.02 | 0.03 | 0.25* | 0.13 | −0.01 | −0.03 | 0.14 |
Digit span | −0.25* | −0.18 | −0.32** | −0.21 | −0.03 | 0.24* | 0.22* | −0.21 | 0.24* | 0.41** |
Matrix reasoning | −0.33** | −0.10 | −0.02 | −0.26* | 0.10 | 0.33** | 0.10 | −0.27* | 0.04 | 0.36** |
Vocabulary | −0.08 | −0.15 | −0.14 | −0.04 | 0.06 | 0.24* | 0.19 | 0.00 | 0.12 | 0.21 |
Arithmetic | −0.15 | −0.13 | −0.28* | 0.02 | 0.15 | 0.22* | 0.20 | −0.18 | 0.24* | 0.34** |
Symbol search | −0.06 | 0.09 | 0.06 | −0.16 | 0.08 | −0.01 | −0.14 | −0.26* | 0.02 | 0.09 |
Visual puzzles | −0.20 | 0.01 | −0.07 | −0.02 | 0.10 | 0.12 | 0.03 | −0.19 | 0.13 | 0.23* |
Information | −0.04 | −0.01 | −0.12 | −0.06 | 0.02 | 0.14 | 0.12 | −0.19 | 0.19 | 0.17 |
Coding | −0.24* | −0.25* | −0.18 | −0.35** | 0.10 | 0.12 | −0.05 | −0.32** | −0.01 | 0.35** |
VCI | −0.09 | −0.08 | −0.12 | −0.05 | 0.04 | 0.24* | 0.13 | −0.11 | 0.13 | 0.19 |
PRI | −0.25* | −0.05 | −0.08 | −0.12 | 0.13 | 0.13 | 0.10 | −0.31** | 0.16 | 0.31** |
WMI | −0.18 | −0.20 | −0.32** | −0.12 | 0.02 | 0.25* | 0.23* | −0.21 | 0.30** | 0.40** |
PSI | −0.16 | −0.16 | −0.08 | −0.36** | 0.10 | 0.12 | −0.12 | −0.38** | 0.01 | 0.31** |
FSIQ | −0.21 | −0.12 | −0.17 | −0.21 | 0.08 | 0.25* | 0.12 | −0.30** | 0.19 | 0.39** |
WAIS-IV subtests indices . | Cogstate Tests . | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
CPAL erra . | DET lmna . | GML tera . | IDN lmna . | ISL corb . | ISLR corb,c . | OCL accb . | OBK lmna . | SECT acca . | COGN scorea . | |
Block design | −0.07 | −0.06 | −0.10 | −0.01 | 0.05 | −0.04 | 0.10 | −0.28* | 0.22 | 0.17 |
Similarities | −0.12 | −0.07 | −0.04 | −0.02 | 0.03 | 0.25* | 0.13 | −0.01 | −0.03 | 0.14 |
Digit span | −0.25* | −0.18 | −0.32** | −0.21 | −0.03 | 0.24* | 0.22* | −0.21 | 0.24* | 0.41** |
Matrix reasoning | −0.33** | −0.10 | −0.02 | −0.26* | 0.10 | 0.33** | 0.10 | −0.27* | 0.04 | 0.36** |
Vocabulary | −0.08 | −0.15 | −0.14 | −0.04 | 0.06 | 0.24* | 0.19 | 0.00 | 0.12 | 0.21 |
Arithmetic | −0.15 | −0.13 | −0.28* | 0.02 | 0.15 | 0.22* | 0.20 | −0.18 | 0.24* | 0.34** |
Symbol search | −0.06 | 0.09 | 0.06 | −0.16 | 0.08 | −0.01 | −0.14 | −0.26* | 0.02 | 0.09 |
Visual puzzles | −0.20 | 0.01 | −0.07 | −0.02 | 0.10 | 0.12 | 0.03 | −0.19 | 0.13 | 0.23* |
Information | −0.04 | −0.01 | −0.12 | −0.06 | 0.02 | 0.14 | 0.12 | −0.19 | 0.19 | 0.17 |
Coding | −0.24* | −0.25* | −0.18 | −0.35** | 0.10 | 0.12 | −0.05 | −0.32** | −0.01 | 0.35** |
VCI | −0.09 | −0.08 | −0.12 | −0.05 | 0.04 | 0.24* | 0.13 | −0.11 | 0.13 | 0.19 |
PRI | −0.25* | −0.05 | −0.08 | −0.12 | 0.13 | 0.13 | 0.10 | −0.31** | 0.16 | 0.31** |
WMI | −0.18 | −0.20 | −0.32** | −0.12 | 0.02 | 0.25* | 0.23* | −0.21 | 0.30** | 0.40** |
PSI | −0.16 | −0.16 | −0.08 | −0.36** | 0.10 | 0.12 | −0.12 | −0.38** | 0.01 | 0.31** |
FSIQ | −0.21 | −0.12 | −0.17 | −0.21 | 0.08 | 0.25* | 0.12 | −0.30** | 0.19 | 0.39** |
Notes: Spearman correlations (two-tailed) were performed.*p < .05, **p < .01CPAL = Continuous Paired Associate Learning Task; DET = Detection Task; GML = Groton Maze Learning Test; IDN = Identification Task; ISL = International Shopping List Task; ISLR = International Shopping List Task -Recall; OCL = One Card Learning Task; OBK = One Back Memory Task; SECT = Social Emotional Cognition Task; COGN_score = Total Cognitive Score; VCI = Verbal Comprehension Index; PRI = Perceptual Reasoning Index; WMI = Working Memory Index; PSI = Processing Speed Index; FSIQ = Full-Scale Intelligence Quotient.
aPartial correlation was used.
bHigher score = better performance.
cLower score = better performance.
WAIS-IV subtests indices . | Cogstate Tests . | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
CPAL erra . | DET lmna . | GML tera . | IDN lmna . | ISL corb . | ISLR corb,c . | OCL accb . | OBK lmna . | SECT acca . | COGN scorea . | |
Block design | −0.07 | −0.06 | −0.10 | −0.01 | 0.05 | −0.04 | 0.10 | −0.28* | 0.22 | 0.17 |
Similarities | −0.12 | −0.07 | −0.04 | −0.02 | 0.03 | 0.25* | 0.13 | −0.01 | −0.03 | 0.14 |
Digit span | −0.25* | −0.18 | −0.32** | −0.21 | −0.03 | 0.24* | 0.22* | −0.21 | 0.24* | 0.41** |
Matrix reasoning | −0.33** | −0.10 | −0.02 | −0.26* | 0.10 | 0.33** | 0.10 | −0.27* | 0.04 | 0.36** |
Vocabulary | −0.08 | −0.15 | −0.14 | −0.04 | 0.06 | 0.24* | 0.19 | 0.00 | 0.12 | 0.21 |
Arithmetic | −0.15 | −0.13 | −0.28* | 0.02 | 0.15 | 0.22* | 0.20 | −0.18 | 0.24* | 0.34** |
Symbol search | −0.06 | 0.09 | 0.06 | −0.16 | 0.08 | −0.01 | −0.14 | −0.26* | 0.02 | 0.09 |
Visual puzzles | −0.20 | 0.01 | −0.07 | −0.02 | 0.10 | 0.12 | 0.03 | −0.19 | 0.13 | 0.23* |
Information | −0.04 | −0.01 | −0.12 | −0.06 | 0.02 | 0.14 | 0.12 | −0.19 | 0.19 | 0.17 |
Coding | −0.24* | −0.25* | −0.18 | −0.35** | 0.10 | 0.12 | −0.05 | −0.32** | −0.01 | 0.35** |
VCI | −0.09 | −0.08 | −0.12 | −0.05 | 0.04 | 0.24* | 0.13 | −0.11 | 0.13 | 0.19 |
PRI | −0.25* | −0.05 | −0.08 | −0.12 | 0.13 | 0.13 | 0.10 | −0.31** | 0.16 | 0.31** |
WMI | −0.18 | −0.20 | −0.32** | −0.12 | 0.02 | 0.25* | 0.23* | −0.21 | 0.30** | 0.40** |
PSI | −0.16 | −0.16 | −0.08 | −0.36** | 0.10 | 0.12 | −0.12 | −0.38** | 0.01 | 0.31** |
FSIQ | −0.21 | −0.12 | −0.17 | −0.21 | 0.08 | 0.25* | 0.12 | −0.30** | 0.19 | 0.39** |
WAIS-IV subtests indices . | Cogstate Tests . | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
CPAL erra . | DET lmna . | GML tera . | IDN lmna . | ISL corb . | ISLR corb,c . | OCL accb . | OBK lmna . | SECT acca . | COGN scorea . | |
Block design | −0.07 | −0.06 | −0.10 | −0.01 | 0.05 | −0.04 | 0.10 | −0.28* | 0.22 | 0.17 |
Similarities | −0.12 | −0.07 | −0.04 | −0.02 | 0.03 | 0.25* | 0.13 | −0.01 | −0.03 | 0.14 |
Digit span | −0.25* | −0.18 | −0.32** | −0.21 | −0.03 | 0.24* | 0.22* | −0.21 | 0.24* | 0.41** |
Matrix reasoning | −0.33** | −0.10 | −0.02 | −0.26* | 0.10 | 0.33** | 0.10 | −0.27* | 0.04 | 0.36** |
Vocabulary | −0.08 | −0.15 | −0.14 | −0.04 | 0.06 | 0.24* | 0.19 | 0.00 | 0.12 | 0.21 |
Arithmetic | −0.15 | −0.13 | −0.28* | 0.02 | 0.15 | 0.22* | 0.20 | −0.18 | 0.24* | 0.34** |
Symbol search | −0.06 | 0.09 | 0.06 | −0.16 | 0.08 | −0.01 | −0.14 | −0.26* | 0.02 | 0.09 |
Visual puzzles | −0.20 | 0.01 | −0.07 | −0.02 | 0.10 | 0.12 | 0.03 | −0.19 | 0.13 | 0.23* |
Information | −0.04 | −0.01 | −0.12 | −0.06 | 0.02 | 0.14 | 0.12 | −0.19 | 0.19 | 0.17 |
Coding | −0.24* | −0.25* | −0.18 | −0.35** | 0.10 | 0.12 | −0.05 | −0.32** | −0.01 | 0.35** |
VCI | −0.09 | −0.08 | −0.12 | −0.05 | 0.04 | 0.24* | 0.13 | −0.11 | 0.13 | 0.19 |
PRI | −0.25* | −0.05 | −0.08 | −0.12 | 0.13 | 0.13 | 0.10 | −0.31** | 0.16 | 0.31** |
WMI | −0.18 | −0.20 | −0.32** | −0.12 | 0.02 | 0.25* | 0.23* | −0.21 | 0.30** | 0.40** |
PSI | −0.16 | −0.16 | −0.08 | −0.36** | 0.10 | 0.12 | −0.12 | −0.38** | 0.01 | 0.31** |
FSIQ | −0.21 | −0.12 | −0.17 | −0.21 | 0.08 | 0.25* | 0.12 | −0.30** | 0.19 | 0.39** |
Notes: Spearman correlations (two-tailed) were performed.*p < .05, **p < .01CPAL = Continuous Paired Associate Learning Task; DET = Detection Task; GML = Groton Maze Learning Test; IDN = Identification Task; ISL = International Shopping List Task; ISLR = International Shopping List Task -Recall; OCL = One Card Learning Task; OBK = One Back Memory Task; SECT = Social Emotional Cognition Task; COGN_score = Total Cognitive Score; VCI = Verbal Comprehension Index; PRI = Perceptual Reasoning Index; WMI = Working Memory Index; PSI = Processing Speed Index; FSIQ = Full-Scale Intelligence Quotient.
aPartial correlation was used.
bHigher score = better performance.
cLower score = better performance.
The strength of the correlations varied between the Cogstate measures and the WAIS-IV subtests and indices from nonexistent to moderate (Table 2):
WAIS-IV, Verbal Comprehension Index (VCI). Overall, the individual Cogstate measures had only low correlations with the three subtests of the VCI. None of the Cogstate measures was significantly related to the whole VCI as such.
WAIS-IV, Perceptual Reasoning Index (PRI). Three of the Cogstate measures (OBK, CPAL, and IDN) had positive, low to moderate correlations with the three subtests of the PRI. There was a moderate correlation between the Cogstate Total Cognitive Score and the PRI (r = .31, p = .006).
WAIS-IV, Working Memory Index (WMI). A wide range of Cogstate measures (CPAL, GML, OCL, OBK, and SECT) showed low to moderate correlations with the two subtests of the WMI. The Cogstate Total Cognitive Score and the WMI were moderately correlated (r = .40, p = .000).
WAIS-IV, Processing Speed Index (PSI). Low to moderate correlations were also noted between the Cogstate measures (CPAL, DET, IDN, and OBK) and both subtests of the PSI. The Cogstate Total Cognitive Score and the PSI correlated moderately (r = .31, p = .005).
WAIS-IV, Full Scale IQ (FSIQ). One Cogstate measure (OBK) was observed to correlate with the FSIQ, derived from all ten WAIS-IV core subtests. Finally, the Cogstate Total Cognitive Score correlated moderately with the FSIQ (r = .39, p = .000).
Discussion
Here, we compared adults’ performance on the computerized neuropsychological Cogstate test battery and a traditional general intelligence measure, WAIS-IV. We found significant and moderate correlations between all WAIS-IV index scores and the Cogstate Total Cognitive Score, with the exception of the VCI/WAIS-IV which did not correlate with Cogstate measures. At the task level, significant weak-to-moderate correlations between individual WAIS-IV subtests and Cogstate tasks were found.
Our main hypotheses were supported at the composite level: the Cogstate Total Cognitive Score correlated significantly with Working Memory and Processing Speed indices, but also with Perceptual Reasoning Index and Full Scale IQ of WAIS-IV. Although significant, the correlations were rather low (i.e. “acceptable”), and our results did not provide significant support for the usability of the Cogstate battery in assessing intelligence/overall cognitive ability.
In earlier studies, individual Wechsler's tasks (e.g. Letter Number Span, Block Design) have shown weak-to-strong correlations with Cogstate memory, attention and learning tasks (Maruff et al., 2009; Overton et al., 2011; Pietrzak et al., 2009). Some have concluded that Cogstate tasks do not adequately correspond to standard neuropsychological tests (Mielke et al., 2015), and that multiple and different cognitive operations contribute to successful task performance (e.g. Hammers et al., 2012). In line with this, we observed significant correlations for domains theoretically analogous between the two tests (i.e. tasks of working memory and processing speed correlating with each other), while some tasks (i.e. CPAL, IDN, ISLR, OBK) correlated across operationally defined cognitive domains. Also, different task requirements, and different processing modalities (e.g. verbal vs. visual working memory) of these two test batteries may contribute to the results.
Both research and clinical fields call for reliable, cost-effective assessment tools. Cogstate has shown good construct validity in assessing core neurocognitive functions (e.g. Maruff et al., 2009; Pietrzak et al., 2009), and it may also be a good candidate for cognitive, neuropsychological screening in large-scale studies (Fredrickson et al., 2010). Therefore, it is of value to know how much the performance in the Cogstate test is related to and dependent on general IQ. Our study suggests that the Cogstate test battery is relatively independent from crystallized, acquired abilities, and relatively language-independent, as can be seen from low correlations with the WAIS-IV verbal tasks. These domains should be assessed separately if needed for the purposes of the research question at hands. Some individual tasks seem to demand multiple cognitive domains, which can be seen from several significant correlations that these tasks have with WAIS-IV tasks.
In this study, almost every participant passed the completion and integrity criteria in every Cogstate task, suggesting that it is well-tolerated. Furthermore, the administration time was half that of WAIS-IV. As the overlap between WAIS-IV and Cogstate was at most moderate, we suggest that test selection should be closely guided by the research question.
Limitations
This study used a sample of convenience, administering the Cogstate assessments performed either during pregnancy or a few months postpartum. As the performance was similar in both groups, and the overall participant numbers were small, we chose to treat these groups as one. However, due to the sample size and the use of non-parametric methods it is possible that our study was underpowered to detect smaller correlations across tests. Additional studies replicating our findings in larger samples outside of the perinatal period, and for example in men, are warranted.
Conclusions
In this study, the correlation between the Cogstate battery and WAIS-IV was modest and the indices of overall performance showed more consistent correlations than individual subtests. As the two tests appear to have somewhat distinct profiles their individual or combined use should be based on the study design and strongly guided by the research questions. Verbal processing domains are largely lacking from Cogstate, so the performance in these domains should be completed by other valid assessment methods, if needed. More studies will be useful in order to evaluate the usability of the Cogstate Composite Score in cognitive screening.
Conflict of Interest
None declared.
Appendix
Cogstate measure . | Cognitive domain . | Outcome measure . |
---|---|---|
Continuous Paired Associate Learning Task, CPAL | Visual learning | Total number of errors made across seven learning trials |
Detection Task, DET | Psychomotor function, processing speed | Speed of performance, log10 milliseconds |
Groton Maze Learning Test, GML | Executive functioning, reasoning | Total number of errors made across five learning trials |
One-Back Memory Task, OBK | Working memory, attention | Speed of performance, log10 milliseconds |
Identification Task, IDN | Visual attention, vigilance | Speed of performance, log10 milliseconds |
International Shopping List Task, ISL | Verbal learning | Total number of correct responses made in remembering the list on three consecutive trials |
International Shopping List Task – recall, ISLR | Verbal learning | Total number of correct responses made in remembering the list after a delay |
One Card Learning Task, OCL | Visual learning, memory | Accuracy of performance, arcsine proportion |
Social Emotional Cognition Task, SECT | Social cognition | Accuracy of performance, arcsine proportion |
Total Cognitive Score | Average of the standardized scores |
Cogstate measure . | Cognitive domain . | Outcome measure . |
---|---|---|
Continuous Paired Associate Learning Task, CPAL | Visual learning | Total number of errors made across seven learning trials |
Detection Task, DET | Psychomotor function, processing speed | Speed of performance, log10 milliseconds |
Groton Maze Learning Test, GML | Executive functioning, reasoning | Total number of errors made across five learning trials |
One-Back Memory Task, OBK | Working memory, attention | Speed of performance, log10 milliseconds |
Identification Task, IDN | Visual attention, vigilance | Speed of performance, log10 milliseconds |
International Shopping List Task, ISL | Verbal learning | Total number of correct responses made in remembering the list on three consecutive trials |
International Shopping List Task – recall, ISLR | Verbal learning | Total number of correct responses made in remembering the list after a delay |
One Card Learning Task, OCL | Visual learning, memory | Accuracy of performance, arcsine proportion |
Social Emotional Cognition Task, SECT | Social cognition | Accuracy of performance, arcsine proportion |
Total Cognitive Score | Average of the standardized scores |
Cogstate measure . | Cognitive domain . | Outcome measure . |
---|---|---|
Continuous Paired Associate Learning Task, CPAL | Visual learning | Total number of errors made across seven learning trials |
Detection Task, DET | Psychomotor function, processing speed | Speed of performance, log10 milliseconds |
Groton Maze Learning Test, GML | Executive functioning, reasoning | Total number of errors made across five learning trials |
One-Back Memory Task, OBK | Working memory, attention | Speed of performance, log10 milliseconds |
Identification Task, IDN | Visual attention, vigilance | Speed of performance, log10 milliseconds |
International Shopping List Task, ISL | Verbal learning | Total number of correct responses made in remembering the list on three consecutive trials |
International Shopping List Task – recall, ISLR | Verbal learning | Total number of correct responses made in remembering the list after a delay |
One Card Learning Task, OCL | Visual learning, memory | Accuracy of performance, arcsine proportion |
Social Emotional Cognition Task, SECT | Social cognition | Accuracy of performance, arcsine proportion |
Total Cognitive Score | Average of the standardized scores |
Cogstate measure . | Cognitive domain . | Outcome measure . |
---|---|---|
Continuous Paired Associate Learning Task, CPAL | Visual learning | Total number of errors made across seven learning trials |
Detection Task, DET | Psychomotor function, processing speed | Speed of performance, log10 milliseconds |
Groton Maze Learning Test, GML | Executive functioning, reasoning | Total number of errors made across five learning trials |
One-Back Memory Task, OBK | Working memory, attention | Speed of performance, log10 milliseconds |
Identification Task, IDN | Visual attention, vigilance | Speed of performance, log10 milliseconds |
International Shopping List Task, ISL | Verbal learning | Total number of correct responses made in remembering the list on three consecutive trials |
International Shopping List Task – recall, ISLR | Verbal learning | Total number of correct responses made in remembering the list after a delay |
One Card Learning Task, OCL | Visual learning, memory | Accuracy of performance, arcsine proportion |
Social Emotional Cognition Task, SECT | Social cognition | Accuracy of performance, arcsine proportion |
Total Cognitive Score | Average of the standardized scores |