Abstract

The Trail Making Test (TMT) is a useful measure of executive dysfunction in elderly subjects. This study aims to investigate the discriminative validity of the oral version of the TMT (OTMT), which can be administered to subjects with visual or motor disabilities, in elderly patients with Mild Cognitive Impairment (MCI; n = 30), Alzheimer's disease (AD; n = 30), and healthy controls (HCs; n = 25). The WAIS-R Digit Span Backwards Subscale, written form of the Trail Making Task, the Clock Drawing Test, the AD Assessment Scale-Cognitive Subscale, and the OTMT were also administered to all participants in order to examine the concurrent validity of the OTMT. The OTMT part B discriminated between patients with MCI, AD, and HC correctly. The OTMT completion time was not correlated with age, but was negatively correlated with education. In conclusion, the OTMT (mostly part B) is a valid and practical measurement tool for different levels of cognitive impairment, especially for patients with visual or motor disabilities for whom the classical written form is not feasible.

Introduction

Mild Cognitive Impairment (MCI) is considered a transitional state between dementia and normal cognitive aging (Petersen & Morris, 2005). Individuals with MCI are at a higher risk for Alzheimer's disease (AD) conversion with annual rates ranging between 10% and 54% (Jack et al., 2005; Petersen et al., 2001). Differentiating healthy elderly people from subjects with MCI is particularly important in determining normative data for cognitive tests (Hashimoto et al., 2006).

The Trail Making Task (TMT) has been widely used in order to evaluate executive functions and the speed of cognitive processing (Arbuthnott & Frank, 2000). The TMT, particularly part B, which is considered to be a sustained attention and sequencing task, requires subjects to alternate their responses to number-letter sequencing. The TMT has two forms; the TMT-A measures attention and processing speed, whereas the TMT-B evaluates switching on mental sets, in other words “cognitive flexibility” and it is considered an executive task (Cullum & Lacritz, 2009). Since 1944, several versions of the task have been developed (Cangöz, Karakoç, & Selekler, 2007). The TMT is sensitive to mild cognitive deterioration and it has been regarded as one of the major neuropsychological tests which identifies patients with MCI. Brief cognitive screening tools for MCI, such as the Montreal Cognitive Assessment, also include a modified version of the TMT-B (Nasreddine et al., 2005). However, it is hard to administer the classical, written form of the TMT which requires intact visual and motor functions. Since some elderly patients have physical restrictions, computerized or paper-pencil tasks have limited use in this group. An oral version of the TMT (OTMT) which was developed by Ricker and Axelrod (1994) removes the visual and motor components of the task in order to allow for the assessment of those who are unable to complete the standard written form. Furthermore, the OTMT is a practical tool and requires shorter time for administration than the classical written form (Ricker and Axelrod, 1994).

Several validity studies were carried out for the OTMT in both normal and clinical samples. It was demonstrated that the OTMT was able to discriminate between patients with different lesion sites at a similar level of sensitivity as the TMT (Ricker, Axelrod, & Houtler, 1996). However, the OTMT was not validated in patients with MCI. In this study, we aimed to examine whether the OTMT discriminated between elderly patients with amnestic type of MCI, AD, and healthy elderly controls (HCs).

Methods

Participants

The study sample consisted of 30 patients with AD (diagnostic and statistical manual of mental disorders-text revised fourth edition) and 30 patients with amnestic type of MCI (Petersen–Mayo Diagnostic Criteria, Petersen et al., 2001) who were consecutively admitted to the Geriatric Psychiatry outpatient clinic of a university hospital between September 2010 and September 2011. According to the Petersen–Mayo Diagnostic Criteria, the patient with amnestic type of MCI should have memory complaint, which is preferably corroborated by an informant, impaired memory function for age and education, preserved general cognitive function, and intact daily activities of living (Petersen et al., 2001). The HCs (n = 25) were elderly people who were admitted to the geriatrics outpatient clinic of the same university hospital for other medical disorders (such as hypertension, diabetes mellitus, congestive heart disease, osteoarthritis). Uneducated subjects (<5 years of education), subjects with other neuropsychiatric disorders (with a history of head trauma, cerebrovascular disorder, seizure disorder, etc.), and subjects who could not complete neuropsychological tests (written form of the TMT, the AD Assessment Scale-Cognitive Subscale [ADAS-Cog], the Clock Drawing Test [CDT], etc.) due to advanced cognitive impairment, hearing loss, or visual or motor deficit were excluded. The Bradykinesia subscale of Unified Parkinson Disease Rating Scale (UPDRS; Akbostancı, Balaban, & Atbasoglu, 2003; Stern, 1988) was applied to all participants and subjects with bradykinesia scores above 2 (moderate–severe bradykinesia) were excluded, because significant bradykinesia can confound tasks like the TMT and the OTMT which assess completion time. The Mini-Mental State Examination (MMSE), the CDT, the ADAS-Cog, the WAIS-R Digit Span Backwards (DSB) Subscale, the TMT-A and B, and the OTMT-A and B forms were administered to all participants. The MMSE, the CDT, and the ADAS-Cog were administered for the diagnosis of dementia and MCI in the outpatient clinic by the clinicians (the authors ETO-K, AS, OA, and UA). Other cognitive tests were administered by the clinical psychologists (the authors GB and SK). Concurrent validity of the OTMT was evaluated by the correlation between the scores of the OTMT, the TMT, the DSB, the ADAS-Cog, and the CDT. In order to control the bias due to sequence/practice effect, the application orders of the tasks (the TMT and the OTMT) were counter balanced (random integers were generated by using a computer program). Application of the whole battery lasted for about 45–60 min. Informed consent was taken from the subjects or their caregivers and the study was approved by the ethics committee of the university. Detailed information about the cognitive battery is given below.

Mini-Mental State Examination

The scale was developed by Folstein, Folstein, and McHugh (1975) as a screening test for the evaluation of cognitive impairment. Turkish standardization of the scale was carried out by Güngen, Ertan, Eker, Yaşar, and Engin (2002). The subscales include time and space orientation, immediate and delayed memory, attention, and language. The maximum score possible on the MMSE is 30. A lower MMSE score means impaired cognitive functioning.

Clock Drawing Test

The CDT has been widely used in dementia screening as a practical and valid tool for assessing various neuropsychological domains like executive and visuospatial functions. The CDT was developed by Goodglass and Kaplan (1983) and many scoring systems requiring different instructions and methodologies were proposed. Turkish versions of the three scoring methods of the CDT (drawing a 10 past 11 clock within a given circle) has been validated and Shulman's method of scoring has been suggested by Can, Özel Kızıl, Varlı, Turan, and Atlı (2010). In the present study, Shulman's scoring method (0, no clock; 1, severe visuospatial disorganization; 2, moderate visuospatial disorganization of numbers such that accurate denotation of “10 after 11” is impossible; 3, inaccurate representation of 10 after 11 when the visuospatial organization is well done; 4, minor visuospatial errors; 5, perfect clock) was used. The CDT score of ≤3 corresponds to “impaired cognition.”

The Alzheimer's Disease Assessment Scale-Cognitive Subtest

This scale was developed by Rosen, Mohs, and Davis (1984) which is used for the diagnosis and course of AD and the evaluation of the treatment response. The ADAS-Cog has 11 subtests: word recall, naming objects/fingers, commands, constructional praxis, ideational praxis, orientation, word recognition, spoken language ability, comprehension of spoken language, word-finding difficulty in spontaneous speaking, and remembering test instructions. The validity and reliability study of the Turkish version of the scale was performed by Mavioglu, Gedizlioglu, Akyel, Aslaner, and Eser (2006). The maximum score possible on the ADAS-Cog is 70 and a higher score means more cognitive impairment.

WAIS-R DSB Subscale

This subscale was reviewed by Wechsler (1981). Subjects are given sets of digits to repeat backwards. The score is the number of the correctly recalled trials under each condition. Lower scores indicate more cognitive impairment. The DSB task has been commonly used to assess attention and working memory in both clinical and non-clinical samples. This is a test of immediate auditory recall and freedom from distraction.

Trail Making Test

The TMT was first developed in 1944, as a part of the Army Individual Test Battery. Reitan (1955) used it for organic brain damage studies. Cangöz and colleagues (2007) conducted the standardization study of the Turkish version of the TMT in elderly people above 50 years of age. This scale has two paper/pencil forms (A and B) and aims to measure several cognitive features such as complex visual screening, cognitive flexibility, motor speed, and sustained attention. For the TMT-A, the participant is asked to sequentially connect a series of 25 numbered targets that are placed in a quasi-random order on a page as quickly as possible. The TMT-B differs in that the participant is asked to alternate between connecting 25 numbers and letters in a progressive sequential order (such as 1, A, 2, B, 3, C, … ). The major dependent variable for this measure is completion time. Before each application, a short trial is applied and the application time for each form is recorded by a chronometer. If there is an error, it is corrected by either the participant or the executive and recorded as an error score. In the present study, we applied the classical, paper–pencil form of the test and also the oral form of the TMT. The OTMT also has two forms (A and B). In the OTMT-A, participants are asked to count numbers from 1 to 25. In the OTMT-B, the participants are asked to count numbers and letters as in the written form. Before each application, a short trial for the OTMT is applied and the application time for each form is recorded by a chronometer. If there is an error, it is corrected by either the participant or the executive and recorded as an error score (Ricker & Axelrod, 1994).

For the analysis of the TMT scores, we used the difference between the completion time for the TMT-A and the TMT-B (TMT B − A) and the sum of the completion time for the TMT-A and the TMT-B (TMT A + B) by referring to the original study (Cangöz et al., 2007).

Statistical Analysis

According to the distribution of the data, either one-way analysis of variance or Kruskal–Wallis tests were used for the comparison of continuous variables (age, years of education, cognitive test scores). Post hoc tests were Bonferroni or Mann–Whitney U-tests. The correlation analysis was carried out by the Spearman rank correlation test. All statistics were carried out using SPSS.

Results

Table 1 presents the comparison of the clinical characteristics of the groups. Patients with AD were older than those with MCI and HC. There was no significant difference between groups in terms of the total years of education. The MMSE, the CDT, and the ADAS-Cog scores of three groups were significantly different. The TMT-B and the OTMT-B also differentiated between three groups. The TMT-A and the OTMT-A scores of the patients with AD were higher than the patients with MCI and HC. However, the TMT-A and the OTMT-A scores of the patients with MCI and HC were similar. The TMT (B − A) and the TMT (B + A) scores, as well as the OTMT (B − A) and the OTMT (B + A) scores of the three groups were significantly different (AD > MCI > HC).

Table 1.

Statistical comparison of the clinical characteristics of the groups

 Healthy controls Patients with MCI Patients with AD Statistical significance Post hoc tests 
Age (mean) 70 ± 6.3 72.6 ± 8.8 76.3 ± 4.54 F = 5.8, p = .004a AD > MCI = HC 
Years of education (median) 7.5 6.5 χ2 = 0.08, p = .95 NS 
MMSE (median) 28 26 24.5 χ2 = 23.5, p < .001b AD < MCI < HC 
CDT (median) 2.5 χ2 = 25.7, p < .001 AD < MCI < HC 
ADAS-cog (total; median) 9.6 10.8 15.4 χ2 = 28.7, p < .001 AD > MCI > HC 
WAIS-R DBS (mean) 3.56 ± 1.45 4.10 ± 1.47 3.17 ± 1.74 F = 2.68, p = .074 NS 
TMT-A (s; median) 67 85.5 128.5 χ2 = 19, p < .001 AD > MCI = HC 
TMT-B (s; median) 156 253 510 χ2 = 27, p < .001 AD > MCI > HC 
OTMT-A (s; median) 10 χ2 = 16.6, p < .001 AD > MCI = HC 
OTMT-B (s; median) 42 63 111 χ2 = 25, p < .001 AD > MCI > HC 
TMT B − A (s; median) 82 178 334 χ2 = 26.9, p < .001 AD > MCI > HC 
TMT B + A (s; median) 218 328 596 χ2 = 26.3, p < .001 AD > MCI > HC 
OTMT B − A (s; median) 36 53 102 χ2 = 23.1, p < .001 AD > MCI > HC 
OTMT B + A (s; median) 48 76 120 χ2 = 26.3, p < .001 AD > MCI > HC 
 Healthy controls Patients with MCI Patients with AD Statistical significance Post hoc tests 
Age (mean) 70 ± 6.3 72.6 ± 8.8 76.3 ± 4.54 F = 5.8, p = .004a AD > MCI = HC 
Years of education (median) 7.5 6.5 χ2 = 0.08, p = .95 NS 
MMSE (median) 28 26 24.5 χ2 = 23.5, p < .001b AD < MCI < HC 
CDT (median) 2.5 χ2 = 25.7, p < .001 AD < MCI < HC 
ADAS-cog (total; median) 9.6 10.8 15.4 χ2 = 28.7, p < .001 AD > MCI > HC 
WAIS-R DBS (mean) 3.56 ± 1.45 4.10 ± 1.47 3.17 ± 1.74 F = 2.68, p = .074 NS 
TMT-A (s; median) 67 85.5 128.5 χ2 = 19, p < .001 AD > MCI = HC 
TMT-B (s; median) 156 253 510 χ2 = 27, p < .001 AD > MCI > HC 
OTMT-A (s; median) 10 χ2 = 16.6, p < .001 AD > MCI = HC 
OTMT-B (s; median) 42 63 111 χ2 = 25, p < .001 AD > MCI > HC 
TMT B − A (s; median) 82 178 334 χ2 = 26.9, p < .001 AD > MCI > HC 
TMT B + A (s; median) 218 328 596 χ2 = 26.3, p < .001 AD > MCI > HC 
OTMT B − A (s; median) 36 53 102 χ2 = 23.1, p < .001 AD > MCI > HC 
OTMT B + A (s; median) 48 76 120 χ2 = 26.3, p < .001 AD > MCI > HC 

The percentages of errors for TMT-A and OTMT-A are presented in Table 2. Table 2 presents the error rates (percentages) of the groups for the TMT and the OTMT. Table 3 presents the correlations between the OTMT-A and B with other executive tasks in the whole sample. The OTMT-A and the OTMT-B scores were also positively correlated with each other (R = .47, p < .001).

Table 2.

Error rates of the groups for the TMT and the OTMT

 Healthy controls (%) Patients with MCI (%) Patients with AD (%) 
TMT-A 20 20 36 
TMT-B 76 86 93 
OTMT-A 
OTMT-B 72 90 93 
 Healthy controls (%) Patients with MCI (%) Patients with AD (%) 
TMT-A 20 20 36 
TMT-B 76 86 93 
OTMT-A 
OTMT-B 72 90 93 
Table 3.

Correlations between the OTMT and other executive tasks

 OTMT-A completion time OTMT-B completion time 
TMT-A R = .46, p < .001 — 
TMT-B — R = .69, p < .001 
CDT R = −.55, p < .001 R = −.60, p < .001 
WAIS-R DBS R = −.19, p = .08 R = −.23, p = .04 
 OTMT-A completion time OTMT-B completion time 
TMT-A R = .46, p < .001 — 
TMT-B — R = .69, p < .001 
CDT R = −.55, p < .001 R = −.60, p < .001 
WAIS-R DBS R = −.19, p = .08 R = −.23, p = .04 

R, Spearman's correlation coefficient.

Age was not correlated with the TMT-A, the TMT-B, the OTMT-A, or the OTMT-B completion times in HC (R = .02, p = .94; R = −.01, p = .95; R = .30, p = .14; R = −.07, p = .74, respectively). However, education was negatively correlated with the TMT-A, the TMT-B, and the OTMT-B scores, but not with the OTMT-A scores in HC (R = −.57, p = .003; R = −.49, p = .01; R = −.45, p = .02; R = −.30, p = .14, respectively).

Discussion

These results indicate that the OTMT, especially part B, is useful for discriminating between patients with MCI, patients with AD, and healthy elderly subjects. The high correlations between the scores of the OTMT and other neuropsychological tests further support the validity of the task. We also found strong correlations between the OTMT-A and the OTMT-B as in the previous studies (Strauss, Sherman, & Spreen, 2006). However, the OTMT-A and the TMT-A did not discriminate between healthy elderly and patients with MCI. Only part B of the TMT and the OTMT differentiated between the groups. In a large sample of 526 subjects, Ashendorf and colleagues (2008) had similar results in favor of the TMT-B. Cercy, Simakhodskaya, and Elliott (2010) studied the diagnostic accuracy of a brief cognitive measure consisting of several tasks including the OTMT. In their study, a receiver operating curve (ROC) analysis revealed that the diagnostic accuracy of the OTMT-B (area under the curve value was 0.911) was the highest, whereas the diagnostic accuracy of the OTMT-A was not significant. Also, they specified cut-off values for the completion times. For the OTMT-B, a cut-off value of >55 s had a sensitivity of 75% and a specificity of 100%, whereas a cut-off value of >42 s had a sensitivity of 85% and a specificity of 75%. We did not perform ROC analysis; however, median values for the OTMT-B completion times in our study were similar to their results (for MCI patients, median = 63 s; for AD patients median = 42 s). Besides, Cercy, Simakhodskaya, and Elliott (2010) took the presence of cognitive impairment instead of exact clinical diagnoses as an inclusion criterion. In a recent study, Bezdicek and colleagues (2012) also reported that TMT indices, with the exception of the TMT-A, might be useful clinical indicators in distinguishing patients with AD and MCI. The discrepancy between the results for parts A and B is probably due to the hierarchical ordering of attention. Patients with severe cognitive impairment often have difficulty with all levels of attention processing beyond the most basic of tasks like forward digit span or sustained attention tasks like the Continuous Performance Test (which measures target–non-target discrimination ability, e.g., mark the letter “X” followed by the letter “A”) and the TMT-A/OTMT-A. However, patients with MCI demonstrate little difference in simple/sustained attention, even though they have difficulty in alternating their attentional focus (Cullum & Lacritz, 2009).

The WAIS-R DSB, which is a commonly used attention task, was also taken as a measure of concurrent validity in our study. There was a negative correlation between the WAIS-R DSB and the OTMT scores as suggested by Kortte, Horner, and Windham (2002). However, the WAIS-R DSB did not discriminate between the groups.

In this study, we also examined whether the OTMT and the TMT performances were correlated with age or education. Although there was a difference between the groups in terms of age (patients with AD were older), the samples were similar in terms of age intervals (all of them were ∼70) and neither the OTMT nor the TMT scores were correlated with age. Some of the previous studies did not find an association between age and the TMT-B performance, either (Abraham, Axelrod, & Ricker, 1996; Ricker & Axelrod, 1994). The result in the present study may also be due to the exclusion of participants with higher UPDRS scores in order to prevent the confounding effects of Parkinsonism which may have occurred in other studies, as previous research has shown greater deficits on the TMT in AD patients with Parkinsonian features than Alzheimer's patients without these features (Merello et al., 1994). On the other hand, some of the previous studies found that the TMT and the OTMT completion times increased with increasing age (Ashendorf et al., 2008; Bezdicek et al., 2012; Cangöz et al., 2007; Mrazik, Millis, & Drane, 2010). However, until the age of 80, normative data for TMT completion times failed to identify significant age-related decrement (Stuss, Binns, Murphy, & Alexander, 2002; Wardill & Anderson, 2008).

In the current study, the OTMT-A scores were not correlated with education, but the OTMT-B scores were negatively correlated with education. This is probably because the OTMT-A is a simple task consisting of only numbers, whereas the OTMT-B has a greater working memory load. Moreover, the error percentages were quite lower for the OTMT-A than the TMT-A, whereas the error percentages were similar for the B forms of the TMT and the OTMT. This is probably because the OTMT-A is a quite easy task of counting numbers sequentially. In addition, the mean education level of the subjects in the present study was also low (corresponding to a primary school degree); therefore, the correlation between education and the OTMT-B scores may be due to such a low levels of education. In other words, subjects with little education may not have perfect knowledge of the alphabet. For samples with higher levels of education such a correlation seems unlikely as reported by Mrazik and colleagues (2010).,Hashimoto and colleagues (2006) also had similar findings; cognitive functions evaluated by the TMT-A and the TMT-B were not affected by aging until the subjects were ≥85 years old. For the TMT-A, an educational effect became apparent when the population included poorly educated subjects, but this part of the test was not affected by the educational level provided that the subjects had some education (>6 years). The completion time for the TMT-B was affected by the educational level; however, when adjusted using the results for the TMT-A (B − A or B/A), the educational effect on executive function disappeared. Thus, the effect of the educational level on executive function was reported as unclear in healthy elderly subjects (Hashimoto et al., 2006). Some of the previous studies reported correlations between education and the TMT/OTMT scores (Ashendorf et al., 2008; Bezdicek et al., 2012; Drane, Yuspeh, Huthwaite, & Klingler, 2002; Ruchinskas, 2003). Ashendorf and colleagues (2008) and Bezdicek and colleagues (2012) reported a negative correlation between both parts of the TMT and education. Ruchinskas (2003) found an impact of education on OTMT-B performance in a sample of older medical patients. However, the authors acknowledged a significant difference in the education levels among the groups, with the control group having almost three more years of education than one of the experimental groups (consisting of neurological patients).

Some of the previous research examined the ratios between the TMT and the OTMT completion times in order to detect whether these tasks were identical. Axelrod and Lamberty (2006) suggested an oral-to-written ratio of 2.5 and the study by Mrazik and colleagues (2010) revealed an overall ratio of 2.1 for the entire sample, with a range 1.7–2.3 for different age groups. Ricker and Axelrod (1994) demonstrated that the OTMT yielded results consistent with an individual's written performance in normal subjects, regardless of age. Although we found high correlations between the completion times of the TMT-B and the OTMT-B and similar error rates, the working memory load seems to be higher in the OTMT-B than in the written form of this task. Because, participants can use the visual cues to track the last number/letter marked in the written form, whereas in the oral form, they must rely on their own memory to do so. Moreover, there were differences between the error rates of the oral and written forms of part A in our study. The written form of part A requires sustained attention and visual search; therefore, there is no doubt that this situation in the TMT-A differs from simply counting from 1 to 25 in the OTMT-A. The idea that the OTMT-A is not an equivalent task to the TMT-A, probably due to the relative simplicity of the task, has also been reported by Mrazik and colleagues (2010). Taken together, the results of the present study and our experience suggest that the OTMT and the TMT should be considered as two independent tasks. High correlations between these tasks are not sufficient to conclude that they are identical and that their scores can be converted. Inconsistent results from the previous studies further support the idea that they are independent. Therefore, we did not calculate the oral-to-written ratio in our sample.

Although the results of the present study support the OTMT (mostly part B) as a valid and practical screening tool for different levels of cognitive impairment like MCI and AD, there are some limitations. For example, in previous studies, TMT-B performance was reported as a significant predictor of time to progression from mild impairment to a clinical diagnosis of AD (Blacker et al., 2007; Rozzini et al., 2007). However, the present study is a cross-sectional one and this should be taken as the major limitation. Therefore, future studies should be carried out by a longitudinal method in order to address the prognostic value of the OTMT. Additionally, the patients with hearing loss or visual/motor deficits were excluded, although this is the subgroup of patients for whom the OTMT will be most useful and applicable. However, we had to perform such a exclusion in order to administer other tasks requiring intact visuomotor abilities. Furthermore, in order to minimize the practice effects, we counterbalanced the administration sequence of the TMT and the OTMT. However, these tasks consist of the same verbal material and we could not completely control the practice effect. This should also be taken as a limitation of the current study.

In conclusion, the OTMT seems to have many advantages. First of all, the OTMT takes less time. Completion times for the OTMT were shorter than those for the TMT. This finding is in parallel with the original study of the OTMT by Ricker and Axelrod (1994) and the study by Mrazik and colleagues (2010), which also reported similar completion times for the OTMT-A (mean = 7 s) and the OTMT-B (mean = 34 s). Second, it is affected by the aging process less than its written counterpart and it can be more suitable for functional imaging research as it requires lesser time and motor activity. A recent study by Jacobson, Blanchard, Connolly, Cannon, and Garavan (2011) showed significantly greater activation in fMRI during the TMT-B relative to the TMT-A in right inferior/middle frontal cortices, right precentral gyrus, left angular gyrus/left middle temporal gyrus. However, the computerized version of the TMT in that study was quite different from the original task. Finally, the OTMT can also be administered via telephone for longitudinal neuropsychological assessment, as performed by Mitsis and colleagues (2010).

We recommend the usage of the OTMT-B, the OTMT (A + B), or the OTMT (B − A) scores instead of the OTMT-A score both for clinical practice and future research. Moreover, although the TMT and the OTMT seem to be similar tests, they are not identical; therefore, the scores of the OTMT cannot be converted to the TMT scores and norm studies for the OTMT should be carried out.

Conflict of Interest

None declared.

References

Abraham
E.
Axelrod
B. N.
Ricker
J. H.
Application of the oral trail making test to a mixed clinical sample
Archives of Clinical Neuropsychology
 , 
1996
, vol. 
11
 (pg. 
697
-
701
)
Akbostancı
M. C.
Balaban
H.
Atbasoglu
C.
Interrater reliability study of motor examination subscale of UPDRS and AIMS
Journal of Parkinson's Disease and Movement Disorders
 , 
2003
, vol. 
3
 (pg. 
7
-
13
)
Arbuthnott
K.
Frank
J.
Trail Making Test, Part B as a measure of executive control: Validation using a set-switching paradigm
Journal of Clinical and Experimental Neuropsychology
 , 
2000
, vol. 
22
 
4
(pg. 
518
-
528
)
Ashendorf
L.
Jefferson
A. L.
O'Connor
M. K.
Chaisson
C.
Green
R. C.
Stern
R. A.
Trail Making Test errors in normal aging, mild cognitive impairment, and dementia
Archives of Clinical Neuropsychology
 , 
2008
, vol. 
23
 (pg. 
129
-
137
)
Axelrod
B. N.
Lamberty
G. J.
Poreh
A. M.
The Oral Trail Making Test
Neuropsychological assessment: A quantified process approach
 , 
2006
Lisse, The Netherlands
Swets & Zeitlinger
(pg. 
45
-
52
)
Bezdicek
O.
Motak
L.
Axelrod
B. N.
Preiss
M.
Nikolai
T.
Vyhnalek
M.
, et al.  . 
Czech version of the trail making test: normative data and clinical utility
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 
8
(pg. 
906
-
914
)
Blacker
D.
Lee
H.
Muzikansky
A.
Martin
E. C.
Tanzi
R.
McArdle
J. J.
, et al.  . 
Neuropsychological Measures in Normal Individuals That Predict Subsequent Cognitive Decline
Archives of Neurology
 , 
2007
, vol. 
64
 (pg. 
862
-
871
)
Can
S. S.
Özel Kızıl
E. T.
Varlı
M.
Turan
E.
Atlı
T.
Psychometric properties of the Turkish versions of three different Clock Drawing Tests in patients with dementia
Archives of Neuropsychiatry
 , 
2010
, vol. 
47
 
2
(pg. 
91
-
95
)
Cangöz
B.
Karakoç
E.
Selekler
K.
Standardization study of “Trail Making Test” for Turkish adults and elderly people (ages 50 and over)
Turkish Journal of Geriatrics
 , 
2007
, vol. 
10
 
2
(pg. 
73
-
82
)
Cercy
S. P.
Simakhodskaya
Z.
Elliott
A.
Diagnostic accuracy of a new instrument for detecting cognitive dysfunction in an emergent psychiatric population: The Brief Cognitive Screen
Academic Emergency Medicine
 , 
2010
, vol. 
17
 
3
(pg. 
307
-
315
)
Cullum
C. M.
Lacritz
L. H.
Weiner
M. F.
Lipton
A. M.
Neuropsychological assessment in dementia
Textbook of Alzheimer disease and other dementias
 , 
2009
The American Psychiatric Publishing
(pg. 
85
-
105
)
Drane
D. L.
Yuspeh
R. L.
Huthwaite
J. S.
Klingler
L. K.
Demographic characteristics and normative observations for derived-trail making test indices
Neuropsychiatry Neuropsychology Behavioral Neurology
 , 
2002
, vol. 
15
 
1
(pg. 
39
-
43
)
Folstein
M. F.
Folstein
S. E.
McHugh
P. R.
“Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician
Journal of Psychiatric Research
 , 
1975
, vol. 
12
 (pg. 
189
-
198
)
Goodglass
H.
Kaplan
B.
Assessment of aphasia and related disorders
 , 
1983
2nd ed.
Philadelphia
Lea & Febiger
Güngen
C.
Ertan
T.
Eker
E.
Yaşar
R.
Engin
F.
Standardize Mini Mental Test'in Türk Toplumunda Hafif Demans Tanısında Geçerlik ve Güvenilirliği
Türk Psikiyatri Dergisi
 , 
2002
, vol. 
13
 (pg. 
273
-
281
)
Hashimoto
R.
Meguro
K.
Lee
E.
Kasai
M.
Ishii
H.
Yamaguchi
S.
Effect of age and education on the Trail Making Test and determination of normative data for Japanese elderly people: The Tajiri Project
Psychiatry and Clinical Neurosciences
 , 
2006
, vol. 
60
 (pg. 
422
-
428
)
Jack
C. R.
Shiung
M. M.
Weigand
S. D.
O'Brien
P. C.
Gunter
J. L.
Boeve
B. F.
, et al.  . 
Brain atrophy rates predict subsequent clinical conversion in normal elderly and amnestic MCI
Neurology
 , 
2005
, vol. 
65
 (pg. 
1227
-
1231
)
Jacobson
S. C.
Blanchard
M.
Connolly
C. C.
Cannon
M.
Garavan
H.
An fMRI investigation of a novel analogue to the Trail-Making Test
Brain and Cognition
 , 
2011
, vol. 
77
 
1
(pg. 
60
-
70
)
Kortte
K. B.
Horner
M. D.
Windham
W. K.
The Trail Making Test, Part B: cognitive flexibility or ability to maintain a set?
Applied Neuropsychology
 , 
2002
, vol. 
9
 
2
(pg. 
106
-
109
)
Mavioglu
H.
Gedizlioglu
M.
Akyel
S.
Aslaner
T.
Eser
E.
The validity and reliability of Turkish version of Alzheimer Disease Assessment Scale-Cognitive Subscale (Adas-Cog) in patients with mild and moderate Alzheimer disease and normal subjects
International Journal of Geriatric Psychiatry
 , 
2006
, vol. 
21
 
3
(pg. 
259
-
265
)
Merello
M.
Sabe
L.
Teson
A.
Migliorelli
R.
Petracchi
M.
Leiguarda
R.
, et al.  . 
Extrapyramidalism in Alzheimer's disease: Prevalence, psychiatric, and neuropsychological correlates
Journal of Neurology, Neurosurgery, and Psychiatry
 , 
1994
, vol. 
57
 (pg. 
1503
-
1509
)
Mitsis
E. M.
Jacobs
D.
Luo
X.
Andrews
H.
Andrews
K.
Sano
M.
Evaluating cognition in an elderly cohort via telephone assessment
International Journal of Geriatric Psychiatry
 , 
2010
, vol. 
25
 
5
(pg. 
531
-
539
)
Mrazik
M.
Millis
S.
Drane
D. L.
The oral trail making test: Effects of age and concurrent validity
Archives of Clinical Neuropsychology
 , 
2010
, vol. 
25
 
3
(pg. 
236
-
243
)
Nasreddine
Z. S.
Phillips
N. A.
Bedirian
V.
Charbonneau
S.
Whitehead
V.
Collin
I.
, et al.  . 
The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment
Journal of American Geriatric Society
 , 
2005
, vol. 
53
 
4
(pg. 
695
-
699
)
Petersen
R. C.
Doody
R.
Kurz
A.
Mohs
R. C.
Morris
J. C.
Rabins
P. V.
, et al.  . 
Current concepts in Mild Cognitive Impairment
Archives of Neurology
 , 
2001
, vol. 
58
 (pg. 
1985
-
1992
)
Petersen
R. C.
Morris
J. C.
Mild Cognitive Impairment as a clinical entity and treatment target
Archives of Neurology
 , 
2005
, vol. 
62
 (pg. 
1160
-
1163
)
Reitan
R. M.
The relation of the Trail Making Test to organic brain damage
Journal of Consulting Psychology
 , 
1955
, vol. 
19
 (pg. 
393
-
394
)
Ricker
J. H.
Axelrod
B. N.
Analysis of an oral paradigm for the Trail Making Test
Assessment
 , 
1994
, vol. 
1
 
1
(pg. 
51
-
55
)
Ricker
J. H.
Axelrod
B. N.
Houtler
B. D.
Clinical validation of the oral trail making test
Neuropsychiatry, Neuropsychology, and Behavioral Neurology
 , 
1996
, vol. 
9
 (pg. 
50
-
53
)
Rosen
W. G.
Mohs
R. C.
Davis
K. L.
A new rating scale for Alzheimer's disease
American Journal of Psychiatry
 , 
1984
, vol. 
141
 
11
(pg. 
356
-
364
)
Rozzini
L.
Chilovi
B. V.
Conti
M.
Bertoletti
E.
Delrio
I.
Trabucchi
M.
, et al.  . 
Conversion of amnestic Mild Cognitive Impairment to dementia of Alzheimer type is independent to memory deterioration
International Journal of Geriatric Psychiatry
 , 
2007
, vol. 
22
 
12
(pg. 
1217
-
1222
)
Ruchinskas
R. A.
Limitations of the Oral Trail Making Test in a mixed sample of older individuals
The Clinical Neuropsychologist
 , 
2003
, vol. 
17
 
2
(pg. 
137
-
142
)
Stern
M. B.
Stern
M. B.
Hurting
H. I.
The clinical characteristics of Parkinson's disease and Parkinsonian syndromes: A diagnosis and assessment
The comprehensive management of Parkinson's disease
 , 
1988
New York
PMA
(pg. 
3
-
50
)
Strauss
E.
Sherman
E. M. S.
Spreen
O.
A compendium of neuropsychological tests: Administration, norms, and commentary
 , 
2006
3rd ed.
New York
Oxford University Press
Stuss
D. T.
Binns
M. A.
Murphy
K. J.
Alexander
M. P.
Dissociations within the anterior attentional system: Effects of task complexity and irrelevant information on reaction time speed and accuracy
Neuropsychology
 , 
2002
, vol. 
16
 
4
(pg. 
500
-
513
)
Wardill
T.
Anderson
V.
Anderson
V.
Jacobs
R.
Anderson
P. J.
Assessment of executive functioning in older adults
Executive functions and the frontal lobes: A lifespan perspective
 , 
2008
Taylor & Francis Group
(pg. 
155
-
179
)
Wechsler
D.
Manual for the Wechsler Adult Intelligence Scale—Revised
 , 
1981
New York
Psychological Corporation