Diagnostic test accuracy of informant-based tools to diagnose dementia in older hospital patients with delirium: a prospective cohort study

Abstract

Background: delirium and dementia co-exist commonly in hospital. Older people with delirium have high rates of undiagnosed dementia, but delirium affects the use of cognitive testing in dementia diagnosis. Novel methods to detect dementia in delirium are needed. The purpose of the study was to investigate the diagnostic test accuracy of informant tools to detect dementia in hospitalised older people with delirium.

Methods: the presence of dementia on admission was assessed using the short form of the Informant Questionnaire of Cognitive Decline in the Elderly (IQCODE-SF) and Alzheimer's Disease 8 (AD8) in people over 70 years old with delirium. Reference standard diagnosis was made using Diagnostic and Statistical Manual of Mental Disorders fourth edition (DSM-IV) criteria at 3 months. The main outcome measures were the diagnostic test accuracy of the IQCODE-SF and the AD8 in diagnosing DSM-IV dementia.

Results: dementia prevalence at 3 months was 61%. The area under the receiver operating characteristic curve (AUROC) was 0.93 (P < 0.0005) for admission IQCODE-SF and 0.91 (P < 0.0005) for admission AD8. An IQCODE-SF test result of >3.82 on admission had a sensitivity of 0.91 (0.79–0.97) and specificity of 0.93 (0.76–0.99) for detecting dementia. An AD8 of >6 had a sensitivity of 0.83 (0.69–0.92) and specificity of 0.90 (0.72–0.97) for detecting dementia.

Conclusion: the IQCODE-SF and AD8 are sensitive and specific tools to detect prior dementia in older people with delirium. The routine use of either tool in practice could have important clinical impact, by improving the recognition and hence management of those with dementia.

delirium, dementia, sensitivity and specificity, screening, diagnosis, older people

Introduction

Dementia is a chronic neurodegenerative disease characterised by progressive cognitive decline, amnesic deficits and functional deficits [1]. Dementia is common among older hospital patients. In acute medical admissions, cognitive impairment affects 50% [2] and dementia up to 42% [3, 4]. Older patients with dementia have higher rates of adverse events in hospital [5] as well as higher mortality and institutionalisation [6]. However, dementia in this population is under-recognised, with up to a half of older patients in hospital with dementia not having a recognised diagnosis [3].

Delirium is a severe neuropsychiatric syndrome characterised by an acute change in cognition, attentional deficits and altered arousal [7]. Delirium is also common in older hospital inpatients, affecting 15–25% of acute admissions [2, 8]. Apart from age, cognitive impairment is the most important risk factor for delirium [9]. Therefore, older patients with delirium are a key target population to improve identification of dementia within the general hospital.

Few traditional performance-based assessments (cognitive tests) for dementia have been validated in the general hospital, and they are not valid in the context of patients with delirium [10]. As delirium, by definition, is associated with an acute cognitive change from baseline, novel strategies to identify dementia in delirium are required. Brief informant-based screening tools with relatives and carers are thoroughly validated for community use [11] but have only been validated in a single study in hospital inpatients [12] and not in those with delirium.

The report aimed to assess the diagnostic test accuracy of two informant tools to detect prior dementia in older patients with current delirium.

Methods

Recruitment

Patients aged 70 and over with an unplanned medical admission to a United Kingdom university hospital between March 2013 and November 2014 were screened for delirium. The screening used the Confusion Assessment Method (CAM) [13], Abbreviated Mental Test Score (AMTS) [14], the Digit Span test and a detailed review of the medical notes. An expert geriatrician (T.A.J.) completed all screening. If participants met the Diagnostic and Statistical Manual of Mental Disorders fourth edition (DSM-IV-TR) criteria for delirium [15], they were eligible for the study. Potential participants who were unable to communicate because of severe sensory impairment or inability to speak in English were excluded, as were those deemed to be at risk of imminent death.

Patients with delirium were then invited to participate. Informed consent was sought from the potential participant if they had capacity. For those who lacked capacity to give informed consent, the next of kin was consulted in accordance with the provisions of the Mental Capacity Act with respect to participation in research, and they were also approached to act themselves as informants in the study. Informed consent was then gained from the informants.

An informant interview enquired about previous diagnosis of dementia or cognitive impairment. The informant was then asked to complete two rating scales in order. First the short form of the Informant Questionnaire of Cognitive Decline in the Elderly (IQCODE-SF) [16] and second, the ‘AD8: the Washington University Dementia Screening Test’, also referred to as the Eight-item Interview to Differentiate Aging and Dementia (AD8) [17]. These were chosen as they are freely available and have face validity. After explanation of the tests, they were then completed in private and placed in an envelope to ensure appropriate blinding.

Informant tools

The IQCODE-SF asks the informant to rate changes in cognition, memory and behaviours over a 10-year period. It has been validated in primary care settings [11], community settings [18] and secondary care settings [19] to diagnose dementia. It takes ∼10 min to complete. The scale consists of 16 items rated 1 to 5 where higher scores indicate a greater degree of cognitive impairment, from which a mean is derived (also ranging from 1 to 5). In instances where ratings were missing for an item, a mean score was calculated from the items that were completed. The introductory wording was changed to ‘Now we want you to remember what your friend or relative was like 10 years ago and to compare it with what he/she was like before they got the illness that brought them to hospital’.

The AD8 is a shorter screening informant interview, asking the informant to rate memory and thinking changes over a few years, rating eight items yes, no or don't know. It takes 5 min to complete and has been found to be sensitive and specific to dementia in a memory clinic population [65]. The test is scored from 0 to a maximum of 8, where higher scores indicate a greater degree of cognitive impairment. Missing values were assumed to be ‘don't know’, so scored as 0.

Reference standard

At 3 months, an assessment in survivors was undertaken to determine a reference diagnosis of dementia at the time of index admission. The assessment was carried out by T.A.J. at the patient's own home or hospital if they were still an inpatient. The assessment was carried out blinded to the results of the IQCODE-SF and AD8. It is not possible to diagnose DSM-IV dementia in the presence of delirium, so follow-up at 3 months was chosen as the best balance to allow recovery from delirium and ensuring an accurate diagnosis of dementia at the index admission. The presence of persistent delirium was first assessed for using DSM-IV-TR criteria. If no delirium was present, a standardised history and examination, including the Addenbrooke's Cognitive Examination III (ACEIII) [20], was performed to establish the presence or absence of dementia at the time of index admission (before the onset of the delirium). Dementia and subtype was diagnosed using the DSM-IV criteria [21]: (i) the development of multiple cognitive deficits, including memory impairment, and (ii) the impairment is sufficiently severe to cause impairment in occupational or social function. To make the diagnosis of dementia at the index admission, the symptoms of cognitive decline had to have been present for at least 6 months prior to the admission with delirium.

Statistical analysis

Receiver operating characteristic curves were generated for both tests and a calculation of the area under the curve (AUROC) was made. Sensitivity, specificity, the positive likelihood ratio (LR+) and the negative likelihood ratio (LR−) were calculated from the best cut-offs. In evaluating the relative misclassification costs in this population, we selected cut-offs that minimised false-positive diagnoses, reducing the burden of potentially stressful and unnecessary further investigation. Data were analysed using IBM SPSS version 20 for Windows.

See Supplementary data, Appendix 1, available in Age and Ageing online for details of the power calculations.

All assessments were carried out by a single trained assessor (T.A.J.). The protocol was assessed and approved by the Bradford Ethics Committee, part of the Yorkshire and Humber National research and Ethics Service (Ref: 12/YH/0534). The study was reported using the Standards for Reporting of Diagnostic Accuracy (STARD) statement with specific reference to studies of diagnostic test accuracy in dementia [22, 23] (check list and flow chart available in Supplementary data, Appendix 2, available in Age and Ageing online).

Results

Delirium was diagnosed in 228 (17.2%) of 1,327 older people screened for delirium. One hundred and twenty-five of this group of 228 (54.8%) were recruited. The main reason for non-recruitment was the lack of an available consultee or informant (57/228, 25%). 22/228 (10%) were deemed at risk of imminent death, 15/228 (7%) were not able to communicate in English, the consultee declined participation in 2/228 (1%) and 7/228 (3%) had been previously recruited.

Of the 125 recruited, 77 (62%) had a full assessment for the reference standard diagnosis. 25/125 (20%) had died, 10/125 (8%) declined the follow-up visit, 8/125 (6%) were not contactable for an assessment and 5/125 (4%) had persistent delirium at assessment. There was no difference in age, gender or admission dementia status between those assessed for dementia at 3 months and those not. The mean age of the final sample assessed for reference criterion diagnosis of dementia was 84.4 and 69% were female. Participant flow through the study is illustrated in Figure 1.

Figure 1.

Flowchart of recruitment through study with demographics of recruited participants. ♀, female.

At 3-month assessment, 47/77 (61%) were diagnosed with dementia, 14/77 (18%) were diagnosed with MCI and 16/77 (21%) had no cognitive impairment. Of those with dementia, this was newly diagnosed in 17/47 (36%).

In diagnosing DSM-IV dementia, the AUROC curve for the admission IQCODE-SF was 0.93 (95% CI 0.86–1.00, P < 0.0005) and for the admission AD8 it was 0.91 (95% CI 0.83–0.98, P < 0.0005). There was no statistical difference between the AUROC curves (z = 0.45, P = 0.65) [24]. The selected cut-off of >3.82 for the IQCODE-SF gave a sensitivity of 0.91, a specificity of 0.93, an LR+ of 13.72 and an LR− of 0.09. The selected cut-off of >6 for the AD8 gave a sensitivity of 0.83, a specificity of 0.90, an LR+ of 8.30 and an LR− of 0.19.

The traditional cut-off of the IQCODE-SF for diagnosing dementia in the general hospital [25] is ≥3.44. Using this cut-off gave a greater sensitivity of 0.98 at the expense of a lower specificity of 0.67.

Table 1 illustrates the diagnostic test accuracy values for the IQCODE-SF and AD8 including confidence intervals at various cut-offs. See Supplementary data, Figure S1, available in Age and Ageing online for the ROC curves for the IQCODE-SF and AD8.

Table 1.

Diagnostic test accuracy of the IQCODE-SF and AD8 at different cut-offs to diagnose dementia

	Test/cut-off	Sens	Spec	LR+	LR−	AUROC
Against diagnosis of dementia	IQCODE ≥3.44	0.98 (0.87–0.99)	0.67 (0.47–0.82)	2.94 (1.77–4.88)	0.03 (0.01–0.23)	0.93 (0.86–1.00, P < 0.0005)
	IQCODE >3.82	0.91 (0.79–0.97)	0.93 (0.76–0.99)	13.72 (3.58–52.50)	0.09 (0.04–0.23)
Prevalence 61%	AD8 >2	0.97 (0.87–0.99)	0.40 (0.23–0.60)	1.63 (1.21–2.19)	0.05 (0.01–0.41)	0.91 (0.83–0.98, P < 0.0005)
	AD8 >6	0.83 (0.69–0.92)	0.90 (0.72–0.97)	8.30 (2.81–24.47)	0.19 (0.09–0.36)

95% confidence intervals in brackets after all values.

Sens, sensitivity; Spec, specificity; LR+, positive likelihood ratio; LR−, negative likelihood ratio; AUROC, area under the receiver operating characteristic curve.

Discussion

The main new finding in this study is that we found excellent diagnostic test accuracy of the IQCODE-SF and AD8 in diagnosing dementia in older people presenting with delirium.

Specifically, we found that the IQCODE-SF and AD8 had excellent sensitivity, specificity and discriminatory ability. An AUROC of >0.9 is considered excellent. Based on the findings from this study, taking 100 older people with delirium, IQCODE-SF at a cut-off of >3.82 would correctly identify 56 of the 61 with dementia and would falsely imply a diagnosis of dementia in 3 of the 39 without dementia. Interpreting the likelihood ratios of the IQCODE-SF at a cut-off of >3.82 to diagnose dementia, a positive test is 13 times more likely to occur in someone with dementia rather than someone without dementia. A negative test is 0.07 times less likely to occur in someone with dementia rather than someone without. This indicates good discrimination after a test has been performed. A positive likelihood ratio of >10 and a negative likelihood ratio of <0.1 indicates a test is useful [26].

Neither the IQCODE-SF nor the AD8 has previously been validated to diagnose dementia in patients with delirium. One previous study has examined the diagnostic test accuracy of the IQCODE-SF to diagnose dementia in older hospital inpatients, excluding those with delirium. This reported a sensitivity of 1.00 and specificity of 0.86 for detecting DSM-IIIR dementia at a cut-off of >3.44 [12]. They report at ≥3.76 cut-off a sensitivity of 0.92 and specificity of 0.93 for the IQCODE-SF, which are very similar to our reported values. It is also worth noting that the prevalence of dementia was much lower (10%) in this study. Although the IQCODE-SF has not been formally assessed in people with delirium, it is commonly used as a proxy for previous cognitive impairment in delirium research using different cut-offs. A cut-off of >3.5 has been used in a general hospital population [27], and our findings suggest these cut-offs may be too low.

Although both tests have similar diagnostic accuracy both have advantages and disadvantages. The AD8 is simpler and quicker to complete; however, it has a higher number of false positives. The IQCODE-SF still only takes a maximum of 10 min to complete and the extra detail gathered may be of clinical relevance. The IQCODE-SF has been translated into 14 languages. Both are freely available without cost [28, 29]. On balance, we would recommend using the IQCODE-SF at a cut-off of >3.82 to detect dementia.

The study has important strengths and limitations. The reported prevalence of delirium of 17.9% is similar to previous studies in acute admissions [8] indicating the screening was robust. We recognise the difficulty in diagnosing dementia among patients in hospital with delirium, so strict reference criteria were applied by an experienced and trained specialist assessor. The assessor was blinded to the index test result. We accept that the diagnosis of dementia before the hospital admission with delirium could be affected by recall bias and the presence of post-delirium cognitive impairment. However, the assessor aimed to take this into account when conducting his assessment. The diagnosis of prior cognitive impairment did not rely exclusively on cognitive testing at 3 months, but also on the clinical history. Therefore, we believe the findings to be robust and are representative of a real world general hospital. The study recruited to the appropriately powered sample size. The study did not recruit all patients diagnosed with delirium, the main reason being the lack of an available consultee and/or informant. However, there were no gender or age differences between those recruited and those not. Not all participants were available for reference diagnosis assessment at 3 months. The main reason was mortality, with 25 (20%) having died prior follow-up. This is in keeping with the expected mortality of delirium [27] and importantly higher mortality would be expected in those with dementia.

The other factor that could reduce the generalisability of our findings is the non-availability of an informant. In our study, one-quarter (57/228) of patients with delirium had no informant available, so this may affect the utility of informant tools in practice. However, the recruitment of an informant in our study was time limited, so we feel in real world practice this proportion would be lower. There was no difference in age and gender between the recruited and non-recruited groups. The informant information may also be affected by recall bias as characteristics of the informant may influence the result such as depression or cognitive impairment. However, the strength of an informant tool is that it to some extent mitigates problems with pre-existing educational background, culture or language difficulties.

The significance of our findings to the research community is that the IQCODE-SF or AD8 at validated cut-offs can now be used to robustly identify prior dementia in studies of delirium cohorts in general hospital.

Our findings could impact significantly on clinical practice, given the high prevalence of delirium and unrecognised dementia in the general hospital. Figure 2 is a flow chart extrapolating our diagnostic and prevalence data to a hypothetical acute hospital with 1,000 beds. Although we acknowledge there are differing views, a timely diagnosis of dementia during hospital admission may ameliorate adverse events associated with a hospital stay [5], allow signposting to a suitable cohort ward [30] or trigger the need for comprehensive geriatric assessment [31]. It could also lead to extra care in communication with patients, especially around issues of capacity assessment and consenting for interventions.

Figure 2.

Flow chart illustrating the use of the IQCODE-SF for screening patients with delirium for dementia in a hypothetical 1,000-bedded hospital and 10,000 non-elective admissions of older people yearly. We expect 1,700 (17%) would have delirium, with 60% of those also having dementia. Thirty-three per cent of that dementia will be unrecognised. We would expect 20% of patients to have no informant available.

Conclusions

Both the IQCODE-SF and AD8 are sensitive and specific tools to detect dementia in older people presenting to hospital with delirium. They are simple and quick to administer as well as being freely available. Given the high prevalence of delirium in older people in hospital, the routine use of either tool in practice will have important clinical impact, potentially improving the formal recognition of dementia as well as having immediate implications for the inpatient care of this vulnerable population.

Key points

• Delirium and dementia are common but undetected in older hospital admissions.

• Delirium precludes traditional performance-based cognitive assessment for dementia.

• Informant tools may offer a pragmatic solution to detect dementia in older hospital patients with delirium.

• IQCODE-SF and AD8 are sensitive and specific tools to detect dementia in patients with delirium.

Conflicts of interest

None declared.

Funding

T.A.J. was supported jointly by the Research into Ageing Fund, a fund set up and managed by Age UK and the British Geriatrics Society (#367). The study sponsor had no role in either study design; the collection, analysis and interpretation of data; the writing of the report and in the decision to submit the paper for publication.

Acknowledgements

We gratefully acknowledge the participants, carers and consultees who took part in the study. We gratefully acknowledge Dr Jenni Harrison of the University of Leicester for her help with statistical analysis.