Abstract

Feedback methods have been studied in medical and psychotherapy settings, but limited research is available in neuropsychology. The purpose of this study was to examine whether supplementing oral feedback with written information would lead to greater retention of information and improved adherence to recommendations. Seventy-two veterans were enrolled in the study and randomized to receive oral feedback only or oral feedback with written information. The participants were then interviewed immediately after feedback and 1 month later by phone. Univariate analyses revealed that the written group freely recalled more recommendations at the phone interview; however, there were no differences in recall of diagnostic information or the number of recommendations attempted. Findings indicate that receiving supplemental written information improves recall of recommendations and that patients prefer to receive written information in addition to oral feedback. Recommendations to improve the retention of feedback information are discussed.

Introduction

Providing patients with feedback of evaluation results and medical information is perhaps one of the most important functions of healthcare providers. In psychology, it is an ethical obligation (American Psychological Association, 2010) that serves as an opportunity to explain the results of evaluation, discuss treatment and rehabilitation, assist in understanding the effects of the condition on every day functioning and decision-making, and provide individual and family support (American Psychological Association, 2012; Gass & Brown, 1992; Poston & Hanson, 2010). Despite the potential impact of this information on patients and their families, many psychologists are not formally trained in feedback or do not consistently provide feedback (Butcher, 1992; Curry & Hanson, 2010; Merker, Hanson, & Poston, 2010; Poston & Hanson, 2010). Furthermore, research examining the benefits of feedback and the utility of different feedback methods is limited (Gass & Brown, 1992; Pope, 1992).

Several studies conducted in medical settings have examined patient satisfaction with different feedback methods and generally found a preference for a combination of oral and written feedback that is personalized and uses plain language (Borgsteede, Karapinar-Carkit, Hoffmann, Zoer, & van den Bemt, 2011; Keatinge, Stevenson, & Fitzgerald, 2009; Rose, Worrall, Hickson, & Hoffmann, 2010). Follow-up studies have shown combined written and oral feedback leads to improved patient knowledge, better adherence to treatment, and enhanced outcomes in both inpatient hospital settings (Ellis, Hopkin, Leitch, & Crofton, 1979; Mittenberg, Tremont, Zielinski, Fichera, & Rayls, 1996; Ponsford et al., 2001; Suner et al., 2010) and outpatient medical settings (Bucker et al., 2010; Culkin, Gabae, & Madden, 2009; Kaariainen, Kukkurainen, Kyngas, & Karppinen, 2011; Wright, Fleming, Sharma, & Battagel, 2010) with few exceptions (Madar, Klepp, & Meyer, 2011).

The psychotherapeutic benefits of receiving oral feedback following psychological and neuropsychological evaluations have also been assessed (Finn & Tonsager, 1992; Halperin & Snyder, 1979; Lopez, Roberts, Tchanturia, & Treasure, 2008; Newman & Greenway, 1997; Poston & Hanson, 2010). Two studies examined psychological feedback alone as a therapeutic intervention and found patients who received feedback endorsed increased self-esteem and decreased symptomatic distress at 2 weeks compared with participants in a wait-list condition (Finn & Tonsager, 1992; Newman & Greenway, 1997). Providing feedback about psychological testing conducted prior to psychotherapy has resulted in improvements in knowledge of psychological condition(s), adherence to treatment, and outcome (Halperin & Snyder, 1979; Poston & Hanson, 2010). Lopez and colleagues (2008) found that information processing biases of two patients with anorexia nervosa were improved by using neuropsychological data to illustrate relevant concepts during feedback.

In contrast to the voluminous research on feedback in medical settings and psychotherapy, research in neuropsychology is lacking. Existing studies have focused primarily on satisfaction with neuropsychological services, including outcomes and recommendations, in adult patients (Bennett-Levy, Klein-Boonschate, Batchelor, McCarter, & Walton, 1994; Donofrio, Piatt, Whelihan, & DiCarlo, 1999; Westervelt, Brown, Tremont, Javorsky, & Stern, 2007), parents (Bodin et al., 2007; Farmer & Brazeal, 1998), and referral sources (Allott, Brewer, McGorry, & Proffitt, 2011; Tremont, Westervelt, Javorsky, Podolanczuk, & Stern, 2002). In general, referral sources tend to have a high degree of satisfaction with recommendations and outcomes (Tremont et al., 2002). Studies examining parent satisfaction of pediatric neuropsychological evaluations have generally indicated that parents feel that they have a better understanding of their child's strengths and difficulties and more resources to address problems and affect outcomes as a result of the assessment (Bodin et al., 2007; Farmer & Brazeal, 1998). Adult patients also indicate a high degree of satisfaction with services (Bennett-Levy et al., 1994; Donofrio et al., 1999; Westervelt et al., 2007); however, follow-through with recommendations was relatively low in one study unless they pertained to safety or specific coping strategies for cognitive difficulties (Westervelt et al., 2007). Reasons recommendations were not followed was not systematically examined, and future studies focusing more directly on barriers to compliance were suggested (Westervelt et al., 2007).

One potential barrier to follow-through with recommendations is poor recall of the recommendations and the feedback session, in general (Bennett-Levy et al., 1994). Forgetting what healthcare professionals say after consultation is not uncommon (Crane, 1997) and may be even more problematic for patients referred for possible cognitive impairment (Diez-Ascaso, Martinez-Sanchez, Fuentes, & Diez-Tejedor, 2011). Bennett-Levy and colleagues (1994) noted that while only 26% of patients in their study received written feedback, 82% said they would have liked written information. To our knowledge, no study has examined the impact of the feedback method on patient recall of results or follow-through with recommended care. Thus, the aim of the present study was to examine the impact of supplemental written information on adult patients' memory for and adherence to recommendations. Specifically, it was hypothesized that providing written and oral information at the time of neuropsychological feedback would improve memory for diagnosis and recommendations, resulting in increased adherence to recommendations compared with patients receiving oral feedback only.

Method

Participants

The sample consisted of 72 veterans (96% men) with an average age of 60.6 years (SD = 14.2, range 22–88) and average education of 12.4 years (SD = 3.1, range 3–21). Average estimated premorbid IQ using the Test of Premorbid Functioning (Pearson, 2009) was 97.6 (SD = 10.0, range 69–121), and all patients indicated that they were comfortable being evaluated in English. The majority was non-Hispanic White (56%) or Hispanic (38%), while African Americans (5%) and Asian Americans (1%) made up the remainder of the sample. The sample represented a wide range of clinical diagnoses that were placed into three diagnostic categories based on DSM-IV-TR diagnostic criteria: Dementia (22%), Cognitive Disorder, Not Otherwise Specified (47%), and no cognitive disorder (31%). Etiologies were non-neurologic (usually multiple) medical conditions (26%), neurodegenerative diseases (15%), stroke (12%), psychiatric disorder (12%), neurologic medical conditions (6%), attention-deficit/hyperactivity disorder (6%), traumatic brain injury (3), and other miscellaneous conditions (20%). Twenty-six percent of patients had two or more current psychiatric diagnoses, 31% had one active psychiatric diagnosis, and 43% had no psychiatric diagnosis. Depression was the most common psychiatric diagnosis (i.e., 36%) with anxiety disorder other than Post-Traumatic Stress Disorder (PTSD) second (31%), followed by PTSD (21%). Patients with memory disorders were not excluded in order to conduct a study that would be representative of most general neuropsychology consultation clinics; however, it is acknowledged that patients with dementia usually would rely on a family member or other caregiver to remember information provided in feedback.

Procedures

Patients evaluated in a Veterans Administration (VA) neuropsychology clinic who were literate and provided credible performance on neuropsychological testing were given the option to participate in the study. After providing written informed consent as approved by the University Institutional Review Board and VA Research and Development Committee, participants were randomized into one of two groups: oral feedback only (Oral) and oral feedback with supplemental written information (Written). The supplemental written information consisted of a two-page letter containing the reason for referral, summary and impressions, and patient-specific recommendations copied verbatim from the neuropsychological report. On the same day, immediately following the feedback session, a research assistant who was not blinded to the aims of the study but who was not involved in the evaluation or feedback process conducted in-person interviews at the neuropsychology clinic. The research assistant assessed immediate recall of diagnostic information and recommendations using a structured interview form that was developed solely for the purpose of this study. The interview questions were: (a) “Did the testing say you had thinking or memory problems?” (b) “If yes, what is causing your thinking or memory problems?” (c) “Are your thinking or memory problems expected to get better, stay the same, or get worse over time?” (d) “Please rate your agreement with the results of the evaluation on a scale of 1 (definitely agree) to 5 (definitely disagree)” and (e) “What were the recommendations you were given?” Participants received a phone call approximately 1 month later to assess for retention of diagnostic information and recommendations as well as adherence to treatment recommendations, consistent with previous studies in feedback retention (Westervelt et al., 2007). The questions at the phone interview were the same as the initial interview, with the addition of cues for recommendations they were unable to freely recall and being asked at the beginning of the call if they remembered the testing session, if they remembered receiving feedback, and if they kept the written information. Participants were cued for recognition of recommendations by asking, “were you given the recommendation to … ” with the recommendation from the report read verbatim. Participants were then asked if they completed each recommendation. Note that participants were not asked to recall recommendations that were directed at healthcare providers. Participants assigned to the Oral group were provided with supplemental written information upon completion of the study.

Data Analysis

Descriptive statistics were used to describe the study sample, and independent samples t-tests and χ2 analyses (for nonparametric variables) were conducted on demographic data to determine between-group differences. Group differences in retention of feedback information and follow-through with recommendations were tested using univariate and multivariate analysis of variance, accounting for covariates when needed. χ2 analyses and Spearman's ρ were used for nonparametric variables, including differences in recall of information provided during feedback and completion rates of recommendations. IBM Statistical Package for the Social Sciences (SPSS) version 19.0 was used for all analyses.

Results

Of the 72 participants initially enrolled in the study, six did not receive feedback due to appointment no-shows and nine could not be reached to complete the 1-month phone interview. Participants who did not attend the feedback session did not differ on any demographic variables from those who completed at least the initial interview. Furthermore, there were no differences in demographic variables or diagnoses between completers and noncompleters of the full study.

Immediate Interview

Of the 66 participants who completed the feedback session and initial interview, 36 were in the Oral group and 30 were in the Written group. The Oral group had significantly more years of education, t(64) = 2.37, p = .02, whereas participants in the Written group were more likely to have a current depression diagnosis, χ2(1) = 4.42, p = .04 (Table 1). The Oral and Written groups did not differ in percentage with a collateral at feedback (56% and 37%, respectively), length of feedback (53.5 min in both conditions), or rates of agreement with results (91% and 94%, respectively).

Table 1.

Demographic characteristics by group at the immediate and phone interviews

Demographic Immediate
 
Phone
 
Oral Written Oral Written 
n 36 30 30 27 
Age (years; mean ± SD62.5 (14.2) 59.9 (13.3) 60.5 (14.1) 60.0 (13.9) 
Edu (years; mean ± SD13.0 (2.6) 11.3 (3.2)* 13.0 (2.3) 11.3 (3.3)* 
TOPF (mean ± SD98.8 (9.2) 96.1 (11.2) 98.6 (9.2) 96.6 (11.5) 
Gender 
 Men 34 (94%) 29 (97%) 28 (93%) 26 (96%) 
 Women 2 (6%) 1 (3%) 2 (7%) 1 (4%) 
Ethnicity 
 Non-Hispanic White 22 (61%) 15 (50%) 19 (64%) 15 (56%) 
 Hispanic 12 (33%) 13 (43%) 9 (30%) 10 (37%) 
 African American 1 (3%) 2 (7%) 1 (3%) 2 (7%) 
 Asian 1 (3%) — 1 (3%) — 
Diagnostic Category 
 Dementia 11 (31%) 5 (17%) 8 (27%) 3 (11%) 
 Cognitive Disorder 16 (44%) 16 (53%) 14 (46%) 16 (59%) 
 No Cognitive Disorder 9 (25%) 9 (30%) 8 (27%) 8 (30%) 
Psychiatric Diagnosisa 
 Anxiety Disorder 10 (28%) 10 (33%) 10 (33%) 8 (30%) 
 PTSD 6 (17%) 6 (20%) 5 (17%) 6 (22%) 
 Depression 9 (25%) 15 (50%)* 8 (27%) 12 (44%) 
Demographic Immediate
 
Phone
 
Oral Written Oral Written 
n 36 30 30 27 
Age (years; mean ± SD62.5 (14.2) 59.9 (13.3) 60.5 (14.1) 60.0 (13.9) 
Edu (years; mean ± SD13.0 (2.6) 11.3 (3.2)* 13.0 (2.3) 11.3 (3.3)* 
TOPF (mean ± SD98.8 (9.2) 96.1 (11.2) 98.6 (9.2) 96.6 (11.5) 
Gender 
 Men 34 (94%) 29 (97%) 28 (93%) 26 (96%) 
 Women 2 (6%) 1 (3%) 2 (7%) 1 (4%) 
Ethnicity 
 Non-Hispanic White 22 (61%) 15 (50%) 19 (64%) 15 (56%) 
 Hispanic 12 (33%) 13 (43%) 9 (30%) 10 (37%) 
 African American 1 (3%) 2 (7%) 1 (3%) 2 (7%) 
 Asian 1 (3%) — 1 (3%) — 
Diagnostic Category 
 Dementia 11 (31%) 5 (17%) 8 (27%) 3 (11%) 
 Cognitive Disorder 16 (44%) 16 (53%) 14 (46%) 16 (59%) 
 No Cognitive Disorder 9 (25%) 9 (30%) 8 (27%) 8 (30%) 
Psychiatric Diagnosisa 
 Anxiety Disorder 10 (28%) 10 (33%) 10 (33%) 8 (30%) 
 PTSD 6 (17%) 6 (20%) 5 (17%) 6 (22%) 
 Depression 9 (25%) 15 (50%)* 8 (27%) 12 (44%) 

Notes: SD = standard deviation; Edu = education; TOPF = test of premorbid functioning; PTSD = post-traumatic stress disorder.

aNot every patient in the study had a psychiatric diagnosis.

*p < .05.

The results of χ2 analyses showed that there were no differences between groups in regard to ability to recall cognitive problems (Question 1), χ2(1) = 0.085, p = .77 (81% vs. 83% in Oral and Written groups, respectively), etiology (Question 2), χ2(1) = 2.449, p = .12 (22% vs. 40% in Oral and Written groups, respectively), or prognosis (Question 3), χ2(1) = 1.779, p = .18 (55% vs. 35% in Oral and Written groups, respectively) (Table 2). Overall, although the majority of participants were able to recall whether or not they had cognitive problems (i.e., 82%), most participants were unable to consistently recall either the etiology (i.e., 30%) or prognosis (i.e., 44%). Education was not significantly correlated to recall of cognitive problems (ρ = 0.16), etiology (ρ = 0.15), or prognosis (ρ = 0.13). Independent samples t-tests showed that the number of recommendations provided to patients did not differ between groups—t(64) = 0.20, p = .84 (Table 3). After controlling for education, there was no significant group difference in the number of recommendations freely recalled—F(1,63) = 0.33, p = .57 (Table 3).

Table 2.

Recall of diagnostic information by group at the immediate and phone interviews

 Oral (n [%]) Written (n [%]) p 
Immediate 
 Cognitive Problems 29 (81) 25 (83) .77 
 Etiology 8 (22) 12 (40) .12 
 Prognosisa 12 (55) 8 (35) .18 
Phone 
 Cognitive Problems 21 (81) 22 (85) .71 
 Etiology 4 (13) 4 (15) .87 
 Prognosisa 7 (26) 11 (48) .11 
 Oral (n [%]) Written (n [%]) p 
Immediate 
 Cognitive Problems 29 (81) 25 (83) .77 
 Etiology 8 (22) 12 (40) .12 
 Prognosisa 12 (55) 8 (35) .18 
Phone 
 Cognitive Problems 21 (81) 22 (85) .71 
 Etiology 4 (13) 4 (15) .87 
 Prognosisa 7 (26) 11 (48) .11 

aNot every participant in the study discussed prognosis with their provider, such as those who did receive a diagnosis of no cognitive disorder, resulting in a smaller sample size for this measure.

Table 3.

Recall and completion of recommendations by group at the immediate and phone interviews

Recommendations Oral (M [SD]) Written (M [SD]) t F 
Immediate 
 Total Given by Provider 4.1 (1.6) 4.0 (1.7) 0.20  
 Number Freely Recalled 1.7 (1.0) 1.7 (1.0)  0.33 
Phone 
 Total Given by Provider 4.2 (1.6) 4.0 (1.8) 0.37  
 Number Freely Recalled 0.7 (0.8) 1.3 (1.2)  5.49* 
 Number Cued Recalled 2.1 (1.3) 2.1 (1.2)  0.05 
 Total attempted/completed 2.2 (1.3) 2.3 (1.6)  0.01 
Recommendations Oral (M [SD]) Written (M [SD]) t F 
Immediate 
 Total Given by Provider 4.1 (1.6) 4.0 (1.7) 0.20  
 Number Freely Recalled 1.7 (1.0) 1.7 (1.0)  0.33 
Phone 
 Total Given by Provider 4.2 (1.6) 4.0 (1.8) 0.37  
 Number Freely Recalled 0.7 (0.8) 1.3 (1.2)  5.49* 
 Number Cued Recalled 2.1 (1.3) 2.1 (1.2)  0.05 
 Total attempted/completed 2.2 (1.3) 2.3 (1.6)  0.01 

Note: M (SD) = mean (standard deviation).

*p < .05.

Phone Interview

Of the 57 participants who completed the phone interview 1 month following the feedback session, 30 were in the Oral group and 27 were in the Written group. The Oral group again had significantly more years of education, t(55) = 2.25, p = .03, but did not differ significantly from the Written group on any other demographic variables (Table 1). The Oral and Written groups did not differ in percentage with a collateral at feedback (50% and 37%, respectively), length of feedback (53.7 and 53.9 min, respectively), or rates of agreement with results (67% and 81%, respectively). Furthermore, there was not a significant difference in the length of time from the immediate interview to the phone interview between groups (35 days for the Oral group and 34 days for the Written group).

The results of χ2 analyses showed that there were no differences between groups in regard to recall of cognitive problems (Question 1), χ2(1) = 0.134, p = .71 (81% vs. 85% in Oral and Written groups, respectively), etiology (Question 2), χ2(1) = 0.026, p = .87 (13% vs. 15% in Oral and Written groups, respectively), or prognosis (Question 3), χ2(1) = 2.585, p = .11 (26% vs. 48% in Oral and Written groups, respectively) (Table 2), and there were no significant correlations with education (ρ = 0.05, 0.02, and −0.15, respectively). Independent samples t-tests again showed that the number of recommendations provided to patients also did not differ between groups—t(55) = 0.37, p = .71 (Table 3). After controlling for education, the Written group freely recalled significantly more recommendations compared with the Oral group, F(2,54) = 5.49, p = .02, but the groups did not differ in the number of cued recommendations recalled, F(2,54) = 0.05, p = .82 (Table 3). In regard to the number of recommendations attempted, there was no significant difference between the Oral and Written groups—F(2,54) = 0.01, p = .91.

Analysis of Recommendations Completed

There were no significant differences between the Written and Oral groups in completion rates of specific recommendations (Table 4). A qualitative review of the reasons for not following through with recommendations revealed that some participants did not agree with the recommendation provided, wanted to try other recommendations first, forgot the recommendation, or did not understand the referral process to obtain the recommendation.

Table 4.

Completion rates of recommendations by group

Recommendation Oral (n [%]) Written (n [%]) χ2 p 
Attempt 
Psychiatry 3 (38) 7 (64) 1.27 .37 
Psychotherapy 3 (27) 9 (60) 2.74 .13 
Patient Educational Material 16 (70) 14 (67) 0.04 .99 
Primary Care/Medication Compliance 14 (74) 12 (80) 0.19 .99 
Change in Environment 3 (60) 2 (50) 0.09 .99 
Change in Driving Status 5 (83) 4 (100) 0.74 .99 
Volunteer/Increase Activity 4 (57) 1 (33) 1.06 .55 
Recommendation Oral (n [%]) Written (n [%]) χ2 p 
Attempt 
Psychiatry 3 (38) 7 (64) 1.27 .37 
Psychotherapy 3 (27) 9 (60) 2.74 .13 
Patient Educational Material 16 (70) 14 (67) 0.04 .99 
Primary Care/Medication Compliance 14 (74) 12 (80) 0.19 .99 
Change in Environment 3 (60) 2 (50) 0.09 .99 
Change in Driving Status 5 (83) 4 (100) 0.74 .99 
Volunteer/Increase Activity 4 (57) 1 (33) 1.06 .55 

Discussion

The utility of written feedback in medical and psychotherapy settings has been well documented; however, research in neuropsychological feedback has been less extensive. Research on aspects of feedback in clinical neuropsychology has been primarily satisfaction based and shown a patient preference for written and oral feedback together. To our knowledge, there are no published studies examining if written information benefits patients following neuropsychological evaluation. Thus, the aim of this study was to examine if providing written and oral information at the time of neuropsychological feedback would improve the recall of diagnostic information and recommendations, potentially resulting in increased adherence compared with participants in the Oral group. Results of the study indicated that the Written group freely recalled more recommendations than the Oral group 1 month after the feedback session. However, the differences between groups were small, overall recommendation recall was low, and participants did not exhibit a better recall of their diagnostic information nor did they exhibit a better recommendation adherence rate, as hypothesized. There are several possible reasons for this finding, including the methodological design of the study which did not fully assess the multitude of patient variables that may impact adherence to treatment recommendations (e.g., attitudes toward recommendations, motivation to complete recommendations, psychosocial support and stressors, etc.) or the effect of collaterals, especially for patients with dementia.

When examining the results of the present study, it is striking that the vast majority of participants were unable to recall their diagnosis or prognosis either immediately after feedback or 1 month later. One reason for this may be that cognitive impairment makes it difficult to remember the amount of information presented in feedback, providing further support for the use of written information and/or the presence of collaterals at feedback. It is also possible that technical language used by neuropsychologists during feedback is not completely understood or easily integrated into patients' lay-language and knowledge. This latter possibility is supported by findings of a recent study showing that while experts and lay individuals tend to use similar language in describing cognitive constructs, agreement dropped when matching cognitive constructs to neuropsychological tests (Schoo, van den Berg, van Zandvoort, & Postma, 2012). Examination of technical language used in describing etiology may also account for differences in understanding between neuropsychologists and laypersons. For instance, it was observed in the Schoo and colleagues (2012) study that many participants were surprised to learn that there were more than one or two types of dementia; most were aware of Alzheimer dementia only. A third possibility is that knowing the diagnosis and prognosis may not have been as immediately important to the patient as knowing whether or not they had cognitive impairment. Importance of information to the participant was not assessed during this study but presents an interesting area of future research.

Similar to patients' poor recall of diagnosis and prognosis, the overall free recall rate of recommendations was low, with several patients failing to freely recall even a single recommendation. However, patients were able to remember more recommendations when cued, and the number of recommendations attempted/completed in each group was not too dissimilar from the total number of recommendations recalled. The overall poor free recall rate suggests several possibilities, the most obvious of which is that cognitive impairment may impact the ability of the patient to take in all of the information presented during feedback. A second explanation is that patients may be more likely to attend and adhere to certain types of recommendations. For example, previous research found higher adherence rates for recommendations involving safety (Westervelt, 2007). The current study also found that recommendations involving patient safety (e.g., change in environment, change in driving, medication compliance) had a higher completion rate compared with mental health treatment recommendations. Furthermore, while agreement with results during the immediate interview was uniformly strong (i.e., 91%–94%), agreement at the phone interview was lower (i.e., 67%–81%) and may have contributed to lower motivation to remember and/or adhere to recommendations. In support of this latter point, when asked about reasons for lack of follow-through, several patients indicated that they wanted to try other recommendations first. As indicated above, there are several factors that may play a role in adherence to recommendations that were not assessed in the current study, and it is possible that poor recall plays a very small part.

Overall, there are interventions that can be implemented to improve the recall of information learned in the feedback session. First and foremost, this study indicates that providing written information in addition to oral feedback leads to improved free recall of recommendations. Not only is this a viable and easy change in current practice, but it is well received by patients, as the majority of patients in the Written group kept their written information and all but one patient in the Oral group requested written information at the completion of the study. Furthermore, considering that all participant groups benefited from cuing, written information serves a second purpose in reminding the patient about the information that they received during the feedback session. A second consideration is adapting the way written information is presented in order to further increase the recall of diagnostic and prognostic information and recommendations. For example, participants in the current study were provided with recommendations addressed to them only versus recommendations for providers; however, this may not be the case in clinical practice. Thus, separating out recommendations into sections based on who they are targeting (e.g., patient, caregiver, referring provider, etc.) may make it easier for the patient to attend to pertinent information. Finally, this study indicated that participants recalled less than two recommendations immediately after the feedback. Therefore, it may be beneficial to provide fewer recommendations, emphasizing only the most important recommendations or providing repetition/rehearsal in session to facilitate encoding. Ultimately, knowing the patients cognitive strengths and weaknesses should help the provider tailor the delivery of the feedback in such a way that it will be meaningful to the patient and allow them to capture the most information (Allen et al., 1986). Furthermore, utilizing lay-friendly language that is concise, free of jargon, and presented in a context that is likely to exert a positive influence on recall of information presented during feedback and adherence to recommendations may be important (Pope, 1992).

Focusing on other ways of delivering feedback information in session may also lead to improved recall of diagnostic and prognostic information and recommendations and thus improved treatment outcomes. For example, Culkin and colleagues (2009) demonstrated that providing a simple pamphlet with educational material and a diary led to improved medical outcomes in patients with intestinal failure, and Ringstrom and colleagues (2010) found similar results in patients with irritable bowel syndrome. In the present study, multiple patients noted that it would be beneficial to receive some of the tools required to start particular feedback recommendations. For example, providing day planners, organizers, and calendars may have a beneficial impact for patients with impaired attention, memory, or executive functioning or possibly improve recall of treatment recommendations and/or adherence rates. Diagnostic and prognostic information could be included in the day planner, organizer, or calendar to facilitate recall.

Future research may also wish to examine if there are benefits to conducting more than one feedback session to reinforce information, such as a feedback “booster” session. Lopez and colleagues (2008) used a three-session assessment and feedback module, with the feedback portion consisting of an immediate feedback session followed up with a second feedback session. In their case presentation of two patients with anorexia nervosa, they found that the two-feedback session method resulted in improved structure and rationale for following through with treatment and that having more than one session allowed them to start some of the recommendations with the patients as behavioral experiments. For patients with cognitive impairment, employing a similar two-feedback session model could help improve recall of diagnostic and prognostic information, as well as recommendations, through repetition of information and “hands-on” implementation of some of the recommendations (e.g., contacting the local support group to gather more information and meeting dates). In situations where two feedback sessions are not feasible, utilizing motivational interviewing techniques in feedback to enhance readiness to change may result in improved recommendation recall and adherence.

Finally, because it was outside the scope of this study, we did not fully examine the impact of collaterals at feedback or patient characteristics that may influence recall of feedback information and follow-through of recommendations. With regard to collaterals, as suggested by Gass and Brown (1992), they may help the patient become involved in recommendations and allow the neuropsychologist further opportunity for normalizing behaviors, increasing support, and making changes to the patient's environment. On the other hand, it is possible that collaterals may relieve some of the pressure on the patient to recall the information or fully engage in the session, contributing to poor recall and/or adherence. At the very least, it is understood that patients are less likely to attempt or complete recommendations that family or primary supports are at odds with (Bennett-Levy et al., 1994). Therefore, having collaterals present to explain the rationale for recommendations may help reduce opposition, and we found a positive relationship between the presence of a collateral and the number of recommendations attempted or completed (data not shown). Future research examining the impact of collaterals at feedback will be vital given the importance of collateral support following onset of a cognitive disorder (American Psychological Association, 2012), and preliminary data suggest that providing written information to collaterals present at feedback enhances memory for recommendations (Meth & Tranel, 2012). With regard to patient characteristics that may be important to study further, we found negative relationships between age and recall of prognosis and the number of recommendations, between dementia diagnosis and recall of etiology, and between current depression and the number of recommendations attempted or completed (data not shown). Examining specific domains of cognitive impairment, such as attention, executive function, and/or memory, may shed additional light on which cognitive characteristics may affect ability to remember and follow-through with information provided during feedback.

In summary, to our knowledge, this is the first published study to go beyond patient satisfaction and explore how written information impacts patients who have undergone neuropsychological evaluation. However, these results must be interpreted within the context of study limitations. Primarily, the sample was relatively small and consisted mostly of male veterans. Because findings may not generalize to other study samples, future research utilizing larger, more diverse samples could further elucidate what patients take away from neuropsychological feedback and what they find most helpful. Second, English language proficiency was not assessed formally and could have impacted understanding and recall of feedback information, although we believe the effect of this limitation to be small since all of our participants served in the military and indicated that they were comfortable being evaluated in English. Third, the research assistant who surveyed the participants was not blind to the aims of the study, which could have affected the results. Fourth, the inclusion of patients with dementia may have minimized differences between feedback methods, as patients with dementia would be expected to have poorer recall of information. The effect of this limitation is also thought to be small because both feedback groups had similar numbers of patients with dementia and post hoc analyses showed that patients with dementia were just as likely to remember (or not) whether they were told they had cognitive impairment and what their diagnoses and prognoses were (data not shown). On the other hand, they freely recalled significantly fewer recommendations than patients diagnosed with Cognitive Disorder, Not Otherwise Specified, although they did not differ significantly from patients without a cognitive disorder diagnosis (these patients were usually given a psychiatric disorder diagnosis). Nevertheless, findings suggested that patients who receive written information in addition to oral feedback have better free recall of recommendations compared with patients who receive oral feedback only. However, greater free recall did not result in better adherence to recommendations and recall of diagnosis and prognosis was poor. Therefore, there is only limited evidence that providing written information results in improved patient care and outcomes. Despite this, the vast majority of patients still requested written feedback, and given that it may benefit free recall of recommendations, professionals are encouraged to utilize written information in addition to oral feedback.

Conflict of Interest

None declared.

Acknowledgements

The authors of this paper would like to acknowledge and thank the individuals who contributed their time and effort to this project, including: Dr. Emily Luther, Dr. Janice Marceaux, Dr. Karin McCoy, Frank Reyes, Dr. Justin O'Rourke, Dr. Russ Pella, and Dr. Nabeel Yehyawi.

References

Allen
J. G.
Lewis
L.
Blum
S.
Voorhees
S.
Jernigan
S.
Peebles
M. J.
Informing psychiatric patients and their families about neuropsychological assessment findings
Bulletin of the Menninger Clinic
 , 
1986
, vol. 
50
 
1
(pg. 
64
-
74
)
Allott
K.
Brewer
W.
McGorry
P. D.
Proffitt
T.
Referrers' perceived utility and outcomes of clinical neuropsychological assessment in an adolescent and young adult public mental health service
Australian Psychologist
 , 
2011
, vol. 
46
 
1
(pg. 
15
-
24
)
American Psychological Association
Ethical principles of psychologists and code of conduct
2010
 
American Psychological Association
Guidelines for the evaluation of dementia and age-related cognitive change
American Psychologist
 , 
2012
, vol. 
67
 
1
(pg. 
1
-
9
)
Bennett-Levy
J.
Klein-Boonschate
M. A.
Batchelor
J.
McCarter
R.
Walton
N.
Encounter with Anna Thompson: The consumer's experience of neuropsychological assessment
The Clinical Neuropsychologist
 , 
1994
, vol. 
8
 
2
(pg. 
219
-
238
)
Bodin
D.
Beetar
J. T.
Yeates
K. O.
Boyer
K.
Colvin
A. N.
Mangeot
S.
A survey of parent satisfaction with pediatric neuropsychological evaluations
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
884
-
898
)
Borgsteede
S. D.
Karapinar-Carkit
F.
Hoffmann
E.
Zoer
J.
van den Bemt
P. M. L. A.
Information needs about medication according to patients discharged from a general hospital
Patient Education and Counseling
 , 
2011
, vol. 
83
 (pg. 
22
-
28
)
Bucker
B.
Butzlaff
M.
Isfort
J.
Koneczny
N.
Vollmar
H. C.
Lange
S.
, et al.  . 
Effect of written information on knowledge and function of patients with acute uncomplicated back pain (PIK Study) [Abstract]
Gesundheitswesen
 , 
2010
, vol. 
72
 
12
(pg. 
78
-
88
)
Butcher
J. N.
Introduction to the special section: Providing psychological test feedback to clients
Psychological Assessment
 , 
1992
, vol. 
4
 
3
pg. 
267
 
Culkin
A.
Gabe
S. M.
Madden
A. M.
Improving clinical outcome in patients with intestinal failure using individualised nutritional advice
Journal of Human Nutrition and Dietetics
 , 
2009
, vol. 
22
 (pg. 
290
-
298
)
Curry
K. T.
Hanson
W. E.
National survey of psychologists' test feedback training, supervision, and practice: A mixed methods study
Journal of Personality Assessment
 , 
2010
, vol. 
92
 
4
(pg. 
327
-
336
)
Crane
J. A.
Patient comprehension of doctor-patient communication on discharge from the emergency department
Journal of Emergency Medicine
 , 
1997
, vol. 
15
 
1
(pg. 
1
-
7
)
Diez-Ascaso
O.
Martinez-Sanchez
P.
Fuentes
B.
Diez-Tejedor
E.
Sociocultural study on the self-perception of stroke and analysis of doctor-patient communication
Neurologia
 , 
2011
, vol. 
26
 
2
(pg. 
81
-
91
)
Donofrio
N.
Piatt
A.
Whelihan
W.
DiCarlo
M.
Neuropsychological test feedback: Consumer evaluation and perception [Abstract]
Archives of Clinical Neuropsychology
 , 
1999
, vol. 
14
 
8
pg. 
721
 
Ellis
D. A.
Hopkin
J. M.
Leitch
A. G.
Crofton
J.
Doctor's orders: Controlled trial of supplementary, written information for patients
British Medical Journal
 , 
1979
, vol. 
1
 
6161
pg. 
456
 
Farmer
J. E.
Brazeal
T. J.
Parent perceptions about the process and outcomes of child neuropsychological assessment
Applied Neuropsychology
 , 
1998
, vol. 
5
 
4
(pg. 
194
-
201
)
Finn
S. E.
Tonsager
M. E.
Therapeutic effects of providing MMPI-2 test feedback to college students awaiting therapy
Psychological Assessment
 , 
1992
, vol. 
4
 
3
(pg. 
278
-
287
)
Gass
C. S.
Brown
M. C.
Neuropsychological test feedback to patients with brain dysfunction
Psychological Assessment
 , 
1992
, vol. 
4
 
3
(pg. 
272
-
277
)
Halperin
K. M.
Snyder
C. R.
Effects of enhanced psychological test feedback on treatment outcome: Therapeutic implications of the Barnum effect
Journal of Consulting and Clinical Psychology
 , 
1979
, vol. 
47
 
1
(pg. 
140
-
146
)
Kaariainen
M.
Kukkurainen
M. L.
Kyngas
H.
Karppinen
L.
Improving the quality of rheumatoid arthritis patients' education using written information [Abstract]
Musculoskeletal Care
 , 
2011
, vol. 
9
 
1
(pg. 
19
-
24
)
Keatinge
D.
Stevenson
K.
Fitzgerald
M.
Parent's perceptions and needs of children's hospital discharge information
International Journal of Nursing Practice
 , 
2009
, vol. 
15
 (pg. 
341
-
347
)
Lopez
C.
Roberts
M. E.
Tchanturia
K.
Treasure
J.
Using neuropsychological feedback therapeutically in treatment for anorexia nervosa: Two illustrative case reports
European Eating Disorders Review
 , 
2008
, vol. 
16
 (pg. 
411
-
420
)
Madar
A. A.
Klepp
K. L.
Meyer
H. E.
The effect of tailor-made information on vitamin D status of immigrant mothers in Norway: A cluster randomized controlled trial [Abstract]
Maternal & Child Nutrition
 , 
2011
, vol. 
7
 
1
(pg. 
92
-
99
)
Merker
B. M.
Hanson
W. E.
Poston
J. M.
National survey of psychologists' training and practice in breaking bad news: A mixed methods study of the MUM effect
Journal of Clinical Psychology in Medical Settings
 , 
2010
, vol. 
17
 (pg. 
211
-
219
)
Meth
M. Z.
Tranel
D.
How well do patients remember and follow recommendations from their neuropsychologist?
The Clinical Neuropsychologist
 , 
2012
, vol. 
26
 
3
(pg. 
416
-
417
)
Mittenberg
W.
Tremont
G.
Zielinski
R. E.
Fichera
S.
Rayls
K. R.
Cognitive-behavioral prevention of postconcussion syndrome
Archives of Clinical Neuropsychology
 , 
1996
, vol. 
11
 
2
(pg. 
139
-
145
)
Newman
M. L.
Greenway
P.
Therapeutic effects of providing MMPI-2 test feedback to clients at a university counseling service: A collaborative approach
Psychological Assessment
 , 
1997
, vol. 
9
 
2
(pg. 
122
-
131
)
Pearson
Advanced Clinical Solutions for WAIS-IV and WMS-IV
 , 
2009
San Antonio, TX
Author
Ponsford
J.
Willmott
C.
Rothwell
A.
Cameron
P.
Ayton
G.
Nelms
R.
, et al.  . 
Impact of early intervention on outcome of mild traumatic brain injury in children
Pediatrics
 , 
2001
, vol. 
108
 
6
(pg. 
1297
-
1303
)
Pope
K. S.
Responsibilities in providing psychological test feedback to clients
Psychological Assessment
 , 
1992
, vol. 
4
 
3
(pg. 
268
-
271
)
Poston
J. M.
Hanson
W. E.
Meta-analysis of psychological assessment as a therapeutic intervention
Psychological Assessment
 , 
2010
, vol. 
22
 
2
(pg. 
203
-
212
)
Ringström
G.
Störsrud
S.
Posserud
I.
Lundqvist
S.
Westman
B.
Simrén
M.
Structured patient education is superior to written information in the management of patients with irritable bowl syndrome: A randomized controlled study
European Journal of Gastroenterology and Hepatology
 , 
2010
, vol. 
22
 (pg. 
420
-
428
)
Rose
T.
Worrall
L.
Hickson
L.
Hoffmann
T.
Do people with aphasia want written stroke and aphasia information? A verbal survey exploring preferences for when and how to provide stroke and aphasia information
Topics in Stroke Rehabilitation
 , 
2010
, vol. 
17
 
2
(pg. 
79
-
98
)
Schoo
L. A.
van den Berg
E.
van Zandvoort
M. J. E.
Postma
A.
Do you know what I mean? Laypeople and experts' concepts of cognition
Archives of Clinical Neuropsychology
 , 
2012
, vol. 
27
 
2
(pg. 
182
-
189
)
Suner
R. S.
Juvinya
D. C.
Bertran
C. N.
Graboleda
C. P.
Brugada
N. M.
Garcia
M. M. G.
Continuity of care and monitoring pain after discharge: patient perspective
Journal of Advanced Nursing
 , 
2010
, vol. 
66
 
1
(pg. 
40
-
48
)
Tremont
G.
Westervelt
H. J.
Javorsky
D. J.
Podolanczuk
A.
Stern
R. A.
Referring physician's perceptions of the neuropsychological evaluation: How are we doing?
The Clinical Neuropsychologist
 , 
2002
, vol. 
16
 
4
(pg. 
551
-
554
)
Westervelt
H. J.
Brown
L. B.
Tremont
G.
Javorsky
D. J.
Stern
R. A.
Patient and family perceptions of the neuropsychological evaluation: How are we doing?
The Clinical Neuropsychologist
 , 
2007
, vol. 
21
 (pg. 
263
-
273
)
Wright
N. S.
Fleming
P. S.
Sharma
P. K.
Battagel
J.
Influence of supplemental written information on adolescent anxiety, motivation and compliance in early orthodontic treatment
Angle Orthodontist
 , 
2010
, vol. 
80
 
2
(pg. 
329
-
335
)

Author notes

Present address: Samaritan Health Services in Corvallis, OR, USA.