Abstract

With the implementation of healthcare reform, it is essential for neuropsychologists to establish themselves as important members of healthcare teams and to work efficiently and effectively. The purpose of this study was to survey the satisfaction of referring providers with neuropsychological services, including turn-around time of reports and their essential elements, within a hospital system that uses electronic medical records. Findings revealed that referral sources are generally satisfied with neuropsychological services and find them valuable. They especially appreciate the neuropsychological report and detailed testing of cognitive and functional abilities and prefer the report within 2 weeks of the patient's appointment. Most find all sections of the report completely essential with the exception of test scores, and most are comfortable with recommendations made by neuropsychologists. With few exceptions, there were no differences among types of referring providers (i.e., physicians, psychologists, and social workers). Future surveys may wish to examine how neuropsychological services affect patient outcomes.

Introduction

In this new era of healthcare reform, the focus is to reduce the overall cost of health care while improving health outcomes and streamlining healthcare delivery. Two primary avenues for achieving these goals include forming Accountable Care Organizations (ACOs), which consist of interdisciplinary, interprofessional teams of healthcare providers, and utilizing electronic medical record (EMR) systems, both of which will facilitate coordinated care and communication among providers and across settings. While joining an ACO is not mandatory, healthcare providers within ACOs will be awarded shared savings when they meet or exceed quality performance standards, and there are incentives, as well as federal and state funding, for implementing EMR systems. EMRs are expected to reduce paperwork and administrative burden, prevent duplicative procedures, and decrease medical errors (see www.healthcare.gov for a more complete review of the new healthcare law, accessed on January 14, 2013).

Given these changes, it is imperative that neuropsychologists establish themselves as valuable members of healthcare teams, particularly since referring providers determine the medical necessity of an evaluation, and work as efficiently and effectively as possible. With regard to the former, we are aware of only two published studies that have investigated the satisfaction of healthcare providers with neuropsychological services (Temple, Carvalho, & Tremont, 2006; Tremont, Westervelt, Javorsky, Podolanczuk, & Stern, 2002). Both studies surveyed physicians and found that they are generally satisfied with neuropsychological services, agree with diagnostic impressions and recommendations, consider the evaluation and report useful, and are likely to refer future patients for neuropsychological evaluation (Temple et al., 2006; Tremont et al., 2002). Temple and colleagues (2006) also inquired about comfort with neuropsychologists making certain recommendations and found that most physicians are comfortable with neuropsychologists making recommendations about behavioral strategies but fewer physicians are comfortable with neuropsychologists recommending medications, consultation to other medical specialties, and neurodiagnostics, lab work, or other medical procedures.

With regard to working as efficiently and effectively as possible, the use of EMR systems may facilitate this for neuropsychologists by allowing them to reduce the length of their reports if relevant information is available in the EMR and to communicate findings in a more timely manner, since reports can be accessed from a referring provider's computer as soon as they are completed. Within the Veterans Administration (VA), use of EMRs has been in place for many years. One effect of EMR use on neuropsychological practice in our VA medical center was the mandate that all consults be completed within 48 h of the patient's appointment. (Note that since our survey was conducted, the mandate for completion of consults has been reduced to within 24 h.) In response, we developed a brief impression consisting of 2–3 paragraphs. The brief impression is followed by the full neuropsychological report, which is typically 4–6 pages, and is electronically attached to the brief impression as an addendum. Ideally, the full report is completed within 2–4 weeks, a little longer than other sites perhaps because ours is a training program.

The purpose of the present study was to replicate and extend the above findings concerning satisfaction with neuropsychological consultative services within a VA medical center that uses EMRs. Replication of prior studies was accomplished by inquiring about satisfaction that referral questions are answered, agreement with diagnostic impressions and recommendations, and comfort with neuropsychologists making certain recommendations using the same scales employed previously (Temple et al., 2006; Tremont et al., 2002). Extension of prior work was achieved in several ways. First, we surveyed all types of referring healthcare providers, not just physicians. We hypothesized that all types of referring providers would be satisfied with neuropsychological services. Second, we inquired about the desired turn-around time of reports within the EMR and their essential elements, since reports are neuropsychologists' primary work product and over 2 h, on average, is spent performing this activity (Sweet, Peck, Abramowitz, & Etzweiler, 2002). It was reasoned that it would be important to know if these work products are valued by referring providers and if certain portions are less valuable and could potentially be left out to save time. Third, because neuropsychologists in the VA can initiate consults to other providers through the EMR system, we explored whether this would be the preference of referring providers or whether they would prefer to initiate recommended consults themselves as is typically done in non-EMR settings employing neuropsychologists as consultants. Finally, we included open-ended items asking specifically about what is useful and worth continuing in the neuropsychology service and what should be changed. An additional study question was whether these factors differed by the type of referring provider. Given the exploratory nature of this work, we had no hypotheses about these latter study goals.

Method

Participants and Procedure

All healthcare providers who had referred patients to the Neuropsychology Service at Audie L. Murphy VA Medical Center between March 1, 2010 and February 29, 2012 were identified as potential study participants. Of the 493 referrals to the Neuropsychology Service during this time frame, 322 were duplicates (i.e., more than one referral made by the same provider) and 55 referring providers no longer had valid e-mail addresses within the local VA system. Thus, the remaining 116 healthcare providers were contacted via e-mail and asked to complete an online survey administered through Survey Monkey. A second request was emailed approximately 1 month later in an effort to remind those who may have meant to respond but had not yet done so.

Survey

A 10-item online survey (see Appendix A) was constructed to include selected items from prior work (Temple et al., 2006; Tremont et al., 2002), as well as new items specific to the study goals and procedures of the Neuropsychology Service. In brief, participants were asked to indicate their profession, estimate the percentage of patients they refer for neuropsychological evaluation, specify a time frame for which they would like a brief impression and then a full neuropsychological report, rate agreement about particular aspects of the report and which sections are essential, indicate comfort level with certain recommendations and how they would prefer neuropsychologists to handle recommended consults within the EMR system, and state what they find useful and worth continuing as well as what they would like to change about the Neuropsychology Service. The survey was restricted to 10 items to keep it brief and because surveys of 10 items or less can be administered by Survey Monkey at no cost. The survey was vetted several times by our neuropsychology research group and reviewed and approved by the VA Office of Labor Relations as potential participants were VA employees. IRB approval was not required because the anonymous survey did not meet the regulatory definition of Human Subjects Research.

Data Analysis

Descriptive statistics were used to describe the study sample and to characterize findings for each survey item. The two open-ended items were analyzed individually and grouped into themes by the senior author (RCH). When an open-ended item response contained content that was consistent with more than one theme, the item was included under all relevant themes. The themes and associated items were reviewed by the co-authors (TLH and KJM) for consensus. When a discrepancy was present, the authors discussed the item in question until a consensus was reached. Differences among types of providers were examined using Chi-square analyses and Fisher's exact test when cell sizes were <5. Given the exploratory nature of these analyses, p-values of ≤.05 were considered statistically significant.

Results and Discussion

Characteristics of Respondents

The survey was completed by 40 healthcare providers for a response rate of 34%. Seventeen were physicians, 13 were psychologists, 9 were social workers, and 1 was a doctor of pharmacy. Among the 17 physicians, six were psychiatrists, four were primary care physicians, four were internal medicine physicians, one was a neurologist, one was a geriatrician, and one was a polytrauma/rehabilitation physician. Seventeen providers indicated that they referred less than 5% of their patients for neuropsychological evaluation, 14 indicated they referred 5%, and 9 indicated they referred greater than 5%. Within this latter group, six indicated that they refer about 10% of their patients, and the remaining three indicated that they refer 20%–50% of their patients. When grouped by the type of provider (i.e., physicians, psychologists, and social workers), there were no significant differences in referral percentages—χ2(4) = 4.4, p = .35.

Time Frame for Receiving Results

When asked for a “useful/acceptable” time frame for receiving an “initial impression,” most respondents (80%) preferred this information within 1 week. Six indicated within 2 weeks, one indicated within 1 month, and one indicated before the patient's next appointment. The percentage of providers preferring the initial impression within a particular time frame did not differ significantly by the type of provider—χ2(4) = 6.5, p = .17.

When asked about a useful/acceptable time for receiving the “full neuropsychological report,” most (69.2%) indicated within 2 weeks. The remaining respondents were split equally between receiving the report within 1 month and before the patient's next appointment. There was a significant difference among type of providers preferring the full report within a certain time frame—χ2(4) = 11.1, p = .03. Follow-up analyses showed a significant difference between social workers and physicians, χ2(2) = 8.2, p = .02, with a greater percentage of social workers preferring the report within 1 week (62.5%) compared with physicians (8.3%; Fisher's exact test, p = .01). There were no significant differences between social workers and psychologists or between psychologists and physicians—χ2(2) = 4.8 and 2.2, p = .09 and .34, respectively. Reasons the majority of social workers prefer the full report within 1 week were not explored in the current study, but one possible hypothesis is that social workers rely on information in the neuropsychological report for treatment planning, whereas most physicians rely on the report for help with diagnosis or baseline functioning (Temple et al., 2006; Tremont et al., 2002), which may not be as time sensitive.

Agreement and Satisfaction with Neuropsychological Report

Respondents were asked four questions concerning agreement and satisfaction with the neuropsychological assessment and report. Almost all respondents (97.5%) indicated that they “very much” or “mostly” agreed that the findings were communicated clearly in the neuropsychological report, and most (95%) were “very much” or “mostly” satisfied that their referral questions were answered. Similarly, a large majority (92.5%) “very much” or “mostly” agreed with the diagnostic impressions, and most (89.7%) “very much” or “mostly” agreed with the recommendations. There were no significant differences among types of providers on any of these four items—χ2(4) = 1.5, 4.8, 2.9, and 1.7, p = .82, .30, .57, and .79, respectively.

Essential Sections of the Neuropsychological Report

Respondents were provided with a brief description of each section of the neuropsychological report (see Appendix A) and asked to indicate if each was “completely essential,” “somewhat essential,” or “not at all essential.” With few exceptions, most sections of the report were rated as “completely essential” or “somewhat essential” by all respondents. Results are displayed in Table 1 and listed in descending order beginning with the section rated by most respondents as “completely essential,” i.e., summary and impressions (100%), to the section rated by the fewest respondents as “completely essential,” i.e., test scores (35%). This latter finding is noteworthy as a survey of report writing practices revealed that approximately 89% of neuropsychologists include test scores in their reports (Donders, 2001b).

Table 1.

Responses concerning the essential sections of the neuropsychological report (N = 40)

 Completely essential (%) Somewhat essential (%) Not at all essential (%) 
Summary and impressions 100 — — 
Test interpretation 95 2.5 2.5 
Behavioral observations/mental status exam 90 10 — 
Recommendationsa 89.7 10.3 — 
Current complaints and relevant history 85 15 — 
Medical history 75 25 — 
Identifying information and reason for referral 67.5 27.5 
Psychosocial historya 59 38.5 2.5 
Test scores 35 52.5 12.5 
 Completely essential (%) Somewhat essential (%) Not at all essential (%) 
Summary and impressions 100 — — 
Test interpretation 95 2.5 2.5 
Behavioral observations/mental status exam 90 10 — 
Recommendationsa 89.7 10.3 — 
Current complaints and relevant history 85 15 — 
Medical history 75 25 — 
Identifying information and reason for referral 67.5 27.5 
Psychosocial historya 59 38.5 2.5 
Test scores 35 52.5 12.5 

aN = 39.

Significant differences among types of providers were found on the medical history section of the report—χ2(2) = 6.0, p = .05. A significantly greater percentage of social workers rated the medical history section of the report as “completely essential” (100%) compared with psychologists (53.8%; Fisher's exact test, p = .05). Psychologists and physicians did not differ significantly on this item, and neither did social workers and physicians (Fisher's exact test, both p's = .26). There were no significant differences among the types of providers for identifying information, current complaints and relevant history, psychosocial history, test scores, and test interpretation, allχ2(4) ≤ 4.2, p ≥ .38, or for behavioral observations or recommendations, χ2(2) = 0.1 and 0.2, p = .93 and .91, respectively. (Note that since 100% of providers rated the summary and impressions as “completely essential,” statistical analysis was not performed on this item.)

These findings revealing high percentages of satisfaction with regard to clarity and essential elements of the neuropsychological report are encouraging because they suggest that this work product is valued by referring providers and the 2+ h taken to write these comprehensive reports is time well spent. It is interesting to note that in spite of the availability of medical and psychosocial information within the EMR, the majority of referring healthcare providers rated the inclusion of these sections in the report as completely essential. Reasons for this may include the convenience of having all relevant information in one document without having to search through the EMR and/or increased understanding and appreciation of the case conceptualization presented in the summary and impressions. Support for the latter conclusion comes from comments to the open-ended question about useful aspects of the neuropsychological service, which included insight and integration of neurobehavioral information. Because this information is also helpful in establishing medical necessity for auditing purposes and teaching report writing to trainees, it is reassuring to know that it is preferred by most referring providers. For neuropsychologists seeking to streamline report writing, results suggest that test scores may be left out in some cases as they are not viewed to be as essential as other sections of the report. However, it is important to note that including test scores may be valuable for continuity of care, particularly in cases where the need for re-evaluation is likely.

Comfort with Neuropsychologists Making Recommendations and Entering Consults into the EMR

Respondents were provided with 11 recommendations frequently made by neuropsychologists and asked to indicate their comfort level according to the following scale: “very comfortable,” “comfortable,” “somewhat comfortable,” or “not at all comfortable.” Findings, which are presented in Table 2 and listed in descending order beginning with the recommendation rated by most respondents as “very comfortable,” showed that recommendations about capacity of the patient to make decisions and live independently had the highest percentage of respondents, indicating that they were “very comfortable” with neuropsychologists making these recommendations (both at 82.5%). Need for supervision and future planning and behavioral strategies were the next two recommendations most frequently rated as “very comfortable” by respondents. Recommendations endorsed by the lowest percentage of respondents who were “very comfortable” were laboratory work, medications, and other medical procedures, which is in agreement with findings of Temple and colleagues (2006).

Table 2.

Respondents indicating comfort with recommendations made by neuropsychologists (N = 40)

 Very comfortable (%) Comfortable (%) Somewhat comfortable (%) Not at all comfortable (%) 
Capacity for patient to make decisions 82.5 12.5 2.5 2.5 
Capacity for patient to live independently 82.5 12.5 2.5 2.5 
Need for supervision and future planninga 76.9 15.4 5.1 2.6 
Behavioral Strategies 70 27.5 2.5 — 
Educational accommodationsa 69.2 25.7 5.1 — 
Consultation to other specialties 67.5 30 2.5 — 
Opinion on driving ability 65 25 7.5 2.5 
Neuroimaging or other neurodiagnostic procedures 62.5 30 2.5 
Laboratory work 47.5 37.5 10 
Medications 27.5 32.5 27.5 12.5 
Other medical proceduresa 25.6 43.6 20.5 10.3 
 Very comfortable (%) Comfortable (%) Somewhat comfortable (%) Not at all comfortable (%) 
Capacity for patient to make decisions 82.5 12.5 2.5 2.5 
Capacity for patient to live independently 82.5 12.5 2.5 2.5 
Need for supervision and future planninga 76.9 15.4 5.1 2.6 
Behavioral Strategies 70 27.5 2.5 — 
Educational accommodationsa 69.2 25.7 5.1 — 
Consultation to other specialties 67.5 30 2.5 — 
Opinion on driving ability 65 25 7.5 2.5 
Neuroimaging or other neurodiagnostic procedures 62.5 30 2.5 
Laboratory work 47.5 37.5 10 
Medications 27.5 32.5 27.5 12.5 
Other medical proceduresa 25.6 43.6 20.5 10.3 

aN = 39.

While the present study showed no differences among the types of providers on any of these items, all χ2(4,6) ≤ 11.7, all p ≥ .07. (We also analyzed each of these items after collapsing “very comfortable” and “comfortable” responses into a “comfortable” category and “somewhat comfortable” and “not at all comfortable” responses into a “not as comfortable” category, similar to the analyses conducted by Temple and colleagues (2006) and still found no significant differences among types of providers on any of the items, all χ2(2) ≤ 3.4, all p ≥ .18.) Temple and colleagues (2006) found that primary care physicians were significantly more comfortable than other physician groups with neuropsychologists recommending laboratory work, medications, and other medical procedures. In addition, neurologists and psychiatrists were significantly less comfortable than other physician groups with neuropsychologists recommending laboratory work and medications, respectively. This discrepancy in study findings likely is due to differences in the types of professionals surveyed. The number of physicians responding to the current survey was not large enough to examine subspecialty differences. However, support for the findings of Temple and colleagues (2006) within the current sample is suggested by a response to one of the open-ended items requesting that neuropsychologists consider providing recommendations about medications to referring providers who are not neurobehavioral specialists.

Finally, when neuropsychologists make recommendations to consult other services, respondents were asked if they prefer (a) to enter the consult request into the EMR themselves, (b) the neuropsychologist enter mental health-related consults into the EMR, or (c) the neuropsychologist enter all recommended consults into the EMR. Slightly over half of respondents (53.8%) indicated that they prefer the neuropsychologist enter all consults into the EMR, 28.2% indicated that they prefer to enter recommended consults into the EMR themselves, and 23.1% indicated that they prefer the neuropsychologist enter mental health-related consults. Insight into reasons for this preference was found in a response to the open-ended item about aspects of the neuropsychology service to change. The respondent requested that neuropsychologists enter consults in the EMR because the primary care provider might not get to it in a timely manner.

Responses on this item differed significantly by the type of provider, χ2(4) = 9.4, p = .05. Follow-up analyses showed that a significantly greater percentage of psychologists preferred the neuropsychologist enter mental health-related consults (54.5%) compared with social workers (0%) (Fisher's exact test, p = .04). There were no significant differences on this item between physicians and psychologists, χ2(2) = 5.0, p = .08, or between physicians and social workers, χ2(2) = 2.1, p = .35. A potential explanation for these findings is that psychologists and neuropsychologists complete similar training and thus may be more likely to agree on the types of mental health consults needed resulting in psychologists feeling very comfortable with neuropsychologists initiating these consults. Another possibility is that social workers perform case management more often than psychologists, so entering mental health-related consults into the EMR may be viewed as a primary job function.

Useful Aspects of the Neuropsychology Service

Thirty-one of the 40 providers responded to the open-ended question asking about useful aspects of the Neuropsychology Service that should be continued. The two aspects that received the most comments as being useful were (a) the neuropsychological report and thoroughness of the evaluation (mentioned by 10 providers) and (b) the detailed nature of the testing which allows for a better understanding of cognitive and functional abilities (mentioned by nine providers). Help with differential diagnosis, insight and integration of neurobehavioral aspects, and recommendations were mentioned by five providers each. These responses provide strong support for the current practice of most neuropsychologists to engage in a thorough evaluation that includes the integration of neurobehavioral and test data into a comprehensive report (Donders, 2001a; Sweet et al., 2002; Sweet, Meyer, Nelson, & Moberg, 2011). Three providers mentioned the availability and timeliness of evaluations and three made more general comments about the usefulness of neuropsychology and the high level of skill required to provide the service. Sample responses under each theme are presented in Table 3.

Table 3.

Sample responses about useful aspects of the Neuropsychology Service

Theme Verbatim response 
Neuropsychological reports and thoroughness of evaluation (N = 10) “comprehensive nature of report”
“specialty knowledge, thorough evaluation, recommendations”
“I think the information provided in the report is extremely valuable.” 
Better understanding of cognitive and functional abilities (N = 9) “really helps define dementia vs depression and give me an idea of IQ and cognitive level of pt”
“specific capacities or lack there of”
“It's very important for the neuropsychologist to directly address pt's ability to make decisions with self care, money management and driving” 
Differential diagnosis (N = 5) “Help with differential dx; help with evaluating memory complaints”
“diagnostic clarification and very customized, specific recommendations about how to support the pt given cognitive deficits, including referral suggestions” 
Insight and integrating neurobehavioral aspects (N = 5) “I appreciate the neurological insight when there are issues of veterans who are unable to take care of themselves due to either/ both biological or mental health concerns. Integrating the neurological vs behavioral outlook is extremely important in helping to determine a positive outcome for the veteran”
“The Neuropsych evals are invaluable in complicated seizure pts especially those with brain trauma/lesions. It provides a different perspective and enables providers in Neuroseizure clinic to more effectly [sic] tx the pts- also helps the [sic] their PCP (I hope) communicate more effectively with the pt” 
Recommendations (N = 5) “I always appreciate the insight and recommendations regarding patient functioning and next course of action”
“I found the Summary and Recommendations the most useful part of the report” 
Availability and timeliness of evaluations (N = 3) “I have used this service on several occasions and have been very satisfied with the availability of appointments and the outcome of the assessments and recommendations”
“timely, useful impressions about pt functioning” 
Theme Verbatim response 
Neuropsychological reports and thoroughness of evaluation (N = 10) “comprehensive nature of report”
“specialty knowledge, thorough evaluation, recommendations”
“I think the information provided in the report is extremely valuable.” 
Better understanding of cognitive and functional abilities (N = 9) “really helps define dementia vs depression and give me an idea of IQ and cognitive level of pt”
“specific capacities or lack there of”
“It's very important for the neuropsychologist to directly address pt's ability to make decisions with self care, money management and driving” 
Differential diagnosis (N = 5) “Help with differential dx; help with evaluating memory complaints”
“diagnostic clarification and very customized, specific recommendations about how to support the pt given cognitive deficits, including referral suggestions” 
Insight and integrating neurobehavioral aspects (N = 5) “I appreciate the neurological insight when there are issues of veterans who are unable to take care of themselves due to either/ both biological or mental health concerns. Integrating the neurological vs behavioral outlook is extremely important in helping to determine a positive outcome for the veteran”
“The Neuropsych evals are invaluable in complicated seizure pts especially those with brain trauma/lesions. It provides a different perspective and enables providers in Neuroseizure clinic to more effectly [sic] tx the pts- also helps the [sic] their PCP (I hope) communicate more effectively with the pt” 
Recommendations (N = 5) “I always appreciate the insight and recommendations regarding patient functioning and next course of action”
“I found the Summary and Recommendations the most useful part of the report” 
Availability and timeliness of evaluations (N = 3) “I have used this service on several occasions and have been very satisfied with the availability of appointments and the outcome of the assessments and recommendations”
“timely, useful impressions about pt functioning” 

Notes: pt = patient; dx = diagnosis; Neuropsych evals = neuropsychological evaluations; tx = treat; PCP = primary care physician.

Aspects of the Neuropsychology Service to Change

Although 31 providers responded to the open-ended question about the aspects of the Neuropsychology Service that needed to be changed, 12 of these stated that they would change nothing and/or provided positive comments (e.g., “nothing—they do a great job and are a great service”). The most frequent comment about aspects to change, made by nine providers, pertained to timeliness of answering the consult question and included both requests for quicker appointment slots and faster turn-around of the full neuropsychological report. Comments about the length and content of the neuropsychological report and expanding services were the next most frequent suggestions made by four providers each. Three providers suggested improving recommendations, one requested more communication with the neuropsychologist, and one recommended providing a diagnostic impression rather than recommending treatment of symptoms (e.g., attention problems). See Table 4 for examples of responses under each of the identified themes.

Table 4.

Sample responses about aspects of the Neuropsychology Service to change

Theme Verbatim response 
Availability and timeliness of evaluations (N = 9) “more rapid response to initial consult”
“No major changes, would like to see reports in CPRS quicker, and the same system for locating those reports (i.e., all final reports are addendums to the original impression note)”
“quicker evals on inpatient unit” 
Neuropsychological reports and thoroughness of evaluation (N = 4) “Their notes are too long and sometimes hard to find the relevant information. If they can provide at the end of the note or consult what their impression is and what do they want us, the referring provider to do in a separate paragraph would be helpful. If they think the pt should not be driving they should make the appropriate referrals. also would be helpful if they make appropriate referrals Instead of waiting for the PCP to read the note and do consults. We may not get it to [sic] in a timely manner. I appreciate all their help. Thanks”
“too inclusive on other things already in chart” 
Expansion of services (N = 4) “perhaps veterans who have been found incapable of managing finances should be directly referred to the VBA fiduciary unit along with the specific findings. This would make appointing a fiduciary guardian much easier and less likely to be overlooked”
“With TBI being a huge risk factor with our returning veterans, I would like to see an additional Neuropsychologist. I would also like to see more marketing within the VA system, for example meeting with Social Work, MHOS or PCT to help these providers understand the importance of your role in the veteran's treatment plan” 
Recommendations (N = 3) “sometimes the recommendations are not clear”
“more specific, tangible recommendations” 
Other (N = 2) “It would be great to have more communication between the referring clinician and the Neuropsychologist before testing and when the final recommendations are being formulated” 
Theme Verbatim response 
Availability and timeliness of evaluations (N = 9) “more rapid response to initial consult”
“No major changes, would like to see reports in CPRS quicker, and the same system for locating those reports (i.e., all final reports are addendums to the original impression note)”
“quicker evals on inpatient unit” 
Neuropsychological reports and thoroughness of evaluation (N = 4) “Their notes are too long and sometimes hard to find the relevant information. If they can provide at the end of the note or consult what their impression is and what do they want us, the referring provider to do in a separate paragraph would be helpful. If they think the pt should not be driving they should make the appropriate referrals. also would be helpful if they make appropriate referrals Instead of waiting for the PCP to read the note and do consults. We may not get it to [sic] in a timely manner. I appreciate all their help. Thanks”
“too inclusive on other things already in chart” 
Expansion of services (N = 4) “perhaps veterans who have been found incapable of managing finances should be directly referred to the VBA fiduciary unit along with the specific findings. This would make appointing a fiduciary guardian much easier and less likely to be overlooked”
“With TBI being a huge risk factor with our returning veterans, I would like to see an additional Neuropsychologist. I would also like to see more marketing within the VA system, for example meeting with Social Work, MHOS or PCT to help these providers understand the importance of your role in the veteran's treatment plan” 
Recommendations (N = 3) “sometimes the recommendations are not clear”
“more specific, tangible recommendations” 
Other (N = 2) “It would be great to have more communication between the referring clinician and the Neuropsychologist before testing and when the final recommendations are being formulated” 

Notes: CPRS = electronic medical record system; eval = evaluations; pt = patient; PCP = primary care physician; VBA = veterans benefits; TBI = traumatic brain injury; VA = veterans affairs; MHOS = Mental Health Outpatient Service; PCT = Posttraumatic Stress Disorder Clinical Team.

Conclusions and Future Directions

The need for neuropsychologists to be viewed as an integral component of an interdisciplinary, intra-professional healthcare team and to work as efficiently and effectively as possible has never been more important. Because referring providers determine the medical necessity of an evaluation, it is imperative that they are satisfied with and value neuropsychological services. Findings of the current study indicate that referral sources, both physicians and non-physicians, are satisfied with neuropsychological services and find them valuable. They especially appreciate the neuropsychological report and detailed testing of cognitive and functional abilities. Most find all sections of the neuropsychological report completely essential, and most are comfortable with recommendations made by neuropsychologists. Within an EMR system that requires consult results be available within 48 h, referring providers indicated access to initial impressions within 1 week is acceptable and access to the full report within 2 weeks to 1 month is useful. Over half prefer all recommended consults be initiated by the neuropsychologist, although preferences differed by the type of provider, suggesting that tailoring this service might be best.

The results of this study must be interpreted within the context of study limitations. First, the practices of our Neuropsychology Consult Service and VA medical center may not apply to other settings. However, they serve as guidance about what may matter to referring providers. Second, our sample size was relatively small, although the response rate was consistent with other neuropsychological surveys of professionals (Rabin, Barr, & Burton, 2005; Sweet et al., 2002) and greater than that of Temple and colleagues (2006), who employed a national survey of physicians. On the other hand, our response rate was approximately half that of Tremont and colleagues (2002), which may be explained by methodological differences. Tremont and colleagues (2002) sent physicians a survey along with the neuropsychological report of a patient referred by that particular physician. Thus, the surveys and reports were individualized in a sense, which may have increased the investment of the physician to complete the survey. Regardless of how the return rate of this study compares to previous studies, the small sample size may have precluded the ability to detect differences among subgroups of providers, resulting in Type II error.

Additional research in other healthcare settings with and without EMRs with larger subsamples of different types of providers and comparing response rates among the types of providers is recommended to replicate and extend these findings. An area worthy of future study is how neuropsychological services affect patient outcomes. For example, future surveys could investigate if the referring provider felt more or less confident in the patient's diagnosis after receiving the neuropsychological report and/or if the course of treatment was determined and/or altered based on neuropsychological findings. Another tactic could be an approach similar to that of Bishop, Temple, Tremont, Westervelt, and Stern (2003) who conducted medical record reviews to determine the use of inpatient neuropsychological evaluations. They discovered that out of 100 inpatients, information from the neuropsychological evaluation was included in 78% of hospital discharge summaries and recommendations for placement post-discharge were followed 80% of the time. Clearly, further empirical study of the value of neuropsychological services to all stakeholders, including referring providers, patients, and care providers is needed, as well as a better understanding of which neuropsychological services and/or procedures are most useful and which, if any, can be reduced or eliminated. Moreover, with the growing use of EMRs, it would be informative to investigate how to capitalize on the benefits of EMRs for improving efficiency and efficacy of neuropsychological practice.

Conflict of Interest

None declared.

Acknowledgements

The authors would like to acknowledge and thank the neuropsychology research team that assisted with refining the survey used in this study, including Dr Robert Fallows, Dr Emily Luther-Worthy, Dr Janice Marceaux, Mr Frank Reyes, Dr Justin O'Rourke, Dr Russell Pella, and Dr Nabeel Yehyawi. The views expressed in this publication do not represent the views of the Department of Veterans Affairs or the United States Government.

Appendix A

Neuropsychology consult service survey

The Neuropsychology Consult Service is conducting a survey of our referral sources in an effort to evaluate and improve the services provided. The survey is 10 questions and will take approximately 5 min to complete. Completion of the survey is completely voluntary. Please click on the link below to take the survey.

1. Which best describes your profession?     2. Please estimate the percentage of patients you refer for neuropsychological evaluation:

  • Primary Care Physician

  • Internal Medicine Physician

  • Neurologist

  • Psychiatrist

  • Physician - other

  • Psychologist

  • Pharmacist

  • Nurse

  • Social Worker

  • Other (please specify):

 3. We typically provide a brief Initial Impression (2–3 paragraphs) within 48 hours of evaluating the patient. What is a useful/acceptable time from when the patient was seen until you receive the Initial Impression?     4. We typically provide a full Neuropsychological Report (4–6 pages) with recommendations within 2 weeks of evaluating the patient. What is a useful/acceptable time from when the patient was seen until you receive the full Neuropsychological Report?     5. With regard to the neuropsychological report, in general, to what extent:

  • 2–4 days

  • 1 week

  • 2 weeks

  • 1 month

  • Before I next see the patient

  • Other:

  • 2–4 days

  • 1 week

  • 2 weeks

  • 1 month

  • Before I next see the patient

  • Other:

 Very Much Mostly Somewhat Not At All 
Are the findings communicated clearly?     
Are you satisfied that the referral questions are answered?     
Do you agree with the diagnostic impressions?     
Do you agree with the recommendations?     
 Very Much Mostly Somewhat Not At All 
Are the findings communicated clearly?     
Are you satisfied that the referral questions are answered?     
Do you agree with the diagnostic impressions?     
Do you agree with the recommendations?     

6. Please rate how essential each of the following sections of the Neuropsychological Report are:

 Completely essential Somewhat essential Not at all essential 
Identifying information (age, sex, education, ethnicity, handedness) and reason for referral (referral source, current/suspected diagnosis, and problem needing evaluation)    
Current complaints and relevant history (patient's complaints; onset, frequency, and course of each complaint; precipitating event; daily functioning; attempts to compensate for cognitive problems; emotional status and relevant history; collateral report)    
Medical history (current and past medical problems; current medications; history of neurological disorders; neuroimaging or neurological findings; prior neuropsychological evaluation findings; psychiatric history; substance use history and family medical history)    
Psychosocial history (birth and developmental history; abuse history; educational achievement; military service and achievement; occupational history; disability status; legal history; social situation)    
Behavioral observations/mental status exam (unusual presentation or symptoms, approach to testing, level of insight/awareness, mood and affect, and personality traits)    
Test scores listed individually including Raw scores, Standard Scores, Scaled Scores, T-scores and Percentiles    
Test interpretation (paragraph or two of cognitive, psychological, and functional test findings)    
Summary and impression (summary of patient's complaints, essential medical and psychosocial information, cognitive performance and psychological status, diagnosis, etiology, prognosis and answer to any specific referral question)    
Recommendations that are individual to the patient's needs    
 Completely essential Somewhat essential Not at all essential 
Identifying information (age, sex, education, ethnicity, handedness) and reason for referral (referral source, current/suspected diagnosis, and problem needing evaluation)    
Current complaints and relevant history (patient's complaints; onset, frequency, and course of each complaint; precipitating event; daily functioning; attempts to compensate for cognitive problems; emotional status and relevant history; collateral report)    
Medical history (current and past medical problems; current medications; history of neurological disorders; neuroimaging or neurological findings; prior neuropsychological evaluation findings; psychiatric history; substance use history and family medical history)    
Psychosocial history (birth and developmental history; abuse history; educational achievement; military service and achievement; occupational history; disability status; legal history; social situation)    
Behavioral observations/mental status exam (unusual presentation or symptoms, approach to testing, level of insight/awareness, mood and affect, and personality traits)    
Test scores listed individually including Raw scores, Standard Scores, Scaled Scores, T-scores and Percentiles    
Test interpretation (paragraph or two of cognitive, psychological, and functional test findings)    
Summary and impression (summary of patient's complaints, essential medical and psychosocial information, cognitive performance and psychological status, diagnosis, etiology, prognosis and answer to any specific referral question)    
Recommendations that are individual to the patient's needs    

7. Indicate how comfortable you are with neuropsychologists recommending:

 Very Comfortable Comfortable Somewhat Comfortable Not at All 
Consultation to other specialties     
Neuroimaging or other neurodiagnostic procedures     
Laboratory work     
Behavioral Strategies     
Medications     
Other medical procedures     
Opinion on ability to work     
Capacity for patient to make decisions     
Capacity for patient to live independently     
Opinion on driving ability     
Need for supervision and future planning     
Educational accommodations     
 Very Comfortable Comfortable Somewhat Comfortable Not at All 
Consultation to other specialties     
Neuroimaging or other neurodiagnostic procedures     
Laboratory work     
Behavioral Strategies     
Medications     
Other medical procedures     
Opinion on ability to work     
Capacity for patient to make decisions     
Capacity for patient to live independently     
Opinion on driving ability     
Need for supervision and future planning     
Educational accommodations     

8. How would you like the consults recommended in the neuropsychological report to be handled?     9. What do you find useful about the neuropsychology service and want to see continued?

  • I prefer entering recommended consults into CPRS myself

  • I prefer the neuropsychologist enter Mental Health consults into CPRS

  • I prefer the neuropsychologist enter all recommended consults into CPRS

10. What would you change about neuropsychological services? Please provide any other comments or concerns.

Thank you. Your participation is greatly valued.

References

Bishop
C. L.
Temple
R. O.
Tremont
G.
Westervelt
H. J.
Stern
R. A.
Utility of the neuropsychological evaluation in an acute medical hospital
The Clinical Neuropsychologist
 , 
2003
, vol. 
17
 (pg. 
468
-
473
)
Donders
J.
A survey of report writing by neuropsychologists, I: General characteristics and content
The Clinical Neuropsychologist
 , 
2001a
, vol. 
15
 (pg. 
137
-
149
)
Donders
J.
A survey of report writing by neuropsychologists, II: Test data, report format, and document length
The Clinical Neuropsychologist
 , 
2001b
, vol. 
15
 (pg. 
150
-
161
)
Rabin
L. A.
Barr
W. B.
Burton
L. A.
Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA Division 40 members
Archives of Clinical Neuropsychology
 , 
2005
, vol. 
20
 (pg. 
33
-
65
)
Sweet
J. J.
Meyer
D. G.
Nelson
N. W.
Moberg
P. J.
The TCN/AACN 2010 “Salary Survey”: Professional practices, beliefs, and incomes of U.S. neuropsychologists
The Clinical Neuropsychologist
 , 
2011
, vol. 
25
 (pg. 
12
-
61
)
Sweet
J. J.
Peck
E. A.
Abramowitz
C.
Etzweiler
S.
National Academy of Neuropsychology/Division 40 of the American Psychological Association practice survey of clinical neuropsychology in the United States, Part I: Practitioner and practice characteristics, professional activities, and time requirements
The Clinical Neuropsychologist
 , 
2002
, vol. 
16
 (pg. 
109
-
127
)
Temple
R. O.
Carvalho
J.
Tremont
G.
A national survey of physicians' use and satisfaction with neuropsychological services
Archives of Clinical Neuropsychology
 , 
2006
, vol. 
21
 (pg. 
371
-
382
)
Tremont
G.
Westervelt
H. J.
Javorsky
D. J.
Podolanczuk
A.
Stern
R. A.
Referring physicians' perceptions of the neuropsychological evaluation: How are we doing?
The Clinical Neuropsychologist
 , 
2002
, vol. 
16
 (pg. 
551
-
554
)