Abstract

It was our aim to develop a questionnaire for patients with chronic musculoskeletal diseases to self-report their health education literacy, to analyse the psychometric properties of the instrument and to test hypotheses concerning sociodemographic predictors of health education literacy. A total of 577 patients with chronic back pain or osteoarthritis who underwent inpatient rehabilitation were surveyed. The resulting ‘HELP questionnaire’ (health education literacy of patients with chronic musculoskeletal diseases) consists of 18 items and three scales (comprehension of medical information, applying medical information, communicative competence in provider interactions). The instrument’s psychometric properties are good (Cronbach’s alpha between 0.88 and 0.95, unidimensionality and Rasch model fit established). Our sample’s average level of self-reported health education literacy is quite high. However, 20–30% of the patients admitted to having difficulty understanding important aspects of health education programmes (i.e. comprehending what medical information means in relation to their disease). The variance explained by sociodemographic and basic medical variables is small (4–8%). Greater effort is required to make health education programmes easier to understand. There is a need for more research on interindividual variability of complex aspects of health literacy.

Introduction

The World Health Organization (WHO) defines health literacy as ‘the cognitive and social skills which determine the motivation and ability of individuals to gain access to understand and use information in ways which promote and maintain good health’ [1]. There is evidence that low health literacy may be associated with worse health outcomes [2], disadvantageous health behaviour [3], lower patient satisfaction [4], and—in certain cases—with mortality [5]. The construct’s relevance is not solely based on relationships between medical important outcomes; in fact, many maintain that low health literacy essentially contributes to existing health disparities [6].

The low health literacy risk factor is widespread: 38% of the US population possesses low health literacy [7]; in the European Health Literacy Survey, nearly every second respondent revealed limited health literacy [8]. Sufficient health literacy is particularly important for the chronically ill as they are more dependent over the long term on support from the healthcare system, and many therapeutic approaches rely on the patient’s personal initiative and efficient self-management. Health education and self-management programmes are essential elements in the therapy of the chronically ill. By ‘health education literacy’ (HEL) we mean (according to the aforementioned WHO definition), the cognitive and social skills that determine the motivation and ability of chronically ill people to understand and use the information as conveyed in health education programmes. Having HEL is a key enhancer of the chronically ill patient’s ability to manage his or her disease and contributes to a more positive outcome. According to the latest theories of health literacy [9, 10, 11], it does not just rely on literacy and understanding basic health information, rather it involves communicative–interactive capacities too in dealing with programme trainers. Depending on a study’s focus, HEL may be regarded as a prerequisite for benefiting from health education, or as the outcome of an intervention.

We consider it worthwhile to capture health literacy according to specific fields, in this case, patients with chronic musculoskeletal diseases (CMD patients), and to restrict ourselves to the skills necessary to understand and apply the information conveyed in health education programmes. The utility of such domain-specific instruments has been verified frequently (e.g. [11]), which has led to the creation of instruments tailored to certain symptoms (e.g. arterial hypertension [12]) and specific media in communicating information (i.e. eHealth [13]).

Our approach is based on an ecologically framed conceptual model described in detail in the paper by Farin et al. [14]. We do not understand health literacy as an exclusively individual construct, but rather as interaction between the person’s abilities and factors in the healthcare system (see also [15, 16]). In terms of health education, this means that there should be a match between two aspects, the comprehensibility of health education programmes and the patient’s HEL. In reference to this ecological model, we developed in an initial step (see [14]) an instrument to capture the comprehensibility of health education programmes (COHEP questionnaire). In this article, we describe an instrument to measure the HEL of CMD patients that corresponds in content with COHEP.

To the best of our knowledge, there is no suitable instrument with which to specifically measure the HEL of the chronically ill. The health literacy instruments of van der Vaart et al. [17] and Weis and Giesler [18] address the chronically ill as well, and extend beyond the measurement of basic skills, but they are not tailored to be used specifically in conjunction with health education programmes. One approach somewhat similar to ours is the ‘Effective Consumer Scale’ by Kristjansson et al. [19] as it was also assessed in the context of self-management interventions. However, the concept measured in that scale is more global than ours, and it is not based on an ecologically framed model. In other words, no instrument with corresponding content exists with which the comprehensibility of an educational intervention can be captured. There is a general need for methodologically sound health literacy instruments as many of those available reveal psychometric weaknesses [20].

This study’s primary aim was to develop and test a new and psychometrically sound instrument to measure the HEL of CMD patients (HELP questionnaire). Its second aim was to investigate the sociodemographic predictors of HEL using this new instrument. One of the reasons this issue is relevant is that if basic sociodemographic variables and HEL reveal obvious interrelationships, it would mean that treatment providers could manage without a time-consuming and detailed assessment and instead address sociodemographic variables in a kind of screening format [21]. Current findings have led us to suspect that HEL can be predicted through general education of the patient [22, 23], financial status [22, 24], age [23, 24] and sex [24].

Methods

Instrument development

We referred to the COHEP questionnaire when designing the HELP instrument in order to assess HEL and comprehensibility of health education programmes in a corresponding way. COHEP was developed in conjunction with CMD patient focus groups in which the comprehensibility problems of health education programmes were discussed [14]. It captures the comprehensibility of health education as perceived by patients, and consists of 30 items and four scales (comprehension-fostering behaviour of programme trainers, transferability to everyday life, comprehensibility of medical information and amount of information). All scales are reliable, unidimensional, and meet the Rasch-model requirements. An example of an item is ‘The education sessions were easy to understand’.

For each COHEP item, we considered which skills or abilities a patient must demonstrate to ensure that that particular comprehensibilty or applicability aspect is achieved. Each skill or ability constituted one item, and to limit the number of questions, only items with clearly different content were generated. For example, the COHEP item “The programme trainers included the patients’ expectations and wishes” became the HELP item ‘How much difficulty did you have communicating your own expectations and wishes in terms of your therapy?’. While the COHEP items address the evaluation of a particular health education intervention, the HELP items aim to capture the patient’s aggregate perception of the difficulties he or she has with comprehension (i.e. understanding foreign words and medical terminology) and communication (i.e. posing questions, expressing needs) in various health education or treatment situations. We compiled and then tested a total of 26 items; the final version with 18 items is illustrated in Table IV.

Sample

This study was approved by the Ethics Committee of the University of Freiburg (approval number 369/10). A total of 577 patients with chronic back pain or osteoarthritis who underwent inpatient rehabilitation were surveyed. The inclusion criteria were illness for at least 6 months (as a criterion for chronicity, e.g. [25]) and patient age at least 18 years. Patients with specific low back pain due to tumours or inflammatory diseases, and those incapable of filling out the questionnaires in German due to cognitive, physical, or language limitations were excluded. Eight rehabilitation centres participated in the survey. The patients were asked by the clinic personnel at the start of their inpatient rehabilitation to fill out the questionnaire on their own and to then turn it back. The percentage of patients that did not fill out the questionnaire was 43.5%. This relatively high percentage can be attributed in part to one centre with a very high non-participation rate. Without that centre, that percentage would have been 37.8%. The reason for patients refusing to participate was generally an unwillingness to fill out the questionnaire. Table I provides information on our study patients.

Table I.

Respondent characteristics (N = 577)

Sociodemographic characteristics
    Age (Mean, SD, range)54.0, 10.5, 17–85
    Sex (%)
        Female55.0
    Level of education (highest level completed) (%)
        Elementary school35.3
        Secondary school42.6
        University-entrance diploma or  technical college qualification20.9
    Employment (%)
        Employed73.8

Medical characteristics

    Disease (%)
        Chronic back pain55.3
        Osteoarthritis35.2
        Missing9.5
    Chronification (%)
        <2 years23.7
        2–5 years23.3
        5–10 years21.1
        >10 years29.6
    General health status (%)
        Very good or good16.2
        Satisfactory46.0
        Not good26.3
        Bad or very bad11.5
Sociodemographic characteristics
    Age (Mean, SD, range)54.0, 10.5, 17–85
    Sex (%)
        Female55.0
    Level of education (highest level completed) (%)
        Elementary school35.3
        Secondary school42.6
        University-entrance diploma or  technical college qualification20.9
    Employment (%)
        Employed73.8

Medical characteristics

    Disease (%)
        Chronic back pain55.3
        Osteoarthritis35.2
        Missing9.5
    Chronification (%)
        <2 years23.7
        2–5 years23.3
        5–10 years21.1
        >10 years29.6
    General health status (%)
        Very good or good16.2
        Satisfactory46.0
        Not good26.3
        Bad or very bad11.5

All data are based on patient reports. General health status was assessed with a one-item measure (How would you rate your health?).

Table I.

Respondent characteristics (N = 577)

Sociodemographic characteristics
    Age (Mean, SD, range)54.0, 10.5, 17–85
    Sex (%)
        Female55.0
    Level of education (highest level completed) (%)
        Elementary school35.3
        Secondary school42.6
        University-entrance diploma or  technical college qualification20.9
    Employment (%)
        Employed73.8

Medical characteristics

    Disease (%)
        Chronic back pain55.3
        Osteoarthritis35.2
        Missing9.5
    Chronification (%)
        <2 years23.7
        2–5 years23.3
        5–10 years21.1
        >10 years29.6
    General health status (%)
        Very good or good16.2
        Satisfactory46.0
        Not good26.3
        Bad or very bad11.5
Sociodemographic characteristics
    Age (Mean, SD, range)54.0, 10.5, 17–85
    Sex (%)
        Female55.0
    Level of education (highest level completed) (%)
        Elementary school35.3
        Secondary school42.6
        University-entrance diploma or  technical college qualification20.9
    Employment (%)
        Employed73.8

Medical characteristics

    Disease (%)
        Chronic back pain55.3
        Osteoarthritis35.2
        Missing9.5
    Chronification (%)
        <2 years23.7
        2–5 years23.3
        5–10 years21.1
        >10 years29.6
    General health status (%)
        Very good or good16.2
        Satisfactory46.0
        Not good26.3
        Bad or very bad11.5

All data are based on patient reports. General health status was assessed with a one-item measure (How would you rate your health?).

Analyses

Psychometric properties

  • Response frequency and ceiling/floor effects: An item was removed if one of the following conditions was fulfilled: (a) more than 5% missing values and (b) ceiling or floor effects (more than 50% of values in the extreme categories).

  • Exploratory factor analysis (software: IBM SPSS Statistics version 21 with extension command SPSSINC HETCOR for analysing polychoric correlations): In order to determine the number of factors to be extracted, the following criteria were used: (a) interpretability, (b) scree test and (c) explained variance of at least 40%. An item was removed if it did not load unambiguously on the extracted factors (factor loading ≥0.50 on exactly one factor; factor loading <0.50 on all other factors).

  • Unidimensionality: Every scale was tested separately to check whether it measures a latent dimension (single-factor confirmatory factor analyses; software: Mplus version 5.1, using the WLSMV estimator). No cross loadings or correlated errors terms were permitted. Model fit was evaluated using the Comparative Fit Index (CFI), Tucker–Lewis index (TLI), root mean square error of approximation (RMSEA) and standardised root mean square residual (SRMR). CFI and TLI values >0.90 are an indication of good fit. RMSEA values <0.10 suggest a moderate fit; values <0.05 are a good fit [26]. The SRMR value should be under 0.08 [26]. Unidimensionality is assumed whenever at least three of four parameters produce good values.

  • Item response theory (IRT) analyses: The one-parameter IRT model (Rasch model, Bond and Fox [27]; software: WINSTEPS version 3.68) was used. Infit and outfit mean square statistics (Infit MNSQ, Outfit MNSQ) were applied as goodness-of-fit statistics. Poor item fit was defined as infit or outfit <0.6 or >1.4 ([27], p. 179).

  • Reliability: Cronbach’s alpha was calculated.

  • Construct validity: We proposed the hypothesis, that all scales of the HELP correlate significantly and substantially (r > 0.40) with general health literacy. This was measured with the three-item scale by Chew et al. [28] that we translated into German (Cronbach’s alpha in our sample: 0.77). An example of an item in the scale is ‘How confident are you filling out medical forms by yourself?’. Furthermore, we expect moderate correlations (about r = 0.30) between the HELP scales and the corresponding COHEP scales as comprehensibility is an ‘amalgam’ [29], that is defined not only by the features of the health education programme but also by the patient’s HEL.

Predictors of HEL

We considered age, sex, partner status, highest level of education, employment status, income, the duration of disease and the self-assessment of health status (one-item measure: ‘How would you rate your health?’) as potential predictors. The first step involved determining bivariate correlations between the predictors and the HELP scales. We eliminated those predictors that correlated weakly with the HELP scales (P > 0.20, corresponding to a correlation of <0.06). With this restriction, more sparse models could be specified and problems of multicollinearity avoided. As the HELP scale’s values were not distributed normally, we carried out both linear regression and ordinal regression analyses. We interpret only those results that concurred with both analyses. A separate model was specified for each of the HELP scales. For the diagnosis of multicollinearity, the variance inflation factor (VIF) was calculated. Values over 5 can be considered as an indication of multicollinearity.

Results

Psychometric properties

Eight of the original 26 items were abandoned as they revealed ceiling effects (50–60% of values are in the most positive category ‘no difficulty’). We identified a three-factor solution with the remaining items after doing an exploratory factor analysis. We call those factors ‘comprehending medical information’ (CMI), ‘applying medical information’ (AMI) and ‘communicative competence in provider interactions’ (CCP). The content of these factors corresponds well with the COHEP scales ‘comprehendsible medical information’, ‘transferability to everyday life’ and ‘comprehension-fostering behaviour of programme trainers’.

Table II illustrates various characteristics of these scales. All three are very reliable (Cronbach’s alpha between 0.88 and 0.95) and unidimensional. Only the RMSEA values in the CMI and AMI scales were not entirely satisfactory; the CFI, TLI and SRMR values proved good, however. The standardized factor loadings generally lie above 0.68. All the items fulfil the Rasch model requirements. Just one item in the AMI scale (‘…applying the medical advice you have received at home on a daily basis’) exceeds the cutoff-value for infit and outfit values narrowly (1.45 and 1.47). We decided to keep this item because we consider its content important and other values (factor loading, item scale correlation) proved satisfactory. The correlations between the HELP scales and the Chew scale were r = 0.58 (CMI), r = 0.45 (AMI) and r = 0.35 (CCP). The correlations between the HELP scales and individual Chew items lay between ρ = 0.38 and 0.61 (CMI), ρ = 0.26 and 0.48 (AMI) and ρ = 0.22 and 0.41 (CCP). Those between the HELP and COHEP scales ranged from r = 0.33 to 0.35. Table III presents the intercorrelations of the HELP scales; they lay between r = 0.64 and 0.76.

Table II.

Scale mean, reliability, range of corrected item scale correlation, unidimensionality and factor loadings of HELP scales

ScaleScale mean (SD, range)Cronbach‘s alphaRange of corrected item scale correlationUnidimensionalityRange of standardized factor loadings
Comprehension of medical information (six items)2.2 (0.84, 1.0–4.7)0.940.75–0.84CFI = 0.9900.84–0.91
TLI = 0.994
RMSEA = 0.162
SRMR = 0.029
Applying medical information (five items)1.8 (0.73; 1.0–4.0)0.880.60–0.80CFI = 0.9920.69–0.92
TLI = 0.994
RMSEA = 0.103
SRMR = 0.022
Communicative competence in provider interactions (seven items)1.9 (0.86; 1.0–4.9)0.950.77–0.86CFI = 0.9950.85–0.93
TLI = 0.998
RMSEA = 0.092
SRMR = 0.016
ScaleScale mean (SD, range)Cronbach‘s alphaRange of corrected item scale correlationUnidimensionalityRange of standardized factor loadings
Comprehension of medical information (six items)2.2 (0.84, 1.0–4.7)0.940.75–0.84CFI = 0.9900.84–0.91
TLI = 0.994
RMSEA = 0.162
SRMR = 0.029
Applying medical information (five items)1.8 (0.73; 1.0–4.0)0.880.60–0.80CFI = 0.9920.69–0.92
TLI = 0.994
RMSEA = 0.103
SRMR = 0.022
Communicative competence in provider interactions (seven items)1.9 (0.86; 1.0–4.9)0.950.77–0.86CFI = 0.9950.85–0.93
TLI = 0.998
RMSEA = 0.092
SRMR = 0.016

Scale range is 1–5 for all scales, lower values indicate higher HEL (1 = no difficulty, 2 = slight difficulty, 3 = moderate difficulty, 4 = serious difficulty, 5 = extreme difficulty).

Table II.

Scale mean, reliability, range of corrected item scale correlation, unidimensionality and factor loadings of HELP scales

ScaleScale mean (SD, range)Cronbach‘s alphaRange of corrected item scale correlationUnidimensionalityRange of standardized factor loadings
Comprehension of medical information (six items)2.2 (0.84, 1.0–4.7)0.940.75–0.84CFI = 0.9900.84–0.91
TLI = 0.994
RMSEA = 0.162
SRMR = 0.029
Applying medical information (five items)1.8 (0.73; 1.0–4.0)0.880.60–0.80CFI = 0.9920.69–0.92
TLI = 0.994
RMSEA = 0.103
SRMR = 0.022
Communicative competence in provider interactions (seven items)1.9 (0.86; 1.0–4.9)0.950.77–0.86CFI = 0.9950.85–0.93
TLI = 0.998
RMSEA = 0.092
SRMR = 0.016
ScaleScale mean (SD, range)Cronbach‘s alphaRange of corrected item scale correlationUnidimensionalityRange of standardized factor loadings
Comprehension of medical information (six items)2.2 (0.84, 1.0–4.7)0.940.75–0.84CFI = 0.9900.84–0.91
TLI = 0.994
RMSEA = 0.162
SRMR = 0.029
Applying medical information (five items)1.8 (0.73; 1.0–4.0)0.880.60–0.80CFI = 0.9920.69–0.92
TLI = 0.994
RMSEA = 0.103
SRMR = 0.022
Communicative competence in provider interactions (seven items)1.9 (0.86; 1.0–4.9)0.950.77–0.86CFI = 0.9950.85–0.93
TLI = 0.998
RMSEA = 0.092
SRMR = 0.016

Scale range is 1–5 for all scales, lower values indicate higher HEL (1 = no difficulty, 2 = slight difficulty, 3 = moderate difficulty, 4 = serious difficulty, 5 = extreme difficulty).

Table III.

Scale intercorrelations

Comprehension of medical informationApplying medical information
Applying medical information0.73
Communicative competence0.640.76
Comprehension of medical informationApplying medical information
Applying medical information0.73
Communicative competence0.640.76

Pearson correlations, Nmin = 547, P < 0.001 for all correlations.

Table III.

Scale intercorrelations

Comprehension of medical informationApplying medical information
Applying medical information0.73
Communicative competence0.640.76
Comprehension of medical informationApplying medical information
Applying medical information0.73
Communicative competence0.640.76

Pearson correlations, Nmin = 547, P < 0.001 for all correlations.

HEL level

Our patients revealed a high level of self-reported HEL. Table IV shows that the average patient responded to the skills addressed in the AMI and CCP scales with ‘slight difficulty’. They responded with ‘slight to moderate difficulty’ in the CMI scale. Table IV demonstrates that every fourth to fifth patient admits to having difficulty (moderate, serious or extreme) with individual skills. This percentage rises to 42% when the comprehension of medical terminology is involved. Patients, 5–14%, display severely limited health literacy as they appear to have obvious difficulty with all the items in each category.

Table IV.

Items of the HELP questionnaire (in order of mean value within scales)

ItemMean (SD)Percentage of persons with difficultiesPercentage of persons with difficulties with all aspects of the scale
How much difficulty do you have in talks with doctors, therapists or nursing staff …
Comprehension of medical information13.7
    … absorbing the amount of new information1.87 (0.83)21.2
    … understanding complex sentences2.12 (0.97)33.0
    … also understanding difficult medical information2.19 (0.99)35.4
    … understanding medical information immediately2.26 (0.91)36.6
    … understanding foreign words2.27 (1.04)37.5
    … understanding medical terminology2.43 (1.06)42.4
Applying medical information4.9
    … distinguishing essential from less important information1.69 (0.80)15.0
    … understanding medical information given to you1.73 (0.84)18.0
    … understanding what the medical information means in terms of your disease1.79 (0.94)20.1
    … understanding the wealth of information conveyed1.83 (0.89)21.0
    … applying the medical advice you have received at home on a daily basis1.83 (0.92)22.1
Communicative competence8.7
    … communicating what you already know and don’t know about your disease to doctors, therapists and nursing staff1.79 (0.93)19.7
    … communicating your own expectations and wishes in terms of your therapy1.86 (0.97)23.8
    … posing very personal questions about your disease1.86 (0.99)22.8
    … approaching staff when a problem of yours has not been adequately addressed1.88 (1.01)24.3
    … making it clear to doctors, therapists and nursing staff how important your questions are for you1.90 (1.01)24.2
    … addressing your own problems and issues1.91 (0.99)26.3
    … talking about your questions2.00 (1.05)29.7
ItemMean (SD)Percentage of persons with difficultiesPercentage of persons with difficulties with all aspects of the scale
How much difficulty do you have in talks with doctors, therapists or nursing staff …
Comprehension of medical information13.7
    … absorbing the amount of new information1.87 (0.83)21.2
    … understanding complex sentences2.12 (0.97)33.0
    … also understanding difficult medical information2.19 (0.99)35.4
    … understanding medical information immediately2.26 (0.91)36.6
    … understanding foreign words2.27 (1.04)37.5
    … understanding medical terminology2.43 (1.06)42.4
Applying medical information4.9
    … distinguishing essential from less important information1.69 (0.80)15.0
    … understanding medical information given to you1.73 (0.84)18.0
    … understanding what the medical information means in terms of your disease1.79 (0.94)20.1
    … understanding the wealth of information conveyed1.83 (0.89)21.0
    … applying the medical advice you have received at home on a daily basis1.83 (0.92)22.1
Communicative competence8.7
    … communicating what you already know and don’t know about your disease to doctors, therapists and nursing staff1.79 (0.93)19.7
    … communicating your own expectations and wishes in terms of your therapy1.86 (0.97)23.8
    … posing very personal questions about your disease1.86 (0.99)22.8
    … approaching staff when a problem of yours has not been adequately addressed1.88 (1.01)24.3
    … making it clear to doctors, therapists and nursing staff how important your questions are for you1.90 (1.01)24.2
    … addressing your own problems and issues1.91 (0.99)26.3
    … talking about your questions2.00 (1.05)29.7

Scale range is 1–5 for all items, lower values indicate higher HEL (1 = no difficulty, 2 = slight difficulty, 3 = moderate difficulty, 4 = serious difficulty, 5 = extreme difficulty). ‘Percentage of persons with difficulties’ indicates the percentage of patients who rate the difficulty as ‘moderate’, ‘serious’ or ‘extreme’.

Table IV.

Items of the HELP questionnaire (in order of mean value within scales)

ItemMean (SD)Percentage of persons with difficultiesPercentage of persons with difficulties with all aspects of the scale
How much difficulty do you have in talks with doctors, therapists or nursing staff …
Comprehension of medical information13.7
    … absorbing the amount of new information1.87 (0.83)21.2
    … understanding complex sentences2.12 (0.97)33.0
    … also understanding difficult medical information2.19 (0.99)35.4
    … understanding medical information immediately2.26 (0.91)36.6
    … understanding foreign words2.27 (1.04)37.5
    … understanding medical terminology2.43 (1.06)42.4
Applying medical information4.9
    … distinguishing essential from less important information1.69 (0.80)15.0
    … understanding medical information given to you1.73 (0.84)18.0
    … understanding what the medical information means in terms of your disease1.79 (0.94)20.1
    … understanding the wealth of information conveyed1.83 (0.89)21.0
    … applying the medical advice you have received at home on a daily basis1.83 (0.92)22.1
Communicative competence8.7
    … communicating what you already know and don’t know about your disease to doctors, therapists and nursing staff1.79 (0.93)19.7
    … communicating your own expectations and wishes in terms of your therapy1.86 (0.97)23.8
    … posing very personal questions about your disease1.86 (0.99)22.8
    … approaching staff when a problem of yours has not been adequately addressed1.88 (1.01)24.3
    … making it clear to doctors, therapists and nursing staff how important your questions are for you1.90 (1.01)24.2
    … addressing your own problems and issues1.91 (0.99)26.3
    … talking about your questions2.00 (1.05)29.7
ItemMean (SD)Percentage of persons with difficultiesPercentage of persons with difficulties with all aspects of the scale
How much difficulty do you have in talks with doctors, therapists or nursing staff …
Comprehension of medical information13.7
    … absorbing the amount of new information1.87 (0.83)21.2
    … understanding complex sentences2.12 (0.97)33.0
    … also understanding difficult medical information2.19 (0.99)35.4
    … understanding medical information immediately2.26 (0.91)36.6
    … understanding foreign words2.27 (1.04)37.5
    … understanding medical terminology2.43 (1.06)42.4
Applying medical information4.9
    … distinguishing essential from less important information1.69 (0.80)15.0
    … understanding medical information given to you1.73 (0.84)18.0
    … understanding what the medical information means in terms of your disease1.79 (0.94)20.1
    … understanding the wealth of information conveyed1.83 (0.89)21.0
    … applying the medical advice you have received at home on a daily basis1.83 (0.92)22.1
Communicative competence8.7
    … communicating what you already know and don’t know about your disease to doctors, therapists and nursing staff1.79 (0.93)19.7
    … communicating your own expectations and wishes in terms of your therapy1.86 (0.97)23.8
    … posing very personal questions about your disease1.86 (0.99)22.8
    … approaching staff when a problem of yours has not been adequately addressed1.88 (1.01)24.3
    … making it clear to doctors, therapists and nursing staff how important your questions are for you1.90 (1.01)24.2
    … addressing your own problems and issues1.91 (0.99)26.3
    … talking about your questions2.00 (1.05)29.7

Scale range is 1–5 for all items, lower values indicate higher HEL (1 = no difficulty, 2 = slight difficulty, 3 = moderate difficulty, 4 = serious difficulty, 5 = extreme difficulty). ‘Percentage of persons with difficulties’ indicates the percentage of patients who rate the difficulty as ‘moderate’, ‘serious’ or ‘extreme’.

Predictors of HEL

The VIF values remained below 1.3, so there is no indication of multicollinearity. The predictors we examined only explain 4–8% of the variance in HEL. Depending on the HELP scale, different sociodemographic variables proved to be predictors (Table V):

  • The comprehension of medical information is lower in patients with a lower educational level.

  • Younger patients tend to rate their HEL lower when AMI.

Table V.

Predictors of health literacy

Comprehension of medical information
Applying medical information
Communicative competence
Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)
Age−0.127 (0.009)−0.020 (0.021)
Level of education
    Elementary school0.118 (0.020)0.199 (0.027)
    University-entrance diploma or technical college qualification−0.111 (0.028)
Income−0.094 (0.044)
Health status
    Very good or good−0.689 (0.029)−0.816 (0.011)−1.135 (<0.001)
    Satisfactory−0.590 (0.030)−0.742 (0.005)
    Not good0.112 (0.023)
    Bad or very bad0.121 (0.011)0.148 (0.002)0.170 (0.001)
Chronification
    5–10 years0.097 (0.042)
R/corrected R20.253 (0.056)0.232 (0.043)0.259 (0.059)
Nagelkerkes R20.0810.0520.060
Comprehension of medical information
Applying medical information
Communicative competence
Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)
Age−0.127 (0.009)−0.020 (0.021)
Level of education
    Elementary school0.118 (0.020)0.199 (0.027)
    University-entrance diploma or technical college qualification−0.111 (0.028)
Income−0.094 (0.044)
Health status
    Very good or good−0.689 (0.029)−0.816 (0.011)−1.135 (<0.001)
    Satisfactory−0.590 (0.030)−0.742 (0.005)
    Not good0.112 (0.023)
    Bad or very bad0.121 (0.011)0.148 (0.002)0.170 (0.001)
Chronification
    5–10 years0.097 (0.042)
R/corrected R20.253 (0.056)0.232 (0.043)0.259 (0.059)
Nagelkerkes R20.0810.0520.060

The polarity is such that predictors with positive coefficients are risk factors for low health literacy. Only significant predictors (P < 0.05) are shown.

Table V.

Predictors of health literacy

Comprehension of medical information
Applying medical information
Communicative competence
Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)
Age−0.127 (0.009)−0.020 (0.021)
Level of education
    Elementary school0.118 (0.020)0.199 (0.027)
    University-entrance diploma or technical college qualification−0.111 (0.028)
Income−0.094 (0.044)
Health status
    Very good or good−0.689 (0.029)−0.816 (0.011)−1.135 (<0.001)
    Satisfactory−0.590 (0.030)−0.742 (0.005)
    Not good0.112 (0.023)
    Bad or very bad0.121 (0.011)0.148 (0.002)0.170 (0.001)
Chronification
    5–10 years0.097 (0.042)
R/corrected R20.253 (0.056)0.232 (0.043)0.259 (0.059)
Nagelkerkes R20.0810.0520.060
Comprehension of medical information
Applying medical information
Communicative competence
Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)Linear Regression: standardized coefficient (P-value)Ordinal Regression: parameter estimate (P-value)
Age−0.127 (0.009)−0.020 (0.021)
Level of education
    Elementary school0.118 (0.020)0.199 (0.027)
    University-entrance diploma or technical college qualification−0.111 (0.028)
Income−0.094 (0.044)
Health status
    Very good or good−0.689 (0.029)−0.816 (0.011)−1.135 (<0.001)
    Satisfactory−0.590 (0.030)−0.742 (0.005)
    Not good0.112 (0.023)
    Bad or very bad0.121 (0.011)0.148 (0.002)0.170 (0.001)
Chronification
    5–10 years0.097 (0.042)
R/corrected R20.253 (0.056)0.232 (0.043)0.259 (0.059)
Nagelkerkes R20.0810.0520.060

The polarity is such that predictors with positive coefficients are risk factors for low health literacy. Only significant predictors (P < 0.05) are shown.

As the relationship between low income and less communicative competence is only significant in linear regression analysis, we did not interpret that result.

Discussion and conclusion

Discussion

Psychometric properties and applicability of the HELP questionnaire

The HELP questionnaire consists of 18 items that economically capture the HEL of CMD patients. The instruments’ psychometric properties are good: reliability is high, there is evidence of unidimensionality, and the fit to the one-parameter IRT model reveals various methodological advantages that are not guaranteed in the health literacy questionnaires thus far available. However, there are only initial indications of the questionnaire’s validity. The correlations with COHEP occurred as anticipated, yet the relationships to the Chew scale fall into the expected range only in the AMI and CMI scales. Moreover, it merits consideration as to whether the brief and not very reliable Chew scale is well-suited for testing the construct validity of HELP—further analyses are thus necessary to assess its validity.

The HELP questionnaire's three scales correlate with each other quite closely. This is in accordance with expectation as CMI is a prerequisite for applying it. As the intercorrelations did not exceed 0.76, we assume, however, that separate concepts are being measured.

We have attempted to ensure content validity by covering all of the COHEP’s contents, which are based on the feedback from patients and providers in focus groups. However, due to ceiling effects, eight of the original 26 items had to be removed. Table VI presents the contents of the items we had to remove and for each, an item we kept with similar content. Face validity shows that no essential aspect is lost.

Table VI.

Items eliminated due to ceiling effects and allocation to items we kept with similar content

Item eliminatedKept items with similar content
… posing questions… talking about your questions
… participating actively in a discussion… addressing your own problems and issues
… understanding the purpose of and reasons behind a planned therapeutic intervention… understanding what the medical information means in terms of your disease
… taking the really important results from a discussion home… applying the medical advice you have received at home on a daily basis
… asking for help when you don’t understand something… approaching staff when a problem of yours has not been adequately addressed
… taking home something really new… applying the medical advice you have received at home on a daily basis
… talking about your experiences with your illness… addressing your own problems and issues
… applying the information you have learned to your own situation… applying the medical advice you have received at home on a daily basis
Item eliminatedKept items with similar content
… posing questions… talking about your questions
… participating actively in a discussion… addressing your own problems and issues
… understanding the purpose of and reasons behind a planned therapeutic intervention… understanding what the medical information means in terms of your disease
… taking the really important results from a discussion home… applying the medical advice you have received at home on a daily basis
… asking for help when you don’t understand something… approaching staff when a problem of yours has not been adequately addressed
… taking home something really new… applying the medical advice you have received at home on a daily basis
… talking about your experiences with your illness… addressing your own problems and issues
… applying the information you have learned to your own situation… applying the medical advice you have received at home on a daily basis
Table VI.

Items eliminated due to ceiling effects and allocation to items we kept with similar content

Item eliminatedKept items with similar content
… posing questions… talking about your questions
… participating actively in a discussion… addressing your own problems and issues
… understanding the purpose of and reasons behind a planned therapeutic intervention… understanding what the medical information means in terms of your disease
… taking the really important results from a discussion home… applying the medical advice you have received at home on a daily basis
… asking for help when you don’t understand something… approaching staff when a problem of yours has not been adequately addressed
… taking home something really new… applying the medical advice you have received at home on a daily basis
… talking about your experiences with your illness… addressing your own problems and issues
… applying the information you have learned to your own situation… applying the medical advice you have received at home on a daily basis
Item eliminatedKept items with similar content
… posing questions… talking about your questions
… participating actively in a discussion… addressing your own problems and issues
… understanding the purpose of and reasons behind a planned therapeutic intervention… understanding what the medical information means in terms of your disease
… taking the really important results from a discussion home… applying the medical advice you have received at home on a daily basis
… asking for help when you don’t understand something… approaching staff when a problem of yours has not been adequately addressed
… taking home something really new… applying the medical advice you have received at home on a daily basis
… talking about your experiences with your illness… addressing your own problems and issues
… applying the information you have learned to your own situation… applying the medical advice you have received at home on a daily basis

We would like to bring the content congruence between the HELP and COHEP questionnaires to the reader’s attention. When both instruments are used in an institution, we obtain a more comprehensive perspective of the patient’s health literacy situation within a socially ecological framework [30]. From the institution’s perspective, a combination of low comprehensibility of educational interventions and a patient’s high HEL would be especially problematic, as that would reveal that the patient believes he or she generally understands medical information well but has difficulty understanding something about the institution’s educational intervention.

As we see it, the HELP questionnaire can be used in research on health literacy in CMD patients, as an outcome indicator when assessing health education programmes and (together with COHEP) in quality management activities, when an institution wishes to identify how successful its personnel is in making health education programmes understandable and useful to patients. A diagnostic application is also conceivable to capture the preconditions a patient population brings to a given health education intervention. Most authors consider, as do we, health literacy (i.e. [31]) not as a trait, but as a state, as it depends on context factors and can be modified by intervention (i.e. [32]). Whenever the HELP is employed as a diagnostic instrument it is advisable to use a long temporal frame of reference (e.g. ‘in the past 12 months’) to be able to make an overall assessment based on past experiences. When applying the HELP as an outcome indicator, we recommend using a briefer temporal frame of reference—depending on the duration of the intervention—to allow pre–post comparisons.

Level of HEL

The appraisal of our results regarding the level of HEL in CMD patients is difficult because there are so few studies targeting health literacy in this particular patient group. Briggs et al. [33] used an instrument that merely measured the ability to read and understand health-related materials (S-TOFHLA), observing that low back pain patients possess adequate health literacy. In their later study with a more differentiated instrument [34], seven of eight health literacy domains revealed no difference between patients with and those without back pain. On one hand, our results are in line with their findings, as the average patient in our sample reported no major health literacy problems. On the other hand, a not inconsiderable proportion of 20–30% of our patients admitted to having at least moderate difficulty with key aspects in health education programmes. As this percentage nearly corresponds to the prevalence of low health literacy in the general European and US populations [5, 7, 8], our cohort probably has a low health literacy problem as well.

Predictors of HEL

Our hypothesis that HEL can be predicted by education, financial status, age and sex was confirmed in part. Sex and financial status were not relevant, whereas age and education were relevant in only one of the three HELP scales. Sociodemographic predictors only became apparent in the CMI and AMI scales, not in the CCP. The observation that communicative skills (as in CCP scale) are less influenced by sociodemographic variables than basic literacy skills (i.e. understanding foreign words or technical terminology) is in line with the result that the acquisition of communicative skills depends on factors beyond cognitive and educational variables, such as emotional, personality and temperament factors [35].

Previous studies reporting an influence of education on health literacy have generally used simple reading and comprehension tests as dependent variables. We replicated that effect in the HELP’s CMI scale measuring the comprehension of medical information. The AMI scale’s results demonstrate, however, that education does not necessarily also serve to predict more complex health literacy skills. In our study, age but not educational level was associated with perceived competence in AMI. The closest relationship between sociodemographic variables and HEL that we observed was that between age and the AMI scale. Surprisingly, the age influence we noted went in a different direction than that reported by other working groups (i.e. [23, 24]): our older patients showed higher HEL in this regard, even after adjusting for the disease duration. This result may have to do with the fact that the patient must possess self-knowledge and everyday wisdom to apply medical information personally in his or her daily routine [36]. These factors are partially independent of educational level and may rise with age. An alternative explanation is that younger persons have more difficulty applying the information transmitted in health education programmes on an everyday basis because they tend to have more work-related and family responsibilities than older individuals, many of whom are retired.

The most important risk factor for low HEL in our data was a poor self-rated health status. That risk factor was revealed in all three HELP scales. There are several possible explanations for this finding: perhaps, poor health makes it more difficult to apply information and to communicate effectively with treatment providers. It is, however, less plausible that this relationship—see our data—would also be revealed in terms of understanding medical information (the CMI scale). We therefore consider it more likely that we are observing a causal path in the opposite direction: that chronically ill patients with low health literacy are at a disadvantage and thus experience a less positive disease course. We also may be observing the influence of another, third variable (i.e. depression or low self-efficacy), which would exert a negative influence on both one’s own health literacy and the self-reported health status.

Limitations

The HELP questionnaire is not an objective test of health literacy, rather it is a self-assessment instrument; thus, its results are subject to response biases. However, the use of economical, objective tests to measure complex aspects of health literacy (i.e. communication skills) is quite unfeasible. The HELP's contents do not appear to apply specifically to patients with back pain and osteoarthritis; thus, we can assume that it can be used in other patients with musculoskeletal diseases as well. This would, however, require empirical examination. Furthermore, the instrument has only been applied in German rehabilitation centres so far; thus, we know nothing about its generalizability to other populations. The HELP's factorial structure has been studied exploratively and should be validated in an independent sample. Another limitation is that we conducted cognitive pretests only with the COHEP items, not with the HELP items. Nevertheless, the low missing value rates indicate the HELP's acceptance and comprehensibility.

We included only sociodemographic and several basic medical predictors of HEL, not psychological variables such as depression or self-efficacy. Moreover, we applied a cross-sectional design. These two factors mean that our ability to elucidate causal paths between influencing factors and HEL is extremely limited. Finally, there are problems associated with assessing health status with a one-item measure. Ideal would be the replication of our findings by using a broader measuring instrument (e.g. SF-36).

Conclusion

Regardless of the aforementioned limitations, our study reveals several relevant findings. There are now available well-tested instruments (HELP and COHEP questionnaire) with which HEL can be assessed in the context of research, evaluation and quality management. However, there is only a psychometrically tested German-language version, and the English-language items in Table IV are simply translations of the tested German items.

Patients, 20–30%, that we queried admitted to having problems with certain key aspects of health education programmes. This indicates that greater effort is required to make health education programmes easier to understand, especially for this subgroup of patients. One means of doing this would be to design and carry out targeted interventions for patients with low health literacy (i.e. [32]). But a study by Pignone et al. [37] has demonstrated that those with sufficient health literacy can also benefit from such education programmes. Furthermore, defining certain target groups can run the risk of stigmatisation. We therefore developed a basic health education programme that takes place at the beginning of rehabilitation and is tailored for all CMD patients. This programme is currently being implemented and evaluated in six orthopaedic rehabilitation centres. Basic concepts of rehabilitation, of the treatment of chronic disease, and patient skills in communication with medical staff are conveyed in this training called ‘Active in rehab: Patient education to enhance health literacy’. The programme aims to enable the participant to get more out of the upcoming rehabilitation and of future educational interventions.

Our results with HEL predictors show that there is a need for more research on interindividual variability of complex aspects of health literacy. Relatively little variance can be attributed to education and other basic sociodemographic variables. Future research efforts should focus more intensively on the social, psychological and medical influencing factors of health literacy.

Acknowledgements

We wish to thank the cooperating rehabilitation centres for their support in data collection: Asklepios Klinik (Schaufling), Breisgau-Klinik (Bad Krozingen), Hedon Klinik (Lingen), MediClin Klinik am Brunnenberg (Bad Elster), MEDIAN Klinik Bad Sülze (Bad Sülze), Orthopädische Klinik ‘Kurköln’ (Bad Neuenahr), Paracelsus Klinik an der Gande (Bad Gandersheim), RehaKlinikum Bad Säckingen GmbH (Bad Säckingen), Salze Klinik (Bad Salzdetfurth).

Funding

This work was conducted in the Project ‘Development and evaluation of a health education program for enhancing health literacy of chronically ill persons’ which is funded in Germany by the Federal Ministry of Education and Research (Grant No.: 01GX1020) as part of the funding priority for ‘Chronic Illnesses and Patient Orientation’ (http://www.forschung-patientenorientierung.de).

Conflict of interest statement

None declared.

References

1
World Health Organization
Health Promotion Glossary
1988
Geneva
World Health Organization
2
Berkman
ND
Sheridan
SL
Donahue
KE
, et al. 
Low health literacy and health outcomes: an updated systematic review
Ann Intern Med
2011
, vol. 
155
 (pg. 
97
-
107
)
3
Osborn
CY
Paasche-Orlow
MK
Bailey
SC
, et al. 
The mechanisms linking health literacy to behavior and health status
Am J Health Behav
2011
, vol. 
35
 (pg. 
118
-
25
)
4
Shea
JA
Guerra
CE
Ravenell
KL
, et al. 
Health literacy weakly but consistently predicts primary care patient dissatisfaction
Int J Qual Health Care
2007
, vol. 
19
 (pg. 
45
-
9
)
5
Bostock
S
Steptoe
A
Association between low functional health literacy and mortality in older adults: longitudinal cohort study
Br Med J
2012
, vol. 
344
 pg. 
e1602
 
6
Hasnain-Wynia
R
Wolf
MS
Promoting health care equity: is health literacy a missing link?
Health Serv Res
2010
, vol. 
45
 (pg. 
897
-
903
)
7
White
S
Assessing the Nation's Health Literacy: Key Concepts and Findings of the National Assessment of Adult Literacy (NAAL)
2008
Chicago, IL
American Medical Association
8
HLS-EU Consortium
Comparative Report of Health Literacy in Eight EU Member States
 
The European Health Literacy Survey HLS-EU. Available at: http://www.health-literacy.eu. Accessed: 24 September 2013
9
Nutbeam
D
The evolving concept of health literacy
Soc Sci Med
2008
, vol. 
67
 (pg. 
2072
-
8
)
10
Frisch
AL
Camerini
L
Diviani
N
, et al. 
Defining and measuring health literacy: how can we profit from other literacy domains?
Health Promot Int
2012
, vol. 
27
 (pg. 
117
-
26
)
11
Mabiso
A
Williams
KP
Todem
D
, et al. 
Longitudinal analysis of domain-level breast cancer literacy among African-American women
Health Educ Res
2010
, vol. 
25
 (pg. 
151
-
61
)
12
Kim
MT
Song
HJ
Han
HR
, et al. 
Development and validation of the high blood pressure-focused health literacy scale
Patient Educ Couns
2012
, vol. 
87
 (pg. 
165
-
70
)
13
Collins
SA
Currie
LM
Bakken
S
, et al. 
Health literacy screening instruments for eHealth applications: a systematic review
J Biomed Inform
2012
, vol. 
45
 (pg. 
598
-
607
)
14
Farin
E
Nagl
M
Ullrich
A
The comprehensibility of health education programs: questionnaire development and results in patients with chronic musculo-skeletal diseases
Patient Educ Couns
2013
, vol. 
90
 (pg. 
239
-
46
)
15
Jordan
JE
Buchbinder
R
Osborne
RH
Conceptualising health literacy from the patient perspective
Patient Educ Couns
2010
, vol. 
79
 (pg. 
36
-
42
)
16
Ruger
JP
Health capability: conceptualization and operationalization
Am J Public Health
2010
, vol. 
100
 (pg. 
41
-
9
)
17
van der Vaart
R
Drossaert
CHC
Taal
E
, et al. 
Validation of the Dutch functional, communicative and critical health literacy scales
Patient Educ Couns
2012
, vol. 
89
 (pg. 
82
-
8
)
18
Weis
J
Giesler
JM
Subjective dimensions of patient competence: relationships with selected healthcare usage behaviors and general features of self-rated competence
Patient Educ Couns
2008
, vol. 
73
 (pg. 
511
-
8
)
19
Kristjansson
E
Tugwell
PS
Wilson
AJ
, et al. 
Development of the effective musculoskeletal consumer scale
J Rheumatol
2007
, vol. 
34
 (pg. 
1392
-
400
)
20
Jordan
JE
Osborne
RH
Buchbinder
R
Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses
J Clin Epidemiol
2011
, vol. 
64
 (pg. 
366
-
79
)
21
Jeppesen
KM
Coyle
JD
Miser
WF
Screening questions to predict limited health literacy: a cross-sectional study of patients with diabetes mellitus
Ann Fam Med
2009
, vol. 
7
 (pg. 
24
-
31
)
22
Ayotte
BJ
Allaire
JC
Bosworth
H
The associations of patient demographic characteristics and health information recall: the mediating role of health literacy
Neuropsychol Dev Cogn B Aging Neuropsychol Cogn
2009
, vol. 
16
 (pg. 
419
-
32
)
23
Martin
LT
Ruder
T
Escarce
J
, et al. 
Developing predictive models of health literacy
J Gen Intern Med
2009
, vol. 
24
 (pg. 
1211
-
6
)
24
Wagner
Cv
Knight
K
Steptoe
A
, et al. 
Functional health literacy and health-promoting behaviour in a national sample of British adults
J Epidemiol Community Health
2007
, vol. 
61
 (pg. 
1086
-
90
)
25
O'Halloran
J
Miller
GC
Britt
H
Defining chronic conditions for primary care with ICPC-2
Fam Pract
2004
, vol. 
21
 (pg. 
381
-
6
)
26
Hu
LT
Bentler
P
Cutoff criteria for fit indices in covariance structure analysis: conventional criteria versus new alternatives
Struct Equ Modeling
1999
, vol. 
6
 (pg. 
1
-
55
)
27
Bond
TG
Fox
CM
Applying the Rasch Model: Fundamental Measurement in the Human Sciences
2001
Mahwah, NJ
Lawrence Erlbaum Associates
28
Chew
LD
Bradley
KA
Boyko
EJ
Brief questions to identify patients with inadequate health literacy
Fam Med
2004
, vol. 
36
 (pg. 
588
-
94
)
29
Luk
A
Aslani
P
Tools used to evaluate written medicine and health information
Health Educ Behav
2011
, vol. 
38
 (pg. 
389
-
403
)
30
Berkman
ND
Davis
TC
McCormack
L
Health literacy: what is it?
J Health Commun
2010
, vol. 
15
 (pg. 
9
-
19
)
31
Peterson
PN
Shetterly
SM
Clarke
CL
, et al. 
Health literacy and outcomes among patients with heart failure
JAMA
2011
, vol. 
305
 (pg. 
1695
-
701
)
32
Sheridan
SL
Halpern
DJ
Viera
AJ
, et al. 
Interventions for individuals with low health literacy: a systematic review
J Health Commun
2011
, vol. 
16
 
Suppl. 3
(pg. 
30
-
54
)
33
Briggs
AM
Jordan
JE
Buchbinder
R
, et al. 
Health literacy and beliefs among a community cohort with and without chronic low back pain
Pain
2010
, vol. 
150
 (pg. 
275
-
83
)
34
Briggs
A
Jordan
J
O'Sullivan
P
, et al. 
Individuals with chronic low back pain have greater difficulty in engaging in positive lifestyle behaviours than those without back pain: an assessment of health literacy
BMC Musculoskelet Disord
2011
, vol. 
12
 pg. 
161
 
35
Beauchamp
MH
Anderson
V
SOCIAL: an integrative framework for the development of social skills
Psychol Bull
2010
, vol. 
136
 (pg. 
39
-
64
)
36
Staudinger
UM
Glück
J
Psychological wisdom research: commonalities and differences in a growing field
Annu Rev Psychol
2011
, vol. 
62
 (pg. 
215
-
41
)
37
Pignone
M
Dewalt
DA
Sheridan
S
, et al. 
Interventions to improve health outcomes for patients with low literacy
J Gen Intern Med
2005
, vol. 
20
 (pg. 
185
-
92
)