Why organizations continue to create patient information leaflets with readability and usability problems: an exploratory study

Abstract

Readability and usability problems with patient information leaflets continue to be reported despite long-standing recognition of their existence and the availability of guidelines for developing health education materials. This exploratory study examined possible causes for such problems, based on interviews with professionals who developed leaflets in large health organizations. Findings suggest that readability is an important concern for developers, but that complex organizational processes are involved in creating leaflets, and that developers face a need to cope with organizational politics, goal conflicts and various other pressures. Six factors were identified, i.e. (1) initiators, (2) intended users and contexts of use, (3) goals, (4) work flow and content decisions, (5) readability considerations, and (6) evaluation practices, that can adversely affect the content, organization, and resulting comprehensibility and usability of leaflets. It is suggested to adopt a broad ecological view of the environments in which patient education materials are created and deployed. Implications for practice and for future related research are discussed.

Introduction

Over the last few decades a voluminous literature has developed regarding the use and characteristics of printed health education materials or ‘patient information leaflets’ (PILs) (Kenny et al., 1998). PILs are widely used by diverse health organizations and professionals as part of patient education or health promotion efforts, in support of preventive, treatment and compliance objectives. Dixon-Woods (Dixon-Woods, 2000), for example, reported that two national databases in Scotland and England together catalogued a total of 4894 health-related PILs produced by 1013 different organizations.

Research has exposed problems with the readability and usability of a wide range of PILs on diverse matters, such as diet and health (Dollahite et al., 1996), cancer prevention and treatment (Glazer et al., 1996), postoperative instructions (Clement and Wales, 2004), AIDS prevention (Johnson et al., 1997), consent forms (Ott and Hardie, 1997) or pharmacy drug leaflets (Kirksey et al., 2004). The overall picture that can be drawn from the hundreds of studies published over several decades (Zorn and Ratzan, 2000) is that a majority of PILs, regardless of their topic, require relatively high reading skills that may not exist in a large proportion of their target populations. Thus, sizable segments of target audiences are not likely to be able to comprehend important aspects of PILs because of their content, writing style or organization (Payne et al., 2000).

The presence of such problems is perplexing, given that guidelines for the design, writing and testing of readable and user-friendly materials have been available for years (Flesch, 1949; Ley, 1972; Duffy, 1985). Research has documented the benefits of implementing such guidelines, e.g. in terms of better content recall by clients (Ley, 1988; Doak et al., 1996; Thompson et al., 2004). Given the awareness that has developed over several decades to the need to improve readability and usability of PILs, and the existence of guidelines for doing so, it is essential to ask why do organizations continue to produce PILs with problematic characteristics that appear ill-suited for their apparent purpose? Surprisingly, this critical question has not been addressed in the research literature on health education. This study was thus designed to examine factors and processes that affect the content and characteristics of PILs, and that may explain the poor readability reported for many PILs. Understanding such factors and processes can inform future research on health education, improve the effective usage of health services and contribute to clients' health.

We conducted exploratory research using semi-structured interviews with 10 professionals involved in the production of leaflets in diverse settings in order to understand their reasoning and considerations. A qualitative approach was deemed essential given the lack of prior research in this area and followed similar efforts to understand work processes in other areas of health education (Glick et al., 1996; Stevenson, 2001).

The interviews were conducted in Israel, whose health system offers several similarities to those of other countries. In Israel, health services are regulated by a national health law. Four non-profit Health Maintenance Organizations (HMOs) operate under this law and provide many standard services, yet they compete with each other, i.e. clients can switch their service providers. Each HMO holds independent health education initiatives either on a national basis or in regional units. The Ministry of Health is also involved in nationwide health education efforts, sometimes in collaboration with the HMOs.

The primary official language in Israel is Hebrew and most PILs created in Israel appear only in this language. (Some PILs are also translated to Arabic and Russian, which are spoken by about one-quarter of the population.) Of importance is that standard readability formulas cannot be used to analyze texts in Hebrew, because of its use of diacritical marks instead of vowel letters and a system of prefixes and suffixes; together, these render English-based formulas relying on syllable counts or word/sentence length useless as proxies for text difficulty. Given that the use of readability measures has been heavily criticized [e.g. (Duffy, 1985; Bruce and Rubin, 1988)], the examination of PIL creation in organizations that cannot use readability formulas offers an interesting natural case for studying the reasoning of health professionals involved in creating PILs and is also of interest to non English-speaking countries.

Method

Informants

Based on reviewing pertinent literature and on a background interview with a health promoter from a large HMO, we identified four types of positions in which workers may be involved in developing PILs: health promoters, medical personnel, public relations and marketing personnel, and managers and administrators. A purposive sample was identified of 10 professionals who participated within 3 years prior to the interview in the creation of at least one PIL. The informants worked in two HMOs or the Ministry of Health in Israel, as follows: two physicians, three regional health promoters, two national managers of HMO health education units, a director of public relations in an HMO and two officials from the Ministry of Health's Department of Health Education. Leaflets developed directly by the informants addressed a range of topics, such as lower back pain, liver inflammation, well-being of newborns, childhood illnesses, diet and nutrition, cholesterol, cigarette smoking, vaginitis, diabetes, and glaucoma.

Procedure

Informants participated in semi-structured interviews which lasted between 60 and 90 min, and covered three areas chosen based on the literature review and background interview: (1) goals, working processes and decisions regarding the content and design of a specific leaflet the informant developed, (2) formative research used during the development of this leaflet, and (3) organizational perspectives regarding development procedures, research practices and approaches towards writing of PILs in general. Overall, the design of questions provided rich details about the ‘life history’ of the creation of a range of specific leaflets, yet also enabled us to sketch in broad strokes the environments within which PILs are developed.

Results and analysis

Interviewees' statements were content-analyzed and resulting themes were grouped into six broad categories. Three (initiators, users and contexts of use, and goals) involve context and organization factors that cause work on a PIL to begin and shape its general characteristics. Three (work flow and decisions, readability considerations, and evaluation practices) involve factors affecting the development process. However, these factors are not independent and interact in complex ways during the process of creating PILs. Below we first sketch each of the six factors, and then present four ways in which these factors, separately or in interaction, can cause readability and usability problems. This section is supported by the seven composite cases in Table I, created on the basis of the life histories of PILs described by informants; we refer to these cases to illustrate contexts within which new PILs emerge.

Table I.

Examples of contexts for development of PILs and their initiators

Case	Scenario
1	A physician becomes aware of a need to educate his/her local patients on preventive and treatment aspects of a condition not addressed in existing materials. A leaflet is developed and given to patients after the physician discusses with them their status. This leaflet is ‘discovered’ by a health promoter who distributes copies ‘as is’ in all regional clinics, for pick-up in waiting rooms.
2	A regional health promoter is told during field visits in clinics of recurring questions regarding a new problem that nurses have trouble answering. He/she designs a dual-purpose PIL, to be used both for educating nurses and informing patients.
3	A manager of a customer hotline reviews a monthly analysis of calls, identifies repeated questions on a certain topic and pushes Marketing to create a PIL to be sent by hotline staff to callers asking about this topic. This PIL will serve to educate customers and staff, market relevant HMO services, and reduce staff workload.
4	The national director of health education at an HMO decides to create a series of leaflets on a ‘theme topic’ chosen to be the focus of this year's system-wide patient education activities. All leaflets will have a unified ‘look and feel’ and showcase the organization's commitment to client wellbeing. Development is co-led by the Public Relations department and is of interest to high management.
5	A health professional comes across a PIL distributed by another organization, and asks Marketing and Public Relations to help his/her department to generate a competing PIL that would be more up-to-date, more thorough and more attractive.
6	A needs assessment by a government health education agency identifies an information gap regarding a topic of national concern, e.g. smoking risks. The agency starts to develop a PIL, yet later finds that an HMO has already started working on a related leaflet; the two organizations decide to join efforts.
7	An outbreak of a communicable disease is identified by public health officials. A PIL is quickly created by the Ministry of Health and distributed through multiple channels in the affected area (HMO clinics, hospitals, school nurses and health fairs). Other health education activities are also implemented to address the situation.

Factors affecting content and development of PILs

Initiators

While PILs are published by organizations, they are always the result of a work process initiated by individuals, who suggest the development of a PIL due to professional or organizational concerns. Cases 1 and 2 in Table I illustrate scenarios involving a single initiator. Cases 3–5 involve intra-organizational initiators and Case 6 an inter-organizational collaboration. Informants suggested that initiators maintain a sense of ownership and can exert some influence over the content of PILs they initiated.

Users and contexts of use

The content of PILs is affected by the target audiences and the environment where PILs are deployed. Informants described two target audiences for PILs, i.e. clients (patients, families, etc.) and health professionals (nurses, physicians, dieticians, etc.). Their information needs are met in four contexts of use: direct distribution (PILs are designed to be picked by clients), mediated distribution (PILs are designed to be handed to clients only after a discussion with a health professional), distribution within a health campaign (a PIL is one of several patient education tools to be used, in addition to, newspaper ads, TV programs, community lectures, etc.; see Cases 4 and 7), and distribution to both clients and health professionals (when both are relatively unfamiliar with a new health concern).

Goals

PILs are created to satisfy a range of goals. They enable health organizations not only to transmit health-related information, but also address marketing/public relations and administrative goals (Cases 3–5). Leaflets make the public aware of the existence of services, encourage prospective clients to approach the appropriate professionals or units, and enable the organization to showcase its intentions to provide vital information and advanced services to its customers. In addition, leaflets were described as a tangible product that can be shown around to enhance a department's prestige, and thus can affect internal politics and possible allocation of resources. PILs can likewise enhance a health organization's image vis-à-vis other organizations, whether competitors or potential sponsors and collaborators.

Work flow and content decisions

The development of a PIL is usually led by a single person, the ‘Developer’ (who is not necessarily the initiator). The development progresses through two general stages, i.e. initial planning and drafting followed by revision of preliminary drafts, and requires many choices regarding parameters such as the topics to be covered, writing style (e.g. factual, humorous, appeal to fear) or the balance of text, graphical and pictorial elements. The eventual length and content of the PIL emerge through a dynamic process that requires resolution of tensions between multiple inputs (e.g. existing materials, and suggestions from experts, managers, collaborating departments), and available time and resources. The developer needs to be able to negotiate conflicting demands and consolidate the various inputs into a coherent whole through the revision stages.

Readability-related considerations

All informants, regardless of their position and training, expressed a strong awareness for the need to create readable documents, and mentioned relevant guidelines such as the need to keep leaflets reasonably short, avoid heavy technical jargon, or use logical text organization or a question and answer structure to maintain readers' interest, help navigation and improve recall. Informants referred to the need to be sensitive to cultural aspects of specific subpopulations, such as minorities or orthodox groups. Yet, only in a few cases did they relate to reading ability of such subgroups. When asked about the education level needed to understand their particular leaflet, most informants were unsure how many years of education are needed; some hesitantly estimated a minimum of 8, 10 or even 12 years of schooling; none offered a clear justification for why this education level was specified. No informant knew of sources with statistics about distributions of relevant variables such as education levels, literacy or numeracy.

Evaluation practices

All informants emphasized the need to obtain formative feedback regarding PILs being developed, such as about the quality of medical advice, administrative information, cultural appropriateness or text clarity. However, informants appeared to rely primarily on reactions and advice from co-workers or specialists which was routinely solicited informally. In a few cases slightly more elaborate feedback processes were involved, such as sending interim drafts for commentary to a steering committee or to decision makers whose formal sign-off was expected. Only four informants described efforts to solicit any feedback from clients, but these varied in scope and formality, from an elaborate pilot with dozens of clients, to asking a few of the developer's neighbors at home for feedback. None of the informants described any procedure for collecting information about how or to whom leaflets were being distributed, or about their impact. The only post-production data collection activity involved monitoring of the number of copies of different PILs ordered each month by different clinics or hospitals. Thus, demand or popularity was the key indicator described as enabling informants to evaluate a PIL's success.

Impact of the six factors on readability and usability

Below is a synopsis of four key ways in which the six factors uncovered could, separately or in combination, adversely affect readability, comprehensibility or usability of PILs. This synopsis is based in part on examples provided by informants, but also reflects our inferences regarding potential adverse effects of certain situations on readability or usability.

Goal conflicts and organizational pressures

The need to thrive in a dynamic and competitive environment forces health organizations to address multiple goals and conflicting demands when creating PILs. It is difficult to quickly produce a user-friendly PIL that is comprehensive, current, and satisfactory to initiators, content experts and various internal or external stakeholders. Readability may suffer for example when initiators who lack sufficient experience in developing PILs continue to exert influence over development. Texts become harder when developers are unable to balance the requests or pressures of decision makers, experts or partners to add or modify materials (Cases 4 and 6). Common situations described in this regard were when experts or others insist on including additional complex texts, or add technical details or clauses and caveats, presumably to improve medical accuracy, augment the perceived sophistication of the PIL and serve public relations goals (Case 5) or improve its legal defensibility.

Assumptions about users and contexts of use

Developers may knowingly create a relatively complex leaflet, without necessarily seeing it as a problem, if they:

• Expect it to be used only with a high-skilled and specific population.

• Assume other products will be available to take care of the needs of low-skilled subpopulations (e.g. when a PIL is part of a health campaign).

• Expect it to be given out by a health professional only after he or she first assesses clients' knowledge and then explains needed details.

• Need to educate both clients and practitioners (Cases 2 and 3), and hence include in a single leaflet more detailed and lengthier texts than would be used if it was designed only for direct distribution to clients.

Based on informants' descriptions, however, it appears that developers write texts depending on how they believe a leaflet will be used, although usually they do not control how it is actually distributed. In connection with the four situations listed above, readability gaps could ensue when leaflets are deployed in a different way than envisioned, such as if they are given to unplanned target audiences (Case 1), not distributed as part of a health campaign (Cases 4 and 7) or not accompanied by a preliminary explanation by a health professional (Case 1).

Assumptions about client skills

Some developers or medical experts do not base design and writing decisions on statistical information about the full distribution of attributes such as literacy, numeracy or educational achievement in the target population, or believe that the average education level is higher than it is according to available data. This leads to creating leaflets that are aimed at a more sophisticated audience than exists in the field.

Lack of adequate pilot-testing

Developers may conduct little or no pilot-testing, and miss on opportunities to receive formative feedback from a sufficiently diverse set of clients that can help point to readability and comprehensibility gaps. This may occur if developers do not think that pre-testing is necessary or worth the time and effort required, e.g. when:

• It is assumed that even if some people will not understand portions of a leaflet, the text should not be simplified any further, as this will prevent the inclusion of vital medical or technical information (Case 4).

• The relative importance of a leaflet compared to other tools on the same topic (e.g. within a health campaign) is perceived to be low.

• A PIL is designed for limited-time distribution or rapid replacement (Case 7).

• A PIL seems easy enough to write and no complications are foreseen (e.g. it follows the same structure and layout as other ‘popular’ PILs).

• Developers believe that their personal skills and experience are sufficient to ensure adequate quality of a new leaflet (i.e. ‘we know what we are doing’).

Discussion and conclusions

Prior research has documented various problems with readability and usability of health education materials (Zorn and Ratzan, 2000), but has paid little attention to the processes that cause problematic PILs to be created. Recommendations for improving text readability and usability have mostly ignored the realities of how PILs are born, developed or implemented in different organizations. The creation of PILs has been viewed primarily as an intellectual and technical exercise in applying writing and development guidelines. Our findings show, however, that multiple organizational and developer-related factors are involved in the creation of PILs and can lead to the creation of problematic PILs.

Developers face pressures and dilemmas during the lengthy and winding development trajectory of a new PIL. In some cases they do reach compromises or make suboptimal choices, although we believe that some do not necessarily recognize their decisions as such. It was interesting to see that while all developers were attentive to readability issues, most have neither reflected on their inherent assumptions about clients' skills (Tagtow and Amos, 2001) nor sought hard facts about the distributions of educational achievement, literacy or numeracy in target populations (Statistics Canada and Organization for Economic Cooperation and Development, 1996). Further, developers often did not seem to be aware of the impact of insufficient pre-testing on their understanding of the suitability of their PIL to the intended audiences. In light of the repeated reports in the literature regarding readability gaps with PILs on a vast array of health-related topics, we have to conclude that at least some developers are overconfident in their ability to create ‘good’ PILs and not fully aware of the detrimental impact of the various organizational factors on the characteristics of their PILs.

The findings from this exploratory study have to be viewed with caution. Although the use of a qualitative approach is well accepted as a first step in exploratory research (Field and Morse, 1998; Polit and Hungler, 1999; McGregor, 2003), an inquiry based on 10 interviews obviously has limited generalizability. Further, it is unclear whether developers working in contexts were readability formulas can be used, unlike in Israel, face the same issues. Nonetheless, the informants used in this study represented diverse roles and had varied experiences in creating PILs in large organizations with national visibility, resembling similar health systems in several other countries. We believe that producers of PILs in many types of for-profit and non-profit organizations are likely to face similar dilemmas and conflicts. For example, organizations such as community coalitions or hospital departments where many PILs are created internally (Payne et al., 2000) are not likely to have workers with extensive experience in developing PILs, may lack resources needed for lengthy development and pre-testing cycles or have to work jointly with other organizations on designing PILs (Shortell, 2000).

Research implications

The overall research picture regarding the existence of readability and usability problems may be partially misleading. Research has focused on examination of technical characteristics of PILs, such as on average readability scores of sample PILs, or (less often) on comparing such scores to average reading or education levels of samples from a presumed target population [e.g. (Davis et al., 1990; Bull et al., 2001)]. Our findings suggest that PILs that present high readability demands may not necessarily be problematic, if their distribution follows the developer's plans. Yet, actual field usage may vary depending on organizational circumstances (see Case 1) or practitioners' characteristics and information-giving strategies (Jaffray et al., 2001; Stevenson, 2001). Consequently, PILs may still be too difficult for a portion of the population that actually receives them. Hence, future research on the readability and usability of PILs should evaluate their characteristics in context, and consider their appropriateness in terms of both their intended and actual deployment.

More broadly, the complex processes and factors documented in this study lead us to conclude that the creation of PILs has to be viewed not only as an intellectual and technical process, but also as an organizational and political process that is replete with conflicts and dilemmas (Nijhuis and Boersema, 1999). We thus suggest that future research adopts a broad ‘ecological perspective’ when considering factors that affect the readability and usability of PILs, and especially when thinking of ways to improve PILs in this regard. Sless (Sless, 1999) argues that theoretical frameworks regarding document design should not rely only on concepts from ergonomics and cognitive information processing, but also consider political issues and subtle relations between stakeholders, organizations and clients. Based on the present study, we argue that descriptive or prescriptive frameworks of the design of patient education materials should refer to concepts listed by Sless (Sless, 1999), such as inferred readers and authors, genres, power, control and resistance, but also to concepts such as goals and goal conflicts, negotiations, document defensibility, field deployment or implementation barriers.

Future evaluations of the effectiveness of PILs should also adopt a more contextual approach. According to client-centered conceptions (Nutbeam, 2000), PILs should be judged in terms of contribution to client knowledge, health literacy or health status. An ecological perspective, however, implies that PILs should also be judged in the context of the organizational systems within which they were created. Relevant criteria for effectiveness could include, for instance, how well PILs satisfy the visions of their initiators or the needs of stakeholders (including marketing and public relations goals), improve utilization of services (Phillips et al., 1998), affect the types of clients who request different services or otherwise reduce work load. It is quite possible that PILs judged to be of questionable readability and usability may still be viewed as successful by their creators, and hence their use could continue to the surprise and dismay of some researchers and practitioners.

Organizational and practice implications

Patient education leaflets are created at a considerable investment of resources by thousands of health providers, organizations and practitioners in the public, commercial and non-profit sectors. Prevention of readability and usability gaps should be of concern to creators of PILs, given the potential contribution of PILs to the missions of their creators, to clients' well-being and to the improvement of the public's health literacy (Nutbeam, 2000). Our findings highlight the importance of emphasizing in guidelines for improving readability and usability not only technical aspects of texts which have traditionally received most attention, such as layout, text comprehensibility or type legibility. There is also a need for developers to:

• Plan leaflets based on valid data about clients' real skills, rather than rely on fuzzy assumptions.

• Plan leaflets based on realistic assumptions about how they will be used, e.g. plan simpler texts on the assumption that they will not be explained to clients.

• Communicate with field personnel regarding the need to deploy PILs in ways consistent with developers' assumptions.

• Plan multiple products to reach populations that will not be helped by complex leaflets.

• Conduct systematic consumer testing, using cost-effective methods that are less demanding than formal sample-based pilot-testing (Dickinson et al., 2001).

• Constantly remind partners who provide inputs or require revisions during development of the impact of such suggestions on readability and usability.

Our interviews with diverse developers have also shown us that the lore of ‘best practices’ in developing PILs resided with a very small number of individuals in each health organization. This raises the possibility that an additional reason for the persistent readability and usability problems noted in the literature is the loss of accumulated wisdom when experienced developers leave or change positions. The quality of development work could fluctuate while new developers slowly build their technical and research skills or improve their status and ability to engage the inevitable political processes accompanying the development of PILs. It follows that health organizations should not only establish adequate design and evaluation procedures, but also consider long-range strategies for capturing and disseminating accumulated wisdom (Wright, 1999), and for enhancing the ability of new developers to engage with organizational conflicts and decision making. Such and related steps could contribute to organizational learning and to the gradual system-wide improvement of patient information materials and health education programs.