A survey on the methodological processes and policies of renal guideline groups as a ﬁ rst step to harmonize renal guidelines

Background. Worldwide, several bodies produce renal guidelines, potentially leading to duplication of effort while other topics may remain uncovered. Nephrology (CSN), European Renal Best Practice (ERBP), Kidney Disease Improving Global Outcome (KDIGO), Kidney Health Australia-Caring for Australians with Renal Insuf ﬁ ciency Kidney Disease Outcome Quality Initiative and United Kingdom Renal Association (UK-RA). Results. Five of the seven groups conduct systematic searches for evidence, two include detailed critical appraisal and all use the GRADE framework. Five have public review of the guideline draft. Guidelines are updated as new evidence comes up in all, and/or after a speci ﬁ ed time frame has passed ( N = 3). Commentaries or position statements on guidelines published by other groups are produced by ﬁ ve, with the ADAPTE framework ( N = 1) and the AGREEII ( N = 2) used by some. Funding is from their parent organizations ( N = 5) or directly from industry ( N = 2). None allow funders to in ﬂ uence topic selection or guideline content. The budgets to develop a full guideline vary from $2000 to $500 000. Guideline development groups vary in size from <5 ( N = 1) to 13 – 20 persons ( N = 3). Three explicitly seek patient perspectives, for example, by involving patients in the scoping process, and four incorporate health economic considerations. All provide training in methodology for guideline development groups and six make their methods public. All try to avoid overlapping topics already planned or published by others. There is no common con ﬂ ict of interest policy. Conclusions. Overall, there is considerable commonality in methods and approaches in renal guideline development by the different organizations, although some procedural differences remain. As the ﬁ nancial and human resource costs of guideline production are high, a collaborative approach is required to maximize impact and develop a sustainable work plan. Coming to consensus on methods and procedures is the ﬁ rst step and appears feasible.

producing renal guidelines: the Canadian Society of Nephrology (CSN), European Renal Best Practice (ERBP), Kidney Disease Improving Global Outcome (KDIGO), Kidney Health Australia-Caring for Australians with Renal Insufficiency (KHA-CARI), Kidney Disease Outcome Quality Initiative (KDOQI), Sociedad Latino-Americano de Nefrologia e Hipertension (SLANH) and United Kingdom Renal Association (UK-RA). Results. Five of the seven groups conduct systematic searches for evidence, two include detailed critical appraisal and all use the GRADE framework. Five have public review of the guideline draft. Guidelines are updated as new evidence comes up in all, and/or after a specified time frame has passed (N = 3). Commentaries or position statements on guidelines published by other groups are produced by five, with the ADAPTE framework (N = 1) and the AGREEII (N = 2) used by some. Funding is from their parent organizations (N = 5) or directly from industry (N = 2). None allow funders to influence topic selection or guideline content. The budgets to develop a full guideline vary from $2000 to $500 000. Guideline development groups vary in size from <5 (N = 1) to 13-20 persons (N = 3). Three explicitly seek patient perspectives, for example, by involving patients in the scoping process, and four incorporate health economic considerations. All provide training in methodology for guideline development groups and six make their methods public. All try to avoid overlapping topics already planned or published by others. There is no common conflict of interest policy. Conclusions. Overall, there is considerable commonality in methods and approaches in renal guideline development by the different organizations, although some procedural differences remain. As the financial and human resource costs of guideline production are high, a collaborative approach is required to maximize impact and develop a sustainable work plan. Coming to consensus on methods and procedures is the first step and appears feasible.
Keywords: evidence-based medicine, practice guidelines as topic

I N T RO D U C T I O N
The number of available diagnostic and therapeutic devices and drugs increases continuously, resulting in an exponential production of scientific publications. This rapidly expanding knowledge base has made clinical decision-making a complex matter. This complexity is further enhanced by the variable scientific quality of the underlying evidence. Systematic reviews, by identifying, appraising and synthesizing evidence, have long since become an integral and efficient tool to inform clinical decision-making [1,2].
In some clinical situations, however, systematic reviews do not provide an appropriate answer. First, because the quality of systematic reviews is variable [3]. Secondly, systematic reviews focus strongly on one specific question in one specific setting, and might thus not be applicable to the situation the clinician has at hand. For these situations, retrieving complimentary information, such as individual study reports, can be very helpful to correctly interpret and frame the findings of the systematic review. Thirdly, for many clinical situations, there is a lack of sufficient evidence to come to a firm conclusion using the systematic review methodology, in which case most systematic reviews leave the question unanswered. Clinical practice guidelines differ from systematic reviews by the fact that they consider a broader more holistic perspective to certain clinical problems, and have increasingly been used by health-care workers to support them in their daily practice [4]. Historically, clinical practice guidelines were mainly based on expert opinion. Ideally, however, they should balance existing evidence (the 'objective' part), with patient perspective and implementability (the 'appreciation' part) to provide guidance. Whereas this makes clinical practice guidelines more clinically relevant than strict systematic reviews, it also induces a greater risk for subjectivity and bias. Modern clinical practice guidelines should thus be based on high-quality systematic reviews, combined with an objective distillation of the views of experts and stakeholders, including consumers, to achieve this balance.
Guidelines should be developed following predefined rigorous methodologies via a transparent process [5]. This requires clinical experts and skilled methodologists, appropriate software support, project management, consumer involvement and implementation strategies. As a consequence guideline development has become a resource and labour intensive enterprise [6][7][8]. Providing guidance specifically for patients with kidney disease is even more challenging, as patients with kidney disease often have complex comorbidities.
Within the Chronic Kidney Disease (CKD) community, the contribution of guidelines to clinical practice has been large. Starting with the Kidney Disease Outcomes Quality Initiative (K-DOQI) in 1997, there has been a largely uncoordinated approach to guideline development, with many jurisdictions developing the capacity for guideline development independently. This is especially problematic because producing and maintaining up-to-date guidelines may not be sustainable for any one group, and readers become confused because of real and apparent inconsistency in the recommendations for the same conditions. These conflicts may be due to real or perceived divergent methods at any step in the evidence synthesis and recommendation process. Other important areas are completely overlooked by all guideline groups, resulting in gaps in guidance in important areas [9][10][11][12]. This spread of guideline production also brings a theoretical vulnerability to commercial influence, as commercial interests could select the guideline groups they sponsor based on a group's willingness to accommodate these interests.
One possible solution to these problems is a formal, collaborative process to the development of CKD guideline from topic selection through publication to implementation, including a common and transparent approach to methods. To begin this process, we undertook a survey to comprehensively identify similarities and discrepancies in the guideline development process among the seven major renal guideline groups (alpha-

M AT E R I A L S A N D M E T H O D S
Design of the survey We used the Institute of Medicine standards on guidelines as a framework to develop an electronic survey, investigating items related to organizational characteristics, procedures and the guideline development methodology [13].
The survey comprised 62 questions grouped into six domains: general information (7 questions), organizational characteristics (11 questions), policies (17 questions), guideline development and updating processes (18 questions), and dissemination and implementation strategies (9 questions). A copy of the survey can be found in Supplementary data.

Data collection and analysis
The survey was sent to the chairs of each participating organization, and either the chair or a member of the senior staff assigned by the chair completed the questionnaire. All guideline bodies answered the questionnaire blinded to responses of the other participating organizations. The collected responses were tabulated and the data were analysed using descriptive statistics. We calculated absolute frequencies for each question to identify discrepancies and similarities.

R E S U LT S
All seven guideline bodies responded. Six chairs filled out the questionnaire themselves and one assigned a collaborator.
General information about guideline organizations Characters, aims, intended users and output of organizations. Five renal guideline bodies represented more than one country; two organizations were national guideline bodies. The guideline organizations published their first guideline between 1997 and 2011. For a list of the most recent guidelines produced by each guideline organization, see Supplementary data. Organizations mostly characterized themselves as a working group which was part of a professional society (N = 5). Four indicated to work not for profit. Improving patient outcomes was the main aim for all participants. In addition, three organizations stated that they aim to stimulate shared decision-making between patients and clinicians, two organizations also aim to inform health-care policy makers and one organization aims to contain costs. All organizations targeted nephrologists and renal nurses. Many also targeted clinicians from other medical specialties involved in renal care (N = 6), policy makers (N = 5) and patients with or at risk for kidney disease and their caregivers (N = 4).
Organizational characteristics Structure of organization and its advisory board. All organizations have an advisory board, with characteristics as depicted in Table 1.
Sources of funding and budgets. Funders are claimed to have no influence on topic selection or guideline content in any of the guideline organizations. Some guideline bodies have more than one source of funding. Table 1 summarizes the funding sources of each organization. Of the seven organizations, four reveal their funding sources explicitly in the public area of their website. Annual budgets of the guideline organizations vary between US $70 000 and $40 000 000. The available budgets to develop one guideline vary between US $2000 and $500 000. Organizations that had a set budget for guideline dissemination and implementation (N = 6) assigned between US $18 000 and $500 000 per guideline for this aspect.

Policies
Declaration of interest. The requirements for declaring interest are comparable for all organizations. Declaration of financial interest is required in six organizations, whereas the declaration of intellectual interest is rarely required. Declaration of interest policies are shown in Figure 1. In four organizations, financial interests of any extent must be declared; two organizations limit the declaration of financial interest to specific amounts. One of these organizations requires specific declaration of all financial interests greater than US $13 000 and another organization requires all financial compensation for travel expenses greater than US $10 000 to be declared, but also any sum for paid work. Three organizations have no time frame for reporting past financial interest and three organizations limit the necessity to declare past financial interest to the preceding 1-5 years.
Composition of guideline development groups. Methods to identify guideline development group members as well as composition of guideline development groups are similar between most organizations and depicted in detail in Table 1.

Consideration of patient perspective and health economics.
Three organizations actively involve patients or their representatives in the guideline development process, for example in the scoping process or formulation of the clinical questions (N = 2) or in reviewing the guideline draft (N = 2). Patients or their representatives are recruited by personal connections (N = 3) or through nominations by patient organizations (N = 2).
Four organizations consider health economics in the guideline development process.
Communication of methodology. All organizations train their guideline development group members in guideline development methodology, six organizations through training sessions and one by providing a manual. Of those organizing training sessions, five have one session before guideline development process commencement, one organization organizes multiple sessions throughout the process and two organizations provide hands-on training to individual guideline development group members. In two organizations, the training sessions are compulsory and in three, they are optional. In addition, some provide manuals (N = 3) and webinars (N = 1).
All but one organization communicate their guideline development methodology to the public via publicly available manuals (N = 5), methods section in guideline documents (N = 1) or Continuing Medical Education courses (N = 1).
Guideline development and updating process Topic selection, systematic review process and recommendation generation process. All organizations consider planned or published guidelines of other renal guideline bodies to avoid overlap. New topics are planned 1-4 years in advance and five Table 1. Shows characteristics of each organization and illustrates how advisory board members are selected, whether advisory board members are appointed for a maximum term, which categories of advisory board members serve in each organization, how guideline organizations and their parent bodies are funded, how guideline development group members are identified and which type of individuals serves on guideline development groups.

N D T P E R S P E C T I V E S
T o w a r d s h a r m o n i z i n g r e n a l g u i d e l i n e s organizations have a specified procedure to select topics for upcoming guidelines, involving the parent body in three cases. The systematic review team is integrated in the guideline development group in all but one organization. The classification of each organization according to the different types of interaction between the systematic review team and the guideline development group as defined by the Institute of Medicine is shown in Figure 2   Updating existing guidelines and processing guidelines produced by others. Guidelines are updated in case of new evidence (N = 7) or after a pre-set timeframe is exceeded (N = 3), using a formal procedure (N = 6). Processing guidelines produced by others includes issuing commentaries or position statements (N = 5), using the AGREE tool (N = 2), adapting the guideline to local circumstances using the ADAPTE framework (N = 1) or another framework, which was not further specified (N = 1) and publishing the guidelines produced by others on the own website (N = 2).

Dissemination and implementation strategies
Publication. All organizations publish their guidelines in a peer-reviewed journal or on their website. One also publishes a printed monograph separately from a journal and one disseminates guidelines also on USB devices. Guidelines are published almost always in the same peer-reviewed journal (N = 6). Besides the full version of the guideline document, 'recommendation-only' versions (N = 5), '1-5-page-summaries' (N = 3) and patient versions (N = 2) are provided. Four organizations support or provide translations of the guidelines in other than the original language.
Implementation. Tools for application to accompany the guidelines, such as algorithms and flow charts (N = 5) and patient leaflets (N = 3) and educational activities to promote the guideline (N = 5), are organized. Three organizations develop guideline-based monitoring or audit criteria, and four organizations promote adherence to the guidelines, e.g. through attendance at national nephrology meetings and workshops or performing studies of adherence to the guidelines.

D I S C U S S I O N
In 2007, a first attempt was made to harmonize guideline production in nephrology [9]. The current survey, based on recommendations of the Institute of Medicine [13], demonstrates that, since then, improvements on the methodology to search evidence has been made by the different renal organizations, and a consensus on a common methodology could be realistic. Some areas still remain somewhat problematic, such as procedures for topic selection, involvement of all stakeholders, selection of guideline group members, strategies to make data extraction available to other guideline groups and end-users and declaration of interest policy. As such, further improvements in harmonization among guideline developing groups should be feasible, provided the political will is present. It is likely that greater harmonization would improve the quality and quantity of nephrology topics covered by guidelines.

The need for harmonization
The core challenges for guideline developers and the scope for harmonization to positively impact these issues are depicted in Table 2.
In the last two decades, the number of publications and available data has grown substantially, so the case for a more digestible guidance format remains compelling [14]. However, as a consequence, the number of guidance documents is increasing, with quality of these materials variable [15,16]. Production of guidelines necessitates a rigorous methodology to ascertain complete and unbiased presentation of evidence in data extraction tables, and transparency, such that, as in basic research, the Table 2. Current problems of guideline production and how guideline harmonization might help

Methodological rigor
A set of minimum requirements related to the production of data extraction tables; these tables form the basis of any high-quality guidance and will allow exchange between guideline groups of this demanding part of guideline production. Providing transparency regarding the methodological approach; this enables reproduction, as well as external assessment of the quality of the guidance.

Huge financial and human resource consumption
Exchange of data extraction tables, the most resource demanding part of guideline production, will reduce financial and human resource cost.

Topic selection
Minimize overlap and redundancy. Increase complementarity. Promote support and ownership by all relevant stakeholders by more correct identification and prioritization of the most relevant topics. Limit the influence of industry or single individuals on topic selection.

Editorial independence
Rules for (transparency on) conflicts of interest for guideline group and advisory board members: Decreases the risk of unwarranted influence of industry on the scope and content of the guidance. Allows guideline users to take into account the potential influence of conflicts of interest on the guideline content. Harmonization avoids that financial support goes to the most 'industry friendly' guideline.

Applicability
Exchange of data extraction tables between guideline groups will allow that local groups can produce guidance that is adapted to local conditions, but still with high-quality standards and based on available evidence. Involvement of different stakeholders will assure that guidelines can be implemented in clinical practice.

N D T P E R S P E C T I V E S
T o w a r d s h a r m o n i z i n g r e n a l g u i d e l i n e s work can be reproduced and verified [13,17]. A lack of sufficient methodological training of guideline development team members is a specific pitfall, especially in guidelines produced by professional specialty societies [16]. Al-Ansary et al. [18] analysed 11 guidance documents on management of hypertension, and found all low methodological quality, probably attributable to the fact that many local or specialty groups wanted to develop their own guideline, without having adequate resources or experience. A solution for this problem might be to engage experienced methodologists to perform the systematic literature review for a group of content experts, who then formulate the actual recommendations (Figure 2). An alternative approach, recommended by a World Health Organization working group [19], is to invest more in the 'hands-on' methodological training of the guidance development team. This approach may result in an improved uptake of the requirements of evidencebased thinking, and the application of such thinking within the specialty. In other models, e.g. used by National Institute for Health and Care Excellence (NICE), the group only consists of methodologists, with assistance provided by medical content specialists who do not have any influence on the final wording of the recommendations [20]. The ADAPTE group [21] postulates that using existing high-quality guidelines and adapting them to local conditions may be an alternative, low cost, resource to de novo development. However, customizing a guideline to local conditions starting from the recommendations themselves bears the risk that the adapted guideline departs from its evidence base, putting back into question the quality and validity of the recommendations. The process of compiling the evidence in a systematic, rigorous way is the part of guidance production that is most suitable for exchange, because the process consists of an objective, unbiased presentation of the available evidence, which is independent of geographic region, professional background, socio-economic conditions or health-care model. Production of high-quality guidelines requires the specific skills of a specialized team, and is labour intensive and thus time consuming and resource demanding. Our survey revealed that the average budget for a guideline, depending on the scale and methodology used, could mount up to hundreds of thousands of dollars, with 5-20 people collaborating on the production. It is not surprising that the seven groups together only issued 27 guidance documents over the last 7 years, and encompassed only 12 different topics. The most ideal model of collaboration between guideline groups might thus be that existing data extraction tables, developed according to a well-defined and accepted methodology, are exchanged and can be used by all groups who want to develop their own guidance. Such a system allows that guidelines are adapted by local stakeholders starting from the available evidence. Such a procedure might enhance the feeling of involvement, and increase the chances that the recommendations will actually be implemented. In addition, such a collaborative model would be likely to reduce costs substantially and liberate work power to cover more guidance without decreasing quality. Several studies reported important inconsistencies between guidelines on the same topic [10,22]. Discrepancies in recommendations can just reflect local circumstances, or local cultural or economic differences in the value placed on specific outcomes and priorities, and are therefore acceptable if adequately explained. However, most of the time, they are just the result of biased compilation and interpretation of evidence. It can thus be hoped that the exchange of high-quality data extraction tables between guideline groups can reduce this reason for discrepant guideline recommendations.

Consensus on policies and procedures
In most assessments of quality of guidelines, editorial independence as defined in the AGREE tool is one of the most problematic domains [15]. External influence can vary from subtle, e.g. by intentionally selecting specific topics (disease mongering) [23], or more straightforward when members of the guideline development group have clear financial conflicts of interest. Our survey shows that all seven guideline groups have rules on how to report financial conflicts of interest. However, the details vary, especially in the way these financial conflicts of interest are dealt with in practice, corroborating previous research [15,16,24]. More problematic is that all groups seem to struggle with dealing with intellectual conflict of interest. It can be hoped that harmonization of the guideline production process will help in this regard. An important step might be to establish a protocol to define and select the topics and scope of the guidelines. As this will need to be done in a mutual agreement, to avoid duplication, a specific procedure and planning will have to be developed. This probably will decrease the number of 'ad hoc' guidelines produced because of commercial interest. Not surprisingly, five guideline groups in our survey have generated guidance documents on anaemia and chronic kidney disease -mineral and bone disease management, two areas where the industry stakes are high. Even if individual guideline group members are not biased towards one specific drug by financial incentives, reliance on direct or indirect funding from pharmaceutical companies is likely to explain why most guidelines (and indeed, most research studies) are focused on drug treatment (rather than diagnostic strategies or lifestyle interventions, for example). Introducing open scoping surveys to clarify which questions need to be addressed with higher priority can probably assure that guidelines with topics that matter more to patients and physicians are prioritized. Such a strategy can also reveal where the largest knowledge gaps reside and, in this way, guideline bodies could steer the future research agenda.
Existing evidence suggests that composition [25,26] of the guideline development group has an impact on the content of the recommendations that are made. However, evidence to support ideal composition of such a group is largely lacking [19].
Most authors agree that groups should be broadly composed and include important stakeholders such as health professionals that work within the relevant area (content experts), patients and managers or policy makers, along with individuals with the necessary technical skills in information retrieval, systematic reviewing, project management, and writing and editing of guideline documents. Stakeholder involvement was more problematic in guidelines issued by (single) specialty societies when compared with other (multidisciplinary) groups [26]. In a systematic review [25], heterogeneity of the guideline group positively influenced identification and critical evaluation of all relevant scientific evidence, but also enhanced the risk of internal conflicts, leading to vague and non-contributive statements. Indeed, it has been demonstrated that specialty groups with no or low involvement of external stakeholders have the tendency to formulate more strong and more invasive recommendations, even in the absence of evidence. In addition, involvement of stakeholders with different backgrounds improves the odds that practical problems to use the guidance will be identified and addressed more early, thus increasing the applicability of the provided guidance. A further advantage of using multidisciplinary groups is that they can create a sense of involvement or 'ownership' among different audiences for the guidelines [21]. In our survey, it was evident that all guideline bodies aimed at a wide inclusion of different subgroups of stakeholders.

CO N C L U S I O N
In view of the results of the survey, concerted effort is required to build a common approach to renal guideline production. Harmonization of guideline production is essential to enhance the efficiency of the sector, by increasing the number of topics covered by guidelines, while at the same time preserving the methodological quality. It would also strengthen the guideline bodies' independence from commercial interests. This survey indicates that methodological barriers for such a harmonization could be overcome, provided the respective organizations show willingness to embark on such a project.