Tuning up the chords of consensus research with the ACcurate COnsensus Reporting Document (ACCORD)

The BJD receives a substantial number of consensus-based papers, yet their quality varies considerably. To address this, the BJD now requires authors to adhere to the ACcurate COnsensus Reporting Document (ACCORD) and include the completed checklist with their submissions.

Consensus methods are extensively utilized for complex scenarios, particularly those involving decision making when objective evidence is limited, contradictory or absent.^1–3 In dermatology, consensus methods are applied for several purposes such as developing clinical practice guidelines, disease classification, core outcome sets and establishing research priorities.^1,4 A range of methods is available, from informal ‘expert consensus meetings’ to more structured methods such as the Delphi method, Nominal Group Technique and RAND/UCLA Appropriateness Method.^1–3 These differ in terms of anonymity, number of participants, face-to-face meetings and required timeframes.

Consensus obtained through formal methods involving a group of experts is generally seen as being more reliable than relying solely on individual opinions and experiences.^1,2 These formal methods mitigate challenges inherent in group decision making by fostering interactions, stimulating critical thinking and questioning conventional viewpoints. Structured approaches minimize risks associated with unstructured group dynamics, such as one individual monopolizing discussions, misrepresenting unanimous agreement when dissenting opinions exist, or allowing group biases to influence decision making.¹

Despite the frequent application of consensus methods, shortcomings in their reporting have been well documented, such as a lack of adequate detail and transparency.^2,3 Many study designs have established reporting guidelines to ensure consistency and quality in research publications.⁴ Examples in the BJD author guidance include the PRISMA and CONSORT guidelines.

Until recently there was no universal reporting guideline applicable to all consensus methods. Two reporting guidelines offer some guidance: Appraisal of Guidelines, Research and Evaluation (AGREE)-II, used for clinical practice guidelines, containing one item on consensus,⁵ and Core Outcome Set–STAndards for Reporting (COS-STAR), for core outcome set development, with three items.⁶ Furthermore, the Conducting and Reporting Delphi Studies (CREDES) guideline is designed for Delphi studies, with a focus on palliative care.⁷ However, ACCORD goes further than just Delphi methods, encompassing all consensus techniques in different health-related domains. The ACCORD checklist comprises 35 items, with the majority focusing on methodological aspects (Table 1).^1,8

Table 1

The ACcurate COnsensus Reporting Document (ACCORD) checklist^1,8

Item no.	Section	Checklist item (help text)a	Page no.
T1	Title	Identify the article as reporting a consensus exercise and state the consensus methods used in the title
		For example, Delphi or nominal group technique
I1	Introduction	Explain why a consensus exercise was chosen over other approaches
I2		State the aim of the consensus exercise, including its intended audience and geographical scope (national, regional, global)
I3		If the consensus exercise is an update of an existing document, state why an update is needed, and provide the citation for the original document
M1	Methods	If the study or study protocol was prospectively registered, state the registration platform and provide a link. If the exercise was not registered, this should be stated
	Registration	Recommended to include the date of registration
M2	Selection of steering committee (SC) and/or panellists	Describe the role(s) and areas of expertise or experience of those directing the consensus exercise
		For example, whether the project was led by a chair, cochairs or an SC, and, if so, how they were chosen. List their names if appropriate, and whether there were any subgroups for individual steps in the process
M3		Explain the criteria for panellist inclusion and the rationale for panellist numbers. State who was responsible for panellist selection
M4		Describe the recruitment process (how panellists were invited to participate)
		Include communication/advertisement method(s) and locations, numbers of invitations sent, and whether there was centralized oversight of invitations or whether panellists were asked/allowed to suggest other members of the panel
M5		Describe the role of any members of the public, patients or carers in the different steps of the study
M6	Preparatory research	Describe how information was obtained prior to generating items or other materials used during the consensus exercise
		This might include a literature review, interviews, surveys or another process
M7		Describe any systematic literature search in detail, including the search strategy and dates of search or the citation if published already
		Provide the details suggested by the reporting guideline PRISMA and the related PRISMA-Search extension
M8		Describe how any existing scientific evidence was summarized and whether this evidence was provided to the panellists
M9	Assessing consensus	Describe the methods used and steps taken to gather panellist input and reach consensus (for example, Delphi, RAND-UCLA, nominal group technique)
		If modifications were made to the method in its original form, provide a detailed explanation of how the method was adjusted and why this was necessary for the purpose of your consensus-based study
M10		Describe how each question or statement was presented and the response options. State whether panellists were able to or required to explain their responses, and whether they could propose new items
		Where possible, present the questionnaire or list of statements as supplementary material
M11		State the objective of each consensus step
		A step could be a consensus meeting, a discussion or interview session, or a Delphi round
M12		State the definition of consensus (for example, number, percentage, or categorical rating, such as ‘agree’ or ‘strongly agree’) and explain the rationale for that definition
M13		State whether items that met the prespecified definition of consensus were included in any subsequent voting rounds
M14		For each step, describe how responses were collected, and whether responses were collected in a group setting or individually
M15		Describe how responses were processed and/or synthesized
		Include qualitative analyses of free-text responses (for example, thematic, content or cluster analysis) and/or quantitative analytical methods, if used
M16		Describe any piloting of the study materials and/or survey instruments
		Include how many individuals piloted the study materials, the rationale for the selection of those individuals, any changes made as a result and whether their responses were used in the calculation of the final consensus. If no pilot was conducted, this should be stated
M17		If applicable, describe how feedback was provided to panellists at the end of each consensus step or meeting
		State whether feedback was quantitative (for example, approval rates per topic/item) and/or qualitative (for example, comments, or lists of approved items), and whether it was anonymized
M18		State whether anonymity was planned in the study design. Explain where and to whom it was applied and what methods were used to guarantee anonymity
M19		State whether the SC was involved in the decisions made by the consensus panel
		For example, whether the SC or those managing consensus also had voting rights
M20	Participation	Describe any incentives used to encourage responses or participation in the consensus process
		For example, were invitations to participate reiterated, or were participants reimbursed for their time
M21		Describe any adaptations to make the surveys/meetings more accessible
		For example, the languages in which the surveys/meetings were conducted and whether translations or plain language summaries were available
R1	Results	State when the consensus exercise was conducted. List the date of initiation and the time taken to complete each consensus step, analysis and any extensions or delays in the analysis
R2		Explain any deviations from the study protocol and why these were necessary
		For example, addition of panel members during the exercise, number of consensus steps, stopping criteria; report the step(s) in which the deviation(s) occurred
R3		For each step, report quantitative (number of panellists, response rate) and qualitative (relevant sociodemographics) data to describe the participating panellists
R4		Report the final outcome of the consensus process as qualitative (for example, aggregated themes from comments) and/or quantitative (for example, summary statistics, score means, medians and/or ranges) data
R5		List any items or topics that were modified or removed during the consensus process. Include why and when in the process they were modified or removed
D1	Discussion	Discuss the methodological strengths and limitations of the consensus exercise
		Include factors that may have impacted the decisions (for example, response rates, representativeness of the panel, potential for feedback during consensus to bias responses, potential impact of any nonanonymized interactions)
D2		Discuss whether the recommendations are consistent with any pre-existing literature and, if not, propose reasons why this process may have arrived at alternative conclusions
O1	Other information	List any endorsing organizations involved and their role in the exercise
O2		State any potential conflicts of interests, including among those directing the consensus study and panellists. Describe how conflicts of interest were managed
O3		State any funding received and the role of the funder
		Specify, for example, any funder involvement in the study concept/design, participation in the SC, conducting the consensus process, funding of any medical writing support. This could be disclosed in the methods or in the relevant transparency section of the manuscript. Where a funder did not play a role in the process or influence the decisions reached, this should be specified

^aACCORD explanation and elaboration provides guidance and examples to support reporting.⁸

Certain aspects of ACCORD align with other reporting guidelines. For instance, the title should clearly indicate the type of research, explicitly stating that the study is a consensus exercise, in addition to specifying the consensus method used. In the introduction, authors must define the aim of the study, the intended audience and the geographical scope. This may seem obvious, but these details are frequently overlooked, leading to ambiguity about who should use the research and where it applies. Authors should identify whether the recommendations are designed for healthcare providers (and specify which type of healthcare provider), patients, researchers, policymakers or a broader mix of stakeholders. Additionally, authors must clarify the geographical scope of their research, i.e. is this study relevant at an international, regional or national level?

Some other key areas that need to be reported to ensure clarity, transparency and integrity are highlighted below.

Protocol registration establishes a clear roadmap for the process minimizing the risk of selective reporting and promoting transparency. If registered, authors should include the platform and provide a link to the registration. This indicates that the study follows a structured approach and allows for independent verification.

Description of the procedure for selecting the steering committee (SC) and panellists contributes to the credibility of the study. Authors should explain who was involved in directing the consensus exercise, their roles and their areas of expertise. The study should describe how the participants were chosen and by whom, and clarify the criteria for panellist inclusion, detailing who was responsible for the selection and how the recruitment process was managed. This level of detail helps readers understand the expertise behind the consensus process, and increases trust in the findings of the study. Furthermore, authors should describe the role of any members of the public, patients or carers involved in the study. Patients and public involvement (PPI) has become increasingly important in health research, providing valuable insights from a user perspective. Some journals require authors to report PPI involvement in their studies.⁸

Another critical aspect is the definition of consensus, as this informs readers about the level of agreement sought among the panel. This definition, often expressed as a percentage of panellists who agree, a certain number of agreeing responses or a categorical scale such as ‘agree’ or ‘strongly agree’, can vary based on the context of the study. For critical topics, e.g. patient safety in dermatological procedures, a higher threshold of agreement might be required. Conversely, more subjective topics, e.g. treatment preferences for conditions such as psoriasis or eczema, might permit a lower threshold owing to a broader range of acceptable opinions and subjective interpretations, allowing for greater flexibility. Therefore, it is important to specify and justify the consensus threshold.

Furthermore, it is crucial to describe how feedback was provided to panellists at each step of the consensus process. This includes clarifying whether the feedback was quantitative, e.g. approval rates per topic or item; qualitative, e.g. through comments; or a combination of both. Authors should indicate whether feedback was anonymized and describe the steps taken to ensure anonymity, specifying who remained anonymous and which methods were used to maintain anonymity. If applicable, authors should indicate whether the SC was involved in decision making alongside the consensus panel, and whether they had voting rights or other forms of influence. Participation can be influenced by various factors, so it is worth noting any incentives provided to encourage responses or participation in the consensus process. Additionally, the study should describe any adaptations made to improve accessibility, such as conducting surveys or meetings in multiple languages or providing translations and plain language summaries to ensure broader understanding and engagement. Finally, conflicts of interest among the SC and panellists must be disclosed, along with measures taken to manage them. Authors should specify the source of funding and the role the funder played in the study, indicating whether the funder had any influence on the design or execution of the study. By focusing on these aspects, researchers can ensure a more transparent and rigorous approach to reporting consensus exercises. The BJD aims to uphold high standards in this area, especially in guideline development, emphasizing ‘the minimization of authors’ conflicts of interest and a clear and detailed conflicts of interest and funding statement’.⁴ ACCORD can be a valuable tool for authors and reviewers to ensure high-­quality, transparent consensus publications.

Acknowledgements

I would like to thank Bernd Arents, Patient Associate Editor at BJD for his valuable feedback.

Funding sources

This editorial received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Data availability

Not applicable.

Ethics statement

Not applicable.

Author notes

Conflicts of interest E.J.v.Z. is a member of the ACCORD development team.