A core domain set for hidradenitis suppurativa trial outcomes: an international Delphi process

Summary

Background

There is no consensus on core outcome domains for hidradenitis suppurativa (HS). Heterogeneous outcome measure instruments in clinical trials likely leads to outcome‐reporting bias and limits the ability to synthesize evidence.

Objectives

To achieve global multistakeholder consensus on a core outcome set (COS) of domains regarding what to measure in clinical trials for HS.

Methods

Six stakeholder groups participated in a Delphi process that included five anonymous e‐Delphi rounds and four face‐to‐face consensus meetings to reach consensus on the final COS. The aim was for a 1 : 1 ratio of patients to healthcare professionals (HCPs).

Results

A total of 41 patients and 52 HCPs from 19 countries in four continents participated in the consensus process, which yielded a final COS that included five domains: pain, physical signs, HS‐specific quality of life, global assessment and progression of course. A sixth domain, symptoms, was highly supported by patients and not by HCPs but is recommended for the core domain set.

Conclusions

Routine adoption of the COS in future HS trials should ensure that core outcomes of importance to both patients and HCPs are collected.

Hidradenitis suppurativa (HS) is a chronic, inflammatory skin disease with an estimated prevalence of 0·1–4% worldwide.1,2,3,4 The primary lesions are inflammatory nodules that may develop into abscesses and sinus tracts with subsequent scarring, affecting flexural sites such as the axillae and groins on a recurrent basis.5,6 Lesions of HS are typically described by patients as painful boils which, along with associated pus and odour, may have a large impact on quality of life.7,8,9,10,11

Interventions for HS are diverse and include topical treatment, systemic antibiotics, retinoids, immunomodulatory oral therapy, biological drugs, laser therapy and surgery.12,13 The level of evidence for existing treatments is low, suggesting a particular need for more clinical trials in HS.13

Validated outcome measure instruments are necessary to ensure that study results are comparable and that, as a consequence, patients receive the most effective interventions. For HS, numerous outcome measure instruments exist, with a total of 30 instruments used in the 12 randomized controlled trials included in the recent Cochrane review.13,14 Heterogeneity of outcome measure instruments in HS limits evidence synthesis, including meta‐analysis13 and likely leads to outcome reporting bias because of selective reporting of more favourable outcomes.15 Because no consensus on core outcomes for HS exists, researchers use various instruments, which may or may not be valid. Furthermore, current instruments emphasize clinical features with limited incorporation of patient‐reported outcomes, despite recommendations emphasizing the importance of the patient perspective in outcomes research.16

To tackle these issues, the HIdradenitis SuppuraTiva cORe outcomes set International Collaboration (HISTORIC) was formed as a collaboration between the International Dermatology Outcome Measures (IDEOM) initiative, the Cochrane Skin Group – Core Outcome Set Initiative and Zealand University Hospital, Roskilde. The first HISTORIC goal was to develop a core outcome set (COS) of domains that is relevant to all major stakeholders, including patients, to be recommended for use in all subsequent HS clinical trials.17,18,19 We performed and moderated an international multiperspective Delphi consensus project with the scope to develop a COS suitable for all HS clinical trials. The COS is intended to suit all types of interventions for all HS patients, regardless of setting or mode of administration.

Materials and methods

The study is reported in accordance with the newly developed Core Outcome Set STAndards for Reporting (COS‐STAR).20 A detailed description of the methods can be found in our protocol article.21 Methodological guidance was followed from Core Outcome Measures in Effectiveness Trials (COMET)19 and Outcome Measures in Rheumatology (OMERACT).22 We were also guided by the Harmonizing Outcome Measures for Eczema (HOME) roadmap.23 The methodology involves a stepwise approach for the development of a COS. The first step is to identify which domains one should measure and report in all clinical controlled trials of a specific condition (what to measure: the core domain set).15 The second step is to identify the instruments that should be used to assess these domains (how to measure: the core outcome measurement set).17,19 The present study achieved the first step, determining what to measure.

Participants

The involvement of multiple stakeholders for the development of a COS is strongly recommended by methodologists.17,19,23,24 Six groups of stakeholders were invited to participate in our development process: patients, dermatology HS experts, surgical HS experts, HS nurses, industry representatives and drug regulatory authorities. Patients were analysed as one stakeholder group and the remaining stakeholder groups were combined into a second group referred to as healthcare professionals (HCPs). The HCP group contained one representative of a drug regulatory authority (the European Medicines Agency) and one industry representative. Other drug regulators and pharmaceutical companies with an interest in HS were contacted but chose not to contribute. The aim was for a 1 : 1 ratio of patients to HCPs.

Patients were identified through patient associations and via dermatologists with a special interest in HS in countries without a formal patient association. HCPs were identified from the community of HCPs working with patients with HS. A clinical background of at least 5 years of experience managing HS was required for all HCPs and publications on HS or participation in scientific meetings on HS was required for dermatologists.

Information sources

The initial list of candidate items was obtained in a three‐step manner.

1. Systematic review of literature. A recent systematic Cochrane review on interventions for HS and another systematic review on outcome measure instruments reviewed the validation evidence for existing instruments and mapped them according to potential domains.13,14

2. Qualitative studies. Semistructured interviews and focus groups were conducted at the Department of Dermatology, Zealand University Hospital, Roskilde, Denmark and the Department of Dermatology, Penn State College of Medicine, Hershey, Pennsylvania, U.S.A. Purposive sampling of a wide diversity of age groups, sex, treatments received and disease severities was employed. Inclusion of patients ceased when saturation was achieved, defined as when no new knowledge was obtained from the subsequent interviews. Patients were identified primarily among those undergoing treatment at the two Departments of Dermatology. Eligibility was based on a confirmed diagnosis of HS and willingness to participate.

All interviews were tape recorded and transcribed verbatim and initially examined for units of meaning, coded as items and grouped into categories. Qualitative interviews do not require ethical approval in Denmark. In the U.S.A. the project was approved by the institutional review board of the Penn State College of Medicine. The two lists of candidate items generated from the Danish and US qualitative studies were combined into one patient‐generated item list.

Identification of items of importance to healthcare professionals. To identify outcomes of importance to HCPs an item generation e‐Delphi round zero was conducted among the HCP stakeholders. Participants were first provided with background information on the rationale for development of an HS COS. They were then asked to list all items that they considered important regarding HS, with items being related to any aspect of the disease, or treatment of the disease.

The steering group reviewed all items suggested by the HCPs and produced a preliminary list of candidate items by combining the results from the systematic reviews, the qualitative studies and the HCPs item generation survey.

Consensus process

A summary of the consensus process is shown in Figure 1. An international steering group (the first 12 authors of this manuscript) consisting of researchers, clinicians and a patient research partner guided development of the COS.

Figure 1

Study summary. HCP, healthcare professional.

Methods to reach consensus on the core domain set

An anonymous Delphi approach was applied to make sure that the views of all participants were obtained. The e‐Delphi survey was delivered using DelphiManager^® (round one and two, COMET Initiative, http://www.comet-initiative.org/delphimanager/) and SurveyMonkey^® (round three to five, SurveyMonkey, San Mateo, CA, U.S.A.) software. A unique identifier code allowed identification of participants completing all rounds of the Delphi survey. Only participants who had completed the previous round of the survey were invited to participate in subsequent rounds. All surveys were pilot‐tested by at least two members of the steering group, including the patient representative, and at least two additional panel members.

Items/domain scoring

Participants were asked to score each item/domain using a modified scale from the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) from one to nine. Explanation was provided that scores of one to three are ‘not important’, scores of four to six are ‘important but not critical’ and scores of seven to nine are ‘critical’ to include.25 From round two of the Delphi onwards, participants were provided with feedback in the form of their own scores in the previous round and the aggregate scores from the previous round, subdivided into the patient and HCP groups.

Definition of consensus

Prespecified consensus end points are outlined in Table 1.

Table 1

Definition of consensus

Consensus classification	Description	Definition
Consensus in	Consensus that the item/domain should be included in the core domain set	70% or more participants scoring 7 to 9 AND < 15% participants scoring 1 to 3
Consensus out	Consensus that the item/domain should not be included in the core domain set	70% or more participants scoring 1 to 3 AND < 15% of participants scoring 7 to 9
No consensus	Uncertainty about importance of the item/domain	Anything else

E‐Delphi round one and two

Participants were provided with background information explaining how the candidate items were identified and were then asked to rate each of the items listed, based on their importance in being measured as an outcome in all clinical trials for HS. Participants were also asked to suggest items not represented in the list. Items suggested were reviewed by the steering group to ensure they represented new items and all items were carried forward to round two. In round two, the number of participants who ranked each item and the distribution of scores (as percentages of the total) by stakeholder group from round one were shown graphically in the survey and participants were asked to consider responses from other panel members and to re‐score all items. All items were carried forward to consensus meeting I and II and e‐Delphi round three.

HIdradenitis SuppuraTiva cORe outcomes set International Collaboration consensus meeting I and II

After the first two e‐Delphi rounds, participants were invited to take part in two in‐person consensus meetings (Vienna, September 2016 and New York, October 2016). In these meetings, patients and HCPs collaborated on nominating items for exclusion based on round two e‐Delphi results and grouping remaining candidate items into domains. The process of defining domains was achieved in small groups using nominal group theory. Prior to the nominal group exercise, participants were made aware that items could form their own stand‐alone domain or be collected into an umbrella domain, if the items were sufficiently congruent and capable of being measured by a single instrument.

As only a subset of the e‐Delphi group was able to attend the in‐person meetings, all decisions taken at the meetings required confirmation by the larger HISTORIC project group in a subsequent online confirmation survey before implementation. Prior to completion of the survey, e‐Delphi panel members were provided with a summary of the in‐person meeting discussions. A detailed description of consensus meeting I and II and the online confirmation survey has been published.26

E‐Delphi round three

In round three, the items were shown under their newly designated domain, following the work from the in‐person meetings. Items that were marked for exclusion at the consensus meetings were shown at the end, under a heading of ‘marked for exclusion’. Items that were marked for exclusion and did not reach ‘consensus in’ were not carried forward to round four.

HIdradenitis SuppuraTiva cORe outcomes set International Collaboration consensus meeting III

After the third e‐Delphi round, it was noted that some items and domains were considered ‘critical’ to include in the COS only by patients or HCPs but not both. These discordant items were discussed by patients and HCPs at a third in‐person consensus meeting (Copenhagen, February 2017). This discussion was followed by nonbinding voting.

E‐Delphi round four and five

In round four, participants voted at the domain level for the first time and voted again on items within each domain that had still not reached clear ‘consensus in’. A summary of the discussion and voting results from consensus meeting III were provided, in addition to the results from e‐Delphi round three. In round five, participants voted on two domains for which consensus was nearly achieved (more than 67% combined critical votes) to determine whether these should be included in the COS.

HIdradenitis SuppuraTiva cORe outcomes set International Collaboration consensus meeting IV

The results of e‐Delphi rounds one to five and HISTORIC in‐person consensus meetings I–III were presented at the annual IDEOM meeting in Washington, DC in May 2017. All meeting participants were asked if they considered the HISTORIC HS COS process to be methodologically robust and inclusive and if the project had developed an appropriate COS. Consensus for the final core domain set was defined as > 70% of all participants voting yes to both these questions.

Results

Participants

Patient characteristics from the qualitative studies are found in Table 2 and demographics of all Delphi participants are found in Table S1 (see Supporting Information). A total of 42 patients participated in the qualitative studies and a total of 93 (41 patients and 52 HCPs) from 19 countries in four continents participated in the first round of the e‐Delphi. In the last round of the e‐Delphi, 78 participants continued to take part, a 16% attrition rate. Of the 15 individuals who dropped out, nine were patients and six were HCPs, with nine from North America and six from other continents.

Table 2

Characteristics of patients with hidradenitis suppurativa, item generation interviews

Variables	North American patients (n = 21)	Danish patients (n = 21)
Age, years	46·8 ± 13·7	37·9 ± 10·8
Women, n (%)	16 (76)	13 (62)
Ethnicity, n (%)
Non‐Hispanic white	13 (62)	21 (100)
Hispanic	3 (14)	0
Black	2 (10)	0
Asian	1 (5)	0
Mixed ethnicity	2 (10)	0
Hurley stage, n (%)
1	0	3 (14)
2	12 (57)	12 (57)
3	9 (43)	6 (29)
Disease durationa, years	20·5 ± 12·7	19·8 ± 10·0

Data is presented as mean ± SD unless otherwise stated. ^aMissing values for three of the Danish patients.

Candidate outcome items and domains

A list of all 56 items initially included in the Delphi exercise is found in Table 3. The systematic review identified 16 potential candidate items, 33 additional items were identified by patients in the qualitative studies and the HCP item generation survey identified seven further items (Table 3). One item suggested by a patient participant in the first e‐Delphi round (number of chronic areas) was judged to represent a new outcome and added to the list of candidate items in round two and subsequent rounds. Item numbers 1, 4, 7, 8, 10, 13, 16, 30, 39, 41, 45, 47, 53 (Table 3) were nominated for exclusion following consensus meetings I and II,26 did not reach ‘consensus in’ in e‐Delphi round three and were therefore excluded. One item, ‘Pain’ was ranked so highly by both patients and HCPs that it was nominated to form a domain in its own right. Other domains were formed by collecting together similar items during the nominal group exercises, as detailed in the report of consensus meetings I and II.26

Table 3

List of all items initially included in the Delphi exercise (item 57 included after the first round)a

Item number	Item name	Help text	Item number	Item name	Help text
1b	Biomarkers	Measures of disease presence or activity in blood samples	30b	Dyspigmentationb	Changes (lighter or darker) to the normal colour of your skin
2	Drainage	Secretion, blood, stains, suppuration	31	Anatomical location	Body areas and number of body areas involved
3	Oedema	Swelling of the skin	32c	Number of inflamed nodules	Number of red, painful or tender nodules
4	Economic burden	Economic burden to the patient related to the disease (e.g. doctor appointments, surgery, medication), management (e.g. bandages, pads, or diet), time lost	33	Psychological functioning	Feelings of depression, apathy, loneliness, suicidal thoughts. Feelings of irritation, anxiety, stress
5	Coping	Being able to handle (cope with) having the disease	34c	Health‐related quality of life	Perceived physical, mental and social health over time
6	Odour	Unpleasant odour	35c	Number of fistulae	Number of connections to skin surface
7c	Satisfaction with treatment	Satisfaction with effectiveness; time spent on treatment	36c	Pain	Pain
8	Adverse effects of surgical treatments	All types of side‐effects from surgical treatments (e.g. bleeding, infection, contractures)	37	Cognition	Impact on concentration (e.g. at work or at school, or in leisure activities)
9c	Number of cysts	Number of sac‐like pockets under the skin which contain fluid or debris from the skin	38	Fatigue	Physical weariness sometimes combined with mental weariness
10	Comorbidities	Associated diseases, e.g. metabolic syndrome, PCOS or other inflammatory diseases	39	Cosmesis	Visual appearance of a person's skin from his/her own perspective related to the disease and surgery for the disease
11	Intimacy	Impact on sexual health, having desire or feeling desired, pain during sexual activity, abstaining from sex, fear of being rejected	40c	Patient global evaluation	Overall assessment of the disease from the perspective of the patient himself or herself, alone and without the influence of anyone else
12	Ability to work or study	Ability to work or study, ability to gain or keep employment, influence on type of job or study, time off from work or study, impact on career	41	Washing or bathing	Ability to wash or bathe oneself; having to frequently wash or bathe oneself
13	Adverse effects of medical treatments	All types of side‐effects from medical treatments	42b	Ulceration	Absence of upper layers of the skin forming an ulcer
14c	Number of noninflamed nodules	Number of skin coloured nodules which may not be painful or tender	43c	Physician global evaluation	Overall assessment of the disease from the perspective of the physician alone
15	Itch	Itch	44c	Number of sinus tracts	Number of tunnel‐like connections between lesions
16	Self‐treatment, not prescribed	Self‐treatment which is not prescribed (e.g. self‐incision to obtain pain relief, placing ice cubes or warm compresses on boils)	45	Scarring from surgery	Scars resulting from surgery
17c	Number of abscesses	Number of collections of pus (sterile or infected)	46b	Complianceb	A patient's adherence to a recommended course of treatment
18c	Total lesion count	Total number of all types of lesions	47	Satisfaction with care	Access to care, satisfaction with the doctor's knowledge of disease, quality of care, feeling supported by medical personnel
19	Psychosocial functioning	Feelings of being accepted by others, nervous to be in public, withdrawn from relationships	48	Independence	Need to be independent, not to be dependent on others
20	Scarring from HS	Scar formation in involved areas	49c	Time to postop recovery	Time to healing after surgery
21	Need for treatment and bandages	Requirements for prescribed treatment, for example acute treatment, pain killers, topic treatment, in‐hospital treatment and bandages	50	Clothing restrictions	Impact on choice of clothing (e.g. choosing clothes that do not irritate lesions, that cover lesions, that cover stains)
22	Surface area	Area of the skin surface involved	51	Flare frequency	Frequency of flares
23	Impact on close relationships	Impact on relationship to partner or family member, neglect of family, poor understanding of disease by family	52c	Inflammatory lesion count	Total number of all red, painful or tender lesions (abscesses or inflamed nodules)
24c	Time to recurrence	Time to reappearance of activity, such as after surgery or after ending medical therapy	53b	Comedones	Appearance of small ‘blackheads’ on the surface of the skin formed by the blockage of pores
25	Emotional well‐being	Feelings of powerlessness, embarrassment, low self‐esteem	54	Constitutional symptoms	The experience of one or more symptom(s) associated with the development of new lesions (e.g. fatigue, fever‐like sensation, headache)
26c	Decreased mobility	Decreased mobility, skin tightness, may be associated with restrictions in exercising, walking, reaching out, standing, sitting, activities of daily living (e.g. housework)	55b	Erythema	Redness of the skin
27	Satisfaction with social roles	Satisfaction with oneself as a partner, parent, family member, friend or colleague	56	Sleep disturbance	Difficulty sleeping, inability to sleep, poor quality of sleep
28b	Progression of course	Worsening of disease, prevention of worsening	57	Number of chronic areas	Number of chronic areas open for more than 6 weeks
29	Recreation and leisure activity	Interference with leisure/recreational activities (e.g. sports, do‐it‐yourself, playing instruments, scouting, hiking or outdoor life). Interference with planning of such activities

^aThe help text was shown with each item in the e‐Delphi. The item numbers were generated at random before round one of the e‐Delphi; ^bitems generated in the healthcare professionals item generation survey; ^citems generated in the review of the literature and re‐found in patient interview qualitative studies. Remaining items were generated in the patient interview qualitative studies.

The final core domain set

The final core domain set is illustrated in Figure 2. After the last e‐Delphi and the final consensus meeting, the participants agreed to include five domains in the HS COS for clinical trials: pain, physical signs, HS‐specific quality of life, global assessment and progression of course. The domains are further defined by the items that were fused in the process of creating domains (Fig. 2). A ‘symptoms’ domain, containing the items ‘drainage’ and ‘fatigue’, was strongly supported by patients but did not quite reach our a priori definition of ‘consensus in’ from the perspective of HCPs.

Figure 2

The final core domain set in an adapted Outcome Measures in Rheumatology (OMERACT) onion model. (a) Inner ring: the core set, domains (in black) and items (in white) that reached ‘consensus in’ for patients and healthcare professionals (HCPs). (b) Middle ring: domains (in black) and items (in white) that reached ‘consensus in’ for patients or HCPs. (c) Outer ring: items that did not reach ‘consensus in’, but were marked at consensus meetings for the research agenda or important in specific trials. HS, hidradenitis suppurativa.

In total, 79% of consensus meeting IV participants considered the HISTORIC COS process to be inclusive and methodologically robust and 82% felt that an appropriate COS had been achieved and voted to ratify the HISTORIC COS.

Protocol deviations

The HISTORIC consensus meeting III was not planned a priori but was added after round three to allow further discussion of some items and domains where disagreement between patients and HCPs was identified. E‐Delphi round five was added after round four to discuss domains where consensus had nearly, but not quite, been achieved. This resulted in inclusion of the ‘progression of course’ domain in the final COS, but did not affect lack of consensus between HCPs and patients regarding the symptoms domain. As no items or domains ever reached the predefined ‘consensus out’ rule in any rounds, the process focused only on the predefined ‘consensus in’ rule. By comparing the proportions voting critical among HCPs and patients, when both proportions were above 70% threshold, these items/domains were considered part of the COS.

Discussion

We used a rigorous, iterative and inclusive approach to identify consensus among an international group of patients with HS and HCPs, producing five core domains relevant for all types of clinical trials for HS, namely pain, physical signs, HS‐specific quality of life, global assessment and progression of course. There was close agreement among all stakeholders to include the final five domains in the COS. Based on our protocol, the symptoms domain is not included as a core domain because it only reached ‘consensus in’ from the patient perspective and support was insufficient from HCPs. However, the HISTORIC steering group reflected that, because symptoms is a patient‐reported domain and was considered critical by our patient participants, the patient view supersedes that of HCPs in this instance. As a result, the HISTORIC steering group agreed that the symptoms domain should be included in step two of the COS process to search for a suitable instrument for the domain.

Limitations to the present study include that our aim to involve a 1 : 1 ratio of patients to HCPs was not completely reached and that we did not succeed in involving participants from the continents of Africa and South America in the project.

The HISTORIC initiative has begun the process to develop a COS for HS trials. The implementation of a COS for HS clinical trials should improve the interpretation and comparison of future studies testing interventions for HS and reduce the risk of outcome reporting bias and heterogeneity across studies. After achieving consensus on what to measure in HS clinical trials, the next step for the HISTORIC initiative will be to reach consensus on the outcome measurement instruments best suited to measure each of the core domains in the COS.

In conclusion, our present study reports on the robust development of a comprehensive COS for use in all trials assessing interventions for HS. The final COS includes five core outcome domains agreed by both patients and HCPs and a sixth domain, symptoms, is recommended by the HISTORIC steering group because it is a patient‐reported domain that received strong support from our patient stakeholder group. The routine adoption of this COS in future HS trials should ensure that outcome domains of importance to both patients and HCPs are included and reported.

Acknowledgments

We thank all Delphi participants for their willingness to participate, time and patience. We thank Selma Bajric, Medical University of Vienna, Department of Dermatology and Amanda Pacia, IDEOM for logistic help at the consensus meetings.

Author notes

Funding sources This work is supported by grants from the International Dermatology Outcome Measures (IDEOM). L.T. is supported by the Region Zealand Research Foundation. Musculoskeletal Statistics Unit, The Parker Institute is supported by grants from The Oak Foundation. J.R.I. is supported by a health fellowship from Health and Care Research Wales.

Conflicts of interest L.T., J.R.I., J.S.K. and G.B.E.J. are or have been involved in the development of instruments that could potentially be used to measure core outcome domains for hidradenitis suppurativa. Specific instruments were, however, not discussed in the present study.

L.T. and J.R.I. contributed equally to the work and should be considered joint first authors.

A.G. and G.B.E.J contributed equally to the work and should be considered joint senior authors.

Plain language summary available online