The benefit–risk balance for biological agents in juvenile idiopathic arthritis: a meta-analysis of randomized clinical trials

Abstract

Objective

To assess the net benefit of biological agents (BA) used in JIA.

Methods

We systematically searched databases up to March 2019 for randomized controlled trials (RCT) performed in JIA disease. Separate random-effects meta-analyses were conducted for efficacy (ACR paediatric score 30%, ACRpedi30) and serious adverse events for safety. In order to standardize the baseline risk, we performed a meta-analysis of baseline risk in the control group (for both efficacy and safety meta-analysis). The net benefit was determined as the risk difference of efficacy subtracted by the risk difference of safety.

Results

We included 19 trials: 11 parallel RCTs (754 patients) and 8 withdrawal RCTs (704 patients). The net benefit ranged from 2.4% (adalimumab) to 17.6% (etanercept), and from 2.4% (etanercept) to 36.7%, (abatacept) in parallel and withdrawal trials assessing non-systemic JIA, respectively. In the systemic JIA category, the net benefit ranged from 22.8% (rilonacept) to 70.3% (canakinumab), and from 32.3% (canakinumab) to 58.2% (tocilizumab) in parallel and withdrawal trials, respectively.

Conclusion

The results suggest that a greater number of patients experienced therapeutic success without serious adverse events in the systemic onset JIA category compared with the BAs for non-systemic JIA categories. Baseline risk, design of trial and JIA categories impact the measure of net benefit of BAs in JIA patients.

juvenile idiopathic arthritis, benefit-risk balance, absolute net benefit, biological agents, meta-analysis

Rheumatology key messages

• Net benefit of biological agents forJIA is favourable, although variable.

• Benefit related to efficacy of biological agents is higher in systemic onset JIA.

• Safety of biological agents is higher in withdrawal trials than in parallel trials.

Introduction

Efficacy (benefit) and safety (risk) of therapeutic interventions are analysed separately in most trials and meta-analyses. Many methods have been proposed to promote simultaneous benefit–risk analysis [1]. In the field of rheumatology, the OMERACT group has identified the need of a simple tool that incorporates both benefits and risks in one scale and proposed a table with three rows for different risk severities and three columns for different benefit levels [2]. This method uses patient-level data from randomized controlled trials (RCTs).

The JIA ACR response (ACRpedi) is the first clinical tool used to asses an improvement or flare in JIA patients enrolled in clinical trials [3]. It is a composite score with six core response variables. The ACR paediatric score 30% (ACRpedi30) response corresponds to a >30% improvement in at least three of the six JIA ACR core response variables without ≥30% worsening in more than one of the remaining JIA ACR core response variables compared with baseline. The ACRpedi30 is accepted by both the US Food and Drug Administration and the European Medicines Agency. Therefore, most RCTs of BAs for JIA use this instrument as an efficacy outcome [4]. RCTs are typically designed to measure the efficacy of an intervention while measuring its safety [5].

Safety events are usually codified in a standardized way using the Medical Dictionary for Regulatory Activities (MedDRA). At the end of a trial, data are categorized and summarized, and adverse events are grouped into five broad categories [6]. In addition, serious adverse events (SAEs) are often assessed independently for causality by the investigator and the sponsor, although it may impact the decision to continue, modify or end the trial [7]. However, MedDRA does not provide severity descriptors of frequency qualifiers. Considering this quantitative issue, it is justified to initiate a process of quantification of adverse events and to assess its impact by modelling the balance of benefit and risk using data from RCTs. Using systematic review and meta-analysis methods, we aim to assess the benefit–risk balance through the net benefit of BAs vs placebo or standard treatment in JIA disease.

Methods

Data sources and searches

The protocol of the review was registered on 5 September 2018 on the International Prospective Register of Ongoing Systematic Review (PROSPERO) database under the register number: CRD42018107592, available at: https://www.crd.york.ac.uk/prospero/display_record.php? RecordID=107592.

We searched MEDLINE via PubMed, the Cochrane Central Register of Controlled Trials and ClinicalTrial.gov register up to 12 March 2019. We restricted our search to reports in English. Conference abstracts and secondary analyses of RCTs were excluded. We also contacted relevant pharmaceutical companies to identify unpublished trial data. The search strategy included keywords related to RCTs, JIA and BAs as presented in Supplementary Tables S1 and S2, available at Rheumatology online. The method used is consistent with the recommendations of the Cochrane Handbook for Systematic Reviews of Intervention [8] and reports according the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) [9].

Study selection

We included single- or double-blinded RCTs of BAs on JIAs fulfilling the established diagnosis criteria by the following international organizations: the ACR since 1997 and the ILAR/EULAR since 2001 [10]. Thus, we included: paediatric population, both sexes, aged <19 years old and diagnosed with JIA disease. All JIA subgroups were eligible.

RCTs comparing BAs alone or in combination with conventional synthetic DMARDs or CS vs placebo or standard treatments were eligible for inclusion. Participants could take other DMARDs, NSAIDs or CS with stable doses, and were then randomly allocated to treatment with or without BAs. The BAs of interest were: etanercept, adalimumab, infliximab, anakinra, canakinumab, rilonacept (not authorized in the European Union), rituximab and abatacept. There were no restrictions regarding dosage or duration of the intervention.

Inclusion criteria were defined a priori, and two outcomes of interests were: (i) the JIA ACR response criteria (ACRpedi30) for efficacy and (ii) SAEs for safety. The efficacy outcome depends on the trial design. For parallel RCTs, the ACRpedi30 was considered according to what was explained in the Introduction section, and in withdrawal RCTs, JIA relapse is defined as 30% or greater worsening in three of the six JIA core response variables without >30% improvement in more than one remaining JIA core response variables.

We have excluded studies when safety and efficacy analyses reported zero events simultaneously in both arms [11, 12].

Data extraction and quality assessment

We extracted study characteristics design, inclusion and exclusion criteria, patient characteristics, drug characteristics and number of events of the two co-primary safety and efficacy outcomes. The quality was assess using the Risk of Bias Tool from The Cochrane Collaboration [13]. Two reviewers (N.C. and G.A.-P.) independently extracted data for all eligible articles and assessed if the internal validity of studies was adequate. Any disagreements were resolved by consensus.

Data synthesis and analysis

First, we conducted separate meta-analyses for efficacy and safety outcomes. The efficacy (response or relapse ACRpedi30, according trial design) and the safety outcomes (SAEs) were chosen based on the confidence and comparability of those measures in RCTs. Summary odds ratios (ORs) for efficacy and safety outcomes were calculated using a random-effects model. For the latter, we used Peto’s methods to pool ORs, which is adequate for very rare events as SAE and which does not require corrections for zero cell count [11]. Secondly, we converted the ORs and Peto’s ORs of the meta-analysis into relative risk in order to project them on the characteristic baseline risks of sub-population (control group). In order to standardize the baseline risk, we performed a meta-analysis in the control group. The pooled results of efficacy and safety responses were used to obtain the risk difference (RD). Thirdly, the RD for efficacy and RD for safety for each BA was calculated. The overall benefits and risk balance, as criterion of the net benefit, were calculated using the difference between the RD of efficacy and RD of safety (i.e. RD efficacy – RD safety), when appropriate. The net benefit shows in absolutes values the number of patients who experience therapeutic success without SAEs.

The random-effect method was chosen because JIA is a disease with different subtypes that can introduce heterogeneity into the results; in addition the random effects method and the fixed effects method will give identical results when there is no heterogeneity between studies. The index I² was estimated to assess the heterogeneity and inconsistency of our meta-analyses. High heterogeneity is usually characterized by an I² of ≥50%, suggesting the presence of substantial statistical heterogeneity [14]. Efficacy meta-analyses were conducted in subgroup analysis depending on the trial design (parallel vs withdrawal RCTs), because withdrawal RCTs select only responding ACRpedi30 patients treated in an open fashion who are then randomized in a double-blind fashion to continue the active treatment or receive placebo, and according JIA disease [systemic-onset JIA (SoJIA) vs non-systemic JIA]. Safety meta-analyses were only separated according to the trial design. Subgroup analyses were also conducted according to the BA type for efficacy and safety meta-analysis. In the presence of heterogeneity, a meta-regression and subgroup analysis were conducted to investigate if studies’ characteristics were associated with the treatment benefit. The parameters included in meta-regression were year of publication, trial design, age of patients, JIA categories, duration of randomized phase, total number of patients in randomized phase, immunosuppressive drug in the control group during the randomized phase, previous BA treatment throughout the randomized phase of trial, and baseline risk for efficacy and safety meta-analysis. The risk of publication bias was determined by visual aspect of funnel plot and the Egger test [15]. All analyses were performed using R Software and its ‘metafor’ package (https://cran.r-project.org/web/packages/metafor/metafor.pdf).

Results

Study selection

A total of 184 citations were retrieved in the initial search through the different databases. Seventy records were excluded on titles and abstracts and 113 studies were assessed for eligibility. During the qualitative synthesis, five additional studies were excluded [16–20] (see Fig. 1).

Fig. 1

PRISMA 2019 flow chart of literature search process of the systematic review

PRISMA: Preferred Reporting Items for Systematic Review and Meta-Analyses.

Characteristics of included studies

We included 18 articles covering 19 trials (involving 1458 patients) conducted on JIA disease. Their characteristics are described in Table 1. Trials were mainly conducted in the OA and polyarticular (PA) JIA categories with two trials including patients with early PA JIA [28, 32]. None of the trials included the undifferentiated JIA category. Among the BAs, trials evaluating anti-IL-1s (anakinra, canakinumab and rilonacept) were conducted exclusively for the SoJIA category, while this category was mostly excluded of trials evaluating anti-TNFs (etanercept, adalimumab and infliximab). Within the non-systemic JIA subgroup, the standard treatments were used in the control arm. In the SoJIA subgroup, the control arm consisted of placebo.

Table 1

Characteristics of included studies

First author, year	Ref.	bDMARD	Trial type	Efficacy outcome	Randomized periodb	JIA subtypes	Randomized patientsc	Age (s.d). (years)	Disease durationd
Lovell et al., 2000	[21]	Etanercept	wRTC	JIA flare	16	OA, PA and SoJIA	51	10.5	5.8
Ruperto et al., 2007	[22]	Infliximab	pRCT	ACRpedi30	14	OA, PA and SoJIA	122	11.2 (4.0)	3.9 (3.5)
Lovell et al., 2008	[23]	Adalimumab	wRTC	JIA flare	32	PA JIA	133	11.2 (3.7)	3.7 (3.7)
Ruperto et al., 2008	[24]	Abatacept	wRTC	JIA flare	24	PA JIA	122	12.3 (3.0)	3.9 (3.6)
Yokota et al., 2008	[25]	Tocilizumab	wRTC	JIA flare	12	SoJIA	44	8.3 (4.4)	2.1 (1)
Ilowite et al., 2009	[26]	Anakinra	wRTC	JIA flare	16	OA, PA and SoJIA	50	11.0	4.1
Quartier et al., 2011	[27]	Anakinra	pRCT	ACRpedi30e	4	SoJIA	24	8.5 (4.5)	3.7 (2.7)
Tynjala et al., 2011a	[28]	Infliximab	pRCTe	ACRpedi30	54	Early PA JIA	60	10.3 (3.3)	0.3 (0.1)
De Benedetti et al., 2012	[29]	Tocilizumab	pRCT	ACRpedi30e	12	SoJIA	112	9.6 (4.5)	5.4 (4.2)
Horneff et al., 2012	[30]	Adalimumab	pRCT	ACRpedi30f	12	Jo AS	32	15.3 (1.6)	3.2 (2.3)
Ruperto et al., 2012	[31]	Canakinumab	pRCT	ACRpedi30e	2	SoJIA	84	8.5	2.2
Ruperto et al., 2012	[31]	Canakinumab	wRTC	JIA flare	56	SoJIA	100	8.0	2.3
Wallace et al., 2012	[32]	Etanercept	pRCT	ACRpedi70	26	Early PA JIA	85	10.5 (4.4)	0.4 (0.1)
Lovell et al., 2013	[33]	Rilonacept	pRCT	ACRpedi30e	4	SoJIA	24	12.6 (4.3)	3.1
Ilowite et al., 2014	[34]	Rilonacept	pRCT	ACRpedi30	4	SoJIA	71	10.0 (4.5)	2.6 (3.4)
Brunner et al., 2015	[35]	Tocilizumab	wRTC	JIA flare	24	OA and PA JIA	166	11.0 (4.0)	4.2 (3.7)
Burgos-Vargas et al., 2015	[36]	Adalimumab	pRCT	ACRpedi30	12	ERA JIA	46	12.9 (2.9)	2.6 (2.3)
Horneff et al., 2015	[37]	Etanercept	wRTC	JIA flare	24	ERA JIA	38	13.4 (2.4)	3.2 (2.8)
Hissink Muller et al., 2017	[38]	Etanercept	pRCT	Inactive disease	12	OA, PA PsA JIA	94	9.2	0.6
Total					360		1458	10.8 (3.7)	3.0 (2.6)

a

RCT in open label fashion; ^btreatment duration in weeks in randomized period of studies; ^ctotal number of patients who were randomized; ^ddisease duration in weeks (s.d.); ^eACRpedi30 including fever; ^fuse ASAS40 for the primary outcome but inform also the ACRpedi30. ACRpedi30: ACR paediatric score 30%; bDMARD: biologic DMARD; pRCT: randomized placebo-controlled trial; wRTC: withdrawal randomized controlled trial. ERA: enthesitis-related arthritis; PA: polyarticular; SoJIA: systemic-onset JIA.

When concomitant medications were used such as NSAIDs and/or conventional DMARDs, doses were stable and distributed in both arms. Twelve trials (63%) used MTX as comparator in the control group, two of which (11%) were performed in early forms of PA JIA [28, 32]. Eleven parallel RCTs (58%) were identified, but 10 were included for efficacy estimates [22, 27–31, 33, 34, 36, 38], as one of them did not report the ACRpedi30 score [32]. However, this one trial was included when exploring safety. All eight withdrawal RCTs (42%) assessed the efficacy outcome of interest [21, 23–26, 31, 35, 37]. Two trials (11%) were excluded from the safety analysis [26, 27] due to zero events in both arms.

Ninety-five percent (18/19) of the trials were double blinded [28]. A high risk of selective reporting bias was found in four trials [22, 26, 28, 34] (see Figs 2 and 3). The visual inspection of funnel plots exploring publication bias was slightly asymmetrical for both efficacy in parallel trials (ACRpedi30) (see Fig. 4). The Egger tests were not in favour of publication bias in efficacy meta-analysis from parallel RCTs (P = 0.41). We were not able to assess the publication bias for the efficacy outcomes in withdrawal RCTs because of the low number of studies or for safety outcomes because Fisher scoring algorithm did not converge (funnel plot is shown in Fig. 4B).

Fig. 2

Review authors’ judgements about each risk of bias item presented as percentages across all included studies

Fig. 3

Review authors’ judgements about each risk of bias item for included studies.

+, low risk of bias; ?, unclear risk of bias; –, high risk of bias.

Fig. 4

Funnel plots of trials

Efficacy outcome for analysis: ACRpedi30 score in parallel RCTs (A), and flare in withdrawal RCTs (B). ACRpedi30: ACR paediatric score 30%; RCT; randomized controlled trial.

Assessment of efficacy of BAs by meta-analysis

In parallel RCTs, the ACRpedi30 response was significantly improved for non-systemic JIA categories (OA or PA JIA, enthesitis-related arthritis and PsA) (OR = 2.19, 95% CI: 1.35, 3.56) in the BAs group compared with standard treatments, as for the SoJIA category (OR = 11.50, 95% CI: 3.37, 39.21) compared with placebo. In withdrawal RCTs, significantly fewer relapses (ACRpedi30 worsening response) occurred for non-systemic JIA categories in the BAs group compared with standard treatment (OR = 0.27, 95% CI: 0.19, 0.39) and for SoJIA (OR = 0.13, 95% CI: 0.03, 0.63) compared with placebo (Table 2 and Supplementary Fig. S1, available at Rheumatology online). Heterogeneity was substantial and statistically significant (I² = 74%, Tau² = 1.37) for the SoJIA category from parallel RCTs. Conversely, we did not detect any significant heterogeneity for the other groups.

Table 2

Net benefit for each biological agent according to juvenile idiopathic arthritis disease and trial design

Outcome/ intervention	N [references]	OR (95% CI)	Anticipated absolute effects, % (95% CI)			N [references]	Peto’s OR (95% CI)	Anticipated absolute effects, % (95% CI)			Net benefit (%)
			Without BAs	With BAs	Difference			Without BAs	With BAs	Difference
Non-systemic JIA
ACRpedi30 in parallel RCT						SAEs in parallel RCT
Infliximab	n = 181 [22, 28]	3.39 (0.48, 24.11)	60.6 (46.1, 73.5)	83.9 (42.5, 97.4)	23.3 (1.1, 56.9)	n = 181 [22, 28]	2.14 (0.21, 21.99)	3.6 (1.7, 6.3)	16.0 (1.6, 164.2)	12.4 (1.3, 64.5)	10.9
Adalimumab	n = 78 [30, 36]	2.06 (0.79, 5.38)		76.0 (54.9, 89.2)	15.4 (1.0, 52.5)	n = 78 [30, 36]	2.22 (0.29, 17.15)		16.6 (2.2, 128.1)	13.0 (1.2, 59.3)	2.4
Etanercept	n = 94 [38]	2.58 (1.00, 6.65)		79.9 (60.6, 91.1)	19.3 (1.2, 58.5)	n = 179 [32, 38]	0.70 (0.08, 6.12)		5.2 (0.6, 45.7)	1.6 (0.2; 8.9)	17.6
Relapses in withdrawal RCT						SAEs in withdrawal RCTa
Etanercept	n = 89 [21, 37]	0.12 (0.05, 0.33)	56.6 (45.7, 67.0)	13.5 (6.1, 30.1)	43.1 (2.7, 134.4)	n = 89 [21, 37]	7.54 (0.77, 73.71)	2.8 (0.07, 6.1)	43.4 (4.4, 424.7)	40.6 (4.1, 206.3)	2.4
Adalimumab	n = 133 [23]	0.31 (0.15, 0.64)		28.8 (16.4, 45.5)	27.8 (0.6, 31.7)	n = 133 [23]	0.13 (0.00, 6.52)		0.7 (3.0, 37.6)	2.1 (0.2, 11.3)	29.9
Abatacept	n = 122 [24]	0.22 (0.10, 0.49)		22.3 (15.5, 39.0)	34.3 (1.2, 59.2)	n = 122 [24]	0.14 (0.01, 2.22)		0.4 (0.0, 6.2)	2.4 (0.3, 17.3)	36.7
Anakinra	n = 50 [26]	0.29 (0.08, 1.09)		27.4 (9.4, 58.7)	29.2 (1.2, 60.8)	–	–		–	–	–
Tocilizumab	n = 163 [35]	0.37 (0.19, 0.72)		32.5 (19.9, 48.4)	24.1 (0.5, 27.5)	n = 206 [25, 35]	1.03 (0.25, 4.19)		2.9 (0.7, 11.7)	0.1 (0.0, 0.7)	24.0
Systemic-onset JIA
ACRpedi30 in parallel RCT						SAEs in parallel RCTb
Anakinra	n = 24 [27]	22.00 (2.05, 236.05)	21.7 (12.0, 36.0)	85.9 (36.2, 98.5)	64.2 (1.4, 72.2)	–	–	3.6 (1.7, 6.3)	–	–	–
Tocilizumab	n = 112 [29]	30.22 (10.25, 89.11)		89.3 (74.0, 96.1)	67.6 (0.8, 38.6)	n = 112 [29]	4.64 (0.56, 38.36)		16.7 (2.0, 138.1)	13.1 (1.2, 62.0)	54.5
Canakinumab	n = 84 [31]	40.47 (11.18, 146.45)		91.8 (75.6, 97.6)	70.1 (0.7, 37.4)	n = 84 [31]	0.95 (0.13, 7.01)		3.4 (0.5, 25.2)	0.2 (0.0, 1.4)	70.3
Rilonacept	n = 95 [33, 34]	3.30 (1.23, 8.87)		47.8 (25.4, 71.7)	26.1 (0.8, 41.1)	n = 95 [33, 34]	1.90 (0.24, 15.19)		6.8 (0.9, 54.7)	3.2 (0.3, 15.9)	22.8
Relapses in withdrawal RCT						SAEs in withdrawal RCTc
Tocilizumab	n = 43 [25]	0.05 (0.01, 0.24)	67.8 (33.2, 89.9)	9.5 (2.1, 33.6)	58.3 (6.2, 310.5)	n = 206 [25, 35]	1.03 (0.25, 4.19)	2.8 (0.07, 6.1)	2.8 (0.7, 11.3)	0.1 (0.0, 0.7)	58.2
Canakinumab	n = 100 [31]	0.26 (0.11, 0.62)		35.4 (18.8, 56.6)	32.4 (0.8, 39.8)	n = 100 [31]	1.00 (0.30, 3.32)		2.8 (0.8, 9.0)	0.1 (0.0, 1.0)	32.3

Withdrawal design estimated the rate of relapse whereas parallels trials estimated the rate of response. ^aSignificant heterogeneity in systemic-onset JIA category from withdrawal RCTs in preliminary efficacy outcomes. ^bAnakinra with simultaneous zero event in both arms. ^cNet benefit for tocilizumab using results of preliminary safety analysis from non-systemic and systemic onset JIA categories. OR: odds ratio; BA: biological agent; ACRpedi30: ACR paediatric score 30%; RCT: randomized controlled trial; SAE: serious adverse event.

In meta-regression analysis, the SoJIA category seems to be associated with better efficacy outcome (coefficient 0.46, R² = 60.2%, Tau² = 0.025), as well as the number of patients with previous BAs in control (coefficient 0.021, R² = 71.9%, Tau² = 0.022) and intervention group (coefficient 0.013, R² = 100%, Tau² = 0.0), for both parallel and withdrawal RCTs. The other tested parameters (year of publication, trial design type, duration of randomized phase, study size, concomitant immunosuppressive treatment and age of patients) were not associated with different treatment effects. These results are shown in Supplementary Table S3, available at Rheumatology online. Also, we have explored the parameters used in meta-regression, in a linear regression model. A forward selection model found that only the SoJIA subtype was correlated with the treatment efficacy (P < 0.05).

We included, as additional analysis, efficacy meta-analysis with ACRpedi50 and ACRpedi70 only with parallel RCTs, and results of both meta-analyses were in line with the results of efficacy meta-analysis with the ACRpedi30 score. Doing this with withdrawal RCTs was not possible because the studies’ outcomes do not allow this. Therefore, it was made only with the studies previously mentioned and when compared, the difference was not large (Supplementary Fig. S2, available at Rheumatology online).

Safety of BAs

There were significantly more SAEs in the BAs group compared with the control group for parallel RCTs (OR = 2.00, 95% CI: 0.94, 4.26). In withdrawal RCTs, the pooled OR was inconclusive (OR = 1.01, 95% CI: 0.45, 2.24). Except for anakinra and abatacept, all BAs had at least one SAE during the randomized period of follow-up (Supplementary Fig. S3, available at Rheumatology online).

Net benefit estimate

There are large variations in RDs between the different BAs for efficacy and safety outcomes. The baseline risk of efficacy outcomes (ACRpedi30 or relapses without BA according to trial design) also varied widely. The net benefit was different according to subgroups delimited by JIA categories and trial design.

In general, BAs seemed to show higher efficacy in SoJIA in withdrawal (range 32.3–58.2%) and parallel (range 22.8–70.3%) RCTs compared with non-systemic JIA in withdrawal (2.4–36.7%) and parallel (range 2.4–17.6%) RCTs (Table 2). However, because of the large CI of estimates, comparisons could not be established.

The two trials assessing anakinra had zero SAEs during the randomized period of interest and the net benefit could not be calculated [26, 27]. The results of the net benefit analysis are summarized in Table 2.

Discussion

We included 19 RCTs involving 1458 patients assessing the efficacy and safety of all BAs approved for treating JIA. To our knowledge, this is the first meta-analysis that simultaneously evaluates the benefits and risks of BAs in one scale in patients with JIA disease. Our results suggest a net benefit in favour of BAs in the short-term follow-up assessed RCTs.

There was consistent evidence of efficacy effects for all approved BAs, to prevent relapses in withdrawal RCTs (ACRpedi score not worsening >30%) and achieve a clinical improvement (ACRpedi30 response) in parallel RCTs. However, the great variability in the estimates of net profit prevents formal comparisons despite having standardized with the control group, the baseline risk of the population. In addition, BAs treatment could not be compared because of high risk of bias related to indirect comparisons.

Differences of treatment effects reported in meta-analysis of safety outcomes were found, suggesting the influence of trial design in the estimate of treatment effect of JIA in BAs. Pooled estimate of SAEs in withdrawal RCTs were inconclusive. Although the net benefit model for this study uses already published trial data, results suggest that there are differences because of BAs and in between BAs depending on the trial design and categories of JIA. The OMERACT group proposes to use individual patient data, but they also used RCTs and the same clinical criteria of efficacy (ACR score) and safety (SAEs) as us [39]. Both outcomes are rigorous enough and clinically relevant to be considered for modelling a net benefit assessment.

Parallel and withdrawal RCTs have different objectives. While parallel RCTs asses the efficacy of BAs to achieve clinical remission, withdrawal RCTs evaluate maintenance of remission and inactive disease only in a specific sub-population (i.e. the ACRpedi30 responders) [40]. On that basis, we decided to analyse these two groups of trials separately. We grouped the non-systemic JIA categories as a different group from the SoJIA, then we analysed the efficacy of each BA. For SAE, we decided to focus on trial design, assuming that the effect-size of safety is a function of each BA and not remarkably related to the underlying disease.

In the meta-analyses for efficacy and safety outcomes, we noted that the effect sizes of clinical responses seem to vary according to the category of JIA in subgroup analyses of BAs. In parallel RCTs, the SoJIA category showed larger effect size to achieve efficacy compared with PA categories of JIA. A possible explanation, supported by the meta-regression results, may be that the efficacy of BAs seems higher in more symptomatic disease, such as the SoJIA category, in comparison with PA JIA. Although direct comparisons between effect size according to JIA categories were not performed, previous meta-analyses have suggested, by indirect comparisons, that the effect size of some BAs could be comparable in the SoJIA category or PA JIA categories. The effect size of tocilizumab did not differ from an anti-TNF (adalimumab) in PA JIA [41] and from two anti-IL-1s (anakinra and canakinumab) in SoJIA categories [42]. In addition, abatacept, anakinra and tocilizumab did not show significant differences in effect size to prevent flare in PA categories of JIA [43]. The effect size variation found could be attributed to many factors such as co-prescription drugs, categories of JIA disease and length of follow-up. The subsequent large variation founded in the absolutes risk, assessed by the RD in efficacy and RD in safety outcomes, justifies the modelling of the net benefit. On the other hand, the ‘number of patients’ previously treated by BAs could explain, at least in part, the heterogeneity found, as suggested by the meta-regression, being significant in both arms (control and intervention).

Differences in baseline risks were found, mainly in SoJIA from withdrawal and parallel trials. In addition, fewer SAEs with two BAs (adalimumab and abatacept in non-systemic JIA categories from withdrawal RCTs). This discrepancy could be attributed to the differences in the previous time of exposure to such BAs in withdrawal RCTs and to the fact that some SAEs occur early in the first months of treatment [44]. For patients who presented an SAE during the first open phase of withdrawal RCTs, we hypothesized that they have not been included in the randomized phase, and thus their safety data were not analysed [5, 45].

In the net benefit model presented here, we assume that an ACRpedi30 has the same weight as an SAE. We decided to use the ACRpedi30 score because most RCTs evaluate it as primary outcome. We used the SAEs for the reason that the causality is methodically verified, and because they are the main way of monitoring safety in RCTs [7]. However, trials are not sufficient to fully determine the potential harmful effects of BAs. Although the accurate coding of the events using MedDRA and the safety evaluation by the sponsor can be difficult, the quality of SAEs reporting in terms of completeness and accuracy are of paramount importance [7]. For SAEs, meta-analysis may be the only way to obtain reliable estimates of safety events occurring in randomized trials [11].

The generalizability of these results is subject to certain limitations. First, the assessment of the benefit–risk balance remains a complex issue with no unique method to analyse it. The weighting of benefit and risk events is an unresolved issue in JIA for the modelization of BA treatment effect. We consider in this study that SAE and JIA relapse had the same clinical weight. Second, the visual asymmetry found in the forest plots were not assumed as a publication bias in this systematic review but are more likely related to the heterogeneity of the JIA, which could have affected the measurements of effect size. To address the issue, we conducted subgroups analyses according to categories of JIA for the efficacy outcome. Third, a risk of selective reporting was present in 19% of trials. Finally, the lack of scoring of SAEs, the short-term follow-up and the relative restricted size of participants may negatively impact the results of the pooled estimates of effect size of BAs. Despite their exploratory nature, our meta-analyses suggest that quantification of the benefit–risk balance of BAs is necessary regardless of the frequency of adverse events.

Conclusion

We present a net benefit model adapted to summarize data from RCTs performed on BAs in JIA disease. The results suggest that a greater number of patients experienced therapeutic success without SAE in the SoJIA category compared with the BAs for non-systemic JIA categories. Baseline risk, design of trial and JIA categories impact the measure of net benefit of BAs in JIA patients.

Acknowledgements

The authors express their sincere thanks to Giuliana Cattivelli (University of Asuncion—Paraguay) and Leticia Segovia-Cabrera (Biostatistics Department, Medicine School of the Catholic University in Asuncion—Paraguay) for help in manuscript preparation. Individual scholarship [Paraguay’s BECAL program (Becas Don Carlos Antonio Lopez—http://www.becal.gov.py)] attributed to a one author (N.C.).

Funding: No specific funding was received from any funding bodies in the public, commercial or not-for-profit sectors to carry out the work described in this manuscript.

Disclosure statement: The authors have declared no conflicts of interest.

Supplementary data

Supplementary data are available at Rheumatology online.

PROSPERO register number: CRD42018107592.