The difference in all-cause mortality between COVID-19 patients treated with standard of care plus placebo and those treated with standard of care alone: a network meta-analysis of randomised controlled trials of immunomodulatory kinase inhibitors

Abstract

Objectives

The aim of this network meta-analysis (NMA) was to assess whether participants assigned to a placebo and standard of care (SoC) group had different major coronavirus disease 2019 (COVID-19)-related outcomes than those assigned to SoC alone.

Design

Frequentist model-based NMA.

Setting

We searched for randomised controlled trials (RCTs) of Janus kinase/Bruton tyrosine kinase inhibitors for the management of COVID-19.

Participants

Patients with COVID-19 infection.

Main outcome measures

The primary outcome was the 28-day all-cause mortality, and secondary outcomes were: (1) use of mechanical ventilation; (2) secondary bacterial infection; (3) acceptability (i.e. drop-out rate); and (4) safety (i.e. serious adverse events). We conducted an NMA using the frequentist model. Effect sizes were estimated using odds ratios (ORs) with 95% confidence intervals (95% CIs).

Results

We identified 14 eligible RCTs enrolling a total of 13,568 participants with COVID-19. Participants assigned to placebo plus SoC had a significantly higher risk of 28-day all-cause mortality than those receiving SoC alone (OR = 1.39, 95% CI = 1.07–1.79). This finding did not change substantially by subgroup analysis stratified by epidemiology factor, pandemic history progression and statistical methodologic consideration. In addition, none of the treatments investigated were associated with a significantly different risk of secondary bacterial infection, acceptability or safety compared with the SoC group.

Conclusions

This NMA suggested a higher all-cause mortality in patients treated with placebo plus SoC compared with those treated with SoC alone. However, caution is advised in interpreting these results due to the absence of a direct head-to-head comparison. Future research should critically evaluate the necessity of placebo administration in COVID-19 RCTs and consider alternative study designs to minimise potential biases.

Trial registration: The current study was approved by the Institutional Review Board of the Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan (TSGHIRB No. B-109–29) and registered in PROSPERO (CRD42022376217).

Keywords

Network meta-analysis mortality COVID-19 immunomodulatory kinase inhibitor placebo

Introduction

To date, the coronavirus disease 2019 (COVID-19) pandemic has resulted in millions of deaths worldwide.^1,2 The immunomodulatory kinase inhibitors, including Janus kinase (JAK) and Bruton tyrosine kinase (BTK) inhibitors, are among the most widely researched agents to reduce mortality related to COVID-19. The rationale for the therapeutical benefit of immunomodulatory kinase inhibitors was based on the observation of immune dysregulation and an associated imbalance in JAK and signal transducer and activator of transcription pathways during severe COVID-19 infection.³ Similarly, the major cause of mortality related to COVID-19 resulted from acute/subacute lung injury, which was linked to over-expression of BTK.⁴ Recently, Ngamprasertchai et al.⁵ published a network meta-analysis (NMA) investigating the efficacy of individual immunomodulators on reducing all-cause mortality in patients with COVID-19, which revealed that, among JAK/BTK inhibitors, baricitinib was the only regimen associated with a lower 28-day all-cause mortality compared with the control group (which included both placebo plus standard of care [SoC] and SoC-only groups). However, a difference in the efficacy between the SoC-only and SoC plus placebo controls on all-cause mortality was observed. Although the authors of the NMA did not address the possible reasons for the difference in all-cause mortality rates between patients with SoC-only or SoC plus placebo, we hypothesised that this difference in efficacy between the SoC-only and SoC plus placebo arms might have arisen for two possible reasons: (1) an actual harmful placebo effect; or (2) methodological weaknesses.

The use of placebo in clinical randomised controlled trials (RCTs) has been a subject of debate, especially with regard to the potential impact related to inadequate placebo application.⁶ Application of placebo would contribute to favourable behavioural change⁷ or unfavourable behavioural changes.⁸ This potential impact by the use of placebo could be found in the major outcomes of certain diseases. In a systematic review of cardiovascular studies using all-cause mortality as the outcome,⁹ the mortality rate in the placebo arm of an indicated study of peripheral arterial disease was increased in comparison with the average mortality in the ordinary peripheral arterial disease population.¹⁰

Second, possible methodological weaknesses might have created bias in the previous NMA.⁵ Specifically, the authors included RCTs without any events in one arm in their analysis. Although the authors did not specify their statistical continuity correction method, we could infer the possible continuity correction method used from the supplementary figures. There was debate about the risk of bias associated with this continuity correction method,¹¹ which might result in a counterintuitive statistical result.¹²

This study aimed to re-examine whether participants assigned to an SoC plus placebo group might have different major outcomes of COVID-19 compared with those assigned to an SoC-only group.

Methods

The detailed information of the method has been described in the supplementary material (Appendix: eMethods and materials). In brief, we followed the PRISMA guideline¹³ (eTable 1) and focused on RCTs of JAK/BTK inhibitors in acute COVID-19 treatment. The detailed search strategies and keywords used in each database are listed in eTable 2. We selected 28-day all-cause mortality as our primary outcome. The reason for choosing 28 days as the period for ascertaining the primary outcome was that the most all-cause mortality occurs during the first month of COVID-19.^14,15 The secondary outcomes included: (1) use of mechanical ventilation; (2) secondary bacterial infection; (3) acceptability; and (4) safety profile. The odds ratio (OR) with 95% confidence intervals (95% CIs) was used as the measure of effect size. We used the frequentist model for NMA.¹⁶ All analyses were performed using network suite for Stata version 16.0 (StataCorp LLC, College Station, TX, USA).¹⁶ Our NMA model is a linear mixed model that combines direct and indirect evidence to compare multiple treatments. To enhance the clinical application of our results, we ranked the treatment probability for the analysis of each outcome to calculate the surface under the cumulative ranking curve (SUCRA). The SUCRA represents the cumulative probability of a specified treatment being superior to a hypothetical treatment, thus indicating the statistical superiority of the specified treatment. We evaluated potential inconsistencies between the direct and indirect evidence using a loop-specific approach, node-splitting method and the design-by-treatment interaction mode. The inconsistency test was another method to test the transitivity assumption. In addition, we evaluated the heterogeneity of tau values among the main results of this study. In the current NMA, we conducted subgroup analyses to eliminate the potential impact of confounding factors. We conducted subgroup analyses considering three aspects: epidemiological factors (age-stratification subgroup); pandemic history progression (subgroup of different publication years); and statistical considerations (subgroup of RCTs without any zero events and subgroup of RCTs with a placebo arm). Specifically, we performed subgroup analysis based on different age-stratification subgroups, that is, mean age less than 60 years versus more than 60 years. Moreover, to account for the potential fluctuation of SoC during the COVID-19 pandemic, along with the wide promotion of COVID-19 vaccine and newly developed anti-viral treatment strategy, we conducted a subgroup analysis according to the publication year of the included RCTs. We compared the difference between RCTs published in 2019 and 2020 versus those published in 2021, 2022 and 2023. This is because the global case fatality rate from COVID-19 dropped significantly since early 2021¹⁷ due to the widespread promotion of the COVID-19 vaccine and newly developed anti-viral treatment strategies in late 2020.¹⁸ Furthermore, we conducted a sensitivity analysis by subgroup restricting to RCTs without any zero events. We conducted another sensitivity analysis by subgroup restricting to RCTs with a placebo arm. We assessed the transitivity assumption by visually evaluating the distribution of the effects of each treatment arm. We conducted an additional assessment of transitivity by comparing the average effects of the common reference treatment in different groups of trials (known as designs in NMA literature) using Comprehensive Meta-Analysis (version 3; Biostat, Englewood, NJ, USA).¹⁹ If there was no statistically significant difference, the assumption of transitivity was considered unlikely to be violated.

Results

Eligibility of the retrieved studies and treatment arms

Figure 1 shows a flowchart of this NMA. After the initial screening procedure, 53 articles were considered for the full-text review, of which 39 were excluded for various reasons (eTable 3). Ultimately, 14 RCTs were included in this study. Figure 2 shows the overall network plot of treatment comparisons for all-cause mortality between JAK/BTK inhibitors, placebo plus SoC and SoC alone.

Figure 1.

Flow diagram of this network meta-analysis.

Figure 2.

Network plot of the primary outcomes: all-cause mortality. The lines between nodes represent direct comparisons in various trials, and the size of each circle is proportional to the size of the population involved in each specific treatment. The thickness of the lines is proportional to the number of trials connected to the network.

Characteristics of the included studies

A total of 13,568 participants (mean age: 58.1 years, range: 46.5–72.5 years; mean female proportion: 35.5%, range: 23.3%–45.6%) were included (Table 3). The mean treatment duration was 2.0 weeks (range: 2.0–4.0 weeks), and the mean overall study duration (treatment and follow-up duration) was 4.2 weeks (range: 4.0–13.0 weeks). All the included RCTs recruited subjects with treatment-naïve patients. With regard to the severity of COVID-19 infection among the recruited participants in the included RCTs, all the RCTs recruited their participants in moderate-to-severe severity of COVID-19 infection in their inclusion criteria.

Primary outcome: all-cause mortality

Compared with the SoC group, only baricitinib was associated with a significantly lower all-cause mortality rate (OR = 0.88, 95% CIs = 0.78–1.00). In contrast, only placebo plus SoC was associated with a significantly higher all-cause mortality rate than SoC alone (OR = 1.39, 95% CI = 1.07–1.79) (Figure 3 and Table 1). According to the SUCRA, high-dose ruxolitinib was ranked best among all interventions (OR = 0.70, 95% CI = 0.33–1.50 compared with the SoC group) (Tables 1 and 2).

Figure 3.

Forest plot of the all-cause mortality in reference to standard of care. The indicated treatment was associated with a significantly lower 28-day all-cause mortality rate than the standard of care alone if the odds ratio is less than 1.95% CI: 95% confidence interval; ADHD: attention-deficit hyperactivity disorder; Bar: baricitinib; BTK: Bruton tyrosine kinase; COVID-19: coronavirus disease 2019; hiRux: ruxolitinib high dosage; Ibr: ibrutinib; JAK Janus kinase; lowRux: ruxolitinib low dosage; NMA: network meta-analysis; OR: odds ratio; Pla: placebo + SoC; RCT: randomised controlled trial; SoC: standard of care; SUCRA: surface under the cumulative ranking curve; Tof: tofacitinib.

Table 1.

League table of all-cause mortality rate.

			League table
hiRux			0.91 (0.49,1.69)		0.44 (0.20,0.94)*
1.03 (0.25,4.26)	Tof				0.49 (0.14,1.66)
0.79 (0.37,1.68)	0.77 (0.22,2.66)	Bar		0.88 (0.78,1.00)*	0.64 (0.51,0.80)*
0.85 (0.47,1.56)	0.83 (0.21,3.28)	1.07 (0.54,2.12)	lowRux		0.63 (0.23,1.72)
0.70 (0.33,1.50)	0.68 (0.19,2.37)	0.88 (0.78,1.00)*	0.82 (0.41,1.64)	SoC
0.50 (0.25,1.04)	0.49 (0.14,1.66)	0.64 (0.51,0.80)*	0.59 (0.31,1.13)	0.72 (0.56,0.94)*	Pla	0.29 (0.01,7.58)
0.15 (0.01,4.13)	0.14 (0.00,4.62)	0.19 (0.01,4.86)	0.17 (0.01,4.77)	0.21 (0.01,5.52)	0.29 (0.01,7.56)	Ibr

Pairwise (upper-right portion) and network (lower-left portion) meta-analysis results are presented as estimate effect sizes for the outcome of all-cause mortality rate. Interventions are reported in order of mean ranking of all-cause mortality rate, and outcomes are expressed as OR (95% confidence interval). For the pairwise meta-analyses, OR of less than 1 indicates that the treatment specified in the row got less all-cause mortality rate than that specified in the column. For the NMA, OR of less than 1 indicates that the treatment specified in the column got less all-cause mortality rate than that specified in the row. Bold results marked with * indicate statistical significance.

Bar: baricitinib; BTK: Bruton tyrosine kinase; hiRux: ruxolitinib high dosage; Ibr: ibrutinib; lowRux: ruxolitinib low dosage; Pla: placebo + SoC; SoC: standard of care; Tof: tofacitinib.

Table 2.

SUCRA of the all-cause mortality rate.

Treatment	SUCRA	OR (95% CI) in comparison with SoC
hiRux	76.4	0.70 (0.33,1.50)
Tof	70.0	0.68 (0.19,2.37)
Bar	64.1	0.88 (0.78,1.00) *
lowRux	63.6	0.82 (0.41,1.64)
SoC	43.0	Reference
Pla	16.6	1.39 (1.07,1.79) *
Ibr	16.3	4.73 (0.18,123.70)

Sorted by efficacy order (the former, the less all-cause mortality rate)

*Achieved statistical significance.

95% CI: 95% confidence interval; Bar: baricitinib; hiRux: ruxolitinib high dosage; Ibr: ibrutinib; lowRux: ruxolitinib low dosage; OR: odds ratio; Pla: placebo + SoC; SoC: standard of care; SUCRA: surface under the cumulative ranking curve; Tof: tofacitinib.

Table 3.

Characteristics of the included studies.

Study name	Design	Diagnosis	Comparison	Subjects	Mean age (years)	Women (%)	Tx duration (day)	Primary outcome^a	Country	Registry
Troseid et al.²⁰	Placebo-controlled	Eligible adults infected with severe/critical COVID-19	Baricitinib 4 mg dailyPlacebo + standard of care	139136	59.060.0	19.427.2	14	1418	Multiple countries	NCT04891133
Coutre et al. (iNSPIRE)²¹	Placebo-controlled	SARS-CoV-2 infection confirmed by reverse transcription PCR test	Ibrutinib 420 mg once dailyPlacebo + standard of care	2224	48.554.5	27.333.3	28	10	USA	NCT04375397
Ely et al. (COV-BARRIER-NIAID-OS-7)¹⁵	Placebo-controlled	SARS-CoV-2 infection, confirmed by PCR test	Baricitinib 4 mg dailyPlacebo + standard of care	5150	58.4 ± 12.458.8 ± 15.2	51.040.0	14	2029	Multiple countries	NCT04421027
Han et al. (RUXCOVID)²²	Placebo-controlled	SARS-CoV-2 infection confirmed by PCR test or another rapid test	Ruxolitinib 10 mg/dayPlacebo + standard of care	286145	56.456.9	43.649.7	14	93	Multiple countries	NCT04362137
Karampitsakos et al.²³	Open-label	Positive PCR test result for SARS-CoV-2	Baricitinib 4 mg dailyStandard of care alone	125126	73.072.0	40.841.3	14	4050	Greece	NCT05082714
Montejano et al. (PANCOVID)²⁴ no TDF/FTC part	Open-label	SARS-CoV-2 detected by PCR or antigenic test	Baricitinib 4 mg dailyStandard of care alone	7571	NA	NA	10	13	Spain	EudraCT: 2020-001156-18
Murugesan et al.²⁵	Open-label	18–65 y/o adults, positive for SARS-CoV2 infection by real-time PCR	Tofacitinib 20 mg/dayStandard of care alone	5050	47.046.0	24.028.0	14	00	India	NA
RECOVERY²⁶	Open-label	Clinically suspected or laboratory-confirmed SARS-CoV-2 infection	Baricitinib 4 mg dailyStandard of care alone	41484008	58.5 ± 15.457.7 ± 15.5	33.934.2	10	514546	UK	NCT04381936
Rein et al. (RUXCOVID-DEVENT)²⁷	Placebo-controlled	SARS-CoV-2 infection confirmed	Ruxolitinib 30 mg/dayRuxolitinib 10 mg/dayPlacebo + standard of care	778547	63.6 ± 12.963.6 ± 12.362.5 ± 13.3	19.536.827.7	14	394533	Multiple countries	NCT04377620
Wolfe et al. (ACTT-4)²⁸	Placebo-controlled	Adults with laboratory-confirmed SARS-CoV-2 infection	Baricitinib 4 mg dailyPlacebo + standard of care	516494	58.2 ± 14.358.5 ± 13.7	41.941.3	14	2730	Multiple countries	NCT04640168
Guimaraes et al. (STOP-COVID)²⁹	Placebo-controlled	SARS-CoV-2 infection as determined by PCR	Tofacitinib 20 mg/dayPlacebo + standard of care	144145	55.0 ± 14.057.0 ± 14.0	34.735.2	14	48	Brazil	NCT04469114
Kalil et al. (ACTT-2)³⁰	Placebo-controlled	18 y/o or older adults, laboratory-confirmed SARS-CoV-2 infection	Baricitinib 4 mg dailyPlacebo + standard of care	515518	55.0 ± 15.455.8 ± 16.0	38.135.7	14	2437	Multiple countries	NCT04401579
Marconi et al. (COV-BARRIER-NIAID-OS-5-6)¹⁴	Placebo-controlled	SARS-CoV-2 infection, confirmed by PCR test	Baricitinib 4 mg dailyPlacebo + standard of care	764761	57.8 ± 14.357.5 ± 13.8	35.937.8	14	62100	Multiple countries	NCT04421027
Cao et al.³¹	Placebo-controlled	COVID-19 diagnosed according to the Chinese management guideline	Ruxolitinib 10 mg/dayPlacebo + standard of care	2021	63.064.0	40.042.9	28	03	China	NA

COVID-19: coronavirus disease 2019; NA: not available; PCR: polymerase chain reaction SARS-CoV-2: severe acute respiratory syndrome coronavirus 2; Tx: treatment.

28-day all-cause mortality.

Regarding the cause of mortality, only a limited number of RCTs provided detailed information, which varied among those RCTs. Therefore, additional analysis of the cause of mortality was not feasible. The most frequently reported cause of mortality was ‘death due to adverse events’, which ranged from 4.0% to 11.1% in placebo-controlled RCTs, whereas the information of ‘death due to adverse events’ was not reported in any open-label RCTs (i.e. RCTs of SoC only). Rather, in the RCTs of SoC only, the most frequent cause of death was COVID-19 infection (12.6%).

Sensitivity analysis by excluding RCTs without zero events in any arm

Compared with the SoC group, only the placebo plus SoC group was associated with a significantly higher all-cause mortality rate than the SoC group (OR = 1.39, 95% CI = 1.07–1.79) (eFigures 1A and 2A, eTables 4A and 5A).

Sensitivity analysis restricted to RCTs with a placebo-control arm

Only baricitinib was associated with a significantly lower all-cause mortality rate than the placebo plus SoC group (OR = 0.64, 95% CI = 0.51–0.80) (eFigures 1B and 2B, eTables 4B and 5B).

Sensitivity analysis by subgrouping RCTs by published date

Because only one RCT was published between 2019 and 2020, we could not perform a subgroup analysis. The result of the subgroup analysis of RCTs published between 2021 and 2023 yielded the same findings that only placebo plus SoC was associated with a significantly higher all-cause mortality rate than SoC alone (OR = 1.39, 95% CI = 1.07–1.79) (eFigures 1C and 2C, eTables 4C and 5C).

Sensitivity analysis by subgrouping RCTs by mean age-stratification

We could not perform a subgroup analysis on those over the age of 60 years, as there were only four RCTs in this age group. However, the result of the subgroup analysis of RCTs with an average age of below 60 years yielded the same findings that only placebo plus SoC was associated with a significantly higher all-cause mortality rate than SoC alone (OR = 1.41, 95% CI = 1.08–1.83) (eFigures 1D and 2D, eTables 4D and 5D).

Assessment of transitivity assumption: comparing the efficacy of JAK/BYK inhibitors in studies with the placebo plus SoC or SoC as the control

The network plot of the primary outcome showed that the indirect comparison between placebo plus SoC vs. SoC only mainly came from the loops of ‘placebo plus SoC versus baricitinib versus SoC only’. Therefore, we will evaluate the transitivity assumption for this treatment in two RCT designs (eFigure 1E).

Specifically, the OR for all-cause mortality in the baricitinib groups was 0.098 (95% CI = 0.050–0.183) in the placebo plus SoC RCTs, and 0.130 (95% CI = 0.047–0.313) in the SoC-only RCTs. The difference in the ORs of these two baricitinib groups was very small and non-significant (p = 0.639) (eFigure 2E). Therefore, there is currently insufficient evidence to support a violation of the transitivity assumption.

To further assess the transitivity assumption, we created a scatterplot (eFigure 2F) illustrating the distribution of the odds of each treatment arm for all-cause mortality. The x-axis represents the study ID and the y-axis displays the odds on the natural log scale. Each treatment arm is denoted by a symbol whose size is inversely proportional to its sampling error. Red circles represent SoC, orange triangles represent placebo and green crosses represent baricitinib. The scatterplot revealed that the odds of all-cause mortality for baricitinib in the three trials (marked by BLUE rectangles) comparing it with SOC were similar to those for baricitinib in the five trials (marked by RED rectangles) comparing it with placebo. The only exception was study ID 10, which had a large sampling error and a small weight.

Sensitivity analysis by excluding RECOVERY trials

Because the RECOVERY trial had the largest sample sizes (8156 subjects) and the broadest COVID-19 severity range,²⁶ we conducted a sensitivity analysis by excluding this trial. The odds ratio of SoC compared with other treatments for all-cause-mortality became non-significant (eFigure 2G).

Secondary outcome: new invasive or non-invasive ventilation usage

None of the investigated treatments were associated with significantly different risks of ventilation usage compared with the SoC group (eFigures 1F and 2H, eTables 4E and 5E).

Secondary outcome: secondary bacterial infection

None of the investigated treatments were associated with a significantly different risk of secondary bacterial infection compared with the SoC group (eFigures 1G and 2I, eTables 4F and 5F).

Secondary outcome: acceptability in terms of drop-out rate

None of the investigated treatments were associated with significantly different drop-out rates compared with the SoC group (eFigures 1H and 2J, eTables 4G and 5G).

Secondary outcome: serious adverse events

None of the investigated treatments were associated with a significantly different risk of serious adverse events (SAEs) compared with the SoC group (eFigures 1I and 2K, eTables 4H and 5H).

Risk of bias and publication bias

Of the 91 items, 63 (69.2%), 19 (20.9%) and 9 (9.9%) of the included studies had an overall low, unclear and high risk of bias, respectively. The unblinding and concealing procedures after randomisation were the main contributors to the high/unclear risk of bias (eFigures 3).

Funnel plots of publication bias and Egger’s test across the included studies (eFigures 4A–J) revealed general symmetry and no significance among the included studies in the NMA. The inconsistency test revealed no significant inconsistencies in this NMA (eTable 6). In addition, there was no significant heterogeneity, defined by the tau value, in the main results of this study (eTable 7). The results of the GRADE evaluation are listed in eTable 8. Briefly, the overall quality of the evidence ranged from low to medium.

Discussion

This NMA suggested that patients with COVID-19 treated with placebo plus SoC appeared to show a statistically significantly higher 28-day all-cause mortality odds than those receiving SoC alone. This difference remained consistent in subgroup analyses stratified by epidemiological factor, pandemic history progression and statistical consideration.

The most intriguing finding of this NMA is the higher odds of all-cause mortality in the placebo arm compared with the SoC arm. This is particularly noteworthy, given that all-cause mortality is considered an objective measure, but not a subjective outcome or minor symptoms, such as pain³² and cough.³³ Furthermore, this difference remained consistent in the subgroup and sensitivity analyses.

There were four possible explanations for this difference in mortality in RCTs on the treatment of COVID-19.

Alteration of the adherence related to placebo use

A previous meta-analysis³⁴ found that the mortality of patients receiving placebo with good adherence was similar to that of patients receiving beneficial drug therapy.³⁴ However, in our study, none of the included RCTs provided information regarding the association between adherence and mortality in individual patients. Furthermore, the acceptability of placebo plus SoC was similar to SoC alone, without a statistically significant difference.

Immunomodulatory effect by the placebo prescription

Placebo administration has been suggested to trigger endogenous neurotransmitter release, consequently mimicking pharmacological effects.³⁵ This is supported by observations during the treatment of asthma³⁶ and chronic cough.³³ However, we did not observe the presence of such pharmacological mimicry, because there were no significant differences in some secondary outcomes (e.g. secondary bacterial infection) or safety profile (e.g. SAEs) between the SoC plus placebo group and the SoC group. Secondary infection and SAEs were considered the indicators of biophysiological harm related to active ingredients.

Behavioural modification associated with placebo administration

The application of placebo could affect the behaviours of the parents of children with attention-deficit hyperactivity disorder.⁷ Further, the attendance in a clinical trial and reception of placebo would contribute to increased medical attention and result in participants’ behaviour change, which is the Hawthorne effect.³⁷ The use of placebo in a statin trial might lead to the false belief that patients could ‘eat anything and stop exercising’, which resulted in worse outcomes than that of patients on the waiting list.⁸ In one systematic review of cardiovascular study regarding outcome of all-cause mortality,⁹ the authors confirmed that the mortality rate in the placebo arm of the study of peripheral arterial disease was increased in comparison with the average mortality in the ordinary population with peripheral arterial disease.¹⁰ That is why, recently, some researchers recommended using the ‘best-available-therapy’ as the control group, rather than placebo.³⁸

Alteration of prognosis because of assignment to SoC-only group

Some previous evidence on the preferences and performance of the patients in an SoC-only arm³⁹ of a weight loss trial suggests that assignment to an SoC-only arm could disappoint patients and promote behaviour change that could subsequently affect their prognosis. Another trial of alcohol abstinence also showed the efficacy of ‘assignment to the waiting-list or SoC-only group’,⁴⁰ which resulted in 20% decreased alcohol consumption.

Strengths and limitations

Our NMA has several strengths to be addressed. First, the distinction between SoC-only and placebo plus SoC groups was made, and this could not be done in a traditional pairwise meta-analysis. Second, we included only RCTs to reduce the potential biases associated with observational studies. Third, we undertook sensitivity analyses by different subgroup analysis to re-appraise the results of this study, which found no evidence of intransitivity.

This NMA also has several limitations. First, some comparisons in our NMA were underpowered because of the heterogeneity in the characteristics of the participants and the small number of trials. Second, when the RECOVERY trial, which had the largest sample size, was excluded from the sensitivity analysis, the difference in the odds of all-cause mortality between patients receiving placebo plus SoC and those receiving SoC alone became smaller and non-significant. This may have been related to disease severity because the RECOVERY trial recruited both moderately and severely ill patients.²⁶ Finally, we distinguished between placebo plus SoC and SoC alone and found a significant difference in the mortality between the two treatment groups. However, this finding is based entirely on indirect evidence, because they have never been directly compared in a trial. Although the transitivity assumption did not appear to be violated, these results should be interpreted with caution.

Implications

This NMA of RCTs found a statistically significant difference in all-cause mortality between COVID-19 patients treated with SoC plus placebo and those treated with SoC alone.

Conclusion and interpretation

In this NMA, we found that COVID-19 patients treated with baricitinib showed a significantly lower 28-day all-cause mortality compared with those treated with SoC. We also found that patients treated with SoC plus placebo showed a higher mortality than those treated with SoC alone. Therefore, future RCTs of COVID-19 treatments should re-evaluate the use of placebo and consider alternative study designs to minimise potential biases.

Supplemental Material

sj-pdf-1-jrs-10.1177_01410768231202657 - Supplemental material for The difference in all-cause mortality between COVID-19 patients treated with standard of care plus placebo and those treated with standard of care alone: a network meta-analysis of randomised controlled trials of immunomodulatory kinase inhibitors

Supplemental material, sj-pdf-1-jrs-10.1177_01410768231202657 for The difference in all-cause mortality between COVID-19 patients treated with standard of care plus placebo and those treated with standard of care alone: a network meta-analysis of randomised controlled trials of immunomodulatory kinase inhibitors by Ping-Tao Tseng, Bing-Syuan Zeng, Chih-Wei Hsu, Trevor Thompson, Brendon Stubbs, Po-Ren Hsueh, Kuan-Pin Su, Yen-Wen Chen, Tien-Yu Chen, Yi-Cheng Wu, Pao-Yen Lin, Andre F Carvalho, Dian-Jeng Li, Ta-Chuan Yeh, Cheuk-Kwan Sun, Yu-Shian Cheng, Yow-Ling Shiue, Chih-Sung Liang and Yu-Kang Tu in Journal of the Royal Society of Medicine

Supplemental Material

sj-pdf-2-jrs-10.1177_01410768231202657 - Supplemental material for The difference in all-cause mortality between COVID-19 patients treated with standard of care plus placebo and those treated with standard of care alone: a network meta-analysis of randomised controlled trials of immunomodulatory kinase inhibitors

Supplemental material, sj-pdf-2-jrs-10.1177_01410768231202657 for The difference in all-cause mortality between COVID-19 patients treated with standard of care plus placebo and those treated with standard of care alone: a network meta-analysis of randomised controlled trials of immunomodulatory kinase inhibitors by Ping-Tao Tseng, Bing-Syuan Zeng, Chih-Wei Hsu, Trevor Thompson, Brendon Stubbs, Po-Ren Hsueh, Kuan-Pin Su, Yen-Wen Chen, Tien-Yu Chen, Yi-Cheng Wu, Pao-Yen Lin, Andre F Carvalho, Dian-Jeng Li, Ta-Chuan Yeh, Cheuk-Kwan Sun, Yu-Shian Cheng, Yow-Ling Shiue, Chih-Sung Liang and Yu-Kang Tu in Journal of the Royal Society of Medicine

Footnotes

Declarations

ORCID iDs

Ping-Tao Tseng

Chih-Wei Hsu

Chih-Sung Liang

References

WHO Coronavirus (COVID-19) Dashboard. See https://covid19.who.int/ (last checked 1 February 2022).

COVID-19 Treatment Guidelines Panel. Coronavirus Disease 2019 (COVID-19) Treatment Guidelines. See www.covid19treatmentguidelines.nih.gov/ (last checked 10 November 2022).

Levy

Guglielmelli

Langmuir

Constantinescu

SN.

JAK inhibitors and COVID-19. J Immunother Cancer 2022; 10: e002838.

Kifle

ZD.

Bruton tyrosine kinase inhibitors as potential therapeutic agents for COVID-19: a review. Metabol Open 2021; 11: 100116.

Ngamprasertchai

Kajeekul

Sivakorn

Ruenroegnboon

Luvira

Siripoon

, et al. Efficacy and safety of immunomodulators in patients with COVID-19: a systematic review and network meta-analysis of randomized controlled trials. Infect Dis Ther 2022; 11: 231–248.

Yetman

Cox

Adler

Hall

Stone

VE.

What do placebo and nocebo effects have to do with health equity? The hidden toll of nocebo effects on racial and ethnic minority patients in clinical care. Front Psychol 2021; 12: 788230.

Waschbusch

Pelham

Jr Waxmonsky

Johnston

Are there placebo effects in the medication treatment of children with attention-deficit hyperactivity disorder?

J Dev Behav Pediatr 2009; 30: 158–168.

Fava

Guidi

Rafanelli

Rickels

The clinical inadequacy of the placebo model and the development of an alternative conceptual framework. Psychother Psychosom 2017; 86: 332–340.

Brass

Hiatt

WR.

Review of mortality and cardiovascular event rates in patients enrolled in clinical trials for claudication therapies. Vasc Med 2006; 11: 141–145.

10.

Ostergren

Sleight

Dagenais

Danisa

Bosch

Qilong

, et al. Impact of ramipril in patients with evidence of clinical or subclinical peripheral arterial disease. Eur Heart J 2004; 25: 17–24.

11.

Cheng

Pullenayegum

Marshall

Iorio

Thabane

Impact of including or excluding both-armed zero-event studies on using standard meta-analysis methods for rare event outcome: a simulation study. BMJ Open 2016; 6: e010983.

12.

Hsu

Tseng

YK.

Associating violence with schizophrenia-risks and biases. JAMA Psychiatry 2022; 79: 739.

13.

Page

McKenzie

Bossuyt

Boutron

Hoffmann

Mulrow

, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71.

14.

Marconi

Ramanan

de Bono

Kartman

Krishnan

Liao

, et al. Efficacy and safety of baricitinib for the treatment of hospitalised adults with COVID-19 (COV-BARRIER): a randomised, double-blind, parallel-group, placebo-controlled phase 3 trial. Lancet Respir Med 2021; 9: 1407–1418.

15.

Ely

Ramanan

Kartman

de Bono

Liao

Piruzeli

MLB

, et al. Efficacy and safety of baricitinib plus standard of care for the treatment of critically ill hospitalised adults with COVID-19 on invasive mechanical ventilation or extracorporeal membrane oxygenation: an exploratory, randomised, placebo-controlled trial. Lancet Respir Med 2022; 10: 327–336.

16.

White

IR.

Network meta-analysis. Stata J 2015; 15: 951–985.

17.

Horita

Fukumoto

Global case fatality rate from COVID-19 has decreased by 96.8% during 2.5 years of the pandemic. J Med Virol 2023; 95: e28231.

18.

Watson

Barnsley

Toor

Hogan

Winskill

Ghani

AC.

Global impact of the first year of COVID-19 vaccination: a mathematical modelling study. Lancet Infect Dis 2022; 22: 1293–1302.

19.

Altman

Bland

JM.

Interaction revisited: the difference between two estimates. BMJ 2003; 326: 219.

20.

Troseid

Arribas

Assoumou

Holten

Poissy

Terzić

, et al. Efficacy and safety of baricitinib in hospitalized adults with severe or critical COVID-19 (Bari-SolidAct): a randomised, double-blind, placebo-controlled phase 3 trial. Crit Care 2023; 27: 9.

21.

Coutre

Barnett

Osiyemi

Hoda

Ramgopal

Fort

, et al. Ibrutinib for hospitalized adults with severe coronavirus disease 2019 infection: results of the randomized, double-blind, placebo-controlled iNSPIRE study. Open Forum Infect Dis 2022; 9: ofac104.

22.

Han

Antila

Ficker

Gordeev

Guerreros

Bernus

, et al. Ruxolitinib in addition to standard of care for the treatment of patients admitted to hospital with COVID-19 (RUXCOVID): a randomised, double-blind, placebo-controlled, phase 3 trial. Lancet Rheumatol 2022; 4: e351–e361.

23.

Karampitsakos

Papaioannou

Tsiri

Katsaras

Katsimpris

Kalogeropoulos

, et al. Tocilizumab versus baricitinib in hospitalized patients with severe COVID-19: an open label, randomized controlled trial. Clin Microbiol Infect 2023; 29: 372–378.

24.

Montejano

de la Calle-Prieto

Velasco

Guijarro

Queiruga-Parada

Jiménez-González

, et al. Tenofovir disoproxil fumarate/emtricitabine and baricitinib for patients at high risk of severe COVID-19: the PANCOVID randomized clinical trial. Clin Infect Dis 2023; 76: e116–e125.

25.

Murugesan

Gauthaman

Nasreen

Santhanam

Gowrishankar

Ravi

, et al. An evaluation of efficacy and safety of tofacitinib, A JAK inhibitor in the management of hospitalized patients with mild to moderate COVID-19 – an open-label randomized controlled study. J Assoc Physicians India 2022; 69: 11–12.

26.

Group

RC.

Baricitinib in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial and updated meta-analysis. Lancet 2022; 400: 359–368.

27.

Rein

Calero

Shah

Ojielo

Hudock

Lodhi

, et al. Randomized phase 3 trial of ruxolitinib for COVID-19-associated acute respiratory distress syndrome. Crit Care Med 2022; 50: 1701–1713.

28.

Wolfe

Tomashek

Patterson

Gomez

Marconi

Jain

, et al. Baricitinib versus dexamethasone for adults hospitalised with COVID-19 (ACTT-4): a randomised, double-blind, double placebo-controlled trial. Lancet Respir Med 2022; 10: 888–899.

29.

Guimaraes

Quirk

Furtado

Maia

Saraiva

Antunes

, et al. Tofacitinib in patients hospitalized with Covid-19 pneumonia. N Engl J Med 2021; 385: 406–415.

30.

Kalil

Patterson

Mehta

Tomashek

Wolfe

Ghazaryan

, et al. Baricitinib plus remdesivir for hospitalized adults with Covid-19. N Engl J Med 2021; 384: 795–807.

31.

Cao

Wei

Zou

Jiang

Wang

Chen

, et al. Ruxolitinib in treatment of severe coronavirus disease 2019 (COVID-19): a multicenter, single-blind, randomized controlled trial. J Allergy Clin Immunol 2020; 146: 137–146.

32.

Price

Finniss

Benedetti

A comprehensive review of the placebo effect: recent advances and current thought. Annu Rev Psychol 2008; 59: 565–590.

33.

Eccles

The powerful placebo effect in cough: relevance to treatment and clinical trials. Lung 2020; 198: 13–21.

34.

Simpson

Eurich

Majumdar

Padwal

Tsuyuki

Varney

, et al. A meta-analysis of the association between adherence to drug therapy and mortality. BMJ 2006; 333: 15.

35.

Pacheco-Lopez

Engler

Niemi

Schedlowski

Expectations and associations that heal: immunomodulatory placebo effects and its neurobiology. Brain Behav Immun 2006; 20: 430–446.

36.

Joyce

Jackevicius

Chapman

McIvor

Kesten

The placebo effect in asthma drug therapy trials: a meta-analysis. J Asthma 2000; 37: 303–318.

37.

McQueen

Cohen

St John-Smith

Rampes

Rethinking placebo in psychiatry: the range of placebo effects. Adv Psychiatric Treat 2018; 19: 162–170.

38.

Castro

Placebo versus best-available-therapy control group in clinical trials for pharmacologic therapies: which is better?

Proc Am Thoracic Soc 2007; 4: 570–573.

39.

McCambridge

Sorhaindo

Quirk

Nanchahal

Patient preferences and performance bias in a weight loss trial with a usual care arm. Patient Educ Couns 2014; 95: 243–247.

40.

Jenkins

McAlaney

McCambridge

Change over time in alcohol consumption in control groups in brief intervention studies: systematic review and meta-regression study. Drug Alcohol Depend 2009; 100: 107–114.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.72 MB

1.28 MB