Suicidality and other severe psychiatric events with duloxetine: Re-analysis of safety data from a placebo-controlled trial for juvenile fibromyalgia

Abstract

BACKGROUND:

In antidepressant trials for pediatric patients with depression or anxiety disorders, the risk of suicidal events and other severe psychiatric adverse events such as aggression and agitation is increased with antidepressants relative to placebo.

OBJECTIVE:

To examine whether largely mentally healthy adolescents treated for a non-psychiatric condition are also at increased risk of suicidality and other severe psychiatric disorders.

METHODS:

This is a re-analysis of a placebo-controlled duloxetine trial for juvenile fibromyalgia based on the main journal article and additional data published in the online supplementary material and on ClinicalTrials.gov. Both serious adverse events related to psychiatric disorders and adverse events leading to treatment discontinuation were defined as severe treatment-emergent psychiatric adverse events.

RESULTS:

We found that a significant portion of adolescents had treatment-emergent suicidal ideation and behaviour as well as other severe psychiatric adverse events with duloxetine, but no such events were recorded on placebo. The incidence of severe treatment-emergent psychiatric adverse events was statistically significantly higher with duloxetine as compared to placebo.

CONCLUSIONS:

Antidepressants may put adolescents at risk of suicidality and other severe psychiatric disorders even when the treatment indication is not depression or anxiety.

Keywords

Duloxetine antidepressant RCT suicidality suicidal ideation suicidal behavior serious adverse event

1. Introduction

In 2004 the Food and Drug Administration (FDA) issued a black box warning for antidepressants, stating that this drug class may cause suicidal ideation and behavior in children and adolescents. This was based on findings from a large internal meta-analysis of published and unpublished trials conducted by the FDA, indicating that the risk of suicidal events is increased with antidepressants relative to placebo in pediatric patients with depression and other psychiatric disorders [1,2]. The results of this FDA meta-analysis were independently replicated by various other research groups [3–6] and are thus considered robust and reliable. Nevertheless, the FDA black box warning of increased suicide risk with antidepressants in pediatric patients has drawn much criticism.

For instance, various authors concluded on the basis of ecological studies that antidepressants would protect against suicide, but ecological studies cannot demonstrate cause-effect relationships and most of these studies were shown to be flawed and/or misrepresented, for example, one influential study misleadingly associated an increase in suicide between 2003 and 2004 to a reduction in antidepressant prescriptions between 2004 and 2005, and another highly cited study assessed intentional and non-intentional poisoning by psychotropic agents instead of suicide attempts, but interpreted the results as if the outcome was the latter [7–10].

A more convincing argument has been that an increased risk of suicidal events with any antidepressant relative to placebo has only been detected by meta-analyzing trials that were neither designed nor powered to detect suicidal events. Although the risk of suicidal events was increased with all new-generation antidepressants, the effect was not statistically significant for individual drugs except for venlafaxine [1]. However, there are also individual trials showing significantly increased suicidal event rates specifically with paroxetine and fluoxetine, but this was first accurately assessed and communicated by independent researchers upon re-analysis of the data [11,12]. These important re-analyses have shown that in the primary trial publications the number of suicidal events were misreported, resulting in a systematic underestimation of the true risk (see also [5,13,14]. Unfortunately, it appears that the misreporting of serious adverse events (SAE) and other safety data is pervasive in publications of drug trials, both in psychiatry [15,16] and in general medicine [17,18]. For a systematic review, see Golder et al. [19].

Another crucial issue with the evaluation of adverse drug effects are false statistical and clinical inferences. Most authors conduct statistical tests to detect differences in adverse events rates between treatment arms, without considering (or acknowledging) that when specific events are rare and the sample size is small, these analyses lack the power to detect even substantial and clinically important differences [20,21]. When a difference between treatment arms is not statistically significant, it is common practice for authors to conclude that there was no difference [21–23]. However, this inference is false, as “absence of evidence is not evidence of absence” [24] and ultimately this common flaw increases the risk that adverse effects of a drug that may cause serious harms are not considered in clinical decision making [21,23,25].

The aim of the present study was to re-analyze the safety data of a duloxetine trial for juvenile fibromyalgia. The study was sponsored and conducted by Eli Lilly, the manufacturer of duloxetine, and the first author was an employee of Eli Lilly. In the main publication of this trial, the authors concluded that “There were no new safety concerns related to duloxetine in the study population”, specifically stating that “the suicidal ideation events reported with duloxetine were not significantly different from placebo-treated patients” [26]. As comprehensive data on suicidality and other psychiatric adverse events were available for this trial from different sources, our aim was to critically re-evaluate the data and the authors’ conclusion. In addition to the availability of comprehensive adverse event data, another reason to examine this particular trial was that the participants had mostly no or only mild psychopathological symptoms at baseline, which facilitates the detection of severe treatment-emergent psychiatric adverse events.

Based on the literature, we hypothesized that close inspection and re-analysis of the data would reveal a higher incidence of treatment-emergent suicidal and other severe psychiatric adverse events with duloxetine relative to placebo.

2. Methods

2.1. Study design and procedure

This is a secondary analysis of a multi-site phase-III trial of duloxetine for the treatment of adolescents with chronic musculoskeletal pain meeting diagnostic criteria of juvenile fibromyalgia. The detailed methods of this trial are reported in the target article [26]. In short, the trial enrolled juvenile patients aged 13–17 years with fibromyalgia from the United States, Argentina, India and Puerto Rico. Exclusion criteria were acute suicidality and suicide attempts in the 12 months prior to entry into the trial as well as any current or past psychiatric disorder, except major depression, generalized anxiety disorder, adjustment disorder, specific phobias, and attention-deficit/hyperactivity disorder (for the latter only when no pharmacologic treatment was required). Likewise excluded were patients who were not likely to benefit from duloxetine treatment, in the opinion of the investigator, and patients who had prior nonresponse or inadequate tolerance to duloxetine for any clinical use. Patients were randomized to duloxetine or placebo for a 13-week double-blind acute treatment period. Participants who completed the acute treatment period entered a 26-week extension period where all participants were treated open-label with duloxetine.

According to Swiss law no approval by an ethics committee was required for this study as it was a secondary analysis of de-identified data freely available in the public domain.

2.2. Data sources and outcomes

For the present re-analysis we combined the data from three different sources: (1) the main article by Upadhyaya et al. published in the journal Pediatric Rheumatology [26]; (2) the supplementary material published online on the journal website (https://ped-rheum.biomedcentral.com/articles/10.1186/s12969-019-0325-6); and (3) the results reported on ClinicalTrials.gov (https://clinicaltrials.gov/ct2/show/results/NCT01237587?view=results). All adverse were extracted from the data for the acute double-blind period and the open-label extension period. Out definition of a severe treatment-emergent psychiatric adverse event included (1) all SAE related to suicidality and other psychiatric disorders according to MedDRA terms and (2) any psychiatric adverse event that lead to treatment discontinuation. In addition to adverse events reports, during the acute treatment period suicidality was also assessed with the Columbia-Suicide Severity Rating Scale (C-SSRS) [27]. Results for the C-SSRS were analyzed separately.

2.3. Statistical analysis

The incidence rate of severe treatment-emergent psychiatric adverse events as defined above was calculated based on the number of events relative to the number of participants at risk. We used Barnard’s test because it is the statistically most powerful test for 2 × 2 contingency tables with two independent binomial distributions and one margin fixed [28]. Fisher’s exact test, despite being frequently used, leads to unnecessarily conservative p-value estimations and, therefore, was considered not optimal for such analyses [28]. We used R’s “Exact” package and calculated both one-tailed and two-tailed results of Barnard’s test. Application of one-tailed significance tests was considered appropriate as our directed hypothesis was that duloxetine increases the risk of severe psychiatric adverse events relative to placebo. The number needed to harm (NNH) was calculated with the formula 1/ARI (Absolute Risk Increase), where ARI = duloxetine event rate minus placebo event rate.

3. Results

3.1. Sample characteristics at baseline

In total n = 91 participants were randomized to duloxetine and n = 93 participants to placebo. Of these, n = 74 (81.3%) and n = 75 (80.6%), respectively, completed the 13-weeks acute double-blind treatment period and entered the 26-weeks open-label duloxetine extension period. Altogether n = 106 participants completed the open-label duloxetine extension period, including n = 50 (66.7%) who were previously treated with placebo during the acute period and n = 56 (75.7%) who had continued duloxetine treatment. Participants mean age at study entry was 15.7 years (SD = 1.4) for the duloxetine arm and 15.3 years (SD = 1.4) for the placebo arm; 80.2% (duloxetine arm) and 69.9% (placebo arm) were female. The severity of mental illness according to Clinical Global Impression (CGI) severity scores was 2.1 (SD = 1.2) for patients randomized to duloxetine and 2.0 (SD = 1.1) for patients randomized to placebo. This indicates that the average participant was borderline ill (CGI score of 2) and that most participants scored between not ill at all (CGI score of 1) and mildly ill (CGI score of 3). Finally, 18.7% (duloxetine arm) and 16.1% (placebo arm) had a lifetime diagnosis of major depressive disorder, 11.0% and 6.5% had a lifetime diagnosis of generalized anxiety disorder, and 4.4% and 7.5% had a lifetime diagnosis of attention deficit disorder.

3.2. Suicidal and other severe psychiatric events in the acute treatment period

During the 13-weeks acute treatment period 2 participants in the duloxetine arm experienced a SAE as compared to 0 in the placebo arm. Of these 2 SAE in the duloxetine arm, 1 was classified as suicidal ideation according to MedDRA terms. In addition, 3 participants in the duloxetine arm had a treatment-emergent psychiatric adverse event leading to discontinuation, of which 1 was due to anxiety, 1 due to depressed mood, and 1 due to suicidal behavior. In the placebo arm none was recorded. Thus, in the duloxetine arm there were in total 4 severe psychiatric adverse events, including 2 suicidal events that were either recorded as serious or that lead to discontinuation as compared to 0 in the placebo arm (see Table 1). The estimated incidence rates for severe treatment-emergent psychiatric adverse events were 4.4% for duloxetine vs. 0% for placebo, resulting in a statistically significant difference according to Barnard’s test (p =0.044 two-tailed and p = 0.034 one-tailed). The NNH was 22.8.

Table 1
Suicidal and other severe psychiatric adverse events in the acute and extension period

Acute period Extension period Entire treatment period

Duloxetine Placebo Duloxetine Duloxetine Placebo

n =91 at risk n = 3 at risk n = 149 at risk^# n = 166 at risk^§ n =93 at risk

Serious adverse events Suicide attempt 0 0 2 2 0

Suicidal ideation 1 0 1^∗ 2 0

Intentional overdose 0 0 1^∗ 1 0

Intentional self-injury 0 0 1^∗ 1 0

Affective disorder 0 0 1^∗ 1 0

Auditory hallucination 0 0 1^∗ 1 0

Treatment discontinuation due to adverse events Suicidal behavior 1 0 0 1 0

Major depression 0 0 1 1 0

Depressed mood 1 0 0 1 0

Anxiety 1 0 0 1 0

Affective disorder 0 0 1^∗ 1 0

Irritability 0 0 1 1 0

Suicidal events 2 0 5 7 0

Suicidal events plus serious adverse events 2 0 7 9 0

All adverse events recorded as serious or leading to treatment discontinuation 4 0 10 14 0

		Acute period	Extension period	Entire treatment period
Serious adverse events	Suicide attempt	0	2	2
	Suicidal ideation	1	1^∗	2
	Intentional overdose	0	1^∗	1
	Intentional self-injury	0	1^∗	1
	Affective disorder	0	1^∗	1
	Auditory hallucination	0	1^∗	1
Treatment discontinuation due to adverse events	Suicidal behavior	1	0	1
	Major depression	0	1	1
	Depressed mood	1	0	1
	Anxiety	1	0	1
	Affective disorder	0	1^∗	1
	Irritability	0	1	1
Suicidal events		2	5	7
Suicidal events plus serious adverse events		2	7	9
All adverse events recorded as serious or leading to treatment discontinuation		4	10	14

Note ^# Includes n = 74 participants from acute duloxetine arm and n = 75 participants from acute placebo arm.^§ Includes n = 91 participants from acute duloxetine arm and n = 75 from duloxetine extension period who were previously in the acute placebo arm.^∗ Events occurred in participants who were previously in the acute placebo arm.

According to the C-SSRS, 6 participants in the duloxetine arm (6.6%) experienced an event, of which all were rated as suicidal ideation and behaviour. In the placebo arm, 4 participants (4.3%) had an event, including 3 participants with suicidal ideation and behaviour and 1 with non-suicidal self-injurious behaviour. Barnard’s test yielded p = 0.510 (two-tailed) and p = 0.260 (one-tailed). When only suicidal events were counted, that is, suicidal ideation and behaviour, then there were 6 events on duloxetine (6.6%) and 3 events on placebo (3.2%), but this difference was not statistically significant according to Barnard’s test (p = 0.304 two-tailed and p = 0.159 one-tailed). The NNH for suicidal events based on the C-SSRS was 29.7.

3.3. Suicidal and other severe psychiatric events in the extension treatment period

During the 26-week open-label duloxetine extension period, 9 SAE were reported, of which 2 were classified as suicide attempts, 1 as intentional overdose, 1 as intentional self-injury, and 1 as suicidal ideation. SAE reports related to other psychiatric disorders according to MedDRA terms included 1 affective disorder and 1 auditory hallucination. In addition, there were 3 psychiatric adverse events that lead to discontinuation, of which 1 was recorded as major depression, 1 as irritability, and 1 as affective disorder (see Table 1). In n = 149 participants at risk the total number of suicidal events with duloxetine was thus 5 (incidence rate: 3.4%) and the total number of severe psychiatric events (SAE and psychiatric adverse events leading to discontinuation) was 10 (incidence rate: 6.7%). Interestingly, 5 of 7 SAE (71.4%) and 6 of 10 severe psychiatric events (60.0%) occurred in participants who were treated with placebo during the acute treatment, that is, in patients who were newly started on duloxetine in the extension period.

3.4. Suicidal and other severe psychiatric events over the entire treatment period

As shown in Table 1, the total number of suicidal events over the entire study period was 7 for patients exposed to duloxetine (n = 166 at risk) and 0 for patients exposed to placebo (n = 93 at risk). The estimated incidence rate of suicidal events was 4.2% for duloxetine as compared to 0% for placebo. Barnard’s test yielded p = 0.054 (two-tailed) and p = 0.024 (one-tailed) and the NNH was 23.7.

With psychiatric SAE added to the equation (+2 events), there were in total 9 events for duloxetine (incidence rate: 5.4%) as compared to 0 for placebo (incidence rate: 0%). These rates differed significantly according to Barnard’s test (p = 0.024 two-tailed and p = 0.008 one-tailed) and the NNH was 18.4. When psychiatric adverse events leading to discontinuation were further added to the equation (+5 events), there were in total 14 events for duloxetine (incidence rate: 8.4%) as compared to 0 for placebo (incidence rate: 0%). These rates differed significantly according to Barnard’s test (p = 0.002 two-tailed and p = 0.001 one-tailed). The resulting NNH was 11.9.

It is possible that one severe psychiatric event that occurred in the extension period (affective disorder) was recorded once as a SAE and once as an event leading to discontinuation. For this reason, we re-ran the last model by counting this event only once, which resulted in 13 events for duloxetine (incidence rate: 7.8%) vs. 0 for placebo (incidence rate: 0%). According to Barnard’s test this difference was statistically significant (p = 0.002 two-tailed and p = 0.001 one-tailed) and the NNH was 12.8.

4. Discussion

According to this re-analysis of a duloxetine trial for juvenile fibromyalgia [26], during the acute placebo-controlled treatment period there were 4 severe treatment-emergent psychiatric adverse events in the duloxetine arm, of which 2 were suicidal events recorded either as SAE or leading to treatment discontinuation. No severe psychiatric events were reported in the placebo arm. The estimated incidence rates were 4.4% vs. 0% and achieved statistical significance with Barnard’s test, indicating that severe treatment-emergent psychiatric adverse events were more likely to occur with duloxetine as compared to placebo. Over the entire study, including acute and extension period, there were in total 7 suicidal events in patients on duloxetine (exposure time 23–39 weeks, depending on whether participants were randomized to placebo for the acute treatment period), yielding an incidence rate of 4.2%. When other severe psychiatric events were added (mostly treatment-emergent mood disorders), then there were in total 14 events recorded as SAE or leading to treatment discontinuation (incidence rate: 8.4%). As detailed above, in patients randomized to placebo (exposure time 13 weeks) no single suicidal or other severe psychiatric event was recorded, yielding an incidence rate of 0%. Barnard’s test indicated that suicidal events (one-tailed test only) and any severe psychiatric adverse event (both two- and one-tailed test) were significantly more likely to occur in patients on duloxetine as compared to placebo, but this should be interpreted with caution due to dissimilar exposure time.

Given that at baseline the participants were not acutely suicidal and mainly mentally healthy (showing no or mild psychopathological symptoms), the estimated incidence rates for suicidal events and any severe treatment-emergent psychiatric adverse event were remarkably high in this duloxetine trial for juvenile fibromyalgia. As duloxetine is supposed to treat as well as to prevent depression symptoms, it is astonishing that several patients on duloxetine developed suicidal ideation and mood disorders, but no patient did so on placebo. In this respect it is also important to note that 6 severe psychiatric adverse events, including 3 suicidal events, were recorded during the duloxetine extension period in participants who were treated with placebo during the acute treatment period. This means that the adolescents had no severe psychiatric events while treated with placebo for 13 weeks, but after duloxetine treatment was initiated several participants newly developed suicidality or mood disorders that were severe enough to be recorded as SAE (n = 5) or to require treatment discontinuation (n = 1). These findings suggest that antidepressants indeed may cause suicidal ideation and behaviour as well as severe emotional disturbances, a notion that is consistent with the results of independent (re-)evaluations of various pediatric depression trials [6,11,12] and the summary estimates for any antidepressant drug derived from meta-analyses of pediatric trials [1,3–5].

However, we acknowledge that according to the assessment with the C-SSRS, during the acute treatment period there were 6 events in the duloxetine arm and 4 in the placebo arm, of which 6 and 3, respectively, were recorded as suicidal ideation and behavior (1 event in the placebo arm was recorded as non-suicidal). The estimated incidence rates for suicidal events were hence 6.6% for duloxetine and 3.2% for placebo. Although the rate of suicidal events is more than two times higher for duloxetine as compared to placebo, this difference was not statistically significant. However, one must not interpret these figures as evidence of no difference simply because the statistical test did not meet the conventional threshold of statistical significance (p < 0.05) [22,24,29]. This false inference is particularly problematic when samples are not adequately powered to detect rare but clinically important harms [20,21,23,25]. Moreover, according to the adverse event reports there was neither a psychiatric SAE nor a psychiatric event leading to discontinuation in the placebo arm. Therefore, we suggest that the 3 suicidal events on placebo according to the C-SSRS were likely not severe. It is also possible that the C-SSRS assessment had produced false-positive results [29,30]. Although the scale has become the gold-standard in suicidology, empirical research has cast doubt on the scale’s validity and its ability to reliably detect active suicidal ideation [30–32].

This study has two major limitations. First and foremost, we had no access to raw individual participant data. Therefore, it was not possible to determine when exactly the adverse events occurred and in which participant. Some participants may have experienced several psychiatric adverse events, and it could also be that the same event was recorded both as an SAE and as an event leading to discontinuation. For instance, during the extension phase there was one SAE recorded as affective disorder, and it could be that this was the same event also recorded as discontinuation due to affective disorder. For this reason, we conducted a sensitivity analysis where this event was counted only once. This change did not affect the interpretation of the main result. Having no access to the raw data also means that we were not able to verify whether all suicidal events were accurately reported by the study sponsor (Eli Lilly, the manufacturer of duloxetine). This is of concern, as independent assessments and re-evaluations of antidepressant trials have consistently shown that suicidal events and other severe psychiatric adverse events are often misreported in industry-sponsored trials [5,12,15].

Second, over the entire study period the exposure time was necessarily higher for duloxetine as compared to placebo, as exposure to placebo was restricted to the acute treatment period. However, 71% of SAE and 60% of any severe psychiatric adverse event that occurred on duloxetine during the extension period were recorded in participants who were randomized to placebo for the acute period, so the exposure to duloxetine was necessarily limited to the extension period in these participants. In addition, given that no single severe psychiatric adverse event was reported for placebo in the acute treatment period, it is unlikely that a placebo extension period would have produced as many severe psychiatric events as recorded in the duloxetine extension period.

5. Conclusion

In conclusion, our re-analysis of the safety data of this duloxetine trial for juvenile fibromyalgia provides further evidence that antidepressants may cause suicidal events and other severe psychiatric adverse events in pediatric patients. Safety plans regarding suicidality should be part of standard practice when prescribing duloxetine for fibromyalgia, especially given the fact that, as opposed to entry criteria into the studies, clinic patients frequently have baseline suicidality [33]. Our findings reinforce the FDA black box warning and further suggest that antidepressants may cause suicidality and other severe psychiatric disorders in adolescents with no or mild psychopathological symptoms at baseline. The benefits of antidepressants for pediatric patients must be weighed against these clinically important harms and it is important to warn mentally healthy children and adolescents that they may develop severe psychiatric disorders after starting an antidepressant.

Footnotes

Conflict of interest

Both authors have no conflict of interest to report.

Funding

No funding was received for this study.

References

Hammad

Laughren

Racoosin

. Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychiatry. 2006;63(3):332–9.

Mosholder

Willy

. Suicidal adverse events in pediatric randomized, controlled clinical trials of antidepressant drugs are associated with active drug treatment: A meta-analysis. J Child Adolesc Psychopharmacol. 2006;16(1-2):25–32.

Dubicka

Hadley

Roberts

. Suicidal behaviour in youths with depression treated with new-generation antidepressants: Meta-analysis. Br J Psychiatry. 2006;189:393–8.

Hetrick

McKenzie

Cox

Simmons

Merry

. Newer generation antidepressants for depressive disorders in children and adolescents. Cochrane Database Syst Rev. 2012;11:CD004851.

Sharma

Guski

Freund

Gotzsche

. Suicidality and aggression during antidepressant treatment: Systematic review and meta-analyses based on clinical study reports. BMJ. 2016;352:i65.

Whittington

Kendall

Fonagy

Cottrell

Cotgrove

Boddington

. Selective serotonin reuptake inhibitors in childhood depression: Systematic review of published versus unpublished data. Lancet. 2004;363(9418):1341–5.

Sparks

Duncan

. Outside the black box: Re-assessing pediatric antidepressant prescription. J Can Acad Child Adolesc Psychiatry. 2013;22(3):240–6.

Spielmans

Spence-Sing

Parry

. Duty to warn: Antidepressant black box suicidality warning is empirically justified. Front Psychiatry. 2020;11:18.

Stone

. The FDA warning on antidepressants and suicidality–why the controversy? N Engl J Med. 2014;371(18):1668–71.

10.

Whitely

Raven

Jureidini

. Antidepressant prescribing and suicide/self-harm by Young Australians: Regulatory warnings, contradictory advice, and long-term trends. Front Psychiatry. 2020;11:478.

11.

Hogberg

Antonuccio

Healy

. Suicidal risk from TADS study was higher than it first appeared. Int J Risk Saf Med. 2015;27(2):85–91.

12.

Le Noury

Nardo

Healy

Jureidini

Raven

Tufanaru

Restoring Study 329: Efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ. 2015;351:h4320.

13.

Jureidini

McHenry

Mansfield

. Clinical trials and drug promotion: Selective reporting of study 329. Int J Risk Saf Med. 2008;20:73–81.

14.

Jureidini

Doecke

Mansfield

Haby

Menkes

Tonkin

. Efficacy and safety of antidepressants for children and adolescents. BMJ. 2004;328(7444):879–83.

15.

de Vries

Roest

Beijers

Turner

de Jonge

. Bias in the reporting of harms in clinical trials of second-generation antidepressants for depression and anxiety: A meta-analysis. Eur Neuropsychopharmacol. 2016;26(11):1752–9.

16.

Hughes

Cohen

Jaggi

. Differences in reporting serious adverse events in industry sponsored clinical trial registries and journal articles on antidepressant and antipsychotic drugs: A cross-sectional study. BMJ Open. 2014;4(7):e005535.

17.

Pitrou

Boutron

Ahmad

Ravaud

. Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med. 2009;169(19):1756–61.

18.

Tang

Ravaud

Riveros

Perrodeau

Dechartres

. Comparison of serious adverse events posted at ClinicalTrials.gov and published in corresponding journal articles. BMC Med. 2015;13:189.

19.

Golder

Loke

Wright

Norman

. Reporting of adverse events in published and unpublished studies of health care interventions: A systematic review. PLoS Med. 2016;13(9):e1002127.

20.

Tsang

Colley

Lynd

. Inadequate statistical power to detect clinically significant differences in adverse event rates in randomized controlled trials. J Clin Epidemiol. 2009;62(6):609–16.

21.

Healy

. The antidepressant tale: Figures signifying nothing? Adv Psychiatr Treat. 2006;12(5):320–8.

22.

Amrhein

Greenland

McShane

. Scientists rise up against statistical significance. Nature. 2019;567(7748):305–7.

23.

Jonville-Bera

Giraudeau

Autret-Leca

. Reporting of drug tolerance in randomized clinical trials: When data conflict with authors’ conclusions. Ann Intern Med. 2006;144(4):306–7.

24.

Altman

Bland

. Absence of evidence is not evidence of absence. BMJ. 1995;311(7003):485.

25.

Young

Nickson

Perner

. When should clinicians act on non-statistically significant results from clinical trials? JAMA. 2020).

26.

Upadhyaya

Arnold

Alaka

Qiao

Williams

Mehta

. Efficacy and safety of duloxetine versus placebo in adolescents with juvenile fibromyalgia: Results from a randomized controlled trial. Pediatr Rheumatol Online J. 2019;17(1):27.

27.

Posner

Brown

Stanley

Brent

Yershova

Oquendo

The Columbia-suicide severity rating scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266–77.

28.

Lydersen

Fagerland

Laake

. Recommended tests for association in 2 × 2 tables. Stat Med. 2009;28(7):1159–75.

29.

Wasserstein

Lazar

. The ASA’s statement on p-values: Context, process, and purpose. Am Stat. 2016;70(2):129–33.

30.

Sheehan

Alphs

Mao

May

Bruer

Comparative validation of the S-STS, the ISST-Plus, and the C-SSRS for assessing the suicidal thinking and behavior FDA 2012 suicidality categories. Innov Clin Neurosci. 2014;11(9-10):32–46.

31.

Giddens

Sheehan

. The Columbia-Suicide severity rating scale (C-SSRS): Has the “gold standard” become a liability? Innov Clin Neurosci. 2014;11(9-10):66–80.

32.

Youngstrom

Hameed

Mitchell

Van Meter

Freeman

Algorta

Direct comparison of the psychometric properties of multiple interview and patient-rated assessments of suicidal ideation and behavior in an adult psychiatric inpatient sample. J Clin Psychiatry. 2015;76(12):1676–82.

33.

Lewcun

Kennedy

Tress

Miller

Sherker

Sherry

. Predicting suicidal ideation in adolescents with chronic amplified pain: The roles of depression and pain duration. Psychol Serv. 2018;15(3):309–15.