Predictors of Placebo Response in Randomized Controlled Trials of Psychotropic Drugs for Children and Adolescents with Internalizing Disorders

Abstract

Objective:

The aim of this study was to assess predictors of placebo response in all available short-term, placebo-controlled trials of psychotropic drugs for children and adolescents with internalizing disorders, major depressive disorder (MDD), obsessive compulsive disorder (OCD,) and anxiety disorders (ANX) exclusive of OCD and posttraumatic stress disorder (PTSD).

Method:

We reviewed the literature relevant to the use of psychotropic medication in children and adolescents with internalizing disorders, restricting our review to double-blind studies including a placebo arm. Placebo response, defined according to each trial's primary response outcome variable and Clinical Global Impressions–Improvement, when available, and potential predictive variables were extracted from 40 studies.

Results

: From 1972 to 2007, we found 23 trials that evaluated the efficacy of psychotropic medication involving youth with MDD, 7 pertaining to youths with OCD, and 10 pertaining to youths with ANX (N = 2,533 patients in placebo arms). For all internalizing disorders combined, predictors of nonresponse to placebo were the percentage of Caucasian patients included in the study and the duration of the disorder: Both variables were negatively correlated with the percent of placebo responders. The type of disorder was found to predict the robustness of placebo response: (OCD < ANX < MDD). For a subset of MDD studies, we found that baseline illness severity tended to be negatively correlated with placebo response. Finally, trial “success” was significantly associated with lower placebo response rate.

Conclusion:

Predictors of placebo response in internalizing disorders of youths parallel those in adult studies, with the exception of race. These predictors should be considered when designing placebo-controlled trials in youths to enhance findings of true drug–placebo differences.

Introduction

I nternalizing disorders are a public health concern because of their frequency and morbidity; child and adolescent depression, in particular, is a concern because of its implication for suicidal acts and youth mortality (Flament et al. 2001; Birmaher et al. 2007). During the 1980s, the arrival of the selective serotonin reuptake inhibitors (SSRIs), which are safer in overdose and associated with fewer side effects than tricyclic antidepressants (TCAs) or monoamine oxidase inhibitors, was viewed as an important step in the treatment of affective and anxiety disorders, first in adults, and then in children and adolescents. In the early 1990s, SSRI use in children and adolescents increased rapidly in developed countries, sometimes to a higher degree than the prevalence rate of depression or anxiety disorder in this age range (Brophy 1995). Prior to the introduction of SSRIs, placebo-controlled trials in youths rarely demonstrated the superiority of TCAs over placebo, owing in part to high placebo response rates, which made the establishment of drug efficacy difficult. The problem of high placebo response rate appears in many SSRI trials in youth major depressive disorder (MDD) trials. In fact, the evidence of SSRI efficacy has been more robustly demonstrated in child and adolescent obsessive compulsive disorder (OCD) (Flament et al. 2000; Geller et al. 2003) and anxiety disorders (ANX) than in MDD (Bridge et al. 2007).

Recently, there has been a substantial increase in interest in placebo response rates in children participating in clinical trials (Rohde et al. 2008; Bridge et al. 2009; Emslie 2009). The focus on placebo response in youth medication trials for MDD has important implications regarding future trial design and selection criteria for study subjects and also has the potential for informing clinical practice decisions regarding when to use medication and in what types of clinical scenarios and patient presentations. We have sought to identify characteristics that may contribute to the placebo response in youths treated with medication for several internalizing disorders in addition to MDD. Previously, we hypothesized that the response to placebo was high in child and adolescent depression because of specific psychopathological factors associated with MDD in youths that promote psychotherapeutic effects in pharmacological trials (Cohen 2007). To test whether response to placebo was different according to type of disorder, we performed a pooled analysis of all randomized double-blind, placebo-controlled studies that have been conducted in the last 25 years on internalizing disorders and reported that the placebo response rates in pharmacological trials for MDD were higher than in those for OCD and other ANX (Cohen et al. 2008).

Identification of other potential predictors of the placebo response in children and adolescents with internalizing disorders could be helpful both in terms of psychopharmacological research, because the placebo response rate is key in detecting drug versus placebo differences within a trial, and in terms of clinical practice, to better detect those children and adolescents that would “truly” benefit from medication. Indeed, in adults, an analysis of the Food and Drug Administration (FDA) Summary Basis of Approval reports for 11 approved antidepressants for MDD between 1985 and 2005 found that the magnitude of placebo response was the single most powerful variable associated with the outcome of an antidepressant trial (Khan et al. 2003). Lower placebo response rates were more likely to result in “positive” treatment trials. For studies with placebo response rate over 30%, the occurrence of significant separation from an antidepressant was 1 in 5; in contrast, for studies with lower placebo response rates, the occurrence of a statistically significance effect favoring drug over placebo was close to 3 in 4.

Placebo response in youths with internalizing disorders has only recently received attention in the literature. Mayes et al. (2007) reported that response to placebo in trials of fluoxetine was more pronounced in children than in adolescents. Bridge et al. (2009) reported in a review of 12 published and unpublished placebo-controlled trials of second-generation antidepressants for child and adolescent depression that the number of study sites was the single best predictor of placebo response. The more study sites, the higher was the placebo response. Baseline severity of illness also emerged as a significant inverse predictor of placebo response. However, to keep the same criterion of response across trials, i.e., Clinical Global Impressions–Improvement (CGI-I), this review did not include all available studies.

In adults, placebo response has received greater attention (Enck et al. 2008), particularly in the context of MDD studies finding a high placebo response rate both in pharmacological and nonpharmacological trials (Brunoni et al. 2009). Adult population predictors of placebo response include younger age, male sex, longer trial duration, fixed-dose protocol (versus flexible-dose), shorter duration of illness, lower baseline severity, fewer prior episodes, study location, and time of research publication (Khan et al. 1991; Wilcox et al. 1992; Charney et al. 2002; Walsh et al. 2002; Stein et al. 2006; Khan et al. 2007; Posternak et al. 2007; Kirsch et al. 2008). Surprisingly, placebo lead-in strategies did not lower subsequent placebo response rates (Trivedi and Trush 1994). This may be a consequence of short intervention lead-in periods. It remains to be seen if alternate lead-in strategies, such as longer periods of placebo therapy or using nonresponse to an initial course of psychotherapy, diminish subsequent placebo response after pharmacological trial inclusion (Gelenberg et al. 2008).

In this article, we assess characteristics and predictors (classified as sociodemographic, methodological, psychopathological, and other) of placebo response in all available short-term, randomized, double-blind, placebo-controlled trials of psychotropic drugs for child and adolescent internalizing disorders, MDD, OCD, and other ANX, exclusive of OCD and posttraumatic stress disorder (PTSD).

Method

Search and study selection

Search strategy and study selection methods have been detailed elsewhere (Cohen et al. 2008). We searched the Medline database for articles describing randomized, placebo-controlled trials of medication for children and adolescents diagnosed with MDD, OCD, and ANX. Searches included combinations of the following keywords: “Major depression,” “obsessive compulsive disorder,” and “anxiety disorders” and/or “children/adolescents” and/or “placebo-controlled.” In addition, references from identified articles and reviews on the same conditions were examined. In particular, recent meta-analyses and reviews that explored the relationship between non-TCAs and suicide were of particular interest, given that some included unpublished trials and detailed response rates according to age (Kapczinski et al. 2003; Ryan 2005; Jureidini et al. 2004; Whittington et al. 2004; Hammad et al. 2006; Hetrick et al. 2007; Bridge et al. 2009). Using these methods, we found 70 publications between January, 1972, and October, 2007. Exclusion criteria were: (1) Crossover design; (2) no indication of the number of responders in either the original report or the available reviews; (3) fewer than 10 individuals in the placebo arms; (4) lack of placebo arms; (5) no randomization; (6) no double-blind evaluation of response; (7) studies on PTSD; and (8) secondary analysis of prior reported trials. Table 1 lists all of the published controlled randomized trials of psychotropic medications that were included in the current analysis. Details on excluded studies and trial flow are given in Cohen et al. (2008).

Table 1.

Placebo Response Rates and Main Characteristics in Double-Blind Placebo-Controlled Trials for Internalized Disorders in Children and Adolescents

Study	Q	N (ages) duration	M	White	Site	U.S. only	Washout Screening	PBO run-in	Drug	Disease duration	First episode	Severity (CGI)	Definition of responders	Placebo Responders	Drug > placebo ^a
Major depressive disorder
Kramer and Feiguine 1981	0.67	20 (13–18) 6 weeks	15	60	1	Yes	No	No	Amitryptiline	36	NA	NA	Authors-scale	6/10 (60%)	No
Preskorn et al. 1987	0.64	22 (6–12) 6 weeks	NA	NA	1	Yes	Yes	Yes	Imipramine	NA	NA	NA	CGI	3/12 (25%)	No
Puig-Antich et al. 1987	0.77	53 (NA) 5 weeks	9.2	59	1	Yes	Yes	No	Imipramine	NA	NA	NA	K-SADS-dep/anhedonia ≤ 2	15/22 (68%)	No
Hughes et al. 1990	0.5	31 (6–12) 6 weeks	NA	NA	NA	Yes	NA	NA	Imipramine	NA	NA	NA	CDRS 50%	7/14 (50%)	No
Geller et al. 1990	0.6	31 (12–17) 8 weeks	14.4	42	2	Yes	Yes	Yes	Nortryptiline	41	NA	NA	CDRS < 25%	4/19 (21%)	No
Geller et al. 1992	0.89	50 (6–12) 8 weeks	9.7	66.7	1	Yes	Yes	Yes	Nortryptiline	40	NA	NA	CDRS ≤ 20	5/24 (17%)	No
Kutcher et al. 1994 *	0.7	42 (15–19) 6 weeks	17.8	30	2	Yes	Yes	Yes	Desipramine	NA	NA	NA	HDRS 50%	9/25 (36%)	No
Kye et al. 1996	0.77	31 (12–18) 6 weeks	15.1	69	1	Yes	Yes	Yes	Amitryptiline	14	NA	3.5	HDRS 50%	11/13 (90%)	No
Emslie et al. 1997	0.97	96 (8–18) 8 weeks	12.5	64	1	Yes	Yes	Yes	Fluoxetine	3.2	48	4.9	CGI ≤ 2	16/48 (33%)	Yes
Birmaher et al. 1998	0.73	27 (13–17) 10 weeks	16.1	29	1	Yes	Yes	No	Amitryptiline	13.6	50	4.6	HDRS 50%	11/14 (79%)	No
Klein et al. 1998	0.83	45 (13–18) 6 weeks	15.7	33	2	Yes	Yes	Yes	Desipramine	9	11	NA	CGI ≤ 2	9/18 (50%)	No
Milin et al. 2000 **	0.97	286 (13–18) 12 weeks	15.6	44	33	No	Yes	Yes	Paroxetine	19.1	70	4.4	MADRS 50%^c	53/91 (58%)	No
Keller et al. 2001	0.85	275 (13–17) 8 weeks	15.1	46	12	No	Yes	No	Paroxetine	13	77	4.4	HDRS <8 or 50%^c	48/87 (55%)	No
Emslie et al. 2002	0.89	219 (8–18) 8 weeks	12.7	51	16	Yes	Yes	Yes	Fluoxetine	14.1	79	NA	CDRS 30%^c	54/101 (54%)	No
Wagner et al. 2003	0.93	376 (6–17) 10 weeks	12	49	53	Yes	Yes	No	Sertraline	22	88	4.5	CDRS 40%^c	105/179 (59%)	Yes^b
March et al. (TADS) 2004	1	439 (12–17) 12 weeks	14.6	46	13	Yes	No	No	Fluoxetine	17.3	86	4.8	CGI ≤ 2	39/112 (35%)	Yes
Wagner et al. 2004	0.67	174 (7–17) 8 weeks	12	47	21	No	Yes	Yes	Citalopram	18.6	81	4.3	CDRS ≤ 28^c,d	20/85 (24%)	Yes
Emslie et al. 2006	0.97	206 (7–17) 8 weeks	12	53	41	No	No	No	Paroxetine	24.9	53	4.3	CGI ≤ 2	46/100 (46%)	No
Von Knorring et al. 2006	0.5	244 (13–18) 12 weeks	16	NA	31	No	No	No	Citalopram	NA	36	NA	K-SADS-dep/anhedonia ≤2	47/77 (61%)	No
Wagner et al. 2006	0.67	268 (6–17) 8 weeks	12.6	48	25	Yes	No	Yes	Escitalopram	15.6	NA	4.2	CGI ≤ 2	69/132 (52%)	No
Emslie et al. 2007	0.8	367 (7–17) 8 weeks	12.2	54	51	Yes	Yes	Yes	Venlafaxine	21.4	86	4.5	CDRS 35%^c,d	99/165 (60%)	No^b
FDA: CN104-141 2007	ND	206 (12–17) 8 weeks	14.7	41	15	Yes	NA	No	Nefazodone	18.5	73	NA	CGI ≤ 2	42/95 (44%)	No
FDA: 003-045 2007	ND	259 (7–17) 8 weeks	12.2	48	34	Yes	NA	No	Mirtazapine	NA	NA	NA	CGI ≤ 2	42/85 (49%)	No
Obsessive compulsive disorder
DeVeaugh et al. 1992	0.79	60 (10–17) 8 weeks	14	55	6	Yes	Yes	Yes	Clomipramine	49	NA	NA	CGI ≤ 2	5/29 (17%)	Yes
March et al. 1998	0.86	189 (6–17) 12 weeks	12.6	NA	12	Yes	Yes	Yes	Sertraline	56	NA	3.8	CYBOCS 25%^c	35/95 (37%)	Yes
Geller et al. 2001	0.92	103 (6–17) 13 weeks	11.4	47	21	Yes	Yes	No	Fluoxetine	NA	NA	4.8	CYBOCS 40%^c	8/32 (25%)	Yes
Riddle et al. 2001	0.86	120 (8–17) 10 weeks	12.7	55	17	Yes	Yes	Yes	Fluvoxamine	41	NA	3.9	CYBOCS 25%^c	17/63 (27%)	No
Liebowitz et al. 2002	0.72	43 (8–17) 8 weeks	12.3	64	2	Yes	Yes	No	Fluoxetine	NA	NA	4.6	CGI ≤ 2	7/22 (32%)	No
Geller et al. 2004	0.96	207 (7–17) 10 weeks	11.6	61	36	Yes	No	No	Paroxetine	53	60	4.5	CYBOCS 25%^c	42/102 (41%)	Yes
POTS 2004	1	112 (7–17) 12 weeks	12.3	50	3	Yes	No	No	Sertraline	NA	NA	4.7	CYBOCS < 10	1/28 (4%)	Yes
Non-obsessive compulsive anxiety disorder
Gittelman-Klein et al. 1973	0.86	35 (6–15) 6 weeks	10.8	46	1	Yes	No	No	Imipramine	NA	NA	NA	School attendance	9/19 (47%)	No
Berney et al. 1981	0.73	46 (9–15) 12 weeks1	NA	41	1	Yes	No	No	Clomipramine	NA	NA	3.8	CGI ≤ 2	10/19 (53%)	No
Klein et al. 1992	0.7	20 (6–15) 6 weeks	9.5	66	1	Yes	Yes	No	Imipramine	NA	NA	NA	Global improvement	4/9 (44%)	No
Simeon et al. 1992	0.43	30 (NA) 4 weeks	11.8	76	2	No	Yes	Yes	Alprazolam	NA	NA	NA	CGI ≤ 2	5/13 (38%)	No
Rynn et al. 2001	0.93	22 (5–17) 9 weeks	11.7	77	1	Yes	Yes	No	Sertraline	NA	NA	4	CGI ≤ 2	1/11 (9%)	Yes
RUPP 2001	0.82	128 (6–17) 8 weeks	10.3	51	5	Yes	Yes	No	Fluvoxamine	NA	NA	NA	CGI ≤ 2	19/65 (29%)	Yes
Birmaher et al. 2003	0.89	74 (7–17) 12 weeks	11.9	46	1	Yes	Yes	No	Fluoxetine	NA	NA	3.9	CGI ≤ 2	13/37 (35%)	Yes
Wagner et al. 2004	0.95	322 (8–17) 16 weeks	13.3	57	38	No	No	No	Paroxetine	NA	NA	4.4	CGI ≤ 2	59/154 (38%)	Yes
Rynn et al. 2007	0.82	323 (6–17) 8 weeks	11.2	52	59	Yes	Yes	Yes	Venlafaxine	39	NA	4.5	CGI ≤ 2	77/159 (48%)	Yes
March et al. 2007	0.95	293 (8–17) 16 weeks	13.6	44	48	Yes	No	No	Venlafaxine	59	NA	4.9	CGI ≤ 2	54/148 (37%)	Yes

*and Boulos et al. 1992.

**and Berard et al. 2006.

Possible superiority of drug vs. placebo concerns the primary variable.

These reports are pooled analysis in which individual trials did not reveal significant treatment effect.

CGI responders are also indicated as secondary variables in these studies.

This study used a continuous variable as the primary outcome variable; the same variable as a dichotomous secondary variable was chosen in the current analysis for definition of responders.

Abbreviations: N = Number of subjects randomized in the study; Q = quality score of the report; NA = not appropriate; M = % of male included in the PBO arm; White = % of white included in the PBO arm; Site = number of sites; Disease duration = in months; First episode = % of first episode included in the PBO arm; CDRS = Children's Depression Rating Scale; CGI = Clinical Global Impression—Severity; HDRS = Hamilton Depression Rating Scale; K-SADS-dep = Schedule for Affective Disorder and Schizophrenia—depression/anhedonia subscore; MADRS—Montgomery—Asberg Depression Rating Scale; CYBOCS = Child Yale-Brown Obsessive Compulsive Scale; Placebo responders = N of responders in the placebo arm/N of subjects randomised in the placebo arm (%).

Table references

Berard et al. J Child Adolesc Psychopharmacol 16:59–75, 2006.

Berney et al. Br J Psychiatry 138:110–118, 1981.

Birmaher et al. J Am Acad Child Adolesc Psychiatry 37:527–535, 1998.

Birmaher et al. J Am Acad Child Adolesc Psychiatry 42:415–423, 2003.

Boulos et al. Psychopharmacol Bull 27:59–65, 1991.

DeVeaugh-Geiss et al. J Am Acad Child Adolesc Psychiatry 31:45–49,1992.

Emslie et al. Arch Gen Psychiatry 54:1031–1037, 1997.

Emslie et al. J Am Acad Child Adolesc Psychiatry 41:1205–1215, 2002.

Emslie et al. J Am Acad Child Adolesc Psychiatry 45:709–719, 2006.

Emslie et al. J Am Acad Child Adolesc Psychiatry 46:479–488, 2007.

FDA: CN104141: www.fda.gov/cder/foi/esum/2004/20152s032_Serzone_clinical_BPCA_FIN.pdf/, 2007.

FDA: 003-045: www.fda.gov/cder/foi/esum/2004/20415SE5_011_Mirtazapine%20MO%20ReviewFIN.pdf/, 2007.

Geller et al. Psychopharmacol Bull 26:85–90, 1990.

Geller et al. J Am Acad Child Adolesc Psychiatry 31:34–44, 1992.

Geller et al. J Am Acad Child Adolesc Psychiatry 40: 773–779, 2001.

Geller et al. J Am Acad Child Adolesc Psychiatry 43: 1387–1396, 2004.

Gittelman-Klein et al. J Nerv Ment Dis 156:199–215, 1973.

Hughes CW et al. Psychopharmacol Bull 26:235–238, 1990.

Keller et al. J Am Acad Child Adolesc Psychiatry 40:762–772, 2001.

Klein et al. J Am Acad Child Adolesc Psychiatry 31: 21–28, 1992.

Klein et al. Depress Anxiety 7:15–31, 1998.

Kramer and Feiguine. J Am Acad Child Psychiatry 20: 636–644, 1981.

Kutcher et al. J Am Acad Child Adolesc Psychiatry 33: 686–694, 1994.

Kye et al. J Am Acad Child Adolesc Psychiatry 35:1139–1144, 1996.

Liebowitz et al. J Am Acad Child Adolesc Psychiatry 41:1431–1438, 2002.

March et al. JAMA 280:1752–1756, 1998.

March J et al. JAMA 292:807–820, 2004.

March et al. Biol Psychiatry 62: 1149–1154, 2007.

Milin R, Simeon J, Spenst W. Brown Univ Child Adolescent Psychopharmacol Update 2:5–7, 2000.

POTS. JAMA 292:1969–1976, 2004.

Preskorn et al. Psychopharmacol Bull 23:128–133, 1987.

Puig-Antich et al. Arch Gen Psychiatry 44: 81–89, 1987.

Riddle et al. J Am Acad Child Adolesc Psychiatry 40:222–229, 2001.

Rynn et al. Am J Psychiatry 158:2008–2014, 2001.

Rynn et al. Am J Psychiatry 164:290–300, 2007.

Simeon et al. J Am Acad Child Adolesc Psychiatry 31:29–33, 1992.

The Research Unit on Pediatric Psychopharmacology Anxiety Study Group. N Engl J Med 344:1279–1285, 2001.

von Knorring et al. J Clin Psychopharmacol 26:311–315, 2006.

Wagner et al. Arch Gen Psychiatry 61:1153–1162, 2004.

Wagner et al. J Am Acad Child Adolesc Psychiatry 45: 280–288, 2006.

Wagner KD et al. JAMA. 290: 1033–1041, 2003.

Wagner KD et al. Am J Psychiatry 161:1079–1083, 2004.

Response criteria

Given that analyses were performed across studies, we kept each trial's definition of responders as indicated in Table 1. Owing to the diversity of definition of “response” across studies, we performed a subgroup analysis that used only the CGI-I, when used in the trials as either a primary or secondary variable (e.g., responder = CGI-I ≤2), as our response criterion.

Predictors of placebo response

We examined the following predictors of placebo response. (1) Sociodemographic variables: Age (mean age of the placebo arm); sex (% of males); race (% of caucasian patients); socioeconomic status (SES) (% of low SES). (2) Methodological characteristics of the trials: N of patients included in the placebo arm; type of active treatment (TCA vs. non-TCA), duration of the trial; number of study sites; number of meetings during the study protocol; Detsky et al.'s short version quality score of the trial that includes items related to randomization, blindness, inclusion/exclusion criteria, outcome measures, treatment description, and statistical analysis (Detsky et al. 1992); use of a placebo run-in; use of a screening and/or washout period; location of the study (U.S. only vs. not U.S. only). (3) Psychopathological characteristics: Disorder type (MDD, OCD, or ANX); duration of the disorder prior to trial entrance; prior history (% of first episodes); CGI–Severity at baseline. We added two other variables: The year of publication, as it has been shown that the time of research publication, with more recent studies showing higher placebo response rates, may be important in determining the placebo response (Keck et al. 2000; Walsh et al. 2002; Sysko and Walsh 2007), and the “positive” (or negative) outcome of the difference between drug and placebo at the end of the trial (Khan et al. 2003). Of note, regarding the year of publication, we chose: (1) The first publication when trials were reported in several papers; (2) the year of FDA or drug company synopsis for unpublished trials.

Statistical analysis

Descriptive statistics were used to characterize the features of each of the included trials. The assumption of normality was assessed using the Shapiro–Wilk test. The assumption of homogeneity of variance was assessed using the Barlett test. Then, depending on the nature of each variable, we used the following method to test statistical links with response to placebo. First, an analysis of variance (ANOVA) was applied for comparison of response rates according to diagnosis (MDD, OCD, and ANX). Second, Pearson's correlations (r) were used for continuous variables that showed a normal distribution. For those that did not show a normal distribution, we examined their link using the Spearman rank correlation (ρ). As a sensitivity analysis, we also iteratively deleted each trial and recomputed the correlation estimates to assess the stability of the results. Finally, two-group comparisons were made using Student t-tests. Secondary analyses following the same methodology were conducted on the subsample of studies that used CGI-I as response criterion (N = 29), on the subsample of studies that included patients with MDD (N = 23).

Results

Among the 70 studies (N = 5,894), we found 23 trials that met our inclusion criteria and evaluated the efficacy of psychotropic medication (mainly non-TCAs) involving youths with MDD, 7 on youths with OCD, and 10 on youths with other ANX. Altogether, the studies included 2,533 patients in placebo arms (n = 1,528 in MDD, n = 371 in OCD, n = 634 in ANX). Table 1 summarizes study characteristics, including potential predictive variables. Our goal to include SES in the potential predictors was unsuccessful owing to both the absence of inclusion in many trials and the diversity of reporting methods in others.

Regarding placebo response rates according to diagnosis, the assumption of homogeneity of variance was assessed with the Barlett test, yielding a p value of 0.53. The assumption of normality was assessed with the Shapiro–Wilk test, which yielded p values of 0.65 (MDD), 0.77 (OCD), and 0.3 (ANX), respectively. Given that it was not possible to reject the hypothesis of the normality of residuals, we tested each potential predictor as indicated in Table 2, depending on the type of variable (continuous vs. dichotomous) and its distribution (normal or not) (see Methods). The same analysis was conducted on the subgroup of studies (n = 29) that used the CGI to indicate responder status. All results are shown in Table 2.

Table 2.

Predictor of Placebo Response Rates in Double-Blind Placebo-Controlled Trials for Internalized Disorders in Children and Adolescents

		Primary variable response		CGI-I response
Variables	Statistics	N of trial	Results	N of trial	Results
Sociodemographic characteristics
Mean age	Pearson	37	r = 0.3 (p = 0.076)	27	r = 0.27 (N.S.)
Male	Pearson	36	r = −0.16 (N.S.)	27	r = −0.4 (p = 0.0538)
White	Spearman	32	ρ = −0.457 (p = 0.006)	24	ρ = −0.52 (p = 0.009)
Methodological characteristics
N placebo arm	Spearman	40	ρ = 0.1 (N.S.)	29	ρ = 0.25 (N.S.)
Study duration	Spearman	40	ρ = −0.21 (N.S.)	29	ρ = −0.17 (N.S.)
TCA vs. other drug	Student	40	t = −0.43 (p = 0.67)	29	t = −0.34 (p = 0.74)
Washout	Student	36	t = 0.84 (p = 0.4)	29	t = 2.4 (p = 0.024)
Placebo run-in	Student	39	t = −0.44 (N.S.)	29	t = 0.21 (N.S.)
U.S. only	Student	40	t = 0.69 (N.S.)	29	t = 1.43 (N.S.)
N of meeting	Pearson	30	r = 0.0 (N.S.)	22	r = −0.323 (N.S.)
Quality score	Spearman	38	ρ = −0.02 (N.S.)	27	ρ = −0.08 (N.S.)
Psychopathological characteristics
Disorder duration	Pearson	23	r = −0.46 (p = 0.027)	19	r = −0.46 (p = 0.04)
First episode	Pearson	14	r = −0.14 (N.S.)	12	r = 0.04 (N.S.)
Baseline CGI-S	Pearson	23	r = −0.16 (N.S.)	19	r = 0.33 (N.S.)
Disease^a	ANOVA	40	F = 6.07 (p = 0.005)	29	F = 7.37 (p = 0.003)
Other variables
Year of publication	Spearman	40	ρ = 0.05 (N.S.)	29	ρ = 0.23 (N.S.)
Drug > placebo^b	Student	40	t = 2.61 (p = 0.013)	29	t = 2.25 (p = 0.033)

Major depressive disorder versus obsessive compulsive disorder versus other anxiety disorder.

Means that the drug was significantly superior to the placebo on the primary response variable.

Abbreviations: CGI-I = Clinical Global Impressions–Improvement; N = Number; N.S. = Not significant; TCA = tricyclic antidepressant; CGI-S = Clinical Global Impressions–Severity; ANOVA = analysis of variance.

Among sociodemographic characteristics, the percentage of Caucasian patients included in the studies was significantly negatively correlated with the percent of placebo responders whatever the definition; i.e., the more Caucasian patients, the lower the placebo response. To assess whether one single trial strongly influenced this finding, we performed a sensitivity analysis by iteratively deleting each trial and recomputing the correlation estimates. The correlation remained significant with a Spearman correlation coefficient ρ ranging from −0.43 to −0.55 (0.003 ≤ p ≤ 0.014), depending on the excluded trial. The percentage of males tended to be negatively correlated with the percentage of placebo responders defined with CGI, i.e., the more male patients, the lower the placebo response.

Among the methodological characteristics, the use of a washout or screening period is the only variable that was significantly associated with the CGI placebo response. When a washout/screening period occurred prior to entry into the trial, the placebo response was lower (33.44% ± 13.5% versus 45.8% ± 10.1% on CGI-I placebo response of 1 or 2, respectively). However, the difference did not reach statistical significance when the definition of responder using other criteria was the primary variable (38.8% ± 20.1% versus 44.3% ± 15.7%).

Among the psychopathological characteristics, two variables were strong predictors of response: The type of disorder and the duration of the index episode of illness prior to trial entrance. Placebo response was lower in OCD than ANX and MDD trials (31% [range, 4–41%] vs. 39.6% [range, 9–53] vs. 49.6% [range, 17–90%], respectively). Disorder duration was significantly negatively correlated with the percent of placebo responders whatever the definition, i.e., the longer the duration of illness, the lower the placebo response. The sensitivity analysis showed that the correlation remained significant with a Pearson correlation coefficient r ranging from −0.42 to −0.56 (0.007 ≤ p ≤ 0.05) depending on the excluded trial (Fig. 1).

FIG. 1.

Proportion of placebo response in child and adolescent internalizing disorders according to the percent of (A) white/Caucasian patients and (B) the disorder duration prior entry. CGI = Clinical Global Impressions.

Superiority of the drug versus placebo on the primary outcome variable was associated with the degree of placebo response whatever the definition of response. The lowest placebo responses yielded trials showing greater superiority of the tested drugs. When the trial conclusion was that the drug effect was superior to the placebo effect, the percentage of placebo response (mean ± SD) was 32.7 ± 16.4 and 33.1 ± 13.4 according to either the primary variable or the CGI-I definitions, respectively. When the trial conclusion was that drug effect failed to separate from the placebo effect, the percentage of placebo response (mean ± SD) was 47 ± 17.1 and 43.6 ± 11.6, respectively. Finally, time of publication was not significantly associated with placebo response.

When the same analysis was conducted on the subgroup of studies testing pharmacological treatments in child and adolescent MDD (n = 23), only baseline CGI–Severity showed a tendency to a significant negative correlation with the percentage of placebo responders. The more severe the disorder at baseline, the lower the placebo response (r = −0.54, p = 0.09, n = 11; and r = −0.61, p = 0.08, n = 9, for definition of response with the primary variable and the CGI-I, respectively).

Discussion

The main limitation of this study is the fact that we pooled all placebo arms from studies that varied in methodology (e.g., inclusion criteria, initial placebo washout period, definition of responders, co-morbidity inclusions, and duration of treatment). In particular, the main analysis was conducted with a variety of outcome variables, and, for a common outcome variable, a diversity of responder definitions were used (e.g., in OCD studies, responders were defined as patients showing a 25% or 40% decrease of the Children's Yale-Brown Obsessive Compulsive Scale [CYBOCS]; see Table 1). However, the fact we found overall similar results with the CGI criterion of response is somewhat reassuring. Most of the studies were sponsored by the pharmaceutical industry, and, in many cases, either broad inclusion criteria (e.g., including youth with co-morbid anxiety in depression trials and vice versa), or narrow inclusion criteria were used (excluding suicidal youth from depression trials), both of which may limit generalizability to real-world clinical populations.

In our study, pooled analysis does not address the fact that MDD is likely to be highly heterogeneous and that some patients may differ in their response to nonspecific professional attention (as evidenced in the wide range of placebo response rates across studies) (Cohen et al. 2008). Three studies conducted before 1980 did not use Diagnostic and Statistical Manual of Mental Disorders, 3^rd edition (DSM-III) (or later) criteria (American Psychiatric Association 1980) and may have had substantial differences in the disorders' definitions. However, they only accounted for 48 patients in the total analysis. Compared to the Bridge analysis (2009), we did not consider predictors of drug response as a comparator to the analyses of predictors of placebo response. This choice was based on our primary hypothesis related to differential placebo response according to psychopathology. We decided to focus on placebo arms only because we hypothesized that response to drug may be dependent on psychopathology and other factors beyond our set of hypotheses as well. Finally, we were limited to trial-level summary data, which may fail to identify important individual patient factors influencing the response to placebo. Therefore, important factors such as SES, psychiatric family history, family adherence, history of sexual abuse or of other maltreatment, and time course of response over study weeks could not be investigated.

Despite these limitations, and in contrast to what has been shown in adult mood disorders (Keck et al. 2000; Walsh et al. 2002; Sysko and Walsh 2007), we did not find a positive correlation between placebo response rate and year of publication despite apparent variability in both placebo rates and publication dates. Our study supports the findings of Bridge et al. (2009) in a wider range of disorders, that increasing placebo response rates over time most likely is a publication artifact, more likely associated with increasing number of study sites over time. Second, age did not appear to modify placebo response rates in children versus adolescents across internalizing disorders (Cohen et al. 2008; Bridge et al. 2009). Most of the data used in these analyses came from studies on SSRIs that were conducted in the late 1990s and early 2000s, after several consensus conferences and guidelines had been done on youth psychopharmacological trials, and the methodologies were very similar for these studies (Bridge et al. 2007). When the CGI-I was used to define placebo response rates (either as the primary or secondary variable), we obtained the same results. There is some evidence that CGI severity scores at baseline may be capturing different aspects of illness severity versus other measures (e.g., Children's Depression Rating Scale, Revised [CDRS-R]) and that the CGI-I may serve to inflate placebo response rates (Bridge et al. 2009).

MDD in children and adolescents appears to be more responsive to placebo than do other internalizing disorders, which may suggest differential features underlying the psychopathology and pathophysiology of the disorders, discussed in depth elsewhere (Cohen et al. 2008). Our findings reveal that, as in adults (Khan et al. 2003), the magnitude of placebo response was the single most powerful predictor of the outcome of an antidepressant trial for all internalizing disorders studied. This is not surprising, given that the percentage of responders in the active compound arm of most studies in youth with internalizing disorders varies fairly widely: 36%–71% (MDD), 21%–65% (OCD), and 56%–91% (ANX). The ranges are narrower when extreme values are excluded: 49%–67% (MDD), 42%–57% (OCD), and 56%–78% (ANX) (Bridge et al. 2007). Therefore, the “success” of a trial is more dependent on the placebo response amplitude than the response to the active treatment itself. This result confirms the Bridge et al. (2009) analysis on trials for youth with depression.

We identified another inverse predictor of placebo response related to psychopathological characteristics: The duration of the disorder prior to trial inclusion. When the analysis was limited to the subgroup of studies on MDD, we only found a tendency for baseline severity of illness to correlate with lower placebo response rate, as in the Bridge et al. (2009) analysis. This is consistent with a recent meta-analysis conducted by Kirsch et al. (2008) showing that drug–placebo differences in antidepressant efficacy in adults increase as a function of baseline severity, and that the relationship between initial severity and antidepressant efficacy was attributable to decreased responsiveness to placebo among very severely depressed patients rather than to increased responsiveness to medication. These last points raise questions about the benefit-to-risk profile of using medications in treating youths with internalizing disorders who have only mild-to-moderate functional impairment. The use of psychotherapy and/or psychosocial approaches as first-line treatment for mild-to-moderate severity pediatric internalizing disorders may be a better first choice (Muratori et al. 2003; Brent et al. 1997; Goodyear et al. 2007). Pharmacotherapy should be restricted to patients showing insufficient improvement of their mood or anxiety symptoms with psychosocial treatment or psychotherapy and to patients showing severe impairments (Cohen 2007; Emslie 2009). Indeed, a recent finding from the Treatment of Adolescent Depression Study (TADS) group (Kennard et al. 2009) is reassuring in that patients who do not initially respond to placebo subsequently respond to active treatment at the same rate as those initially assigned to active treatment and that there were no differences between these groups in terms of rates of suicidal events. For children and adolescents with low-to-moderate levels of internalizing symptoms, it seems that being in regular clinical care, and not medication treatment per se, is the most important initial treatment intervention.

Methodological characteristics were not significant predictors of placebo response. Use of a placebo run-in, study duration, and Detsky et al. quality scores all failed to emerge as predictors. The only moderate contribution to reducing placebo response was the use of a washout–screening period, which was significantly associated with a lower placebo CGI-I response in the subgroup analysis. However, this result should be regarded with caution because it was not confirmed in the larger analysis. This may be a consequence of the brief washout–screening and placebo run-in periods, typically lasting less than 2 weeks. In contrast to the Bridge et al. (2009) report on MDD, we did not find a correlation of placebo response with the number of study sites, neither in our large analysis, nor in the MDD subanalysis. This may be due to our larger sample of studies (23 in MDD in our sample vs. 12 in Bridge et al.). Although as expressed by these authors, the tendency to increase the number of study sites in trials for youth MDD to enhance the number of patients, and therefore the statistical power, has been remarkable in recent years, has resulted in the tendency to include less severe patients and may hinder the ability to detect true drug–placebo differences in future studies in youth.

Finally, an intriguing result of the present analysis is the strong negative correlation found between the percentage of Caucasian patients included in the trials and the magnitude of the placebo response. This correlation appeared with both definitions of response, as defined by the primary study variable and CGI-I responder status, respectively (ρ = −0.457, p = 0.006; ρ = −0.52, p = 0.009). We cannot account for this finding based on genetic differences between Caucasians and other racial groups because we are not aware of such findings having been reported in either the child or adult depression treatment literature. We hypothesize that Caucasian race may serve as a proxy for other socioeconomic variables rather than the construct of ethnicity per se. Indeed, low SES and early life adversities are risk factors for internalizing disorders (Goodyear et al. 2007) and may serve as response predictors to different types of treatment interventions. Alternatively, the meaning of medication treatment may be perceived differently across cultural/ethnic/racial groups, leading to different expectations and placebo response rates. For example, differences in placebo response rates by the origin of patients has been reported in adult migraine prophylaxis (Macedo et al. 2008) and by patient's personal expectations in chronic pain (Linde et al. 2007). In a recent study, Stevens et al. (2009) showed that African-American parents may hold more negative perceptions than other parents regarding antidepressant prescription. Or, some other factor, such as unidentified co-morbidities (e.g., conduct disorder), was differentially represented across ethnic groups in the trials, the latter of which we were unable to capture in our analysis.

Our study has several implications for future research and for clinical decision making. Because differences in drug effectiveness appear to be due more to placebo rather than active treatment factors and to patient characteristics such as illness severity and duration, greater attention should be paid to recruiting study subjects with severe levels of illness. The use of longer placebo run-ins or assessment periods should be considered prior to actual drug treatment in clinical trials. Studies with patients resistant to psychotherapy should be considered as well. Consideration should be given to limiting future clinical trials to fewer sites using highly trained researchers to more tightly manage recruitment, assessment, and conduct of treatment methodology. Future studies should also examine the time course of response to identify potential differences in the time–action effect of placebo versus drug. The true differences in placebo responsivity between adolescents and preadolescents remains an open question, and future stratified study designs may elucidate developmental characteristics important to ultimate treatment response. So too, there have been almost no direct drug–drug–placebo comparator trials in child and adolescent populations. Undertaking comparator trials may begin to elucidate true differences in efficacy between the various drugs on the market for particular types of patients and disorders. Greater attention should be paid to systematically assessing socioeconomic and ethnic variables because this may identify modifiers of treatment response to drug versus placebo in select groups.

Finally, clinicians need to understand the strengths and limitations of research on placebo response. Children and adolescents with mild-to-moderate levels of internalizing symptoms, or of short duration illness, may be better served, and not appreciably harmed, by more conservative approaches to initial therapy using psychosocial treatments before consideration of medication. This was recently confirmed in the analysis of long-term outcomes of adolescents initially treated with placebo in the TADS short-term trial (Kennard et al. 2009).

Footnotes

Disclosures

Previously, David Cohen was an investigator in two industry-sponsored trials, DEROXADO and ADOKOTE, conducted by SmithKline Beecham and Sanofi-Synthélabo, respectively. He also received honoraria for speaking fees from Janssen. Craig Donnelly serves as a consultant for Eli Lilly, and has received grant and research funds from Eli Lilly. He has also served as a consultant to Johnson & Johnson, is on a speakers' bureau for Johnson & Johnson, and serves on a speakers' bureau for Shire. David Cohen, Angèle Consoli, Nicolas Bodeau, Diane Purper-Ouakil, Emmanuelle Deniau, Jean-Marc Guilé report no competing interests during the last 36 months.

Acknowledgments

The authors would like to thank the Centre d'Activités et de Recherches en Psychiatrie Infanto-Juvénile (CARPIJ), the Programme Hospitalier de Recherche Clinique (PHRC) (2006AOM098), and the Fondation Whyeth for their support in promoting research on child and adolescent mood disorders.

Dr. Donnelly has received research funding from the National Institute for Mental Health (NIMH) (STAART [Citalopram for Children With Autism and Repetitive Behavior] study #U54 MH066398-04).

References

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 3rd edition (DSM-III) Washington (DC): American Psychiatric Association, 1980.

Birmaher

, Brent

. AACAP Work Group on Quality Issues et al: Practice parameter for the assessment and treatment of children and adolescents with depressive disorders. J Am Acad Child Adolesc Psychiatry, 46:1503–1526. 2007.

Brent

, Holder

, Kolko

, Birmaher

, Baugher

et al. A clinical psychotherapy trial for adolescent depression comparing cognitive, family and supportive treatments. Arch Gen Psychiatry, 54:877–885. 1997.

Bridge

, Iyengar

, Salary

, Barbe

, Birmaher

, Pincus

, Ren

, Brent

. Clinical response and risk for reported suicidal ideation and suicide attempts in pediatric antidepressant treatment: A meta-analysis of randomized controlled trials. JAMA, 297:1683–1696. 2007.

Bridge

, Birmaher

, Iyengar

, Barbe

, Brent

. A Placebo response in randomized controlled trials of antidepressants for pediatric major depressive disorder. Am J Psychiatry, 166:42–49. 2009.

Brophy

. Kindergarteners in Prozac nation. US News and World Report, 13:96–97. 1995.

Brunoni

, Lopes

, Kaptchuk

, Fregni

. Placebo response of non-pharmacological, pharmacological trials in major depression: A systematic review, meta-analysis. PlosOne, 4:e4824. 2009.

Charney

, Nemeroff

, Lewis

, Laden

, Gorman

, Laska

, Borenstein

, Bowden

, Caplan

, Emslie

, Evans

, Geller

, Grabowski

, Herson

, Kalin

, Keck

Jr. , Kirsch

, Krishnan

, Kupfer

, Makuch

, Miller

, Pardes

, Post

, Reynolds

, Roberts

, Rosenbaum

, Rosenstein

, Rubinow

, Rush

, Ryan

, Sachs

, Schatzberg

, Solomon

, Consensus Development

. National Depressive and Manic-Depressive Association consensus statement on the use of placebo in clinical trials of mood disorders. Arch Gen Psychiatry, 59:262–270. 2002.

Cohen

. Should the use of specific serotonine reuptake inhibitors be banned? Psychother Psychosom, 76:5–14. 2007.

10.

Cohen

, Deniau

, Maturana

, Tanguy

, Bodeau

, Labelle

, Breton

, Guile

. Why is placebo response higher in major depression than in anxiety disorders in children and adolescent? PlosOne, 3:e2632. 2008.

11.

Detsky

, Naylor

, O'Rourke

, McGeer

, L'Abbe

. Incorporating variations in the quality of individual randomized trials into meta-analysis. J Clin Epidemiol, 3:255–265. 1992.

12.

Emslie

. Editorial: Understanding placebo response in pediatric depression trials. Am J Psychiatry, 166:1–3. 2009.

13.

Enck

, Benedetti

, Schedlowski

. New insights into the placebo and nocebo responses. Neuron, 59:195–206. 2008.

14.

Flament

, Cohen

. Childhood obsessive compulsive disorders. Obsessive-Compulsive Disorder: Evidence and Practice. Maj

, Sartorius

. Genève: Word Psychiatric Association, 147–183. 2000.

15.

Flament

, Cohen

, Choquet

, Jeammet

, Ledoux

. Phenomenology, psychosocial correlates and treatment seeking in major depression and dysthymia in adolescence. J Am Acad Child Adolesc Psychiatry, 40:1070–1078. 2001.

16.

Gelenberg

, Thase

, Meyer

, Goodwin

, Katz

, Kraemer

, Potter

, Shelton

, Fava

, Khan

, Trivedi

, Ninan

, Mann

, Bergeson

, Endicott

, Kocsis

, Leon

, Manji

, Rosenbaum

. The history and current state of antidepressant clinical trial design: A call to action for proof-of-concept studies. J Clin Psychiatry, 69:1513–1528. 2008.

17.

Geller

, Biederman

, Stewart

, Mullin

, Martin

, Spencer

, Faraone

. Which SSRI? A meta-analysis of pharmacotherapy trials in pediatric obsessive-compulsive disorder. Am J Psychiatry, 160:1919–1928. 2003.

18.

Goodyear

, Dubicka

, Wilkinson

, Kelvin

, Roberts

, Byford

, Breen

, Ford

, Barrett

, Leech

, Rothwell

, White

, Harrington

. Selective serotonin reuptake inhibitors (SSRIs) and routine specialist care with and without cognitive behaviour therapy in adolescents with major depression: Randomized controlled trial. Br Med J, 335:142. 2007.

19.

Hammad

, Laughren

, Racoosin

. Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychiatry, 63:332–339. 2006.

20.

Hetrick

, Merry

, McKenzie

, Sindahl

, Proctor

. Selective serotonin reuptake inhibitors (SSRIs) for depressive disorders in children, adolescents. Cochrane Database Syst Rev, 3:CD004851. 2007.

21.

Jaffee

, Moffitt

, Caspi

, Fombonne

, Poulton

, Martin

. Differences in early childhood risk factors for juvenile-onset and adult-onset depression. Arch Gen Psychiatry, 59:215–222. 2002.

22.

Jureidini

, Doecke

, Mansfield

, Haby

, Menkes

, Tonkin

. Efficacy and safety of antidepressants for children and adolescents. BMJ, 328:879–883. 2004.

23.

Kapczinski

, Lima

, Souza

, Schmitt

. Antidepressants for generalized anxiety disorder. Cochrane Database Syst Rev, 2:CD003592. 2003.

24.

Keck

Jr. , Welge

, McElroy

, Arnold

, Strakowski

. Placebo effect in randomized, controlled studies of acute bipolar mania and depression. Biol Psychiatry, 47:748–755. 2000.

25.

Kennard

, Silva

, Mayes

, Rhode

, Hugues

, Vitiello

, Kratochvil

, Curry

, Emslie

, Reinecke

, March

. Assessment of safety and long-term outcomes of initial treatment with placebo in TADS. Am J Psychiatry, 166:337–344. 2009.

26.

Khan

, Dager

, Cohen

, Avery

, Scherzo

, Dunner

. Chronicity of depressive episode in relation to antidepressant-placebo response. Neuropsychopharmacology, 4:125–130. 1991.

27.

Khan

, Detke

, Khan

et al. Placebo response and antidepressant clinical trial outcome. J Nerv Ment Dis, 191:211–218. 2003.

28.

Khan

, Schwartz

, Kolts

et al. Relationship between depression severity entry criteria and antidepressant clinical trial outcomes. Biol Psychiatry, 62:65–71. 2007.

29.

Kirsch

, Deacon

, Huedo-Medina

, Scoboria

, Moore

, Johnson

. Initial severity and antidepressant benefits: A meta-analysis of data submitted to the Food and Drug Administration. PLoS Med, 5,2:e45. 2008.

30.

Linde

, Witt

, Streng

, Weidenhammer

, Wagenpfeil

, Brinkhaus

, Willich

, Melchart

. The impact of pateint expectations on outcomes in four randomized controlled trials of acupuncture in patients chronic pain. Pain, 128:264–271. 2007.

31.

Macedo

, Banos

, Farre

. Placebo response in the prophylaxis of migraine: A meta-analysis. Eur J Pain, 12:68–75. 2008.

32.

Mayes

, Tao

, Rintelmann

, Carmody

, Hughes

, Kennard

, Stewart

, Emslie

. Do children and adolescents have differential response rates in placebo-controlled trials of fluoxetine? CNS Spectr, 12:147–154. 2007.

33.

Muratori

, Picchi

, Bruni

, Patarnello

, Romagnoli

. A two-year follow-up of psychodynamic psychotherapy for internalizing disorders in children. J Am Acad Child Adolesc Psychiatry, 42:331–339. 2003.

34.

Posternak

, Zimmerman

. Therapeutic effect of follow-up assessments on antidepressant and placebo response rates in antidepressant efficacy trials: Meta-analysis. Br J Psychiatry, 190:287–292. 2007.

35.

Rohde

, Silva

, Tonev

, Kennard

, Vitiello

, Kratochvil

, Reinecke

, Curry

, Simons

. March JSAchievement and maintenance of sustained response during TADS continuation and maintenance therapy. Arch Gen Psychiatry, 65:447–455. 2008.

36.

Ryan

. Treatment of depression in children and adolescents. The Lancet, 366:933–940. 2005.

37.

Stein

, Baldwin

, Dolberg

, Despiegel

, Bandelow

. Which factors predict placebo response in anxiety disorders and major depression? An analysis of placebo controlled studies of escitalopram. J Clin Psychiatry, 67:1741–1746. 2006.

38.

Stevens

, Wang

, Fan

, Edwards

, Campo

, Gardner

. Parental attitudes toward children's use of antidepressants and psychotherapy. J Child Adolesc Psychopharmacol, 19:289–296. 2009.

39.

Sysko

, Walsh

. A systematic review of placebo response in studies of bipolar mania. J Clin Psychiatry, 68:1213–1217. 2007.

40.

Trivedi

, Trush

. (1994) Does a placebo run-in or a placebo treatment cell affect the efficacy of antidepressant medications? Neuropsychopharmacology, 11:33–43. 1994.

41.

Walsh

, Seidman

, Sysko

, Gould

. Placebo response in studies of major depression: Variable, substantial, and growing. JAMA, 287:1840–1847. 2002.

42.

Whittington

, Kendall

, Fonagy

, Cottrell

, Cotgrove

, Boddington

. Selective serotonin reuptake inhibitors in childhood depression: Systematic review of published versus unpublished data. Lancet, 363:1341–1345. 2004.

43.

Wilcox

, Cohn

, Linden

, Heiser

, Lucas

, Morgan

, DeFrancisco

. Predictors of placebo response: A retrospective analysis. Psychopharmacol Bull, 28:157–162. 1992.