Abstract
Introduction
Medication is currently the most empirically supported and mainstay treatment for adults with ADHD (Asherson, Buitelaar, Faraone, & Rohde, 2016). However, not all patients can or desire to be medicated, and of those medicated, not all respond to medication (Wilens, Spencer, & Biederman, 2002) or persist with it long term (Darredeau, Barrett, Jardin, & Pihl, 2007; Olfson, Marcus, Zhang, & Wan, 2007). Furthermore, up to 50% of adult patients show less than 30% decrease in symptoms with medication (Castells et al., 2011; Hazell et al., 2011).
In addition to the core ADHD symptoms, many adults with ADHD experience marked impairments in key areas of daily functioning, such as occupational, social, and family functioning, as well as psychiatric comorbidity (Barkley, Murphy, & Firscher, 2008; Weiss & Hechtman, 1993). Most studies of medication effects have focused on symptom reduction, rather than change in impairments beyond the core symptoms (Adler et al., 2009; Buitelaar et al., 2009; Faraone, Spencer, Aleardi, Pagano, & Biederman, 2004; Medori et al., 2008; Peterson, McDonagh, & Fu, 2008). Recent evidence suggests that although medication does improve daily functioning in adults with ADHD (Buitelaar et al., 2012; Matza, Stoeckl, Shorr, & Johnston, 2006; Rösler et al., 2013; Rösler et al., 2010), medicated adults continue to suffer from significant impairments, particularly in work and interpersonal domains (Safren, Sprich, Cooper-Vince, Knouse, & Lerner, 2010). This highlights the need for additional interventions to improve functioning, as well as symptoms.
Cognitive behavioral interventions appear well suited to addressing ADHD-related impairments in adults. Behavioral skill training can assist daily functioning through developing effective compensatory strategies (e.g., organization, planning, time management strategies). Cognitive approaches can increase awareness of maladaptive patterns, as well as target dysfunctional thoughts, cognitions, and negative emotions that may result from repeated failures and contribute to maintenance of maladaptive behaviors (e.g., avoidance, procrastination). Studies of cognitive behavioral therapies (CBTs) in adult ADHD have yielded promising results. Several studies in samples of mostly medicated adults with ADHD have reported improvements following CBT, relative to either baseline or waitlist control; improvements were reported both in ADHD symptoms and in functional measures, such as executive functioning, self-esteem, comorbid symptoms, and global functioning (Bramham et al., 2009; Hesslinger et al., 2002; Rostain & Ramsay, 2006; Safren et al., 2005; Solanto, Marks, Mitchell, Wasserstein, & Kofman, 2008; Stevenson, Whitmont, Bornholt, Livesey, & Stevenson, 2002; Virta et al., 2008). More significantly, two large randomized controlled trials, one in medicated adults with ADHD (Safren et al., 2010), and the other in a mixed sample of medicated and unmedicated adults (Solanto et al., 2010), have demonstrated superiority of cognitive behavioral interventions to control treatments for addressing ADHD symptoms. Only one of these studies (Solanto et al., 2010) examined outcomes beyond core ADHD symptoms (i.e., comorbid depression and anxiety symptoms) but failed to find significant treatment effects.
Although there is now convincing evidence of CBT being an efficacious treatment for adult ADHD, this evidence comes primarily from studies of medicated or mostly medicated patient samples (Safren, Sprich, Mimiaga, et al., 2010; Solanto et al., 2010). In studies that included both medicated and unmedicated patients (Hesslinger et al., 2002; Solanto et al., 2008; Solanto et al., 2010; Stevenson et al., 2002; Virta et al., 2008), medication status was not randomly assigned, and the effects of medication could not be examined rigorously. To our knowledge, only one trial has looked at the effects of medication on a psychotherapy for adults with ADHD (Philipsen et al., 2015). This trial featured a four-group comparison between (a) a group therapy grounded in dialectical behavior therapy (DBT) and CBT approaches combined with methylphenidate, (b) clinical management (non-directive supportive counseling) combined with methylphenidate, (c) the group therapy combined with placebo, and (d) clinical management combined with placebo. Patients treated with methylphenidate showed greater reduction in ADHD symptoms and Clinical Global Impression scores than those treated with placebo; however, the group therapy was not superior to the clinical management, except for some Clinical Global Impression–related secondary outcomes. Thus, the effect of medication on CBT, which is the type of treatment that has been demonstrated to outperform non-specific comparator treatments in randomized clinical trials, still remains largely unclear. It therefore remains unknown whether CBT can be a viable standalone treatment for patients who cannot or do not wish to be medicated and how it compares with the combined treatment that is now supported by evidence. For example, it is possible that ADHD symptoms would interfere with CBT in unmedicated patients, rendering the treatment ineffective. Alternatively, symptom management with medication may not be critical for patients to derive benefits from CBT; it is possible that CBT itself may mitigate symptoms sufficiently to allow treatment engagement and further benefits.
The goal of the current study was to compare the effects of CBT alone versus CBT combined with stimulant medication (CBT + M) on core ADHD symptoms and functional outcomes. Because both medication and CBT have been shown to improve ADHD symptoms, we hypothesized that both CBT alone and CBT + M would reduce the core symptoms relative to baseline. Because there is some evidence for improvement in functional outcomes with both medication and CBT, we hypothesized that both CBT alone and CBT + M would improve organizational, psychological, and global functioning relative to baseline. However, we hypothesized that CBT + M would result in greater improvements in core ADHD symptoms and functional outcomes than CBT alone, reflecting benefits from both treatment approaches in the combined treatment. Finally, we expected that treatment benefits would be maintained over a 6-month follow-up.
Method
The data presented here are part of an un-blinded randomized controlled trial with block-randomized (blocks of 10 participants) 1:1 group allocation without stratification. Random group allocation sequence was generated by the principal investigator and was unknown to prospective participants. Group assignment was revealed to participants only after the trial coordinator had enrolled them in the trial and prior to baseline measures. The mean number of days between eligibility assessment and the start of treatment did not differ significantly between the two groups (CBT: 167.04 ±171.77; CBT + M: 223.5 ±271.99, p = .26). This long time period between eligibility assessment and treatment was due to the necessity to accumulate blocks of ≈10 eligible and willing participants to begin a group. Data collection was carried out at the McGill University Health Centre, Montreal, Canada. Study enrollment began on November 9, 2005, and ended on November 1, 2011; the date of last follow-up assessment was January 23, 2013. ADHD symptoms were designated as our primary outcome; functional measures were examined as secondary outcomes. Assuming a medium effect size for the group difference in our primary outcome (ADHD symptoms), we estimated (using G*Power) that a sample of 40 to 50 participants in each group would give us 80% power to detect a significant difference at the alpha level of .05.
Participants
Eighty-eight adults meeting the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) criteria for ADHD were block-randomized to either group CBT alone (n = 46) or group CBT + M (n = 42; see Figure 1 for flow of participants through trial). Participants were recruited through the adult ADHD research program of the Montreal Children’s Hospital, where they were referred from community health centers, outpatient psychiatry clinics, college and university health and mental health centers in the Montreal area, or were self-referred from the community. The study was approved by the research ethics board of the McGill University Health Centre, where it was conducted. Written informed consent was obtained by a trial coordinator, who first reviewed the consent form with the prospective participants and answered questions. Participants did not receive monetary compensation but were reimbursed for travel-related expenses upon request.

Flow of participants through trial.
Participants underwent a standard diagnostic procedure to ascertain ADHD, described in detail elsewhere (Cumyn, French, & Hechtman, 2009). Briefly, structured Conners’ Adult ADHD Diagnostic Interview for DSM-IV (CAARD-D)–Parts I and II (Epstein, Johnson, & Conners, 2001) were used to assess, respectively, (a) the developmental course the patient’s ADHD-related problems and associated risk factors, and (b) the presence of current and childhood DSM-IV ADHD symptoms, their age of onset, pervasiveness, and resulting impairment. CAARD-D was supplemented with self-report and collateral questionnaire ratings from an informant (who knew the participant well, such as a parent and/or a spouse). Participants and informants rated current ADHD symptoms via Barkley Current ADHD Symptoms Scale (Barkley & Murphy, 1998) and Conners’ Adult ADHD Rating Scale (CAARS; Conners, Erhardt, & Sparrow, 1999). They rated childhood ADHD symptoms via Barkley Childhood ADHD Symptoms Scale (Barkley & Murphy, 1998) and the Wender Utah Rating Scale (WURS; Ward, Wender, & Reimherr, 1993). In addition, Structured Clinical Interviews for DSM-IV Axis I and Axis II disorders were used to consider potential alternative causes of ADHD-type symptoms and to evaluate comorbidities. Evaluation also included a cognitive assessment, comprising an abbreviated version of either Wechsler Adult Intelligence Scale–Revised (WAIS-R; Reynolds, Willson, & Clark, 1983; n = 33) or Wechsler Adult Intelligence Scale–III (WAIS-III; Pilgrim, Meyers, Bayless, & Whetstone, 1999; n = 55), and portions of the Wechsler Memory Scale–III (Wechsler, 1997) and of the Wide Range Achievement Test–3 (Wilkinson, 1993). Assessments were conducted by trained professionals with a PhD in clinical psychology or a related discipline, and all diagnoses were confirmed by the senior author. Eligibility criteria were age between 18 and 60 years and the absence of (a) alcohol and substance dependence or abuse within the past 6 months, (b) history of psychosis, (c) illicit substance use or current psychotropic medication, (d) psychiatric comorbidities judged by the treatment team to require immediate treatment, (e) neurological illnesses, (f) organic disorders preventing effective participation in interventions (e.g., deafness, blindness), (g) estimated IQ <85, (h) medical conditions for which stimulants are contraindicated, and (i) pregnancy/breastfeeding. Participants were not undergoing any other concurrent psychological or psychiatric treatment during the study. Due to our exclusion of participants with serious comorbidities requiring urgent attention, this sample can be considered relatively non-comorbid and high-functioning.
CBT
Twelve 1.5-hr weekly CBT sessions were administered by a PhD-level therapist trained to deliver the treatment as per the manual; five different therapists administered the intervention over the course of the study (Therapist 1 for n = 9 participants; Therapist 2 for n = 5; Therapist 3 for n = 61; Therapist 4 for n = 5; Therapist 5 for n = 8). Each new therapist was trained by the previous therapist and by listening to audio-recordings of the previous therapists’ sessions. The mean group size was 6.92 ± 1.73 before and 6.50 ± 1.57 after attrition. Medicated and unmedicated participants were in separate groups.
Our trial was designed before other CBT protocols for ADHD appeared in the literature; therefore, we developed our own protocol to target key functional impairments in adults with ADHD, namely, long-term goal attainment, executive functioning (organization, time management, planning), self-esteem, psychological/emotional functioning, impulse control, and relationships. The program was aimed at acquisition and practice of adaptive cognitive and behavioral skills to foster improved functioning in these areas. This was approached through didactic material, in-group problem solving using the principles of collaborative empiricism, and homework aimed at implementation, practice, and generalization of the learned skills. Homework was emphasized to participants as a critical component of the program. Coaches (research assistants with at least a BA in psychology) were introduced at the start of the program as an important resource to work on homework and help practice, implement, and generalize the learned skills and strategies on an ongoing basis. Coaching was also meant to prevent and counteract difficulties participants may have completing their homework and practicing skills in-between the sessions due to ADHD-related deficits. Each participant received twice weekly telephone coaching and a weekly face-to-face coaching session (10-15 min each). Coaches were supervised weekly by the group therapists, including review of coaching transcripts. Group CBT sessions began with homework review and problem solving: The attempts at skill implementation by group members were discussed, and collaborative problem solving was used to arrive at optimal skill-implementation solutions for each participant and overcome any obstacles encountered during homework. This was followed by presentation and discussion of the didactic material on the session’s topic. The session ended with homework assignment.
The content of the CBT program included the following topics: (a) psychoeducation about ADHD, focusing on symptoms, functional impairment, diagnosis, developmental course, and treatment and relating this information to the participants’ personal experiences; (b) goal attainment, with a focus on implementation intentions (Gollwitzer & Brandstätter, 1997), strategies for self-motivation, setting deadlines, breaking down tasks into manageable components, measuring outcomes, and modifying and updating implementation plans; (c) organization and time management, focusing on strategies for prioritizing, scheduling, realistic time estimation, and planning, as well as on addressing common cognitive distortions and behaviors that can interfere with these processes (e.g., perfectionism, procrastination); (d) stress management, focusing on stress inoculation training (Michelbaum & Deffenbacher, 1988), effective coping, and using techniques for decreasing physiological arousal (e.g., breathing, relaxation); (e) cognitive distortions and restructuring, including learning about maladaptive cognitive schemas, automatic thoughts, common cognitive distortions, and strategies for identifying and challenging these distortions; (f) anger management, focusing on increasing awareness of one’s own anger triggers and underlying causes, learning to anticipate anger-provoking situations and identify anger arousal early on, and learning reflective (as opposed to impulsive) responding to provocation and adaptive means of anger expression, such as calm assertion; (g) impulse control, focusing on self-regulation strategies, such as observing and “riding-out” urges, stimulus control, pre-commitment strategies, and strategies for tolerating delays (e.g., distraction); (h) self-esteem, focusing on understanding the implications of ADHD-related functional limitations for self-esteem, and fostering realistic self-views that emphasize personal worth, while acknowledging certain limitations and embracing personal agency and responsibility; and (i) relationships, focusing on understanding of common relationship problems in ADHD and developing solutions to the specific relationship problems of the participants in the group relying on the Seven Principles outlined by Gottman (Gottman & Silver, 1999).
The didactic session content was determined by the standard set of slides for every session, which the therapist was to cover. To maximize treatment fidelity, if some portions of the material were not covered in a given session due to time constraints, the therapist was to cover that content in the following session. Participants were given copies of the slides to review as part of homework. In the event that a participant missed a session, he or she listened to an audio-recording of that session and met individually with the therapist or a coach to review the missed material. A random selection of 15 audio-recordings were rated by the senior author for treatment fidelity using checklists, which were developed as part of the manual and reflected each session’s content. These ratings showed 80% to 90% fidelity. Following 12 weeks of CBT, participants received 3 monthly booster sessions dedicated to review and troubleshooting. Telephone coaching continued throughout the booster phase on either a weekly or bi-weekly basis, depending on the participant’s choice.
Medication
Each participant was initially started on a low dose of a long acting methylphenidate (Biphentin or Concerta) or amphetamine medication (Adderall XR or Vyvanse). The dose was gradually increased on a weekly or bi-weekly basis depending on feedback of efficacy and side effects (via the CADDRA side effects rating scale, http://www.caddra.ca/pdfs/caddraGuidelines2011_Toolkit.pdf), as well as objective measurements of heart rate, blood pressure, and weight. The optimal dose was considered one where further increase in dose did not result in greater efficacy, while side effects were well tolerated. If the initial stimulant product proved ineffective or produced significant side effects, titration with a product from the other stimulant class was initiated. Of the patients who completed titration, 65.8% were on a methylphenidate product (38.12 ± 15.69 mg) and 34.2% were on an amphetamine product (20.19 ± 6.81 mg). Once the optimal dose was reached, participants were transferred to their community physicians for medication follow-up. They were to remain on the optimal dose of their medication throughout the CBT and the follow-up and to inform the study team of any issues/problems around medication, which were promptly addressed.
Assessment of Treatment Response
Assessments focused on core ADHD symptoms and functional domains targeted by CBT. They were conducted (a) at baseline, (b) post-medication optimization/ immediately prior to CBT for the CBT + M group—to assess the effect of medication, (c) after 12 weeks of CBT, (d) after 3-month follow-up with boosters and coaching, and (e) after another 3 months of follow-up without boosters or coaching. Each assessment included the following measures.
Current DSM-IV ADHD symptoms were evaluated via Barkley’s Current ADHD Symptoms Scale (CSS; Barkley & Murphy, 1998). The scale contains the 18 symptoms of ADHD from DSM-IV in the form of a 4-point scale (never or rarely, sometimes, often, very often). The scale has age- and sex-based norms and both self-report and observer-report forms.
Organizational skills were assessed via an eight-item Organization and Activation for Work (OAW) scale, with items presenting statements regarding aspects of organizational functioning (items endorsed on a 4-point scale) designed locally by LC and HG, see “Supplement.” The locally designed scale was used due to the absence of validated organizational skills measures for adults with ADHD at the time of study onset. Cronbach’s α was .85 in our sample.
Self-esteem was assessed via the Index of Self-Esteem (ISE). This global measure of self-esteem presents 25 statements that participants endorse on a 5-point scale (rarely or none of the time to most or all of the time) and has reliability ≥ .90 and good to excellent content, construct, concurrent, and factorial validity (Abell, Jones, & Hudson, 1984).
Symptoms of non-ADHD psychopathology were measured via the Beck Depression Inventory (BDI; Beck, 1987), the Beck Anxiety Inventory (BAI; Beck & Steer, 1993), and the Global Severity Index of the Symptom Check List 90 (SCL-90 GSI; Derogatis, 1986; Derogatis, 2000). BDI and BAI are widely used 21-item measures, with items endorsed on a 4-point scale. Both measures have good psychometric properties (Beck, Epstein, Brown, & Steer, 1988; Beck, Steer, & Carbin, 1988). SCL-90 is a 90-item scale designed to measure point-in-time psychological symptom status in psychiatric and medical patients along nine symptom dimensions (somatization, obsessive–compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism). Items are endorsed on a 4-point scale (none to extreme). The GSI is the average score across all items. SCL-90 has internal consistency in the range of .77 to .90 and test–retest reliability in the range of .68 to .80 over 10 weeks. The instrument has good validity as a measure of general symptom severity and changes in symptom severity and is well suited for measuring general mental health outcomes (Derogatis, 2000).
Global functional impairment from ADHD was assessed via the Sheehan Disability Scale. This scale measures the degree of functional impairment from psychiatric or medical symptoms and comprises three 10-point visual analogue scales: impairment at work/school, impairment in social life, and impairment in family life/home responsibilities (Sheehan, Harnett-Sheehan, & Raj, 1996). In adult ADHD patients, the scale has shown internal consistency in the range of .79 to .91, test–retest reliability of .72, and good construct validity and sensitivity to change (Coles, Coon, DeMuro, McLeod, & Gnanasakthy, 2014).
Anger expression was measured via the Anger Expression Index (AXI) of the State Trait Anger Expression Inventory–II (STAXI-II; Spielberger, 1999). STAXI-II is a 57-item multi-dimensional measure of anger expression/control; it is commonly used in anger treatment outcome research. Items are endorsed on a 4-point scale (not at all/almost never to very much so/almost always). STAXI-II scales have been demonstrated to be valid and internally consistent (α .73-.94; Spielberger, 1999). AXI provides an overall estimate of a person’s tendency to express anger.
The CSS was completed by the participant and a participant-selected informant who saw the participant on a regular basis (e.g., a spouse or a parent); the other measures were self-report only. The questionnaires were administered by the study coordinator. Changes in ADHD symptoms were the primary outcome measures; the other outcomes were secondary.
Data Analyses
The data were analyzed using SPSS Version 21 (IBM Corporation). CBT and CBT + M groups were compared on baseline characteristics using t tests and Mann–Whitney U tests for continuous variables and chi-square tests for categorical variables. Analyses looking at the effects of the two treatments on change in symptom and functional outcome scores over time were conducted using mixed linear models (MLM), specifying the assessment point as a repeated measure, treatment (CBT vs. CBT + M) as a fixed effect, Assessment point × Treatment interaction as a fixed effect, and cohort (treatment was administered in groups of 5 to 10) and therapist as random effects, and controlling for baseline severity on each outcome measure. MLM imputes missing data using the slope up to the point of discontinuation. If a significant Assessment point × Treatment interaction was observed, separate MLMs (same as above but excluding the Treatment and Assessment point × treatment terms) were run within each group, and Bonferroni-adjusted post hoc pairwise comparisons were used to assess the significance of change between assessment points.
Additional MLMs were run to examine possible moderating effects of baseline functioning, sex, age, estimated IQ, years of education, comorbidity, session attendance, and coaching participation on outcomes by testing the significance of the additional Assessment point × Treatment × Moderator interaction terms. If an interaction term was non-significant at the p value of .05, the potential moderator was removed from the model and no longer considered in the analyses.
The significance threshold for the analyses of ADHD symptoms (primary outcomes) was set at .05. The significance threshold for secondary outcome analyses was set at .007 (Bonferroni-corrected to keep the family-wise error rate at .05, considering seven secondary outcomes). Secondary outcome findings with p values > .007 and < .05 are reported as trends. Because scores on the Sheehan Disability Scale, BDI, BAI, and SCL-90 GSI deviated substantially from normality, Box–Cox transformations were applied before analysis.
Results
Participants
The CBT and CBT + M groups did not differ on any demographic variables except male to female ratio, which was significantly higher in the CBT + M group (35/7) than in the CBT only group (27/19; Table 1). In terms of clinical characteristics, participants in both groups had ADHD of moderate severity, but depression, anxiety, and global psychopathology symptoms scores in the non-clinical range. However, the CBT-only group had higher baseline anxiety symptoms than the CBT + M group. The groups did not differ on any other baseline measures.
Sample Demographics and Baseline Scores.
Note. CBT = cognitive behavioral therapy; OAW = organization and activation for work; ISE = Index of Self-Esteem; BDI = Beck Depression Inventory; BAI = Beck Anxiety Inventory; SCL-90 = Symptom Check List–90.
Treatment Participation
The mean proportion of CBT sessions attended did not differ significantly between the two groups (CBT: 81.05 ± 21.55%; CBT + M: 77.98 ± 27.1%, p = .56). Participants in the CBT group attended on average a higher proportion of booster sessions than CBT + M group during the 3 months following CBT (CBT: 39.13 ± 35.34%; CBT + M: 24.60 ± 31.28%), t(86) = 2.03, p = .05. Coaching participation, quantified as the number of coaching contacts, did not differ between the groups in either the treatment phase (CBT: 13.70 ± 6.40; CBT + M: 14.76 ± 7.65), t(86) = 0.71, p = .48), or the booster phase (CBT: 6.17 ± 6.19; CBT + M: 7.02 ± 6.81, t(86) = 0.61, p = .54). Cumulative attrition rates were as follows: CBT: 8.70%, CBT + M: 19.05% after CBT; CBT: 17.39%, CBT + M: 28.57% after 3-month follow-up; CBT: 26.09%, CBT + M: 38.09% after 6-month follow-up (see Figure 1 for n remaining). Treatment attrition did not differ significantly between the CBT versus CBT + M groups after CBT, 3-month follow-up, or 6-month follow-up (ps ≥ .22). Patients retained did not differ from those lost during treatment on any baseline scores (ps ≥ .06), except for lower self-esteem in those lost (64.58 ± 22.07) than in those retained (39.16 ± 14.66), t(86) = 5.18; p < .0005. In addition, those lost during follow-up were younger (29.65 ± 8.18) than those retained (36.66 ± 11.11), t(86) = 2.90; p = .002. Characteristics of dropouts did not differ between the two groups (ps ≥ .08).
One CBT + M participant did not complete titration, and two stopped medication during follow-up, not finding it helpful. Five CBT-only participants requested to initiate medication during follow-up; although group allocation was to be maintained throughout follow-up, these patients’ requests were granted in keeping with best clinical practices. Analyses were conducted both including and excluding the follow-up data from these participants. Data exclusion did not affect the findings, so we report the analyses including all the data.
CBT Alone Versus CBT With Medication
A significant Assessment point × Treatment type interaction, F(3, 194.55) = 6.73, p < .0005, revealed greater improvement from baseline in self-reported ADHD symptoms with CBT + M than with CBT alone, which was evident following treatment (b = 6.98; 95% confidence interval [CI] = [3.89, 10.06], t(198.18) = 4.46, p < .0005, d = .99); after 3-month follow-up (b = 4.56; CI = [.71, 8.41], t(266.56) = 2.33, p = .02, d = .64); and after 6-month follow-up (b = 4.49; CI = [.26, 8.73], t(272.08) = 2.09, p = .04, d = .65) (Figure 2, Table 2). Post hoc pairwise comparisons of symptom scores between assessment time points based on separate MLMs within each group showed that in the CBT only group, self-reported symptoms decreased significantly following CBT (p < .0005), decreased further after the first 3 months of follow-up (p = .004), and showed no significant change (improvement or detriment) after the subsequent 3 months (p = .48). In the CBT + M group, symptoms decreased significantly following medication optimization (p < .0005) and decreased further following CBT (p = .003), with no subsequent change over the 6-month follow-up (ps ≥ .66). Thus, relative to the CBT-only group, CBT + M group showed greater and more rapid improvement in self-reported symptoms.

CBT versus CBT + medication.
Statistics for the Mixed Linear Models Revealing Differential Treatment Effects of CBT Versus CBT + M.
Note. Means and SDs are estimated marginal means and error estimates from the mixed linear models; Cohen’s ds are calculated using these estimates. Higher scores represent greater symptom levels and poorer functioning. For the CSS, the score of 24 is a clinical cutoff, representing 1.5 SD above the mean of non-ADHD participants within the same age bracket. Scores below this cutoff can be considered in the non-clinical range. b = estimated parameter for treatment effect; 90% CI = 95% confidence interval of the parameter; d = Cohen’s d for the difference between treatments for each time point from the model; CSS = Current ADHD Symptoms Scale; CBT = cognitive behavioral therapy; OAW = organization and activation for work; ISE = Index of Self-Esteem.
Significant change over successive evaluation points assessed by post hoc pairwise comparisons.
For observer-rated ADHD symptoms, there was also an Assessment point × Treatment interaction, F(3, 159.72) = 3.62, p = .02, with CBT + M resulting in greater improvement than CBT alone following CBT (b = 5.14; CI = [1.25, 9.03], t(155.98) = 2.06, p = .01, d = .64), and after 3-month follow-up (b = 4.70; CI = [.00, 9.39], t(208.54) =1.97, p = .05, d = .56), but not after 6-month follow-up (p = .61, d = −.14). In the CBT-only group, observers rated symptoms as significantly reduced after CBT (p < .0005) with no further significant changes over the 6-month follow-up (ps > .05). In the CBT + M group, observers rated symptoms as significantly reduced after medication optimization (p < .0005) with no further changes following CBT or over the 6-month follow-up (ps ≥ .11). Thus, according to observer ratings, CBT + M effects were superior to those of CBT alone following treatment, but this was no longer the case after 6-month follow-up.
For organizational skills, a significant Assessment point × Treatment interaction, F(3, 203.79) = 5.91, p = .001, showed that CBT + M produced greater improvement than CBT alone in organizational skills, which was significant following CBT (b = 4.20; CI = [2.23, 6.17], t(207.16) = 4.21, p < .0005, d = 1.02); marginally significant (after Bonferroni correction for the number of secondary outcomes) at 3-month follow-up (b = 2.86; CI = [0.02, 5.26], t(207.16) = 4.34, p < .02, d = .69); and non-significant at 6-month follow-up (p = .10, d = .53). CBT-only group showed a significant improvement in organizational skills following CBT (p < .0005) with no further significant change over the 6-month follow-up (ps ≥ .23). CBT + M group showed a significant improvement in organizational skills following medication optimization (p < .0005) and a further improvement following CBT (p = .008), with no further significant change over the 6-month follow-up (ps > 1.0).
For self-esteem, there was likewise an Evaluation point × Treatment type interaction, F(3, 200.54) = 4.04, p = .008, which did not reach the corrected significance threshold when considering the number of secondary outcomes. CBT + M produced a significantly greater improvement in self-esteem than CBT alone following CBT (b = 6.72; CI = [2.76, 10.69], t(204.68) = 3.34, p = .001, d = .67), which was marginally significant after 3-month follow-up (b = 4.97; CI = [-0.01, 9.95], t(265.05) = 1.97, p = .05, d = 0.48), and also marginally significant after 6-month follow-up (b = 6.02; CI = [0.59, 11.45], t(268.91) = 2.18, p = .03, d = 0.61). In the CBT-only group, there was a gradual improvement in self-esteem, with only marginal improvement evident following CBT (p = .07), which became significant at the 3-month follow-up point (p = .001), with no further significant change. In the CBT + M group, there was an immediate improvement in self-esteem after medication optimization (p = .005), with a further improvement following CBT (p = .01) and no further change (ps ≥ .84).
Efficacy of CBT + M versus CBT alone did not differ significantly for the Sheehan Disability Scale, BDI, BAI, SCL-90 GSI, or AXI measures (ps ≥ .19). The assessment point terms in these models were significant, with improvements relative to baseline evident following CBT both with and without medication on the Sheehan Disability Scale, SCL-90 GSI, and AXI measures (ps ≤ .002), and marginal improvements observed on the BDI and the BAI (ps ≤ .05 and > .0007, trends after Bonferroni correction). In sum, there were either significant or trend-level improvements from baseline on the above measures, which did not differ as a function of treatment type.
Moderation of Treatment Effects
Significant Assessment point × Treatment × Baseline score interactions suggested moderating effects of baseline scores for observer-rated ADHD symptoms, F(3,152.78) = 3.50, p = .02, and self-esteem, F(3,198.44) = 5.07, p = .002. For observer-rated ADHD symptoms, baseline severity was a stronger predictor for post-treatment improvement in the CBT + M than in the CBT-only group (b = 0.53; CI = [0.17, 0.90], t(151.04) = 2.88, p = .005; CBT + M: r(29) = .71, p < .0005; CBT: r(32) = .29, p < .10). Similarly, lower self-esteem at baseline was a stronger predictor of self-esteem improvement with CBT + M than with CBT alone post treatment (b = 0.44; CI [0.18, 0.69], t(224.08) = 3.42, p = .001; CBT + M: r(33) = .60, p < .0005; CBT: r(33) = .21, p = .21); and at 6-month follow-up (b = 0.41; CI [0.07, 0.75], t(259.91) = 2.39, p = .02; CBT + M: r(28) = .67, p < .0005; CBT: r(28) = .43, p = .01).
Estimated IQ moderated the effect of treatment type on self-reported ADHD symptom improvement regardless of assessment point, Treatment × IQ interaction: F(1, 101.17) = 6.87, p = .01. In the CBT-only group, higher IQ predicted greater symptom improvement, controlling for baseline severity, r(39) = .42, p = .006, whereas in the CBT + M group this association was not significant, r(30) = −.20, p = .23.
ADHD subtype moderated the effect of treatment on self-esteem improvement at all assessment points, Time × Treatment × Subtype interaction: F(3, 198.32) = 3.60, p = .01; post treatment: b = −10.22, CI = [−17.91, −2.52], t(201.49) = −2.62, p = .01; 3 months: b = −12.23, CI = [−21.74, −2.72], t(262.44) = −2.53, p = .01; 6 months: b = −16.15, CI = [−26.51, −5.80], t(271.54) = −3.07, p = .002. Participants with the combined subtype showed significantly greater self-esteem gains with CBT + M than with CBT alone, controlling for baseline self-esteem, post treatment: F(3, 17) = 8.67, p = .008; at 3-month follow-up: F(3, 17) = 4.35, p = .05; and at 6-month follow-up: F(3, 17) = 6.75, p = .02; for predominantly inattentive participants, self-esteem gains did not differ as a function of treatment type (ps ≥ .54). There were no other significant moderators of relative treatment efficacy (p ≥ .07).
None of the treatment participation variables (sessions attended, booster sessions attended, or number of coaching contacts) moderated group differences in treatment outcomes (ps ≥ .17 for interactions terms for treatment and assessment point with participation variables). This is despite the fact that participants in the CBT group attended on average a higher proportion of booster sessions than CBT + M group participants during the 3 months following CBT (CBT: 39.13 ± 35.34; CBT + M: 24.60 ± 31.28, t(86) = 2.03, p = .05).
Discussion
We found that our manualized group CBT, both with and without medication, resulted in significant improvements relative to baseline in ADHD symptoms and functional outcomes. As hypothesized, CBT + M was superior to CBT alone in reducing self- and observer-reported ADHD symptoms. However, for the observer-rated symptom scores, treatment effects were no longer different at the 6-month follow-up assessment point. For both self- and observer-rated symptom scores, the improvements were more immediate and pronounced in the CBT + M group, while the CBT-only group improved more gradually.
A similar pattern emerged for functional outcomes. CBT + M produced greater improvements than CBT alone in organizational skills and self-esteem. Assessments after medication optimization suggest that medication alone produced significant gains in both these areas (Table 2), which were then augmented by CBT. To our knowledge, improvements in organizational skills and self-esteem in adults with ADHD as a result of medication have not been reported previously. Again, the CBT + M group showed more immediate improvements, while the CBT-only group showed more gradual improvements, rendering the initial treatment differences trend-level or non-significant for these functional outcomes.
For the CBT-only participants, we saw evidence of continued improvement over follow-up in terms of ADHD symptoms and self-esteem. Given that participants continued receiving telephone coaching and booster sessions during the follow-up, it is unsurprising that the CBT-only participants continued improving. Ongoing coaching in particular could have played a role, as it may have offered participants an additional opportunity to work on their problem areas after the end of active treatment. This additional practice with the help of the coaches may have resulted in greater proficiency in applying the learned skills and strategies in daily life to improve functioning. It is unclear why we did not see a similar continued improvement in the CBT + M group after the end of CBT. Notably, we did consider booster attendance and coaching participation as possible moderators or group differences in outcomes, and they did not emerge as significant outcome moderators at any assessment point (although CBT only participants attended more boosters than did CBT + M participants). A remaining explanation is that CBT + M participants had reached their full improvement potential (or a level of functioning they found satisfactory) by the end of CBT, making further improvement unlikely. CBT-only participants, however, may not have yet achieved their maximal improvement by the end of active treatment and may therefore have used the follow-up period to make further gains, perhaps through continued individual practice with the coaches.
We found no differences between CBT and CBT + M in terms of their effects on global functional impairment, depression and anxiety symptoms, global psychopathology, and anger expression. Our data do not speak directly to why CBT + M did not produce superior outcomes in these areas. However, it should be noted that symptom scores for depression, anxiety, and global psychopathology were in the non-clinical range and anger expression scores were in the normal range (mean t scores slightly exceeding 50) for both groups before treatment (Table 1). Thus, there was limited room for improvement in these areas, and the lack of treatment differences may have been due to a ceiling effect. The findings may have been different in a sample with higher levels of comorbidity. However, lack treatment difference for global functional impairment (Sheehan disability scale) cannot be attributed to ceiling effects.
Our findings are generally in agreement with the study by Philipsen et al. (2015), which showed higher efficacy of group therapy combined with medication relative to that of group therapy combined with placebo. However, unlike Philipsen et al., we observed the superiority of CBT + M diminishing over follow-up. The major difference in design was our lack of a placebo control for the CBT-only group, but this is unlikely to have caused this discrepancy. We would expect larger improvements with CBT if it were combined with a medication placebo—due to placebo effects—and hence smaller differences between treatments; if placebo effects were to diminish over time, we would expect treatment differences to become larger rather than smaller during follow-up. As mentioned earlier, ongoing telephone coaching during follow-up may have facilitated continued improvement in our CBT-only group. In the Philipsen et al. trial, there was no coaching, although participants continued to receive monthly booster sessions. The effects of coaching as an adjunct to CBT in adults with ADHD merit further investigation.
Aside from the coaching, which is the distinguishing feature of our CBT protocol from the others investigated thus far, there are other notable differences between the CBT protocol we used here and the protocols used by other major published studies. The Philipsen et al. trial used a group therapy protocol based on DBT with elements of CBT. Although this protocol was similar to ours in some ways, such as dealing with organization, impulse control, stress management, and self-respect, it also differed substantially from ours in that it emphasized mindfulness as an intervention for achieving improved attentional control, and used behavioral analysis and DBT strategies for emotion regulation. In addition, it had modules devoted specifically to comorbid depression and substance use problems (Philipsen et al., 2007). Thus, our program could be described as having proportionally more emphasis on strategies for remediating functional deficits, whereas the Philipsen et al.’s program focused more on self-regulation, including emotional regulation. Probably the most similar CBT-like group protocol to ours is the meta-cognitive therapy by Solanto et al. (2010). This protocol shares much of our content and has similar emphasis on executive strategies. The main difference from our protocol is the structure of the program, with the skills introduced in the first half of the program and the second half being devoted to integration and implementation of these skills. Our program did not feature any sessions devoted specifically to integration and implementation; rather, coaching was used for simultaneous practice, implementation, and cumulative integration. The CBT protocol by Safren et al. (2010) was also quite similar to ours in terms of content; however, this was an individual therapy.
Moderation analyses suggested greater benefits of CBT + M than CBT alone for ADHD symptoms and self-esteem for patients with poorer baseline scores. Conversely, higher IQ predicted greater improvement in ADHD symptoms with CBT alone but did not predict improvement with CBT + M. Our exclusion of participants with estimated IQ <85 ensured that patients were cognitively capable of engaging in CBT. Yet, those with higher IQs may have been more capable of relying solely on compensatory CBT strategies, reducing the need for medication to target their core symptoms directly. Finally, ADHD subtype moderated treatment effects on self-esteem. Those with the combined subtype benefited more from CBT + M than CBT alone, possibly because stimulants directly target impulsivity, which can have high personal, interpersonal, and social costs.
The main limitation of this work is the absence of blinded independent evaluations of treatment outcomes. However, the two previous large studies of CBT in medicated patients, as well as the recent four-group trial, reported effects of very similar magnitude for independent ratings and self-report ratings of ADHD symptoms (Philipsen et al., 2015; Safren, Sprich, Mimiaga, et al., 2010; Solanto et al., 2010). Also, although not blinded to treatment assignment, observer ratings corroborated and paralleled self-report findings. This is particularly reassuring given that young adults with ADHD underestimate their symptoms, and hence, collateral report is considered a more reliable source of information (Barkley, Fischer, Smallish, & Fletcher, 2002; Barkley et al., 2008).
There were also significant differences between the CBT-alone and CBT + M groups in sex ratio and baseline anxiety symptoms. However, these differences are unlikely to have biased our conclusions, as our analyses controlled for baseline symptoms, and sex did not emerge as a moderator of any outcomes. Variation in therapist (one therapist ran the majority of the groups, and four others ran one group each) is also unlikely to have affected the results, as the therapist random effect term was not significant in any of the models.
Another notable limitation is the lack of a “sham treatment” control group. However, having such a group would mean that some patients would not receive any active treatment for 9 months. Our institutional ethics board strongly favors comparator trials and does not typically allow controls that offer no empirically supported treatment, if such treatments exist. Having a control group that receives sham group therapy combined with medication could be one ethically acceptable alternative, but this design introduces a challenge of parsing out the effects of sham group therapy from those of medication, especially if the two interact. We also felt that waitlist would be a poor control for group CBT, as CBT comprises many non-specific and potentially beneficial elements beyond expectation of therapeutic benefit. It should be noted that in both CBT-only and CBT + M groups, there was a fairly lengthy waiting period between the initial eligibility assessment and start of treatment. This waiting period was similar to a waitlist control, although its length varied between participants. Symptom scores on CSS were virtually identical for both groups at the time of the assessment versus treatment baseline (CBT [assessment]: 32.24 ± 7.63; CBT [baseline]: 31.39 ± 7.80; CBT + M [assessment]: 32.00 ± 8.57; CBT + M [baseline]: 31.62 ± 7.56, ps ≥ .50). In addition, although some of the benefits seen following treatment relative to baseline may be due to positive expectancy/placebo effects, the absence of a “sham treatment” group does not bear on the conclusions regarding the relative efficacy of CBT alone versus CBT + M, which was the primary research question. Nonetheless, we believe that there is a need for a randomized clinical trial with placebo and comparator psychological treatment controls to examine the effects of medication on CBT delivered via a protocol with proven efficacy.
Finally, generalizability of our findings is somewhat limited by our relatively high-functioning sample without severe comorbidities or substance abuse. Including participants whose comorbid conditions require immediate attention could entail either (a) having them engage in concurrent treatments for their comorbidities, which would make it difficult to discern the effects of our studied interventions, or (b) not referring them to treatment for their comorbidities, which would be ethically and clinically unsound. An effectiveness trial of our CBT intervention as a standalone treatment and in combination with medication in the community would help determine how suitable this intervention is for the general population of adults with ADHD, which includes highly comorbid individuals.
In conclusion, CBT combined with medication attained greater and more rapid improvement in ADHD symptoms, organizational skills, and self-esteem than CBT alone. However, a similar degree of symptom and functional improvement can be reached with CBT alone (given in our format, with ongoing regular coaching) more gradually. Thus, CBT without medication may be a viable option for those who do not wish to take medication or experience prohibitive side effects, especially those with higher cognitive functioning.
Footnotes
Authors’ Note
Outside the submitted work, L. Hechtman has received research support, served on advisory boards, and has been a speaker for Ely Lilly, GlaxoSmithKline, Ortho Janssen, Purdue, and Shire. Other authors have nothing to disclose.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was partially funded through the Montreal Children’s Hospital Foundation via the Shire Endowment Fund.
