Telepsychiatrists' Medication Treatment Strategies in the Children's Attention-Deficit/Hyperactivity Disorder Telemental Health Treatment Study

Abstract

Objective:

The purpose of this study was to examine the prescribing strategies that telepsychiatrists used to provide pharmacologic treatment in the Children's Attention-Deficit/Hyperactivity Disorder (ADHD) Telemental Health Treatment Study (CATTS).

Methods:

CATTS was a randomized controlled trial that demonstrated the superiority of a telehealth service delivery model for the treatment of ADHD with combined pharmacotherapy and behavior training (n=111), compared with management in primary care augmented with a telepsychiatry consultation (n=112). A diagnosis of ADHD was established with the Computerized Diagnostic Interview Schedule for Children (CDISC), and comorbidity for oppositional defiant disorder (ODD) and anxiety disorders (AD) was established using the CDISC and the Child Behavior Checklist. Telepsychiatrists used the Texas Children's Medication Algorithm Project (TCMAP) for ADHD to guide pharmacotherapy and the treat-to-target model to encourage their assertive medication management to a predetermined goal of 50% reduction in ADHD-related symptoms. We assessed whether telepsychiatrists' decision making about making medication changes was associated with baseline ADHD symptom severity, comorbidity, and attainment of the treat-to-target goal.

Results:

Telepsychiatrists showed high fidelity (91%) to their chosen algorithms in medication management. At the end of the trial, the CATTS intervention showed 46.0% attainment of the treat-to-target goal compared with 13.6% for the augmented primary care condition, and significantly greater attainment of the goal by comorbidity status for the ADHD with one and ADHD with two comorbidities groups. Telepsychiatrists' were more likely to decide to make medication adjustments for youth with higher baseline ADHD severity and the presence of disorders comorbid with ADHD. Multiple mixed methods regression analyses controlling for baseline ADHD severity and comorbidity status indicated that the telepsychiatrists also based their decision making session to session on attainment of the treat-to-target goal.

Conclusions:

Telepsychiatry is an effective service delivery model for providing pharmacotherapy for ADHD, and the CATTS telepsychiatrists showed high fidelity to evidence-based protocols.

Introduction

Telehealth is an emerging service delivery model to help rectify disparities in access to evidence-based mental healthcare for children and adolescents who are not well served by traditional models of care. The Health Resources and Services Administration (HRSA) (Health Resources and Services Administration 2015) broadly defines telehealth as “The use of electronic information and telecommunications technologies to support and promote long-distance clinical health care, patient and professional health-related education, public health, and health administration.” When telehealth relies on synchronous (interactive), real-time technologies, such as videoconferencing, to deliver medical care directly to patients, the Center for Medicare and Medicaid (CMS) (http://www.cms.gov/Telemedicine/) uses the term “telemedicine,” and when that care involves psychiatric services, the term “telepsychiatry” is generally used. Pharmacotherapy is one of the most frequently requested telepsychiatry services (Hilty et al. 2006; Lau et al. 2011); however, there are no studies describing child and adolescent psychiatrists' pharmacologic management during telepsychiatry practice, such as their adherence to guideline-based care and decision making in medication adjustments, and whether such expert treatment delivered through videoconferencing produces better outcomes than treatment provided in primary care. The current study describes child and adolescent psychiatrists' approach to pharmacological treatment delivered through telepsychiatry.

Data are drawn from the recently completed Children's Attention-Deficit/Hyperactivity Disorder (ADHD) Telemental Health Treatment Study (CATTS). CATTS was a 5 year, community-based, randomized controlled trial (RCT) funded by the NIMH to test the effectiveness of a telehealth service delivery model in improving the mental healthcare and outcomes for children with ADHD (Myers et al. 2013; Myers et al. 2015; Vander Stoep and Myers 2013). Participants resided in non-metropolitan communities that are underserved in their access to the child psychiatric workforce (Thomas and Holzer 2006).

Based on experience in our telepsychiatry clinic (Myers et al. 2010), the increasing expectations for primary care providers (PCPs) to manage uncomplicated cases of ADHD (Epstein et al. 2010; American Academy of Pediatrics 2011), and the high rate of comorbid oppositional defiant and anxiety disorders (AD) in community studies of children with ADHD (MTA Cooperative Group 1999b; Jensen et al. 2001; Chronis et al. 2004), we anticipated that PCPs would preferentially refer children with ADHD and comorbid disorders to the trial. As the guidelines for ADHD with comorbid disorders recommend combination treatment with pharmacotherapy and behavioral interventions (American Academy of Child and Adolescent Psychiatry 2007; American Academy of Pediatrics 2011), the CATTS trial delivered a short-term combined intervention. Children's treatment was then returned to the care of their referring PCPs with recommendations for ongoing care.

Children randomized to the CATTS combined intervention received six sessions, spaced 3–4 weeks apart over 22 weeks, consisting of pharmacotherapy delivered through telepsychiatry by child and adolescent psychiatrists located at an academic center, followed immediately by caregiver behavior training delivered in person by a local therapist who was trained and supervised remotely by the study's research psychologist (McCarty et al. 2014; Myers et al. 2015). Participants randomized to the comparison condition received a telepsychiatry consultation session followed by management in primary care (augmented primary care).

Results of the trial indicated that participants randomized to the CATTS combined intervention showed greater reductions in caregiver-reported ADHD symptoms, oppositional defiant disorder (ODD) behaviors and impairment, and increased role performance than children randomized to the augmented primary care comparison condition (Myers et al. 2015). The design of the CATTS trial precludes the ability to parse the relative contributions of each intervention component to these outcomes. However, ADHD symptoms appear to respond preferentially to pharmacotherapy (MTA Cooperative Group 1999a; Jensen et al. 2001; Connor et al. 2010) by addressing central neuropsychiatric deficits (Schulz et al., 2012; Czerniak et al. 2013; Wang et al. 2013), whereas psychosocial interventions appear more geared to promoting associated prosocial behaviors (Daley et al. 2014; Evans et al. 2014), we describe telepsychiatrists' strategies used to contribute to the reduction in ADHD symptoms.

The current study addressed three hypotheses regarding telepsychiatrists' strategies used to treat children randomized to the CATTS combined intervention condition. First, we hypothesized that telepsychiatrists would provide guideline-based care, as was evidenced by 90% fidelity to treatment guidelines. Second, we hypothesized that the CATTS intervention condition would be more likely than the augmented primary care condition to achieve 50% reduction in ADHD symptoms across comorbidity status. Third, we hypothesized that telepsychiatrists' decision making regarding medication adjustments during treatment sessions would be predicated on characteristics of children's disorder and response to treatment at each session, including: 1) More severe ADHD symptoms at baseline, 2) one or more comorbid disorders diagnosed at baseline, and 3) <50% reduction in ADHD symptoms from the prior session.

Methods

The CATTS trial was approved by the Institutional Review Board of Seattle Children's Research Institute and monitored by a data safety and monitoring board. The methodology (Myers et al. 2013; Vander Stoep and Myers 2013) and results (Myers et al. 2013; Myers et al. 2015) for the full CATTS trial have been described elsewhere. The current study describes the medication strategies that telepsychiatrists used to treat participants randomized to the CATTS intervention.

Study setting, referral source, and sample

Eighty-eight PCPs in seven underserved communities in Washington and Oregon successfully referred 223 boys and girls, 5.5–12 years of age to the trial: 111 participants were randomized to the CATTS combined intervention condition and 112 were randomized to the augmented primary care comparison condition. Telepsychiatrists were located at an academic center and participants were located at community health centers that were 120–450 km from the academic center (Myers et al. 2013) and had preexisting videoconferencing capability with high bandwidth (≥386 kb/sec) connectivity. Children were eligible if they met diagnostic criteria for ADHD, were 5.5–12.9 years of age, resided with English-speaking legal guardians (caregivers), and attended school (Myers et al. 2013; Vander Stoep and Myers 2013). Exclusion criteria included lack of involvement of the caregivers, or the child being a ward of the state or having medical, developmental or psychiatric disorders that required interventions outside the scope of the study.

Diagnostic determination and enrolment

Eligibility for the CATTS trial was determined in a three step process: 1) Medical records were reviewed to determine exclusionary conditions, 2) caregivers then completed the Child Behavior Checklist (CBCL) (Achenbach and Rescorla 2001) online, and 3) caregivers of children with t score ≥65 on the CBCL's ADHD diagnostic subscale met in person with a local CATTS therapist who administered informed consent and three modules of the Computerized Diagnostic Interview Schedule for Children (CDISC) (Shaffer et al. 1996) to confirm a diagnosis of ADHD and to determine the presence of selected comorbid conditions. Consistent with prior approaches (Jensen et al. 2001; Lewczyk et al. 2003; Rockhill et al. 2013) AD were considered present if endorsed to criteria on the CDISC-IV modules for generalized AD or if the Anxiety Problem Subscale on the CBCL showed a t score ≥70. A similar approach was used to determine the presence of ODD. We created three comorbidity groups: Youth with 1) ADHD alone, 2) ADHD with one comorbidity (ODD or AD: “ADHD plus 1”), and 3) ADHD with two comorbidities (ODD and AD: “ADHD plus 2”). Child participants were then administered assent. Caregivers completed a baseline assessment. Child participants, along with one caregiver, were then randomly assigned within site and age blocks (5.5–9 years old and 10–12 years old) to the CATTS intervention (n=111) or to the augmented primary care comparison condition (n=112).

Telepsychiatry intervention strategies

At each medication session, the telepsychiatrist used two strategies in medication decision-making: 1) Guideline-based pharmacotherapy using the Texas Children's Medication Algorithm Project (TCMAP) and session-specific intervention checklists; and 2) adaptation of the treat-to-target model to titrate medication dosage to a targeted level of improvement (Katon et al. 2009; Lin et al. 2012; Solomon et al. 2014).

First strategy: Guidelines of care

The TCMAP algorithms for ADHD provide a consensus-based approach to medication management. They also provided investigators a mechanism to track and measure telepsychiatrists' adherence to the prescribing protocol. The TCMAP for ADHD includes five algorithms: 1) ADHD alone; and ADHD with comorbidities of 2) anxiety, 3) depression, 4) tics, or 5) aggression (Pliszka et al. 2003, 2006; Wagner et al. 2014). These consensus-based algorithms were developed to be used flexibly in conjunction with clinical judgment, and to include additional medications as new agents become available. We added guanfacine (Intuniv^®) and clonidine hydrochloride (Kapvay^®) in algorithm 1 (ADHD without comorbidity) at the same stage of inclusion as atomoxetine, and in algorithm 5 (ADHD with aggression) stage 2 in place of behavioral treatment, as all participants in the CATTS intervention received caregiver behavior training. Algorithms 2–5 offer the option to start treatment with the same steps as algorithm1 before considering the comorbid condition. Therefore, starting with algorithm 1 is appropriate for many youth with comorbid disorders, and the telepsychiatrists were encouraged to optimize treatment for the referral condition, ADHD, before considering pharmacologic treatment of comorbidities.

We included the algorithms as part of a web-based decision-making and tracking tool termed WebCATTS (Vander Stoep and Myers 2013). Telepsychiatrists decided the algorithm and the stage within the algorithm at which to start treatment. They could change their algorithm choice during the course of treatment. For example, the telepsychiatrist may initiate treatment with algorithm 1 for a child with ADHD and anxiety focusing on the referral issue of ADHD, and then change to algorithm 3 if pharmacologic treatment of anxiety was indicated during the course of treatment. Telepsychiatrists documented all decisions in WebCATTS. If they decided to deviate from the algorithms, they documented a “protocol deviation” and justification for the deviation in WebCATTS. All prescriptions were documented in WebCATTS and automatically recorded in the hospital's electronic medical record upon writing a prescription. One of the investigators then audited WebCATTS and the medical records to determine telepsychiatrists' adherence to their selected algorithm.

Telepsychiatrists were provided a checklist outlining the essential treatment components and psychoeducation to address during each session. All sessions were recorded. Two telepsychiatry sessions per family were randomly selected to rate the telepsychiatrist’ coverage of these essential components.

Second strategy: Treat-to-target model

We adapted the treat-to-targetmodel (Katon et al. 2009; Lin et al. 2012; Solomon et al. 2014) to encourage telepsychiatrists to assertively adjust medications to achieve a predetermined goal of 50% reduction in total ADHD symptoms. The treat-to-target model was developed to address the suboptimal treatment of chronic medical conditions in primary care, and has been adapted to care of patients with combined medical and psychiatric disorders. The model advocates close monitoring of treatment response and assertive medical interventions and medication adjustments to reach an indicated level of improvement in patients' symptoms (Von Korff and Tiemens 2000; Ngo et al. 2013; Solomon et al. 2014). The treat-to-target model has an evolving evidence-based in mental health care settings, but has not been described with youth.

Following the approach of Lin and colleagues (Lin et al. 2012), we coded six categories of medication adjustments that telepsychiatrists made during the CATTS trial: 1) Changing the daily dose of a current medication, 2) increasing the number of medication classes prescribed (e.g., adding a nonstimulant while continuing a stimulant), 3) switching to a medication in the same class (e.g., switching from one methylphenidate product to another such as from Ritalin LA® to Concerta®), 4) switching to a medication in a different class (e.g., switching from methylphenidate to an amphetamine or a nonstimulant), 5) adding a medication in the same class (e.g., adding an after-school dose of methylphenidate, or a mixed amphetamine salt, or guanfacine), and 6) stopping a medication because of side effects, without replacement. One of the authors coded whether any of these categories of changes occurred during each of the six intervention sessions. Each change was coded dichotomously as 0 (no) or 1 (yes). Mean number of medication changes throughout the six sessions was the average number of total medication changes across all six categories of medication changes over all sessions attended. A second author audited 10% of the ratings. Agreement between the coders was excellent (κ=0.94).

To implement the treat-to-target model, we examined telepsychiatrists' decision making regarding the number and timing of medication adjustments in relation to a preset outcome goal of 50% reduction in total ADHD symptoms by the final (sixth) session. We chose this goal for two reasons. First, there is no consistent definition of remission of ADHD. Investigations have typically assessed outcomes according to symptom reduction on a standardized rating scale that indicates no more than “mildly symptomatic,” often in conjunction with a global rating of impairment to assist in determining “caseness” (Steele et al. 2006). Second, a consensus group that met at the European College of Neuropsychopharmacology International meeting in 2003 proposed a 30% reduction of ADHD as an indicator of reaching an optimal medication dose (Buitelaar et al. 2003). The MTA study determined a “best dose” for each child during a 30 day dose titration (Greenhill et al. 1996) based on parent and teacher ratings of inattention and hyperactivity/ impulsivity subscales on a standardized ADHD rating scale (Swanson 1992), and their primary outcome after 14 months of intervention was an ∼50% decrease in inattention and hyperactivity-impulsivity symptoms in the medication only and medication plus behavioral therapy groups (MTA Cooperative Group 1999a). As we did not deem a month-long dose titration to be feasible in a novel telehealth trial, and we wanted to ensure that telepsychiatrists assertively treated children's ADHD symptoms per the treat-to-target model, we chose the more rigorous goal of 50% reduction in total ADHD symptoms.

To help the telepsychiatrists to determine whether ADHD symptoms had reached the treat-to-target goal, caregivers completed a rating scale at each telepsychiatry session. We modified the Inattention/Overactivity With Aggression (IOWA) Conners' Teacher Rating Scale (Loney and Milich 1982) by adding five items (3 inattention and 2 hyperactivity items) to yield a final 15 item Caregiver ADHD Treatment Target Summary (CATTS) Rating Scale that provided equal coverage of the cognitive symptoms of ADHD, the motor symptoms of ADHD, and ODD behaviors. If ADHD symptoms on the CATTS Rating Scale had decreased at least 50%, telepsychiatrists used clinical judgment regarding further medication adjustments. If ADHD symptoms had decreased <50%, there was “room for improvement” and telepsychiatrists were advised to adjust medication toward the goal of further symptom reduction.

Telepsychiatrists partnered with caregivers in decision making at each session. A partnered approach aimed to facilitate rapport building in delivering care remotely through telehealth (Glueck 2013), to engage caregivers in understanding the role of medication in addressing the neurobiological deficits in ADHD, and to explain the rationale for assertive medication management. We graphically displayed caregivers' responses on the CATTS Rating Scale to provide a visual stimulus regarding children's progress. Caregivers also completed the Clinical Global Impressions (CGI) of Improvement Scale modified to caregivers' report Guy 1976). Telepsychiatrists discussed with the caregivers the scores on the two scales, any discrepancy between their ratings, and the goal of medication adjustment to reach the treat-to-target goal of 50% reduction in ADHD symptomatology on the CATTS Rating Scale. They then discussed treatment options to reach this goal.

Study measures

The primary outcome measure testing the effectiveness of the CATTS service delivery model was caregiver-rated improvement in ADHD-related behaviors (http://clinicaltrials.gov/show/NTC0083 0700) assessed with the Vanderbilt ADHD Parent Rating Scale (VADPRS) (Jellinek et al. 2002). VADPRS items are based on the criteria for ADHD specified in the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (American Psychiatric Association 1994). The VADPRS assesses inattention (9 items); hyperactivity/impulsivity (9 items); oppositional and defiant behaviors (8 items); and role performance per academic, classroom, and interpersonal functioning (8 items) and shows solid psychometric properties (Wolraich et al. 2003). In the CATTS trial, the concurrent validity of the item total of the VADPRS and the ADHD symptom scores on the CDISC-IV was high (r=0.79). Caregivers completed the VADPRS at baseline, and 4, 10, 19, and 25 weeks postrandomization. Responses were entered remotely from personal computers into the outcomes database (Geyer et al. 2011). Overall, caregivers completed a mean of 4.8±0.69 of the five assessments; 86.5% of caregivers in the intervention arm and 90.2% in the control arm completed all five of the assessments, whereas 91% and 94.7%, respectively, completed at least four assessments. The VADPRS data were used to test hypothesis 2; that is, whether telepsychiatrists were more likely than PCPs to attain the treat-to-target goal of 50% reduction in ADHD-related symptoms.

To test hypothesis 3, we assessed the association between change in caregivers' ratings on the CATTS Rating Scale with telepsychiatrists' decision making regarding medication adjustment.

Analytic plan

To test hypothesis 1, we evaluated the telepsychiatrists' adherence to their intervention protocol in two ways. To determine whether the telepsychiatrists achieved 90% adherence to all treatment components specified on the session checklists, we coded recordings of two telepsychiatry sessions per family, and reported fidelity as the average percent of completed components. We used a treatment reviewer to determine the correspondence between the telepsychiatrists' indicated TCMAP algorithm recorded in WebCATTS and medication prescribed recorded in the electronic health record. We reported percentage agreement.

To test hypothesis 2, we calculated the cumulative 50% reduction in total VADPRS scores for the CATTS intervention (n=111) and the augmented primary care (n=112) groups at each of the five assessments. We tested the differences between treatment conditions at the end of the trial using χ² analysis to compare VADPRS scores for the total sample and then by the three comorbidity groups: ADHD alone without comorbid disorder, ADHD and one comorbid disorder (ODD or AD), and ADHD and two comorbid disorders (ODD and AD).

To address hypothesis 3a, we defined baseline severity of ADHD symptoms as quartiles for the CATTS Rating Scale scores. We then used mixed-model regression analyses to compare the four ADHD severity groups on the mean number of total medication changes across the six sessions, controlling for age group and gender. We used mixed-model regression analyses to test hypothesis 3b, that comorbidity status would predict the total number of medication changes after accounting for clustering by telepsychiatrists. To test hypothesis 3c, we calculated the percent reduction in the CATTS Rating Scale scores at sessions two through six. We used logistic multiple regressions controlling for baseline CATTS Rating Scale scores and comorbidity status to estimate the odds ratios (OR) and 95% confidence intervals (95% CI) for the association between failure to achieve the target 50% reduction in cumulative CATTS Rating Scale scores and telepsychiatrists' decision to change medication at each of sessions two through five.

Results

Characteristics of participants in the RCT

Table 1 summarizes the demographic characteristics of the CATTS sample at baseline by treatment condition. The sample consisted of boys and girls with mean age of 9.25 (±2.0) years, and was predominantly Caucasian, non-Hispanic. The sample was well balanced across the two treatment conditions. Table 2 shows the clinical characteristics of the sample at baseline by treatment condition. Comorbidity was prevalent. Seventy-five percent (168/223) of participants had a comorbid disorder, either ODD or AD, or both comorbidities. The occurrence of two comorbid disorders was higher in the CATTS intervention condition. ADHD symptom severity was comparable across the two treatment conditions.

Table 1.

Demographic Characteristics of the CATTS Sample at Baseline

Demographic variables	CATTS combined intervention(n=111)	Augmented primary care(n=112)
Child age mean (SD)	9.2 (2.0)	9.3 (2.0)
Child gender (%)
Male	76 (68.5%)	87 (77.7%)
Female	35 (31.5%)	25 (22.3%)
Child race (%)
White	104 (93.7%)	100 (89.3%)
Black	1 (0.9%)	1 (0.9%)
Asian	0 (0%)	2 (1.8%)
NH/PI	0 (0%)	4 (3.6%)
AI/AN	4 (3.6%)	2 (1.8%)
Other	2 (1.8%)	3 (2.7%)
Child ethnicity (%)
Hispanic	10 (9.0%)	19 (17.0%)
Non-Hispanic	101 (91.0%)	93 (83.0%)
Parents' marital status
Married/Cohabitating	76 (68.5%)	83 (74.8%)
Other	35 (31.5%)	28 (25.2%)
Primary caregiver
Biological/Step-parent	92 (82.9)	99 (88.4)
Adoptive parent/Other	19 (17.1)	13 (11.6)
Household income
<$35k	41 (36.9%)	36 (32.1%)
$35k-75k	28 (25.3%)	42 (37.5%)
$75k-100k	22 (19.8%)	13 (11.6%)
>$100k	20 (18.0%)	20 (17.9%)
Unknown		1 (0.9%)

CATTS, Children's ADHD Telemental Health Treatment Study; NH/PI, Native Hawaiian/Pacific Islander; AI/AN, American Indian/Alaska Native.

Table 2.

Clinical Characteristics of the CATTS Sample at Baseline

	CATTS intervention group (n=111)	Active control group (n=112)
Comorbid symptoms	Number and percent of children by comorbidity status	Number and percent of children by comorbidity status
ADHD alone	20 (18.0)	35 (31.2)
ADHD+ODD behaviors	44 (39.7)	49 (43.8)
ADHD+AD symptoms	7 (6.3)	6 (5.4)
ADHD+ODD+AD ^**	40 (36.0)	22 (19.6)

Vanderbilt ADHD Parent Rating Scale	VADPRS mean (SD)	VADPRS mean (SD)
Combined ADHD	2.19 (0.56)	2.09 (0.66)
Inattention	2.33 (0.50)	2.27 (0.51)
Hyperactivity/Impulsivity	2.05 (0.61)	1.91 (0.71)
ODD	1.68 (0.69)	1.51 (0.70)

p<0.01

CATTS, Children's Attention-Deficit Hyperactivity Disorder Telemental Health Treatment Study; ADHD, attention-deficit/hyperactivity disorder; ODD, oppositional defiant disorder; AD, anxiety disorder; VADRS, Vanderbilt ADHD Rating Scales.

Telepsychiatrists' pharmacotherapy in the CATTS intervention

Participants in the CATTS intervention group (n=111) attended an average of 5.2 (range 0–6) telepsychiatry sessions. Attendance did not differ by comorbidity group: 4.8 for ADHD alone, 5.5 for ADHD plus one comorbidity, and 5.1 for ADHD plus two comorbidities (ANOVA, df=2, F=1.39, p=0.25).

Telepsychiatrists' fidelity to intervention protocols

Telepsychiatrists completed the essential steps for each session with 91.6% (±9.5%) fidelity across sessions and telepsychiatrists. The telepsychiatrists' indicated that TCMAP algorithm and stage within the algorithm agreed with the prescribed medication for 98.2% of sessions. Among the 108 subjects who completed two or more intervention sessions, the telepsychiatrists used algorithm 1 (ADHD alone) most frequently, regardless of comorbidity status: 77.8% (n=14/18) for youth with ADHD alone; 57.8% (n=52/90) for youth with ADHD and any comorbidity. Algorithms 2–5 were used for 22.2% (n=4/18) of youth with ADHD alone and 42.2% (n=38/90) for youth with a comorbidity. Comparison of whether algorithms 2–5 were used more often for youth with a comorbidity versus youth with ADHD alone approached statistical significance (χ²=2.52, df=1, Fisher exact test p=0.06).

Of the 574 telepsychiatry sessions conducted during the trial, 29 protocol violations were identified upon audit, indicating that the telepsychiatrists did not prescribe according to their chosen algorithm. Only one of these protocol deviations was not documented and justified by the telepsychiatrist. Protocol deviations included: Algorithm order (n=13), combining agents (n=7), not increasing medication because of parent preference (n=5), using a medication not in the algorithm (n=3), and continuation of medications started by PCP (n=1). Overall, these results support hypothesis 1, that telepsychiatrists adhered to guideline-based care.

The telepsychiatrists' specific medication changes are described in Table 3. The most common medication adjustments were starting a methylphenidate product and combining a stimulant and a nonstimulant. Combining medications was common. At entry to the study, PCPs were prescribing two or more psychiatric medications for14 participants (13.0%) and at the end of the study, telepsychiatrists prescribed two or more medications for 44 (40.7%) of the participants. Of these, 30 (27.8%) were prescribed two ADHD medications, 12 (11.1%) were prescribed one or more ADHD medication plus an antidepressant or an antipsychotic, and 2 participants (1.9%) received a stimulant plus a nonstimulant plus antidepressant.

Table 3.

Medications for the CATTS Intervention Group Prior to and at the End of Study

Type of medication	Prior to study enrollment	End of study
No medication	35	3
Methylphenidate alone	31	41
Amphetamine alone	22	14
Atomoxetine alone	3	3
Alpha-agonist alone	3	3
Stimulant+nonstimulant	8	30
Stimulant+SSRI	2	3
Stimulant+antipsychotic	3	6
Nonstimulant+SSRI	0	1
Nonstimulant+antipsychotic	1	2
Stimulant+nonstimulant+SSRI	0	2
Total	108	108

Sample size indicates participants in the CATTS combined intervention with two or more sessions.

CATTS, Children's ADHD Telemental Health Treatment Study; SSRI, selective serotonin reuptake inhibitor.

Attainment of the treat-to-target goal in the randomized trial

Figure 1 a–d presents the cumulative attainment during the RCT of 50% reduction in the VADPRS scores for the two treatment conditions, stratified by comorbidity group. Overall, 46.0% of participants in the CATTS intervention versus 13.6% of those in the augmented primary care condition achieved the treat-to-target goal of 50% ADHD symptom reduction by the end of the trial (see Fig. 1a) (χ²=25.08; df=1; p=0.000). Comparable results were found for participants in the ADHD plus one comorbidity group (χ²=17.56; ; df=1; p=0.000) and the ADHD plus two comorbidity group (χ²=4.54; df=1; p=0.033), but participants in the ADHD alone group showed less separation between the two conditions (χ²=3.33, df=1; p=0.07). It is of note that the CATTS intervention condition continued to demonstrate goal attainment throughout the trial, whereas the augmented primary care condition never exceeded 20% reduction in VADPRS scores. These findings support the second hypothesis, that the significantly better outcomes for the CATTS intervention condition in the randomized trial reflected greater attainment of the treat-to-target goal across comorbidity groups.

FIG. 1.

(a) Percentage of participants achieving 50% reduction in attention-deficit/hyperactivity disorder (ADHD) symptoms by comorbidity and treatment condition: All ADHD participants. (b) Percentage of participants achieving 50% reduction in ADHD symptoms by comorbidity status and treatment condition: ADHD alone groups. (c) Percentage of participants achieving 50% reduction in ADHD symptoms by comorbidity and treatment condition: ADHD with one comorbidity. (d) Percentage of participants achieving 50% reduction in ADHD symptoms by comorbidity and treatment condition: ADHD with two comorbidities.

Telepsychiatrists' decision making to meet the treat-to-target goal (decision making per baseline ADHD severity)

Telepsychiatrists made successively more medication changes for youth with higher quartile CATTS Rating Scale scores at baseline, controlling for sex and age group. The baseline quartile of CATTS Rating Scale severity scores were significantly associated with a greater number of medication adjustments controlling for sex and age group (first quartile reference group; second quartile: Coefficient=0.76; 95% CI=0.32, 1.219, p<0.001; third quartile: Coefficient=1.44; 95% CI=0.64, 2.23; p<0.001; fourth quartile: Coefficient=2.07; 95% CI=1.17, 2.97; p<0.001). Compared with the lowest quartile, participants in the second quartile received 0.76 more medication changes, participants in the third quartile received 1.44 more medication changes, and participants in the fourth quartile received 2.07 more medication changes.

Telepsychiatrists' decision making to meet the treat-to-target goal (decision making per comorbidity status)

Figure 2 shows the categories of medication adjustments stratified by comorbidity group.

FIG. 2.

Medication changes made for the Children's Attention-Deficit/Hyperactivity Disorder Telemental Health Treatment Study (CATTS) intervention participants, stratified by type of change and comorbidity group.

Telepsychiatrists' assertiveness in medication management varied with the number of comorbid conditions. Telepsychiatrists made 1.25 more medication changes for participants with ADHD plus two comorbidities than for participants with ADHD alone (coefficient=1.25; 95% CI=0.81, 1.69; p<0.001). There was no significant difference in the number of medication changes made for the ADHD plus one comorbidity group compared with the ADHD alone group (coefficient=0.51; 95% CI=−0.43, 1.45; p=0.29).

Telepsychiatrists' decision making to meet the treat-to-target goal (decision-making per symptom reduction)

Table 4 shows the number and percentage of participants at each intervention session who did not demonstrate at least a 50% reduction in the CATTS Rating Scale scores, and for whom telepsychiatrists made a medication adjustment. As the sample sizes in each sequential cell are small, and power calculation is low at <40%, these findings are presented for descriptive purposes. Multiple regressions controlling for baseline CATTS Rating Scale scores and comorbidity group indicated that telepsychiatrists were significantly likely to make a medication change for participants who had not achieved 50% improvement in CATTS Rating Scale scores at session two (OR=1.06; 95% CI=1.00, 1.12; p=0.05); session three (OR=1.06; 95% CI=1.00, 1.11; p=0.04); and session four (OR=1.06; 95% CI=1.00, 1.12; p=0.05). This association was not significant at session five (OR=1.00; 95% CI=0.95, 1.06; p=0.96) and was not tested at session six, as participants were preparing to transition back to their PCPs, and medication change was not encouraged.

Table 4.

Telepsychiatrists' Medication Adjustments for Participants Who Have Not Attained the “Treat to Target” Goal of 50% Reduction in ADHD Symptoms

	Percentage of subjects without 50% improvement based on CATTS Rating Scale who received a medication change
ADHD subtype	Session 2^ % (*n)	Session 3^ % (*n)	Session 4^ % (*n)	Session 5 % (n)	Session 6 % (n)
ADHD alone (n=17)	68.8 (11)	23.1 (3)	28.6 (2)	33.3 (3)	10.0 (1)
ADHD+1 comorbidity (n=48)	64.4 (29)	58.3 (21)	43.3 (13)	26.9 (7)	12.5 (3)
ADHD+2 comorbidities (n=34)	90.6 (29)	70.8 (17)	60.0 (12)	66.7 (16)	14.3 (3)
All subgroups (n=99)	74.2 (69)	56.2 (41)	47.4 (27)	44.1 (26)	12.7 (7)

p<0.05 for comparison among the three groups at each session.

CATTS, Children's Attention-Deficit Hyperactivity Disorder Telemental Health Treatment Study; ADHD, attention-deficit/hyperactivity disorder; ADHD+1, ADHD with a comorbid oppositional defiant disorder behavior or anxiety disorder symptoms; ADHD+2, ADHD with two comorbidities; i.e., oppositional defiant disorder behaviors and anxiety disorder symptoms.

Overall, these findings support our third hypothesis that telepsychiatrists' assertiveness in decision making was associated with baseline ADHD severity, the presence of comorbidity, and the occurrence of <50% reduction in ADHD severity scores.

Discussion

The CATTS trial demonstrated that a brief telehealth service delivery model with combined pharmacotherapy and caregiver behavior training improved ADHD outcomes compared with treatment in an augmented primary care model (Myers et al. 2013; Myers et al. 2015). The present study describes telepsychiatrists' decision-making strategies for pharmacotherapy using guidelines of care to ensure adherence to intervention protocols and a novel treat-to-target model to provide evidence-based care.

The CATTS trial is one of few studies to use the TCMAP algorithms to guide medication decision making (Pliszka et al. 2006; Wagner et al. 2014). The high concordance of the chosen algorithm and stage within the algorithm with medications prescribed, the few protocol violations, and their documentation indicate that telepsychiatrists can adhere to guideline-based care and, more broadly, that the algorithms can be used to guide and track psychiatrists' decision making in practice. These findings and the telepsychiatrists' adherence to the treatment session guidelines support our first hypothesis.

In the CATTS randomized trial, the greater attainment of the treat-to-target goal of 50% reduction in ADHD symptoms across comorbidity groups by the CATTS combined intervention condition supports our second hypothesis and demonstrates the “added value” to youth and their PCPs of providing a short term telepsychiatry intervention by expert child and adolescent psychiatrists, compared with a telepsychiatry consultation model. Participants in the augmented primary care condition never surpassed a 20% reduction in ADHD symptoms and these gains plateaued or decreased by week 19. These findings are consistent with prior studies suggesting inadequate management of chronic conditions in primary care (Lin et al. 2012), and support use of the treat-to-target model for pharmacotherapy of youth with ADHD. The lower attainment of the target goal of 50% reduction in ADHD symptoms for the ADHD only group compared with the ADHD plusone comorbidity group was an unanticipated, and unintuitive, finding, and not caused by a difference in severity of ADHD symptoms (Rockhill et al. 2013). Anecdotally, the telepsychiatrists reported that engaging caregivers as partners in decision making may have curtailed their assertiveness in medication management. Many of the children had several medication trials prior to enrollment in the study, and some caregivers wanted to curtail medication adjustments to focus on the caregiver training component of the intervention. Future work should focus on how the presence of comorbidity may influence telepsychiatrists' and caregivers' thinking about medication use.

Finally, only 46% of participants in the CATTS combined intervention group attained the treat-to-target goal of 50% reduction in ADHD-related symptoms, and telepsychiatrists' assertiveness in using the treat-to-target model was evident early and midtrial, but diminished toward the end of the trial, at session five. Perhaps 50% symptom reduction was an overly ambitious goal, given the prior recommendation that 30% reduction is clinically relevant (Buitelaar et al. 2003) and that the MTA study's attained 50% reduction after 14 months of intensive treatment and titration to an optimal individual dose (The MTA Cooperative Group, 1999) compared with the 25 weeks of the CATTS trial. It would be helpful to know whether impending referral back to referring PCPs or the telepsychiatry modality influenced decisions regarding prescribing practices over time.

The telepsychiatrists' most common medication changes were dosage adjustment and the addition of a second agent, consistent with the recent literature suggesting that polypharmacy for the treatment of ADHD is becoming common (Kreider et al. 2014) and is effective (Wilens et al. 2009; Cutler et al. 2014; Gadow et al. 2014; Hirota et al. 2014). Although further research is needed to clarify telepsychiatrists' decision making in treating ADHD with comorbid disorders, the current findings indicate that telepsychiatrists use medication consistent with usual clinical practice.

Limitations

There are several limitations to the current study. Referring PCPs and families may represent a selected group that was amenable to treatment through telehealth and/or to participation in research. The study is a secondary data analysis that focused on the CATTS intervention group and is, therefore, largely descriptive. We did not have comparable information about the prescribing practices of the PCPs in the augmented primary care condition to compare specific prescribing practices. The study design did not allow an evaluation of pharmacotherapy effects apart from the caregiver behavior training. Therefore, the current study provides an initial approach to examining telepsychiatry practice and serves as a stimulus to further study.

Conclusions

This study demonstrates the use of TCMAP algorithms to guide medication decision making (Pliszka et al. 2006; Wagner et al. 2014), and shows that the telepsychiatrists demonstrated high concordance of the chosen algorithm and stage within the algorithm with medications prescribed. We advocate for the use of such algorithms to guide and track psychiatrists' decision making in practice. The telepsychiatrists were successful in improving symptoms by prescribing medications FDA-approved for use in ADHD, most commonly by adjusting dosage or with the addition of a second FDA-approved agent. Use of a short-term direct psychiatry treatment model, in which the telepsychiatrist assumes medication prescribing for several months in combination with short-term, evidence-based caregiver behavior training, shows significant benefit in reduction in ADHD symptoms compared to a telepsychiatry consultation model in which the telepsychiatrist provides recommendations to the PCP.

Clinical Significance

The current study contributes new information to the emerging literature on providing treatment to youth through telepsychiatry. Telepsychiatrists can deliver guideline-based care to youth that brings “added value” over treatment in primary care, and offers support to PCPs' efforts to manage children with ADHD in their practices. Our modified “treat-to-target” model demonstrates how telepsychiatrists can implement evidence-based care and guide their decision making in medication adjustments. Further work should build on this approach to educate caregivers on the goal of pharmacotherapy, and to closely monitor children's treatment response over time. Our findings indicate that telepsychiatry is an effective service delivery model to provide evidence-based pharmacologic treatment to children with ADHD who do not have usual access to expert psychiatric care.

Footnotes

Acknowledgments

We thank Sarah Grover for her work on an earlier version of this article, which was presented as a poster at the American Academy of Child and Adolescent Psychiatry 58th annual meeting in Toronto in October 2011. We acknowledge the contributions of the CATTS team, including Gina Kim and Heather Violette, and the children, families, and PCPs who participated in the CATTS clinical trial. The Children's Attention-Deficit/Hyperactivity Disorder Telemental Health Treatment Study (CATTS) trial is registered with Clinical Trials: http://clinicaltrials.gov/show/NCT00830700.

Disclosures

Dr. Rockhill is a subinvestigator on an NIMH-funded study of the effectiveness of extended release guanfacine in patients with autism and ADHD symptoms, which can be considered a conceptual conflict of interest. The other authors have nothing to disclose.

References

Achenbach

, Rescorla

: Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2001.

American Academy of Child and Adolescent Psychiatry: Practice parameter for the assessment and treatment of children and adolescents with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry, 46:894–921, 2007.

American Academy of Pediatrics, Subcommittee on Attention-Deficit/Hyperactivity Disorder and Steering Committee on Quality Improvement and Management: ADHD: Clinical practice guideline for diagnosis, evaluation, and treatment of attention-deficit/hyperactivity disorder in children and adolescents. Pediatrics, 128:1007–1022, 2011.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 4th ed.: Washington, DC: American Psychiatric Association; 1994.

Buitelaar

, Montgomery

, van Zwieten–Boot

: Attention deficit hyperactivity disorder: Guidelines for investigating efficacy of pharmacological intervention. Eur Neuropsychopharmacol, 13:297–304, 2003.

Chronis

, Fabiano

, Gnagy

, Onyango

, Pelham

, Lopez–Williams

, Chacko

, Wymbs

, Coles

, Seymour

: An evaluation of the summer treatment program for children with attention-deficit/hyperactivity disorder using a treatment withdrawal design. Behav Ther, 35:561–585, 2004.

Connor

, Steeber

, McBurnett

: A review of attention-deficit/hyperactivity disorder complicated by symptoms of oppositional defiant disorder or conduct disorder. J Dev Behav Pediatr, 31:427–40, 2010.

Cutler

, Brams

, Bukstein

, Mattingly

, McBurnett

, White

, Rubin

: Response / remission with guanfacine extended-release and psychostimulants in children and adolescents with attention-deficit/hyperactivity disorder. J Am Acad of Child Adolesc Psychiatry, 53:1092–1101.

Czerniak

, Sikoglu

, King

, Kennedy

, Mick

, Frazier

, Moore

: Areas of the brain modulated by single-dose methylphenidate treatment in youth with ADHD during task-based fMRI: A systematic review. Harv Rev Psychiatry, 21:151–162, 2013.

10.

Daley

, van der Oord

, Ferrin

, Danchaerts

, Doepfner

, Cortese

, Sonuga–Barke

EJS

, the European ADHD Guidelines Group. Behavioral interventions in attention-deficit/hyperactivity disorder: a meta-analysis of randomized controlled trials across multiple outcome domains. J Am Acad Child Adolesc Psychiatry, 53:835–847, 2014.

11.

Evans

, Owens

, Bunford

. Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. J Clin Child Adolesc Psychol 43: 527–551, 2014.

12.

Epstein

, Langberg

, Lichtenstein

, Altaye

, Brinkman

, House

, Stark

: Attention-deficit/hyperactivity disorder outcomes for children treated in community-based pediatric settings. Arch Pediatr Adolesc Med, 164:160–165, 2010.

13.

Gadow

, Arnold

, Molina

, Findling

, Bukstein

, Brown

, McNamara

, Rundberg–Rivera

, Li

, Kipp

, Schneider

, Farmer

, Baker

, Sprafkin

, Rice

, Bangalore

, Butter

, Buchan–Page

, Hurt

, Austin

, Grondhuis

, Ama

: Risperidone added to parent training and stimulant medication: effects on attention-deficit/hyperactivity disorder, oppositional defiant disorder, conduct disorder, and peer aggression. J Am Acad Child Adolesc Psychiatry, 53:948–959, 2014.

14.

Geyer

, Myers

, Vander Stoep

, McCarty

, Palmer

and DeSalvo

: Implementing a low-cost web-based clinical trial management system for community studies: A case study. Clin Trials, 8:634–644, 2011.

15.

Glueck

: Establishing therapeutic rapport in telemental health. In: Telemental Health: Clinical, Technical, and Administrative Foundations for Evidence-based Practice. Edited by Myers

, Turvey

C.L.

Waltham, MA: eElsevier Insight. 2013. Pp 29–46.

16.

Greenhill

, Abikoff

, Arnold

, Cantwell

, Conners

, Elliott

, Hechtman

, Hinshaw

, Hoza

, Jensen

, March

, Newcorn

, Pelham

, Severe

, Swanson

, Vitiello

, Wells

: Medication treatment strategies in the MTA study: Relevance to clinicians and researchers. J Am Acad Child Adolesc Psychiatry, 34:1304–1313, 1996.

17.

Guy

: ECDEU Assessment Manual for Psychopharmacology, 2nd ed. Rockville, MD: United States National Institute of Health, Psychopharmacology Research Branch; 1976.

18.

Hirota

, Schwartz

, Correll

: Alpha-2 agonists for attention-deficit/hyperactivity disorder in youth: a systematic review and meta-analysis of monotherapy and add-on trials to stimulant therapy. J Am Acad Child Adolesc Psychiatry, 53:153–73, 2014.

19.

Health Resources and Services Administration, Rural Health, United States Department of Health and Human Services: Glossary and Acronyms, 2015. Available at http://www.hrsa.gov/ruralhealth/about/telehealth/glossary.html. Accessed November 15. 2014 .

20.

Hilty

, Yellowlees

, Nesbitt

: Evolution of telepsychiatry to rural sites: Changes over time in types of referral and in primary care providers' knowledge, skills and satisfaction. Gen Hosp Psychiatry, 28:367–373, 2006.

21.

Jellinek

, Patel

, Froehle

: Bright Futures in Practice: Mental Health—Volume II. Tool Kit. Arlington, VA: National Center for Education in Maternal and Child Health; 2002.

22.

Jensen

, Hinshaw

, Kraemer

, Lenora

, Newcorn

, Abikoff

, March

, Arnold

, Cantwell

, Conners

, Elliott

, Greenhill

, Hechtman

, Hoza

, Pelham

, Severe

, Swanson

, Wells

, Wigal

, Vitiello

: ADHD comorbidity findings from the MTA study: comparing comorbid subgroups. J Am Acad Child Adolesc Psychiatry, 40:147–158, 2001.

23.

Katon

, Russo

, Lin

, Heckbert

, Karter

, Williams

, Ciechanowski

, Ludman

, Von Korff

. Diabetes and poor disease control: Is comorbid depression associated with poor medication adherence or lack of treatment intensification?. Psychosom Med, 71:965–972, 2009.

24.

Kreider

, Matone

, Bellonci

, Feudtner

, Huang

, Localio

, Rubin

: Growth in the concurrent use of antipsychotics with other psychotropic medications in Medicaid-enrolled children. J Am Acad Child Adolesc Psychiatry, 53:960–970, 2014.

25.

Lau

, Way

, Fremont

: Assessment of Suny Upstate Medical University's child telepsychiatry consultation program. Int J Psychiatry Med, 42:93–104, 2011.

26.

Lewczyk

, Garland

, Hurlburt

, Gearity

, Hough

: Comparing DISC-IV and clinician diagnoses among youths receiving public mental health services. J Am Acad Child Adolesc Psychiatry, 42:349–356, 2003.

27.

Lin

, Von Korff

, Ciechanowski

, Peterson

, Ludman

, Rutter

, Oliver

, Young

, Gensichen

, McGregor

, McCulloch

, Wagner

, Katon

: Treatment adjustment and medication adherence for complex patients with diabetes, heart disease, and depression: A randomized controlled trial. Ann Fam Med, 10:6–14, 2012.

28.

Loney

, Milich

: Hyperactivity, inattention and aggression in clinical practice. In: Advances in Developmental and Behavioral Pediatrics. Edited by M. Wolraich

D.K.

Routh Greenwich, CT, JAI Press; 1982; pp. 113–147.

29.

McCarty

, Vander Stoep

, Violette

, Myers

: Interventions developed for psychiatric and behavioral treatment in the Children's ADHD Telemental Health Treatment Study. J Child Fam Stud, 24:1735–1743, 2015.

30.

MTA Cooperative Group. A 14-month randomized clinical trial of treatment strategies for Attention Deficit Hyperactivity Disorder. Arch Gen Psychiatry 56: 1073–86, 1999a.

31.

MTA Cooperative Group: Moderators and mediators of treatment response for children with attention-deficit/hyperactivity disorder: The Multimodal Treatment Study of children with Attention-deficit/hyperactivity disorder. Arch Gen Psychiatry, 56:1088–1096, 1999b.

32.

Myers

, Vander Stoep

, Lobdell

: Feasibility of conducting a randomized trial of telemental health with children in underserved communities. J Child Adolesc Psychopharmacol, 23:372–378, 2013.

33.

Myers

, Vander Stoep

, McCarty

, Klein

, Palmer

, Geyer

, Melzer

. Child and adolescent telepsychiatry: variations in utilization, referral patterns and practice trends. J Telemed Telecare, 16:128–33, 2010.

34.

Myers

, Vander Stoep

, Zhou

, McCarty

, Katon

: The Children's Attention Deficit Hyperactivity Disorder Telemental Health Treatment Study (CATTS): A population-based trial. J Am Acad Child Adolesc Psychiatry, 54:263–274, 2015.

35.

Ngo

, Rubinstein

, Ganju

, Kanellis

, Loza

, Rabadan–Diehl

, Daar

: Grand challenges: Integrating mental health care into the non-communicable disease agenda. PLoS, 10:e1001443, 2013.

36.

Pliszka

, Crismon

, Hughes

, Corners

, Emslie

, Jensen

, McCracken

, Swanson

, Lopez

, The Texas Consensus Conference Panel on Pharmacotherapy of Childhood Attention-Deficit/Hyperactivity Disorder: The Texas Children's Medication Algorithm Project: Revision of the algorithm for pharmacotherapy of attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry, 45:642–657, 2006.

37.

Pliszka

, Lopez

, Crimson

, Torpac

, Hughes

, Emslie

, Boemer

: A feasibility study of the Children's Medication Algorithm Project (CMAP) algorithm for the treatment of ADHD. J Am Acad Child Adolesc Psychiatry, 42:279–287, 2003.

38.

Rockhill

, Violette

, Vander Stoep

, Grover

, Myers

: Caregiver's distress: Youth with ADHD and comorbid disorders assessed via telemental health. J Child Adolesc Psychopharmacol, 23:379–385, 2013.

39.

Schulz

, Fan

, Bedard

,. Clerkin

, Ivanov

, Tang

, Halperin

, Newcorn

: Common and unique therapeutic mechanisms of stimulant and nonstimulant treatments for attention-deficit/hyperactivity disorder. Arch Gen Psychiatry, 69:952–961, 2012.

40.

Shaffer

, Fisher

, Dulcan

, Davies

, Piacentini

, Schwab–Stone

, Lahey

, Bourdon

, Jensen

, Bird

, Canino

, Regier

: The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): Description, acceptability, prevalence rates, and performance in the MECA Study. Methods for the Epidemiology of Child and Adolescent Mental Disorders Study. J Am Acad Child Adolesc Psychiatry, 35:865–877, 1996.

41.

Solomon

, Bitton

, Katz

, Radner

, Brown

, Fraenkel

: Review: Treat to target in rheumatoid arthritis: Fact, fiction or hypothesis?. Arthritis Rheumatol, 66:775–782, 2014.

42.

Steele

, Jensen

, Quinn

: Remission versus response as the goal of therapy in ADHD: A new standard for the field?. Clin Ther, 28:1892–1908, 2006.

43.

Swanson

: chool-Based Assessments and Interventions for ADD Students. Irvine, CA: KC Publications; 1992.

44.

Thomas

, Holzer

3 : The continuing shortage of child and adolescent psychiatrists. J Am Acad Child Adolesc Psychiatry, 45:1023–1031, 2006.

45.

Vander Stoep

, Myers

: Methodology for conducting the Children's Attention-Deficit Hyperactivity Disorder Telemental Health Treatment Study in multiple underserved communities. Clin Trials, 10:949–958, 2013.

46.

Von Korff

, Tiemens

: Individualized stepped care of chronic illness. West J Med, 172:133–137, 2000.

47.

Wagner

, Yang

, Xing

, Chen

, Liu

, Cuo

: Treatment receipt and outcomes from a clinic employing the attention-deficit/hyperactivity disorder treatment guideline of the children's medication algorithm project. J Child Adolesc Psychopharmacol, 24:472–480, 2014.

48.

Wang

, Yang

, Xing

, Chen

, Liu

, Luo

: Altered neural circuits related to sustained attention and executive control in children with ADHD: An event-related fMRI study. Clin Neurophysiol, 124:2181–2190, 2013.

49.

Wilen

, Hammerness

, Utzinger

, Schillinger

, Georgiopoulous

, Doyle

, Martelon

, Brodziak

: An open study of adjunct OROS-methylphenidate in children and adolescents who are atomoxetine partial responders: I. Effectiveness. J Child Adolesc Psychopharmacol, 19:485–492, 2009.

50.

Wolraich

, Lambert

, Doffing

, Bickman

, Simmons

, Worley

: Psychometric properties of the Vanderbilt ADHD diagnostic parent rating scale in a referred population. J Pediatr Psychol, 28:559–567, 2003.