Abstract
Keywords
ADHD is a neurodevelopmental disorder that affects approximately 5% of school-age children (DuPaul, Reid, Anastopoulos, & Power, 2014; Polanczyk, Willcutt, Salum, Kieling, & Rohde, 2014) and often involves delays or inadequacies in the development of higher order cognitive processes including the ability to sustain, direct, and shift attention as well as monitor and self-regulate behavior (Barkley, 1997; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). ADHD is a major public health problem as it is associated with impaired child and family functioning and high societal costs and often leads to academic underachievement, behavior problems, parent–child conflicts, and peer relationship difficulties (Barkley, Anastopoulos, Guevremont, & Fletcher, 1992; Bernfort, Nordfeldt, & Persson, 2008; Efron et al., 2014; Loe & Feldman, 2007; Marton, Wiener, Rogers, & Moore, 2015). ADHD is considered a chronic disorder and, when it is not diagnosed and treated early on, it may result in various negative developmental outcomes (e.g., occupational problems, other psychiatric disorders, substance abuse, criminal activity) across the life span as well as an increased rate of mortality (Cherkasova, Sulla, Dalena, Pondé, & Hechtman, 2013; Dalsgaard, Østergaard, Leckman, Mortensen, & Pedersen, 2015; Knecht, De Alvaro, Martinez-Raga, & Balanza-Martinez, 2015; Kooij et al., 2012; Kuriyan et al., 2013).
Neurobiology of ADHD
Neuroimaging studies have shown that there are various structural and functional brain abnormalities in children with ADHD (Cortese et al., 2012; Ellison-Wright, Ellison-Wright, & Bullmore, 2008). Specifically, children with ADHD have delayed cortical development, cortical thinning, and reductions in the volume of several brain regions including the prefrontal cortex and the temporal and parietal lobes (Fair et al., 2012; Shaw et al., 2007; Shaw et al., 2012; Shaw et al., 2009), which correspond to brain networks that support the regulation of attention, emotions, and behavior (Purper-Ouakil, Ramoz, Lepagnol-Bestel, Gorwood, & Simonneau, 2011). Moreover, the brain activity of children with ADHD when performing higher order cognitive tasks is most similar to their younger and typically developing peers (Fernández et al., 2009; Hart, Radua, Mataix-Cols, & Rubia, 2012). In fact, these higher order cognitive processes, particularly response inhibition, sustained attention, and working memory have been found to be impaired in children with ADHD (Crippa et al., 2015; Pennington & Ozonoff, 1996; Willcutt et al., 2005). These underdeveloped brain regions, functional network abnormalities, and associated cognitive impairments may provide important neurocognitive targets for interventions.
Past research has shown that environmentally initiated training (i.e., sustained practice with a musical instrument or athletic activities) may lead to changes in regional brain volume and inter-regional connectivity patterns (Schlaug, 2001; Wang et al., 2013). Computerized cognitive remediation training (CCRT) is a treatment approach based on a similar premise in that such training may result in enhancements in cortical activation and strengthening of cortical connections. In fact, CCRTs have been successful in adults by enhancing recovery from stroke, reducing cognitive impairments in schizophrenia, and addressing other neurological illnesses (Hildebrandt et al., 2007; Raskin & Sohlberg, 2009; Westerberg et al., 2007; Wexler, Anderson, Fulbright, & Gore, 2000). Given the short- and long-term negative outcomes associated with ADHD, it is imperative to develop and evaluate the potential benefit of such interventions for ADHD in children as young as developmentally possible with the goal of improving their neural and behavioral development to prevent later maladaptive outcomes.
Current ADHD Treatments
Both psychopharmacological (Biederman, Spencer, & Wilens, 2004; Swanson, Baler, & Volkow, 2011) and non-pharmacological (Evans, Owens, & Bunford, 2014; Faraone & Antshel, 2014) interventions have shown to be efficacious in the treatment of ADHD. Although pharmacological treatment with psychostimulants are relatively safe and lead to symptom relief for most children (Biederman et al., 2004; Swanson et al., 2011), not all difficulties (e.g., cognitive deficits) associated with ADHD remit and the long-term benefits are limited (Molina et al., 2009; van de Loo-Neus, Rommelse, & Buitelaar, 2011). Alternatively, psychosocial and behavioral treatments, which primarily involve the systematic use of reinforcements (i.e., contingency management), have shown short-term behavioral gains equivalent to low-dose medication, yet these gains disappear once contingencies are removed (Carlson, Pelham, Milich, & Dixon, 1992; Pelham et al., 1993; Sagvolden, Johansen, Aase, & Russell, 2005). In outpatient behavior therapy, improvements over baseline are usually seen, but the resulting treatment effects are not as large as those in medication trials (Daly, Creed, Xanthopoulos, & Brown, 2007; Hinshaw, Klein, & Abikoff, 1998; Pelham, Wheeler, & Chronis, 1998; Van der Oord, Prins, Oosterlaan, & Emmelkamp, 2008). When combining both treatment modalities (i.e., medication plus behavior therapy), improvements in multiple areas of functioning (i.e., academics, comorbid psychiatric conditions, social skills, parent–child relationships) have been found and medication dosages may be lowered while maintaining these beneficial effects (Jensen, 1999; Van der Oord et al., 2008). Despite these advantages, neither treatment option alone or in its combined form completely addresses the neurocognitive pathology of ADHD, so it is possible that the long-term course of the disorder may remain unaltered.
With this in mind, several CCRT interventions have been developed to target working memory as it is considered a fundamental higher order cognitive function underlying other cognitive functions and is essential for goal-directed behavior (Klingberg et al., 2005; Molfese & Molfese, 2002). It has been suggested that training working memory may improve working memory capacity, which in turn may positively affect other cognitive functions and ADHD symptoms (e.g., Chacko et al., 2014; Klingberg et al., 2005; van Dongen-Boomsma, Vollebregt, Buitelaar, & Slaats-Willemse, 2014). Indeed, several randomized controlled trials have shown improvements in working memory following treatment with CCRTs in children and adolescents with ADHD (Beck, Hanson, Puffenberger, Benninger, & Benninger, 2010; Chacko et al., 2014; Cortese et al., 2015; Gray et al., 2012; Green et al., 2012; Holmes et al., 2010; Johnstone et al., 2012; Klingberg et al., 2005; Sonuga-Barke et al., 2013). However, treatment effects have been less consistent for other non-trained neurocognitive outcome measures (e.g., sustained attention, response inhibition, processing speed) and for parent ratings of ADHD symptoms (Beck et al., 2010; Chacko et al., 2014; Gray et al., 2012; Green et al., 2012; Holmes et al., 2010; Johnstone et al., 2012; Klingberg et al., 2005; Shalev, Tsal, & Mevorach, 2007; van Dongen-Boomsma et al., 2014). Moreover, improvements in ADHD symptomatology and neurocognitive functioning as assessed by teacher and clinician ratings (i.e., blinded raters) have not typically been found (Beck et al., 2010; Chacko et al., 2014; Klingberg et al., 2005; van Dongen-Boomsma et al., 2014). It is possible that prior research has failed to find robust treatment effects for CCRTs because most have been designed to train only one higher order cognitive function (i.e., working memory), which may have limited transfer effects. In fact, Rapport, Orban, Kofler, and Friedman (2013) found in their meta-analysis that CCRTs showed evidence of near-transfer effects (e.g., trained cognitive domains), but not far-transfer effects (e.g., non-trained cognitive domains, ADHD symptomatology). Considering ADHD impairs functioning across multiple domains, it stands to reason that interventions must target the core cognitive deficits associated with ADHD in addition to other areas of impairment (e.g., behavioral, motor, social) if treatment effects are expected to generalize.
In an effort to promote transfer effects, a new neuroscience-inspired intervention was designed to target eight cognitive functions (i.e., sustained attention, response inhibition, speed of processing, cognitive flexibility, multiple simultaneous attention, working memory, category formation, pattern recognition), which have been implicated in ADHD and form the groundwork of learning (Barkley, 1997; Crippa et al., 2015; Huang-Pollock, Maddox, & Tam, 2014; Pennington & Ozonoff, 1996; Willcutt et al., 2005). Considering a separate literature suggests that physical exercise can also improve these cognitive functions (Grassmann, Alves, Santos-Galduróz, & Galduróz, 2017; Kamp, Sperlich, & Holmberg, 2014) and ADHD symptomatology (Abramovitch, Goldzweig, & Schweiger, 2013; Smith et al., 2013; Verret, Guay, Berthiaume, Gardiner, & Béliveau, 2012), sport activities designed to train the same cognitive abilities in the context of whole body activity and social activation also comprised this intervention. All in all, the training of both the mind and body was anticipated to maximize neurocognitive benefits so treatment effects would generalize to ADHD symptomatology.
Integrated Brain, Body, and Social (IBBS) Intervention
The purpose of this study was to evaluate the efficacy of a new IBBS intervention for children (aged 5-9 years) with ADHD. This multi-site randomized wait-list controlled study funded by the National Institutes of Health (NIH) Transformative Research Program was conducted in the United States and China (Title: Integrated Brain, Body, and Social (IBBS) intervention for Attention-Deficit/Hyperactivity Disorder; http://clinicaltrials.gov/ct2/show/NCT01542528). This treatment is innovative in that it integrates two modalities of training (i.e., computerized cognitive training and physical exercises) and targets several higher order cognitive functions known to be implicated in ADHD with the goal of enhancing the activity of supporting brain networks to ameliorate symptoms of ADHD. Furthermore, the brain component of treatment builds upon past CCRTs as the difficulty level adjusts to the performance of each child by means of graduation and plateau criteria and online corrective messages help children adopt new strategies to achieve success. Finally, behavioral techniques (i.e., Good Behavior Game [GBG]) are employed to assure children’s engagement in the brain and body components of treatment. In fact, previous studies have used reward systems to maximize compliance of children participating in clinical trials of CCRTs (e.g., Chacko et al., 2014).
Given the integrative nature of IBBS (i.e., comprised of multiple modalities of training) and the enhancements made to the brain component of treatment over and above existing CCRTs (i.e., difficulty of training matches performance level, online corrective messaging), it was hypothesized that children receiving IBBS would show significantly greater improvement than children receiving treatment-as-usual (TAU) in ADHD symptomatology and global clinical functioning as rated by blinded clinicians. These treatment effects would also be observed on parent and teacher ratings of ADHD. Finally, children in the IBBS condition following treatment would outperform children in the TAU condition on neurocognitive tests that were specifically trained by IBBS.
Method
Participants
Children were eligible to participate in the study if the following criteria were met: (a) age between 5 and 9 years; (b) a diagnosis of ADHD (i.e., Combined, Predominantly Inattentive, and Predominantly Hyperactive-Impulsive types) according to the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; American Psychiatric Association, 2000) or children at subthreshold for ADHD, defined as one symptom below diagnostic criteria; (c) an intellectual quotient of at least 80; and (d) on a stable dose of medication for at least 4 weeks (if on medication for ADHD). Children were excluded if they had a severe or impairing comorbid psychiatric diagnosis such as major depression, bipolar disorder, psychotic disorder, or acute behavior problems (e.g., temper tantrums, aggression, self-injury) that required immediate therapeutic attention or a documented physical disability or injury that would prevent them from participating in the IBBS treatment. A comorbid diagnosis of an autism spectrum disorder was not deemed exclusionary for study participants if their level of functioning did not interfere with their ability to participate in all aspects of the program.
A total of 112 children in the United States and China were assessed for eligibility. Of this potential participant pool, 92 children (72 from United States and 20 from China; M age = 7.4 ± 1.1; 70% male) were enrolled into the study. Year of implementation determined to which cohort study participants belonged. Cohort 1 comprised of students from four elementary schools in a northeastern town in the United States as well as children living in a predominantly urban district of Beijing, China. Cohort 2 consisted of children enrolled in the remaining three elementary schools from the same U.S. school district as Cohort 1. See Table 1 for demographic and clinical characteristics of the sample disaggregated by treatment group. Of note, nine children in each group (i.e., IBBS, TAU) were determined to be subthreshold for ADHD. With regard to medication status, 12 children (six in each group) were receiving one or more ADHD medications at baseline and were required to remain on the same type and dose of medication throughout the duration of the study.
Demographic and Clinical Characteristics of IBBS and TAU Groups for Study Completers.
Note. IBBS = Integrated Brain, Body, and Social intervention; TAU = treatment-as-usual; n = number of participants; SNAP = Swanson, Nolan, and Pelham Rating Scale; CGI-S = Clinical Global Impression–Severity scale; SSRIs = selective serotonin reuptake inhibitors.
Data available for U.S. sample only.
Fisher’s exact test.
IBBS Intervention
IBBS is comprised of three components including CCRT (i.e., brain component), physical exercise (i.e., body component), and a classroom-based behavior management strategy (i.e., social component). Each component is described in its respective section below.
Brain component
Three computer games were developed for the purpose of the IBBS intervention. At the most basic level, the first computer exercise resembled the continuous performance task and required children to click on presented stimuli when it changed from a default color to a target color. The cognitive capabilities required by this exercise included sustained attention, response inhibition, working memory, directed attention, attentional switching, and divided attention. The second exercise was most similar to inhibitory control paradigms such as the Go/No-Go task and instructed children to identify and click on members of a target category before they moved off the screen. Some categories were considered “natural” or “basic” (e.g., animals, furniture), whereas some fell on different ends of a continuum within the same category (e.g., small animals vs. large animals), and still others were regarded as “functional” or “temporary” categories (e.g., modes of transportation, household items). This computer exercise required the use of sustained attention, response inhibition, directed attention, visual searching, and category formation. The third exercise resembled the Wisconsin Card Sorting Task and had children determine the relationship among stimuli in the first row to make a selection from stimuli in the second row that completed the sequence in the first row. The cognitive abilities that were employed by this task included sustained attention, response inhibition, working memory, speed of processing, and category formation.
In all the games, several parameters of the program (e.g., number of targets on the screen, rate at which targets moved, complexity of rules to identify targets) were adjusted to increase the level of difficulty and were based on the performance of that child. This novel approach was performed by way of graduation and plateau criteria, such that children were moved through exercises quickly in areas of their strength and continued to work in areas of their weaknesses, but moved on once their maximum gain was reached. All responses made were recorded so progress was easily tracked and online corrective strategy messages were used to facilitate growth. In addition, program teachers were instructed to provide encouragement to support children’s efforts during these exercises and children were awarded points within the game framework based on individual performance, which could then be exchanged for virtual prizes. Each computerized cognitive training session lasted 30 minutes and all three computer games were played within this time frame.
Body component
The body component of IBBS was intended to target the same cognitive functions as the brain component with the intent to increase the likelihood of activating and engaging attentional networks that support these functions. The decision to include physical exercise as part of the IBBS program was based on emerging evidence in the extant literature that physical exercise improves neurocognitive functioning in children with ADHD (Grassmann et al., 2017; Halperin, Berwid, & O’Neill, 2014; Kamp et al., 2014). In fact, two recent studies found that children with ADHD who were randomized to an exercise condition showed better accuracy and quicker performance on neurocognitive tasks (i.e., selective attention, sustained attention, cognitive flexibility, working memory) than children in the control condition (Chang, Liu, Yu, & Lee, 2012; Verret et al., 2012). One study also found transfer effects as parent and teacher ratings of inattention and social problems improved for children in the exercise condition (Verret et al., 2012).
The exercises designed for the IBBS program progressed gradually from single to multiple tasks and from simple to more complicated movements with different requirements for reaction time, speed of processing, and hand-eye-body coordination. These physical activities were designed to target specific cognitive functions thereby enhancing children’s abilities in these areas. Each IBBS exercise drew on several cognitive functions, with some depending more heavily on a particular cognitive ability than others (e.g., patterns through an agility ladder, ball skill acquisition, hula-hoop, juggling). See Supplementary Appendix S1 for further details concerning the body component of treatment. Study participants were closely monitored by instructors and provided feedback and support for developing these skills. Children participated in the body component of IBBS for 45 minutes each session.
Social component
The social component of IBBS was deemed important, as it would help limit behaviors that could potentially interfere with its successful implementation and would also facilitate participation among all children in the program. The GBG was selected for this purpose because it is the most thoroughly studied classroom-based strategy and has the most documented impact on the prevention of emotional and behavioral disorders and related sequelae according to the Institute of Medicine Report on Prevention (O’Connell, Boat, & Warner, 2009). In fact, two large-scale randomized controlled trials revealed that children who participated in the GBG for the entire academic year showed slower growth rates of hyperactive and oppositional symptoms (Leflot, Van Lier, Onghena, & Colpin, 2010) and significant decreases in disruptive behavior, as rated by their teachers across the study period (Huizink, Van Lier, & Crijnen, 2008).
The version of the GBG that was used in the IBBS program was the response cost design. The majority of published studies employ this version and past research has found no differential benefits of allocating attention to rules violated (i.e., response cost) versus rules followed (i.e., reinforcement; Tanol, Johnson, McComas, & Cote, 2010). The GBG was played 3 to 5 times for 15 to 30 minutes (length of games gradually increased as disruptive behaviors decreased) each session during the brain and body components of IBBS.
During the first week of the IBBS program, behavioral observations were made to get a baseline assessment of how often program rule violations occurred and the frequency with which each child violated the rules or displayed disruptive and/or off-task behaviors. Based on this information, students were divided into two comparable teams and a maximum number of rule violations were agreed upon by teachers implementing IBBS. The same four rules used in the GBG manuals were employed, with one additional rule, “we will try our best,” inspired by the brain component of the IBBS protocol. This was included to promote positive engagement and to minimize avoidance behaviors when completing cognitively taxing activities. To win the GBG, teams had to receive fewer rule violations than the maximum number permitted by their teachers. Winning teams were rewarded with immediate behavior rewards (e.g., follow the leader, “Simon says,” red light green light) lasting at most 5 minutes and more long-term tangible rewards (e.g., trip to the prize box) were given to the team that won the most games each week.
Treatment-As-Usual
Treatement-As-Usual (TAU) consisted of whatever the child was receiving at the time of study entry, which included (but was not limited to) psychosocial and/or psychopharmacological interventions for ADHD. As mentioned previously, six children in the TAU group were receiving one or more ADHD medication(s) at study entry. With regard to psychosocial services offered within the school, 12 had a one-to-one school aide, six had a behavior support plan, 10 received school counseling/psychological services, and 16 received some other type of service (i.e., speech therapy, occupational therapy, tutoring). Outside of school, eight were receiving psychotherapy and 12 received some other type of service. Families were instructed to not make any changes to their children’s treatment regimens throughout the duration of the study.
IBBS Training and Treatment Integrity
In the United States, the IBBS intervention was delivered by classroom teachers and mental health professionals (e.g., school counselors, social workers) who were employed by the school district in which the IBBS program was taking place. Graduate and advanced undergraduate students (i.e., juniors and seniors) assisted program teachers in implementing the IBBS intervention. Prior to treatment delivery, teachers and assistants received 6 hours of training by the U.S. research team, which included the developers of the IBBS program. This training was supplemented by the research team modeling all elements of the treatment once the program began. The modeling phase of training ended once teachers and assistants were able to implement the program with full integrity and with limited feedback from the research team. Moreover, the research team held weekly meetings with program teachers to discuss implementation difficulties, behavior problems that interfered with treatment, and to plan upcoming sessions. In China, treatment was implemented by a research team consisting of seven highly trained graduate students enrolled in leading physical education programs because the IBBS program could not be implemented within the schools or by school personnel to respect the privacy of participating families. This research team was supervised and trained by the co-developer of IBBS and trainings were augmented with bi-weekly (then monthly) consultations with the U.S. research team during the first year of implementation.
In both the United States and China, a multi-faceted approach was taken by the research team to ensure treatment integrity. For the brain component of treatment, each participant received a uniform training package, as the length and order of the computer games across all 15 weeks were pre-determined prior to the start of the program. Compliance with the computer exercises was defined as the number of hours played, which is discussed further in the results section. The body component of treatment was also standardized so that the description, length, and order of the physical exercises were detailed in the IBBS manual. On a weekly basis, the research team observed its implementation and, if deviations from the manual occurred, feedback was immediately provided so corrective actions could be taken. As the social component of treatment was less structured and required more flexibility on behalf of program teachers, the research team also completed integrity ratings on a weekly basis where all components of the GBG were rated as either present or absent to obtain an overall score. This score was rated on a 5-point Likert-type scale where 1 indicated not implemented and 5 indicated full integrity (scale adapted from Tanol et al., 2010). Overall, the GBG was implemented with adequate integrity (M = 4.05, SD = .71) and there were no significant differences in GBG integrity across implementation site or cohort. Following each assessment, feedback was provided to program teachers and further training was offered if necessary (i.e., if teachers’ received a score of 1 [not implemented], 2 [minimal integrity], or 3 [half/partial integrity]).
Procedure
All procedures were approved by the human investigation committee or by the scientific research ethics committee at the universities conducting this research. Informed consent was obtained from parents or guardians, and all participants gave informed assent prior to engaging in any study procedures. Families whose children were randomized to the IBBS and TAU groups were compensated US$40 for their time following each assessment visit.
Recruitment of study participants in the United States was achieved by means of a school mailing, which included a letter describing the IBBS research study and a measure assessing ADHD symptomatology (Swanson, Nolan, and Pelham Rating Scale [SNAP]; Swanson, 1992). If the parent/guardian completed and returned the rating scale, the SNAP was also completed by the child’s teacher. A child was considered “screen-positive” and invited to participate in a baseline assessment if the average rating per item was greater than 1.2 on the parent or teacher SNAP (Bussing et al., 2008). At baseline, child participants were evaluated using the Kiddie Schedule for Affective Disorders and Schizophrenia–Present and Lifetime Version (K-SADS-PL), which was administered by doctorate- or master-level clinicians as an interview with the child’s parent/guardian. Following a review of all available information (i.e., results of K-SADS-PL, prior psychological/medical history, scores on parent and teacher SNAPs), best estimate DSM-IV-TR diagnoses were assigned after consensus agreement was achieved by the research team. Neurocognitive tests were administered to participants by trained research assistants. Intellectual functioning was assessed by means of the Kaufman Brief Intelligence Test (K-BIT; Kaufman & Kaufman, 2004). During baseline evaluations, parent and teacher rating scales were also completed.
Once participant eligibility was confirmed, children with ADHD or at subthreshold for ADHD were randomly assigned to either IBBS or TAU (IBBS = 48; TAU = 44). A method of randomly permuted blocks (block size = 4) stratified for treatment site (i.e., United States vs. China) and medication status (see Figure 1 for CONSORT diagram) was employed. The allocation scheme and random assignment of study participants was carried out using an online randomization generator (www.randomization.com) by the study biostatistician who was not involved in data collection. The IBBS treatment condition consisted of a total of 60 sessions delivered in an after-school program format 4 days per week, 2 hours a day for 15 weeks. After the first year of IBBS implementation, it was decided to reduce the program to 3 days a week to allow for a planning day for program teachers and to offer child participants time to engage in other extracurricular activities besides IBBS (a concern voiced by parents after soliciting feedback regarding their children’s participation in the first year of implementation). Therefore, participants in the second cohort received 45 sessions of IBBS treatment. The TAU condition lasted for the same duration (i.e., 15 weeks) as the IBBS treatment condition.
CONSORT 2010 flow diagram.
Endpoint assessments and rating scales were completed within 1 month of the last day of the program. This window of time was necessary to offer families a reasonable amount of options to schedule appointments with the assessment team outside of school hours. Clinicians and teachers evaluating ADHD symptomatology over time were blind to treatment assignment. To ensure blinding was maintained on behalf of study clinicians who assessed symptomatology post-treatment, research assistants reminded families to not mention their children’s treatment assignment to study clinicians immediately before meeting with them for endpoint evaluations. Children in the TAU group were offered the IBBS intervention following endpoint assessments.
The procedures followed by the assessment and implementation team in China were almost identical to the procedures outlined above with a few exceptions. These differences were driven by the need to be sensitive to cultural differences. Considering psychiatric disorders are not publicly discussed and parents do not inform teachers or school personnel of their children’s mental health diagnoses in China, recruitment through the school system was not an option. Instead, parents were informed about the study by means of delivering brochures, paid advertising, and announcements on social media. Moreover, in an effort to respect the privacy of parents, the IBBS program was not conducted in the same schools to which children were enrolled. As a result, the intervention took place at an off-site facility outside of school hours 3 times a week, 90 minutes a day, for 15 weeks. The blinded evaluations were completed by an assessment team comprised of psychiatric doctors and researchers in the areas of pediatrics and psychology.
Behavioral Outcomes
ADHD symptoms were assessed using the SNAP. The SNAP has been used repeatedly in the extant literature as a primary outcome measure (e.g., MTA Cooperative Group, 1999a, 1999b; Swanson et al., 2011). It is composed of 26 items where 18 items assess ADHD symptoms (nine inattentive, nine hyperactive/impulsive) and eight items assess oppositional defiant disorder (ODD) symptoms. Blinded clinicians, parents, and teachers were asked to rate symptoms on a 4-point scale (0 = not at all, 1 = just a little, 2 = quite a bit, 3 = very much) at baseline and endpoint. A total score was calculated by summing all 18 items specific to ADHD. For the purposes of this study, the clinician total score on the SNAP was treated as a primary outcome measure. Cronbach’s alphas ranged from .86 to .94 for the inattentive subscale and from .84 to .94 on the hyperactive/impulsive subscale across all three raters (i.e., blinded clinicians, parents, teachers).
The Clinical Global Impression–Improvement (CGI-I) scale also served as a primary outcome measure to assess improvement in ADHD symptom severity following treatment (Guy, 1976). The CGI-I is a single item scale where improvement was rated on a 7-point scale by blinded clinicians. Responders to treatment were defined as improved, much improved, or very much improved.
Neurocognitive Outcomes
A neurocognitive assessment battery lasting approximately 45 minutes was administered at baseline and endpoint by highly trained research assistants supervised by a licensed clinical psychologist. This battery included the children’s version of the California Verbal Learning Test (CVLT; Delis, Kramer, Kaplan, & Ober, 1987) to measure verbal learning and memory, the Wide Range Assessment of Memory and Learning–Second Edition (WRAML-2) Finger Windows forward (Adams & Sheslow, 1990) and backward (Bedard, Jain, Johnson, & Tannock, 2007) to assess spatial working memory and spatial storage manipulation, and the Flanker Task (Eriksen & Eriksen, 1974) to assess sustained attention and response inhibition. It is important to note that the assessment battery used by the research team in China did not include the Flanker as they did not have access to the computer program supporting this paradigm (i.e., E-Prime). In addition, children from the U.S. sample who received a Flanker accuracy score of less than .70 on congruent trials were excluded from analyses for the Flanker outcome variables only because these participants were thought to be disengaged from the task or not following test instructions, thus invalidating their performance. See Supplementary Appendix S2 for a description of each neurocognitive measure and what variables were identified a priori as capturing treatment effects.
Statistical Analyses
Baseline characteristics between the IBBS intervention and TAU groups as well as differences between sites (i.e., China vs. United States) were compared using t tests or ANOVAs for continuous variables and chi-square tests for categorical variables. ANCOVAs were used to test the difference between groups following treatment for all primary and secondary outcome measures using SAS Version 9.3. For each outcome, the model included the endpoint measurement as the dependent variable, group (i.e., IBBS vs. TAU) as the independent variable, and baseline score, age, medication status, ADHD subtype, and treatment site (i.e., China vs. U.S. sites) as covariates. Non-parametric tests were used for those outcome measures that did not meet parametric assumptions (e.g., variable was not normally distributed). Clinical Global Improvement scores were evaluated using chi-square tests. Missing data were imputed using the last observation carried forward (LOCF) or intent-to-treat approach as well as multiple imputation (MI) as both methods have been used extensively in treatment outcome studies to handle missing data (e.g., Chacko et al., 2014; van Dongen-Boomsma et al., 2014). As there were no differences in results when imputing (using LOCF or MI) or not imputing missing values, the results of study analyses are presented using the raw data of all study completers (i.e., those participants who were randomized and completed endpoint assessments). To control for multiple comparisons, the linear step-up (LSU) procedure was adopted (Maxwell & Delaney, 2004). Following this procedure, the p value obtained from all analyses testing the main study hypotheses, starting with the smallest value, is compared with the equation (i/m) α, where i is assigned a value of 1 and increases by 1 until reaching m, m is the total number of comparisons, and α is equal to .05. If the observed p values from these analyses are less than the corrected alpha level, the findings are deemed significant. For ease of interpretation, the corrected p values will be presented when reporting significant findings. Finally, Cohen’s d (i.e., the difference between the change scores of each group divided by the pooled standard deviations at baseline) was calculated as a measure of effect size.
Results
Demographic Characteristics
No significant differences between groups (IBBS vs. TAU) were detected at baseline with respect to demographic or clinical characteristics (see Table 1). However, significant differences were found between the U.S. and China participants on the baseline clinician SNAP, t(90) = 2.83, p = .006; parent SNAP, t(90) = 2.49, p = .014; and teacher SNAP scores, t(90) = 2.84, p < .001, when not subdivided according to treatment assignment. In all instances, the participants from China had higher mean scores than the U.S. participants (clinician SNAP: U.S. M = 28.8, SD = 6.9; China M = 34.1, SD = 9.2; parent SNAP: U.S. M = 27.7, SD = 11.2; China M = 34.5, SD = 9.3; teacher SNAP: U.S. M = 23.0, SD = 12.8; China M = 34.5, SD = 7.9). As expected from previous research, clinician, parent, and teacher SNAP scores were all significantly correlated (rs ranging from .29 to .73) with the most modest correlation found between parent and teacher SNAPs, r = .29, p = .005 (see Narad et al., 2015).
Treatment Compliance
On average, children played the IBBS computer games for 18.1 hours (SD = 4.3) across the 15-week IBBS treatment period with a target playing time of 20 hours. Previous studies have defined treatment compliance as completing 80% of the total number of training periods (e.g., Chacko et al., 2014; Klingberg et al., 2005; van Dongen-Boomsma et al., 2014). Therefore, we defined treatment compliance as completing 80% of the target playing time (i.e., 16 hrs). With this criterion in place, 25 out of 32 children were identified as treatment compliers in the IBBS group. In the following paragraphs, the results for the treatment compliers and all study completers (i.e., children who were randomized and completed endpoint evaluations) are presented.
Behavioral and Neurocognitive Outcomes for Study Completers
All results of group-wise treatment differences for the behavioral and neurocognitive outcome measures for study completers are presented in Table 2. There were no treatments effects found for clinician, teacher, or parent SNAP scores. Although there were no significant treatment group differences on the CGI-I scale, 25 out of 42 participants (60%) were classified as responders in the IBBS group and 21 out of 38 participants (55%) were classified as responders in the TAU group, as determined by scores on the CGI-I outcome measure.
Test of Group-Wise (IBBS vs. TAU) Treatment Differences.
Note. IBBS = Integrated Brain, Body, and Social intervention; TAU = treatment-as-usual; n = number of participants; SNAP = Swanson, Nolan, and Pelham Rating Scale; CVLT = California Verbal Learning Test; RT = reaction time; Error = directional errors; RTCV = reaction time coefficient of variation; CGI-I = Clinical Global Impression–Improvement.
Data available for U.S. Sample only.
A significant treatment group difference in favor of IBBS was found on CVLT total learning (p = .05); however, this finding did not survive the LSU corrected significant level of .004. No other treatment effects were found on any of the remaining neurocognitive outcome measures. The covariates of interest (i.e., age, medication status, ADHD subtype, treatment site) had no significant effect on study outcomes. Finally, it should be noted that the results for treatment compliers versus all study completers on the treatment outcome measures did not dramatically differ with the exception of the CVLT total learning score. For treatment compliers, the treatment group difference in favor of IBBS only approached significance, F(1, 60) = 2.96, p = .09.
Discussion
The purpose of this study was to evaluate the efficacy of a new non-pharmacological intervention for young children with ADHD. A multi-site, randomized, wait-list controlled design was used to evaluate the effects of the IBBS intervention on outcome measures of ADHD symptomatology, global clinical functioning, and neurocognitive abilities. The results of this study revealed no statistically significant treatment effects on any of the primary outcome measures. In terms of secondary outcome measures, the IBBS group showed significant improvement on a verbal working memory task; however, this result did not survive correction for multiple group comparisons.
Given the integrative nature of IBBS (i.e., comprised of multiple modalities of training) and the enhancements made to the brain component of this intervention (i.e., difficulty of training matches performance level, online corrective messaging), it was predicted that IBBS would not only improve trained cognitive domains, but also generalize to symptoms of ADHD and global clinical functioning. However, the results of this study were not supportive of these hypotheses. Instead, these results were consistent with previous studies and meta-analytic reviews of CCRTs where moderate near-transfer effects (i.e., effects on trained tasks) are consistently found for working memory, but evidence of far-transfer effects (i.e., effects on untrained tasks) on blinded ratings of ADHD symptoms and untrained neurocognitive measures are lacking (Chacko et al., 2014; Rapport et al., 2013; Sonuga-Barke et al., 2013; van Dongen-Boomsma et al., 2014).
The lack of significant treatment effects, especially when IBBS is comprised of intervention components that have shown promise in improving ADHD symptoms and untrained cognitive abilities, is worthy of attention. First, the cognitive processes trained by the brain component of IBBS may not be engaging the right neural pathway to result in improvements at the symptom level considering other circuitry deficits have been implicated in ADHD (see Bush, 2010, for a review) or perhaps engagement without real-world application is not enough. It is also possible that the time spent playing video games or using the computer outside of IBBS sessions may have differentiated groups (IBBS vs. TAU), as this was not monitored or controlled for during the study. Second, the body component of IBBS may not have produced the expected treatment effects because the physical activities shown to improve cognitive functioning and parent/teacher ratings of ADHD in previous studies required whole body movements and great amounts of energy expended with less of a focus on specific skill acquisition (e.g., Chang et al., 2012; Verret et al., 2012). This highlights a limitation of our study, as we did not capture the energy exerted by study participants during the body component of treatment or control for physical exercise acquired outside of the IBBS program. Third, the GBG was used as a tool by the implementation team to limit disruptions and to promote engagement in the IBBS treatment protocol, but was not intended to be an active component of treatment. However, the GBG is considered an evidence-based prevention strategy in its own right based on its long-term impact on impulsive, disruptive, and antisocial behaviors (see Embry, 2002). It should be noted that in studies finding treatment effects, the GBG is played throughout the school day for an entire academic year and with classmates who serve as strong social reinforcers because of their large and influential role in children’s social networks. Fourth, the IBBS treatment dosage and timing (i.e., after-school setting) may not have been optimal. Initially, children were expected to participate in the IBBS after-school program 4 days a week for a period of 15 weeks. However, the number of days per week were reduced for the second cohort after receiving feedback from parents and teachers that children often returned home exhausted, were resistant to completing important school-related responsibilities (i.e., homework), and it made their participation in other after-school activities impossible. Thus, longer treatment periods especially in an after-school format may not have been feasible or well-tolerated. Fifth, it is possible that other neuropsychological measures (e.g., Continuous Performance Task) may have been sensitive to treatment change. Finally, the heterogeneity of our sample (i.e., subthreshold ADHD presentation, inclusion of most comorbidities) may have limited our ability to detect significant group differences following treatment.
In sum, the results of this study did not find evidence for the benefits of IBBS over TAU based on our primary outcome measures. This suggests that expanding cognitive training to multiple domains by means of two training modalities (i.e., computerized and physical exercises) does not lead to generalized improvement of ADHD symptomatology. Giving these findings and the recently published data of other treatment outcome studies for CCRTs, it appears that CCRTs without real-world application should not be recommended as a treatment for young children with ADHD. However, the possibility that cognitive training and physical exercise may help reduce symptoms of ADHD should not be ruled out completely. Although research on physical exercise and its impact on ADHD is still in its infancy stages, there is some evidence to suggest that moderate-to-vigorous exercise does translate into ADHD symptom reduction (Smith et al., 2013; Verret et al., 2012). However, more work is needed to determine the type and dose of physical activity that is required to produce these positive outcomes. With regard to cognitive training, future studies should consider establishing individual neurocognitive profiles of children and then tailoring the cognitive training to meet their needs in real-world situations or settings. Such an approach may better reflect the potential of cognitive training if ADHD symptoms or level of impairment are shown to be beneficially affected following such a treatment approach.
Footnotes
Acknowledgements
We wish to thank the co-developers of IBBS, Dr. Bruce Wexler (brain component) and Dr. Jinxia Dong (body component), who so graciously allowed us to evaluate the efficacy of this intervention. It should also be noted that the brain component of IBBS was programmed by software engineers at C8 Sciences.
Additionally, the authors would like to thank the personnel of the Hamden Public Schools for their invaluable contribution to this project. In particular, we are grateful to Robin Riccitelli, the Coordinator for Elementary Special Education Services, and Chris Brown (her predecessor) as well as the more than 20 teachers and staff members who implemented the IBBS after school program. Finally, we would also like to thank Rachel Kuschner who served as the project coordinator during the first year of IBBS implementation.
Declaration of Conflicting Interests
The author(s) disclosed receipt of the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Jinxia Dong is a co-founder and holds equity in C8 Sciences, which developed and sells the brain training program evaluated by the research described in this paper. All of the remaining authors have declared that they have no competing or potential conflicts of interest.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study has been funded by the Award Number R01HD070821 from the Director’s Office at the National Institutes of Health. The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of NIH.