Abstract
Selecting relevant information, maintaining focus on a task, and inhibiting impulsive responding are all critical skills that gate cognitive development, peer relations and learning (Diamond, 2013; Gomes & Livesey, 2008; Ponde, Cruz-Freire, & Silveira, 2012). These skills typically referred to as attention, emerge in infancy and become progressively more robust from the preschool years onward (Steele, Karmiloff-Smith, Cornish, & Scerif, 2012). Although there is no single definition of attention, Posner and Petersen’s (1990) suggestion that attention is an anatomical network that influences multiple processing systems and comprises three attentional processes is still widely accepted 20 years on from original publication (Petersen & Posner, 2012). The proposed attention network includes selective attention or orienting (e.g., prioritizing information), sustained attention or alerting (e.g., maintaining vigilance during a task), and executive attention (e.g., controlling one’s attention). Disruption to these central attention processes can result in behavioral symptoms of inattention (e.g., distractibility, poor concentration) and hyperactivity (e.g., impulsivity).
A wealth of literature indicates that early attention is strongly associated with emergent language (Mahurin-Smith, DeThorne, & Petrill, 2017), cognitive development (Diamond, 2013), behavioral regulation (Moffitt et al., 2011), literacy (Sims & Lonigan, 2013), and numeracy skills (Steele et al., 2012). Longitudinal studies demonstrate that children’s early attention span significantly predicts long-term reductions in academic achievement and vocational outcomes (McClelland, Acock, Piccinin, Rhea, & Stallings, 2013). Attention skills are integral in allowing children to navigate successfully through the early school years by equipping them with the ability to focus on teacher instructions, to complete tasks, and to follow rules within the classroom. Given the critical importance of attention in providing the foundation for learning, cognitive development, and behavioral regulation, one might assume that strengthening attention may offer the potential to enhance academic and cognitive abilities, as well as reduce problem behaviors in early childhood.
Computer-based training to strengthen underlying brain networks and cognitive functions, including attention (Kirk, Gray, Ellis, Taffe, & Cornish, 2016; Tamm, Epstein, Peugh, Nakonezny, & Hughes, 2013), has provided promising results. The premise of cognitive training is that repeated practice of a skill results in activation of neural regions associated with that skill (Olesen, Westerberg, & Klingberg, 2004) and, therefore, strengthens the cognitive function supported by the targeted neural network. Subsequently, enhancement of other untrained domains associated with the trained cognitive skill may also occur. Attention training studies have generally focused on children with well-documented attention deficits, such as those with attention deficit hyperactivity disorder (ADHD) or intellectual disabilities, and have shown improvements in cognitive attentional processes (Kirk et al., 2016; Kray, Karbach, Haenig, & Freitag, 2011; Tamm et al., 2013) as well as inattentive and hyperactive behaviors (Shalev, Tsal, & Mevorach, 2007; Steiner, Sheldrick, Gotthelf, & Perrin, 2011). Despite these reported improvements, few training studies have assessed transfer of attention training to other domains, in particular academic domains. In the only randomized controlled trial to date examining the effects of attention training on academic performance in early childhood, Kirk and colleagues (2016) evaluated the benefits of a gamified tablet-based attention training program (Tali Train) compared with a placebo control, in 77 children (4-13 years) with intellectual disabilities. Following 5 weeks of adaptive home-based training, selective attention performance increased, and this benefit persisted up to 3 months compared with controls. Furthermore, at 3 months, children who received training showed gains in numeracy skills compared with controls (Kirk, Gray, Ellis, Taffe, & Cornish, 2017). Although these results suggest promising benefits of Tali Train on attention and academic skills in primary school children with intellectual disabilities, the potential effects of attention training in children without intellectual disabilities have not yet been evaluated.
Investigating the effects of attention training more broadly in primary school children, and not just in clinical populations, has important implications. If attention training can enhance cognitive abilities, improve academic achievement, and reduce problem behaviors in children with a wide range of attentional abilities, and not just a subgroup of children with attentional deficits (e.g., children with intellectual disabilities—1.04% global prevalence (Maulik, Mascarenhas, Mathers, Dua, & Saxena, 2011), or clinically diagnosed ADHD—5% global prevalence (Polanczyk, de Lima, Horta, Biederman, & Rohde, 2007)), the flow on effect for the development of children’s other cognitive functions would be significant and such programs might provide evidence-based tools for teachers in the classroom to support learning. Only one trial has investigated the effects of attention training in a nonclinical childhood population (Rueda, Checa, & Combita, 2012). This study assigned a group of 5-year-olds (N = 37) to either 10 sessions of computerized attention training or a control condition (watching cartoons). This small nonrandomized study demonstrated that, after training, children had faster and more efficient activation of executive attention networks, measured by electroencephalogram (EEG), than control children. This effect was sustained up to 2 months posttraining. Although these results provide promising evidence that attention training enhances targeted neural networks, gains in untrained, yet associated, skills such as emerging cognitive or academic skills were not assessed. Therefore, further investigations are necessary to establish whether enhanced attentional processes as a result of attention training subsequently extend to gains in cognition, behavior, and academic achievement—particularly in the primary school years when attention processes are undergoing dramatic development (Steele et al., 2012).
Although attention training has largely been delivered in home settings, recent studies suggest teacher-administered cognitive training may enhance intervention compliance and promote more generalized and robust gains in areas such as academic achievement (Holmes & Gathercole, 2014). As children typically spend a substantial amount of time at school, this setting is key for building cognitive, behavioral, and academic skills. Furthermore, a classroom-based delivery approach offers the potential to integrate training into daily practice and ensures every student has equal access to early intervention.
The aim of the present study was to determine the efficacy of attention training (Tali Train) delivered in the primary school classroom compared with two control arms: a placebo program or a no-contact control condition. We examined the immediate and long-term (6-months posttraining) effects of attention training on attentional processes (primary outcome), problem behaviors (e.g., inattention and hyperactivity), cognition (e.g., working memory), and academic achievement (e.g., numeracy). We expected significant training-related improvements in selective attention (Kirk et al., 2016) as well as closely related attention behaviors (e.g., inattention and hyperactivity), but not in sustained attention or executive attention. We also expected transfer of attention training to numeracy, but not working memory (Kirk et al., 2017).
Method
Design
This was a parallel condition cluster-randomized controlled trial. Assessments occurred at baseline, posttraining, and 6-month follow-up. Cluster randomization occurred at the class level to one of three conditions: Tali Train, placebo, or no-contact control. Researchers collecting data, parents, and the data analyst were blinded to condition assignment. Children and teachers were blind to whether their class was receiving Tali Train or the placebo. We were unable to conceal condition assignment from children and teachers in classes assigned to the no-contact control. The trial was prospectively registered with the Australian Clinical Trials Registry (ACTRN12616001111460) and approved by the Monash University’s Human Ethics Research Committee (MUHREC) and Catholic Education Melbourne]. See Table S2 for the CONSORT checklist.
Participants
Three mainstream primary schools within metropolitan Melbourne participated in the trial that commenced in Term 4, 2016 (baseline and posttraining), and finished in Term 2, 2017 (6-month follow-up). Children were eligible for the trial if they (a) were in participating preparatory, Grade 1, or Grade 2 classes (aged 5-9 years); (b) were fluent in English; and (c) did not have an intellectual disability based on parent reports and confirmed by IQ Composite scores >70 on the Kaufman Brief Intelligence Test (KBIT-2) at baseline (Kaufman & Kaufman, 2004). Exclusion criteria were any reported visual, auditory, or motor impairments that would prevent participation in the assessments or training program. No further exclusion criteria were applied to ensure the sample was as representative as possible of a mainstream primary school classroom. Two children did not meet inclusion criteria due to intellectual disability and lack of English fluency. Furthermore, nine participants did not complete the baseline assessment due to absence or time constraints and were removed from the trial. See Table 1 for sample characteristics and Figure 1 for participant flow.
Baseline Child Characteristics.
Note. IQ = intelligence quotient as measured by the Kaufman Brief Intelligence Test (KBIT-2); SRS = Social Responsiveness Scale; ASD = autism spectrum disorder.
Seven parents did not complete the Conners-3 or SRS questionnaires at baseline: Tali Train (n = 5), placebo (n = 2), no-contact control (n = 0).
Three children had a parent-reported diagnosis of ASD: Tali Train (n = 1), placebo (n = 1) and no-contact control (n = 1). One child with ASD (Tali Train) was prescribed medication to treat sleep disturbances (melatonin, 4 mg), mood problems (Endep, 75 mg), and irritability (risperidone, 0.5 mg).

CONSORT flow diagram.
Interventions
Tali Train
Tali Train is a game-based attention training program involving 25 daily training sessions over a 5-week period (Kirk et al., 2016). Each session is 20 min in duration and involves training on four exercises delivered on a touchscreen tablet. Each exercise lasts 4 min and targets one of three core cognitive attention processes described by Posner and Petersen (1990): selective attention/orienting, sustained attention/alerting, and executive attention. The difficulty level of each exercise is adapted to the performance of the child on a level-by-level basis. To increase engagement with Tali Train, all exercises are set within engaging visual environments, for example, underwater, at a circus, on a pirate ship, or in space (see Figure 2). A reward system also encourages motivation, where children obtain tokens for each level completed and virtual toys at the end of each exercise. These virtual toys accumulate over the course of training and children can interact with them at the end of each training session.

Tali Train exercises.
The selective attention/orienting exercise is based on a visual search task (Scerif, Cornish, Wilding, Driver, & Karmiloff-Smith, 2004) and focuses on strengthening the child’s ability to prioritize certain sensory input. This exercise requires children to locate predefined targets among a series of distractors that differ from the target in size, color, pattern, and orientation. Difficulty is increased by adding more distractors that are harder to distinguish from the targets. The sustained attention/alerting exercise focuses on the ability to maintain alertness and involves a vigilance task, where the child is instructed to monitor a moving target and select it by touching the screen when it momentarily stops moving. Difficulty is increased by prolonging the time before the target stops and by reducing the time the target remains stationery. Due to the complexity of executive attention, two exercises were designed to target this process. The first exercise focuses on interference control and requires the child to make a response (left or right) depending on the direction a predefined target is facing. Difficulty level increases by introducing flanking nontargets that provide either congruent (e.g., facing the same direction as the target) or incongruent (e.g., facing the opposite direction) cues. The second executive attention exercise targets response inhibition and requires the child to press on the screen when a target appears, but to withhold responses when a nontarget appears.
In this study, Tali Train was delivered in the classroom by teachers as part of the school day. Students were provided with 7-in. touchscreen tablets and completed training simultaneously, with teachers providing occasional support when required. Completion of 20 out of 25 training sessions was required to constitute training compliance.
Placebo control
The placebo program was designed to control for the experience of using a touchscreen tablet, and to maintain blinding. As with Tali Train, the placebo program consists of four game-based exercises each 4 min in duration, and includes an inbuilt reward system. In contrast to Tali Train, the placebo exercises are nonadaptive and were designed to involve minimal attention skills. The placebo exercises involved (a) popping balloons on the screen, (b) dragging shapes across the screen, (c) pinching shapes to make them smaller, and (d) rotating shapes by pinching and spinning them on the screen. The reward system and motivational features in the placebo program were identical to those included in the Tali Train program. In the current study, teachers delivered the program as part of their usual teaching via 7-in. touchscreen tablets, with all children participating in the sessions concurrently.
No-contact control
Children assigned to this condition continued with usual classroom teaching and did not use any form of cognitive training program in the classroom during the study period.
Measures
Child characteristics
ADHD symptoms
The Conners’ 3 Parent Rating Scale–Long Form (Conners, 2008) is a 108-item standardized screening instrument of ADHD symptomology for children aged 6 to 18 years. Parents rated their child’s behavior over the past month on a 4-point scale on subscales relating to inattention and hyperactivity behavior (32 items). Standardized scores above 70 indicate very elevated ADHD symptoms. This measure has good internal consistency (Cronbach’s α = .71-.98; Conners, 2008).
Autism spectrum disorder (ASD) symptoms
The Social Responsiveness Scale (SRS; Constantino & Gruber, 2005) assesses ASD symptomology and is designed for children 4 to 18 years old. Parents rated their child’s behavior over the past 6 months on 65 items on a 4-point scale. Standardized scores above 76 are deemed as elevated.
Primary outcome measures
Cognitive attention
Five subtests of the Test of Everyday Attention for Children–Second Edition, (TEACh-2; Manly, Anderson, Crawford, George, & Robertson, 2017), which is designed for children aged 5 to 15 years, measured cognitive attention processes. Visual selective attention was measured by two subtests from the junior version of the TEACh-2 J: (a) Balloon Hunt involved four trials in which participants located and marked as many balloons as they could on a piece of paper in 15 s. The mean number of balloons located across trials was calculated (maximum of 48). (b) Hide & Seek Visual involved two trials in which participants inspected a series of panels and reported whether a target (red ball) was either present or absent. Participants had a 60-s time limit for each trial and the mean number of correct responses across the trials was calculated (maximum of 20). Auditory selective attention was measured by the Hide & Seek Auditory subtest, which asked participants to listen to 14 sound clips and indicate when they heard a dog bark by pressing a keyboard spacebar. Mean response time weighted for accuracy was calculated. Sustained attention was measured by the Simple Reaction Time (SRT) subtest and required participants to respond as quickly as possible when a blue blob appeared on the screen by pressing the keyboard spacebar. The mean response time in milliseconds was recorded. The Sustained Attention to Response Task (SART) subtest was used to assess executive attention. Although this subtest is described as a sustained attention task in the TEACh-2 J (which does not have any specific subtests to assess executive attention), the SART is commonly used to assess aspects of executive attention such as response inhibition (Johnson et al., 2007). The SART involved the random presentation of shapes on the screen at a regular pace. Participants responded to each shape by pressing the spacebar (go trial), but were instructed to withhold a response if the shape was a triangle (no-go trial). The total number of responses to no-go trials was recorded (commission errors). The raw score for each subtest was used in analysis.
Secondary outcome measures
Inattention and hyperactivity
The Strengths and Weaknesses of ADHD symptoms and Normal behavior scale (SWAN; Swanson et al., 2012) was used to assess inattention and hyperactivity. The SWAN consists of 18 items that map onto symptoms of ADHD and has been used in children between the ages of 4 and 18 years. Parents and teachers rated children’s behavior on each item over the last week on a 7-point scale, ranging from 3 = far below average to −3 = far above average. The first nine items relate to inattention and the last nine items relate to hyperactivity. A raw score for each domain was generated by totaling responses in each section, with higher scores indicating greater symptoms of either inattention or hyperactivity (maximum of 27).
Working memory
Subtests of the Automated Working Memory Assessment (AWMA; Alloway, 2007) were used to measure working memory skills. This computerized battery is designed for individuals aged 4 to 22 years. Verbal working memory was measured by the Backward Digit Recall subtest that required participants to recall progressively longer sequences of digits in the reverse order to that verbally presented. Visuospatial working memory was measured by the Odd One Out subtest, which required the participant to identify the “odd one out” in a set of three shapes. The participant was then presented with three empty boxes on the screen and asked to indicate where the odd shape had been located.
Numeracy
The Test of Early Mathematics Ability–III (TEMA-III; Ginsburg & Baroody, 2003) was used to measure mathematical abilities by assessing numbering skills, counting, number comparisons, calculation skills, and understanding concepts. This standardized measure contains 72 items and is designed for children between the ages of 3 and 8 years. A total raw score was calculated by totaling all correct responses.
Procedures
Principals from six primary schools within a 50-km radius of Monash University, Clayton campus, were approached, of which three consented to participate. Eight class teachers gave consent and their classes were randomized to one of three conditions (Tali Train, placebo control, no-contact control) stratified by school. Cluster randomization meant that once a class was randomized to a condition, all children within the class participated in the same condition. Randomization was performed by an independent project manager. Parents of students in participating classes were invited to provide informed consent. Classes ranged in size from five to 24 students, with a mean class size of 20 students. At baseline, participating students completed assessments of attention, working memory, and numeracy at school during class time, and teachers/parents completed questionnaires. Nonparticipating students in classes assigned to Tali Train or placebo conditions were given different activities (e.g., reading) during the 5-week training period. Within a week of training completion, posttraining assessments occurred, which were identical to the baseline assessments. Follow-up assessments occurred 6 months after training completion at which time principals, teachers, and parents were informed of condition assignment. Teacher questionnaires were completed by the child’s class teacher at the time of assessment. As the trial spanned two school years, the follow-up questionnaires were completed by a different teacher to the baseline and posttraining questionnaires.
Data Analysis
The data had three levels: observations over time, nested within children, and nested within classrooms. Although there were sufficient children to allow random effects by child, with only eight classrooms, random effects by classroom would be highly unstable. Therefore, latent growth models, fit within a structural equation modeling framework, were selected as the ideal analysis approach. The latent growth models included a latent intercept with freely estimated variance (functioning equivalently to a random intercept by child in a multilevel model framework). This accounts for nonindependence of observations over time within children. Two slopes (latent change scores) were estimated capturing changes (a) from baseline to posttraining and (b) from baseline to follow-up. These latent slopes incorporate baseline values so that any baseline differences in outcomes are accounted for by the model. Both slopes were allowed to differ by condition, to capture the Time × Condition interaction. Latent growth models fit within a structural equation modeling framework were chosen over multilevel models because, in addition to modeling change over time and accounting for individual differences by child, we could also (a) use full information maximum likelihood, a state-of-the-art approach to address missing data both on outcomes (which multilevel models also allow), and predictors or covariates (which multilevel models typically do not allow), and (b) use clustered standard errors rather than an additional level of random effects by classroom. Use of clustered standard errors by classroom rather than random classroom effects was needed due to unstable random effects with eight classrooms. However, to provide some indication of the relative variability between and within classrooms, intracluster correlation coefficients at each time point are presented in Table S1. Analyses were performed on an intention-to-treat basis, and models were estimated using robust maximum likelihood using MplusAutomation (Hallquist & Wiley, 2018) with Mplus version 8 and R version 3.5.1 (R Development Core Team, 2012).
Following recommendations for clinical trials (Feingold, 2009), effect sizes are reported as the difference in estimated means between conditions at posttraining and follow-up divided by the baseline standard deviation. To examine whether the trial had adequate power to detect significant changes in the primary outcome measure from baseline to posttraining, we conducted a post hoc power analysis. The power was 0.98 to detect a large effect (f = 0.40) and 0.71 to detect a medium effect (f = 0.25) with a total sample of 98 children. Our sample size of 29 to 38 children per group is well above the recommended minimum of 20 observations per group to assess the effects of training over time (Redick, Shipstead, Wiemers, Melby-Lervag, & Hulme, 2015; Simmons, Nelson, & Simonsohn, 2011) and is larger than recent classroom-based training studies (24-25 children per group; Landis, Hart, & Graziano, 2019).
Results
Child Characteristics
A total of 98 children aged between 5 years 6 months and 9 years 1 month (Mage = 7 years 8 months) were included in the analysis. The final data set contained 14.6% of data missing at random across all time points and conditions, indicating good retention and low concern for study validity (<20%; Schulz & Grimes, 2002). Differences in child characteristics at baseline were assessed across conditions using analyses of variance (ANOVAs; see Table 1), revealing no significant differences in IQ, ADHD or ASD symptoms. Each condition had mean standardized scores in the average range for ADHD and ASD symptoms (<60; see Table 1). Three participants had a parent-reported diagnosis of ASD, random allocation resulted in these participants each being assigned to one of the three conditions. Comparisons between children- with parent-reported ASD (n = 3) and without (n = 88) revealed expected differences in parent-reported ASD symptoms (M = 72.67 vs. M = 50.28, respectively; p = .001) and ADHD symptoms (M = 70.67 vs. M = 55.89, respectively; p = .024), but no significant differences in IQ or performance on primary (cognitive attention) or secondary outcome measures (working memory and numeracy) at baseline. Similarly, comparisons between children- with parent-reported very elevated (n = 10), elevated (n = 13), high average (n = 12), and average (n = 56) ADHD symptoms revealed no difference in IQ, or performance on primary or secondary outcome measures at baseline.
Caregiver responders consisted of mothers (78%), fathers (12%), and other legal guardians (2%). Of these responders, 75% had completed university education, and 2% had not completed high school education. The majority of caregivers were employed, in full-time (35%), part-time (32%), or casual (8%) employment, and those not in active employment were either full-time carers (14%) or unemployed and not seeking work (3%). Responder type, education level, and employment status of caregivers did not differ significantly across the three conditions.
Children in the Tali Train condition were significantly younger than children in either control condition (p = .001); therefore, analyses were conducted both with and without (unadjusted) age at baseline as a covariate. There were no significant differences in baseline performance on the primary outcome measures. However, differences across conditions were observed on the following secondary outcome measures: parent-rated hyperactivity (p = .032, placebo vs. no-contact control), verbal working memory (p = .034, Tali Train vs. no-contact control), and numeracy (p = .037, Tali Train vs. no-contact control). As the analyses estimated change scores from baseline to posttraining and baseline to follow-up, these differences at baseline were accounted for by the model.
Intervention Compliance
The average number of completed Tali Train sessions was 19 (range = 11-22), slightly below the recommended 20 sessions. Of the 38 children assigned to Tali Train, 22 children (58%) completed ≥20 training sessions. The main barriers to compliance related to the school environment and involved reasons such as school trips and child absence due to illness or holiday. Compared with noncompliers, compliers were younger (Mage = 6.06 years vs. Mage = 7.27 years, p < .001), and had lower ratings of inattention (M = 50.73 vs. M = 59.41, p = .02) and hyperactivity at baseline (M = 48.45 vs. M = 57.64, p = .008) although were still within the average range. There were no further differences between training compliers and noncompliers.
Intervention Effects
Mean scores and standard errors for the outcome measures at each time point are presented for the Tali Train and two control conditions in Table 2 (primary outcome measures) and Table 3 (secondary outcome measures). The regression coefficients for all outcome measures as functions of time, condition, and Time × Condition interaction (controlling for performance at baseline) are in Table 4. Results did not differ for unadjusted and adjusted (covarying for age at baseline) analyses and, therefore, unadjusted results are reported.
Means, Standard Errors, and Mean Difference Across Time for Primary Outcome Measures.
Note. All presented scores are raw scores from the Test of Everyday Attention for Children–Second Edition (TEACh-2). Diff. = between condition difference in the change over time; CI = confidence interval; SRT = Simple Reaction Time; SART = Sustained Attention to Response Task.
p < .05. **p < .01. ***p < .001.
Means, Standard Errors, and Mean Difference Across Time for Secondary Outcome Measures.
Note. All presented scores are raw scores. Working memory was assessed by the Automated Working Memory Assessment (AWMA). Parent- and teacher-rated hyperactivity and inattention were assessed by the Strengths and Weaknesses of ADHD symptoms and Normal behavior scale (SWAN). Numeracy was assessed by the Test of Everyday Mathematic Abilities (TEMA). Diff. = between condition difference in the change over time; CI = confidence interval.
p < .05. **p < .01. ***p < .001.
Regressions of Primary and Secondary Outcome Measures on Time, Condition, and Time × Condition Interaction.
Note. Attention was assessed by the Test of Everyday Attention for Children–Second Edition (TEACh-2). Parent- and teacher-rated hyperactivity and inattention were assessed by the Strengths and Weaknesses of ADHD symptoms and Normal behavior scale (SWAN). Working memory was assessed by the Automated Working Memory Assessment (AWMA). Numeracy was assessed by the Test of Early Mathematics Abilities (TEMA-II). Diff. = between condition difference in the change over time; CI = confidence interval; SRT = Simple Reaction Time. SART = Sustained Attention to Response Task.
p < .05. **p < .01. ***p < .001.
Primary outcomes
Selective attention
Significant improvements in visual selective attention performance were observed for all conditions from baseline to posttraining (p < .001). At follow-up, gains in performance on the Balloon Hunt subtest remained present for all conditions, and gains in performance on the Hide & Seek Visual subtest were present for the two control conditions (Table 2). There was no significant interaction between time and condition, indicating that across all conditions, children improved their visual selective attention at a comparable rate (Table 4). Significant improvements in auditory selective attention performance from baseline to posttraining, and baseline to follow-up were only observed for the no-contact control condition. A significant interaction effect from baseline to posttraining indicated the no-contact control condition made greater gains in auditory sustained attention than the Tali Train condition (p = .002), resulting in a medium to large effect size (d = 0.72) at posttraining.
Sustained attention
All conditions experienced a significant decline in sustained attention performance (SRT) from baseline to posttraining. This significant decline in performance remained at the follow-up assessment for the no-contact control condition (p < .001; Table 2). A significant interaction effect from baseline to follow-up indicated the Tali Train condition made greater gains in visual sustained attention than the no-contact control condition (p = .011) with a medium to large effect size (d = 0.61) at follow-up.
Executive attention
Significant gains in executive attention performance (SART) were observed for all conditions at each time point. No significant interaction effects were identified, suggesting that regardless of condition, all children improved in executive attention performance.
Secondary outcomes
Hyperactivity
Teacher-rated hyperactivity significantly declined in the Tali Train and no-contact control conditions from baseline to posttraining (p < .001) and from baseline to follow-up (p < .001; Table 3). A significant interaction effect from baseline to posttraining indicated greater reductions in teacher-rated hyperactivity for Tali Train than either placebo (p = .003) or the no-contact control (p = .001; Table 4), with medium to large (d = 0.71) and negligible (d = 0.08) effect sizes posttraining, respectively. Furthermore, significant interactions showed Tali Train to have greater reductions in teacher-rated hyperactivity from baseline to follow-up (p = .04, small to medium effect size, d = 0.29) and in parent-rated hyperactivity from baseline to posttraining (p = .005, negligible effect size d = 0.07) compared with the no-contact control condition.
Inattention
Tali Train was the only condition to experience a significant reduction in teacher-rated inattention from baseline to posttraining (p < .001) and from baseline to follow-up (p = .004). Significant reductions in parent-rated inattention were only observed in the placebo condition at posttraining (p < .001). Significant interaction effects from baseline to posttraining indicated greater reductions in teacher-rated inattention for Tali Train compared with either placebo (p < .001, large effect size d = 0.92) or no-contact control conditions (p < .001, small effect size d = 0.20). No interaction effects were present at follow-up.
Working memory
No improvements in working memory were observed for any condition from baseline to post. At follow-up, significant improvements in verbal working memory were observed for the no-contact control condition (p < .001) and visuospatial working memory for Tali Train (p = .009). There were no significant interaction effects between condition and time.
Numeracy
Numeracy skills significantly improved for Tali Train and the no-contact control from baseline to posttraining (p < .001). All conditions showed significant gains at follow-up (p < .001). No interaction effects were present, suggesting all children improved at a similar rate.
Sensitivity Analysis
Per-protocol analyses conducted with training compliers only (Tali Train; n = 22) showed significant interaction effects persisted for teacher-rated hyperactivity and inattention from baseline to posttraining for Tali Train compared with both the no-contact control (hyperactivity: p = .001, d = 0.24; inattention: p < .001, d = 0.06) and placebo (hyperactivity: p = .009, d = 0.55; inattention: p < .001, d = 0.79). Furthermore, the significant interaction effect for parent-rated hyperactivity from baseline to posttraining for Tali Train compared with the no-contact control condition (p = .003, d = 0.26) also persisted. The direction and magnitude of all persisting per-protocol interaction effects were consistent with the pattern of observed intention to treat effects. However, the interaction effect for teacher-rated hyperactivity from baseline to follow-up for Tali Train compared with the no-contact control condition was no longer significant during per-protocol analysis (p = .51, d = 0.45). Furthermore, the interaction effects between Tali Train and the no-contact control condition for sustained attention from baseline to follow-up (p = .22, d = 0.57) and auditory selective attention from baseline to posttraining (p = .09, d = 0.50) were no longer significant and the effect size much reduced.
Discussion
This is the first study to investigate the effects of classroom-based attention training in young primary school children. The results indicated that children who received attention training (Tali Train) demonstrated greater gains in one of the three aspects of attention studied, sustained attention, in the long term (6-month follow-up) relative to the no-contact control condition. Furthermore, attention training promoted greater reductions in inattentive and hyperactive behaviors in the classroom compared with both control conditions. The decline in hyperactive behavior in the classroom persisted in the long term (6-month follow-up) and was reduced in the home environment in the short term, relative to the no-contact control condition. There was no evidence that attention training in the classroom led to improvements in selective attention, executive attention, inattentive behavior at home, numeracy, or working memory, in the short- or long term. All children, regardless of condition, showed improvements in their visual selective attention, executive attention, and numeracy over the course of the trial.
Our findings are promising, suggesting that problem behaviors such as inattention and hyperactivity may be reduced in early childhood through intensive computerized attention training. These findings are consistent with prior research examining the efficacy of school-based attention training programs on inattentive and hyperactive behaviors in older children with developmental disorders (e.g., ADHD; Steiner et al., 2011; Tamm et al., 2013) and children with elevated attention difficulties (Rabiner, Murray, Skinner, & Malone, 2010). Our study extends these past findings by identifying the potential benefits of attention training when incorporated in to whole class activities for primary school children.
Despite the observed reductions in inattentive and hyperactive behaviors, we did not find transfer effects of attention training to our primary outcomes of selective and executive attention performance. Although small gains in sustained attention were observed in the attention training condition, these were only present at follow-up. These findings contrast previous investigations of attention training in children with developmental disorders and clinical attention difficulties, which have shown immediate and short-term (3-month follow-up) benefits to selective attention (Kirk et al., 2016). These discrepancies in near transfer to cognitive attention processes across studies may be due to fundamental differences in the participants who were investigated, such as children with or without an intellectual disability. There is increasing discussion in the literature that training effects may be influenced by moderating factors such as preexisting cognitive and attentional abilities, age, motivation (Jaeggi, Buschkuehl, Shah, & Jonides, 2014), ASD symptoms (de Vries, Verdam, Prins, Schmand, & Geurts, 2018), and sleep (Zinke, Noack, & Born, 2018). For example, some studies have found that children with reduced ability in the trained skill at baseline show greater training gains (Kray et al., 2011; Rabiner et al., 2010). Thus, it may be the case that attention training is most effective in promoting cognitive gains when implemented as a form of rehabilitation to alleviate cognitive impairment, rather than as an enhancer to strengthen cognitive skills. However, to date, very few attention training studies in children with sufficiently large sample sizes have examined the impact of individual differences on training effects (Lövdén, Brehmer, Li, & Lindenberger, 2012). Future research is required to determine the role of individual differences in children’s abilities on training performance, as well as outcome measures.
Training-related improvements in cognitive skills, such as attention, are suggested to occur as a result of intensive repeated practice on a specific task, which leads to a causal sequence of: Neural changes via plasticity, improvements in the trained cognitive domain (near transfer), and gains in untrained domains such as behavior and academic performance (far transfer). However, the neurobiological mechanisms that underlie this proposed transfer are still unknown (Wolf et al., 2018). The results of our study indicate no immediate gains in cognitive attention processes, broader cognitive or academic skills; yet, gains in behaviors such as inattention and hyperactivity were present. Thus, the mechanisms that underpin far transfer are likely to be complex and dependent on a number of internal and external factors. For instance, it is possible that students assigned to attention training were more engaged than children randomized to either control arm, due to the adaptive game elements of the training exercises, which, in turn, may have facilitated the observed gains in behavior. Future research is needed to test these assumptions regarding the potential influence of training features on training-related improvements to guide the design of future interventions aimed at promoting cognitive development. Further in this study, we measured select aspects of selective attention and executive functions. Thus, it is possible that changes in other unmeasured skills and domains occurred and promoted the behavioral training effects we observed.
The findings of this study should be considered in the context of its limitations. The trial was conducted over two school years, resulting in the majority of teachers who completed the follow-up questionnaire differing from the teachers who completed the baseline and posttraining questionnaires. This change in responders, although a common challenge in longitudinal research, may have affected results at follow-up because the timing of this assessment was at the beginning of the academic year when the teacher would have been less familiar with the child’s behavior. We acknowledge the subjective nature of behavioral rating scales and the need for caution when interpreting the results from such measures. Studies conducted within the same school year utilizing teacher-rated outcomes together with behavioral observation would be preferable to avoid this limitation. Second, for ecological validity and practicality, cluster randomization was used instead of individual randomization, which may have introduced complexity into the design, due to a lack of independence within each cluster. However, clustering was taken into consideration in the analyses, and our trial included a larger sample size than recent individual cluster training trials in children (Laine, Fellman, Waris, & Nyman, 2018; Landis et al., 2019). The variability in training dose (i.e., completed training sessions, which ranged from 11 to 22) across students may have affected training effects. Indeed, some training effects reduced when only training compliers were included in the analysis. Although the importance of dose for cognitive training effects, particularly in childhood, is still unclear (Ball, Ross, Roth, & Edwards, 2013), recent studies in adults suggest that cognitive training is likely to improve cognition in a dose-responsive manner (Bamidis et al., 2015). As such, systematic delineation of optimal training dosage across a range of settings and populations is a worthwhile pursuit for future research and will help inform the threshold for training success. Furthermore, variability in our sample of children may have affected our training outcomes. Although all children attended a mainstream primary school and none had the support of an aide in the classroom, three children had a parent-reported developmental disorder (e.g., ASD), and, as such, may have responded differently to the intervention. However, as this is the first study to trial whether a classroom-based attention training intervention could be used by teachers in whole classes of children, it was important to assess whether the program was appropriate for children in a typical classroom with a range of abilities. Future research with larger samples sizes of children should endeavor to explore potential differential training responses in subgroups of children.
The current study has several important strengths: Attrition was low, with 90% of children in the training condition attending each assessment, and compliance was high, with an average of 76% training sessions completed. The design of this trial implemented several of the recommendations suggested by Simons et al (2016) in their extensive review of cognitive training studies: (a) Classes were randomly assigned to conditions to reduce the influence of uncontrolled factors; (b) two control groups were employed to determine training effects, including a placebo that was closely matched to the training condition, and a no-contact control condition; (c) participants and researchers were blinded to condition allocation to reduce bias; (d) outcome measures distinct from the training tasks were conducted at baseline, as well as short- and long-term follow-ups; and (e) the trial was preregistered and did not deviate from the proposed plan. Furthermore, implementation fidelity (the degree to which the program was delivered as intended) was high, with all training sessions being conducted within the classroom, and the target for training adherence of five sessions per week over the 5-week training schedule being largely achieved. These factors provide encouragement that the training program could be implemented outside of a highly controlled randomized controlled trial and applied as a group-based activity in an educational setting.
In summary, our findings provide initial evidence that computerized attention training conducted in the school environment can lead to greater short-term reductions in inattentive and hyperactive behavior in the classroom for children in mainstream primary school compared with a no-contact control or a placebo control. Furthermore, attention training promoted modest reductions in hyperactive behavior at home in the short term when compared with a no-contact control condition. However, our findings provide little evidence that attention training can enhance performance in trained cognitive attention processes or untrained executive or academic functions among primary school children. Collectively, these findings indicate that attention training may have select benefits for improving attentive behaviors, and are important in highlighting the feasibility of implementing cognitive training in groups within a primary school setting. It would be informative for future studies to systematically measure the role of individual differences and training dosage, as predictors of training-related improvements, to understand whether certain subgroups of children experience greater benefits and whether this is influenced by duration and frequency of training.
Supplemental Material
Supplementary_File_JAD_20190618_ab – Supplemental material for Gamified Attention Training in the Primary School Classroom: A Cluster-Randomized Controlled Trial
Supplemental material, Supplementary_File_JAD_20190618_ab for Gamified Attention Training in the Primary School Classroom: A Cluster-Randomized Controlled Trial by Hannah E. Kirk, Megan Spencer-Smith, Joshua F. Wiley and Kim M. Cornish in Journal of Attention Disorders
Footnotes
Acknowledgements
The authors thank the families and schools who participated; Catholic Education Melbourne for their support; Richard Meagher, Yan Yang, and Rosemary Yates for their assistance with data collection; Sally Richmond for her support with data analysis; and the research interns, Alice MacDonald, Ashley Grigoriadis, Eugenie Edillo, Gabriel Rae, Grace Evans, and Orianne Rais, for their dedication to the project.
Declaration of Conflicting Interests
The author(s) declared potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The first author (Kirk) and last author (Cornish) are listed as co-inventors on an international patent for the Tali Train program. The research was conducted in the absence of any commerical or financial relationship that could be construed as a potential conflict of interest.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding was provided by the Australian Department of Innovation, Industry and Science.
Supplemental Material
Supplemental material for this article is available online.
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
