Abstract
Executive function challenges are commonly reported in the home setting for children with an autism spectrum disorder diagnosis (hereafter, autism), but little is known about these challenges in the school setting. A total of 337 youth (autism, N = 241 and typically developing, N = 96) were assessed using Behavior Rating Inventory of Executive Function ratings from home and school settings. Within each setting, we examined differences in specific executive function skills between diagnostic groups. Then, we examined if the autism group showed similar peak executive function impairments, associations with age, and relationships with adaptive behavior across settings. Finally, we examined inter-rater reliability. Autism and typically developing groups differed on all Behavior Rating Inventory of Executive Function scales in both settings. The Shift scale was the peak impairment in the autism group in both settings. There was also an effect of age on executive function impairment in both settings, and executive function ratings in both settings significantly predicted individual adaptive behavior domains. Inter-rater correlations for autistic participants were similar to inter-rater reliability correlations from the Behavior Rating Inventory of Executive Function standardization sample. This study shows that autistic children experience similar but not identical real-world executive function challenges across school and home settings and that supports may vary by setting.
Lay abstract
Executive functioning skills are important for children to be able to control thoughts, emotions, and actions, and for their ability to adapt to different settings. Previous studies have found that autistic children have difficulty with executive functioning, but most of these studies took place in the home setting. Executive functioning skills in autistic children have not yet been fully examined in the school setting. This is a knowledge gap that needs to be addressed because the school setting is potentially one of the most demanding in terms of executive functioning and is a setting where executive function interventions are already being used for autistic children. In order to learn more about executive functioning for autistic children in the school setting, this study compared ratings of executive function from autistic children’s caregivers and school professionals. The study found that autistic children experience similar but not identical executive function challenges across school and home settings and that supports may differ in each setting. Having information about differences in executive functioning challenges and supports for autistic children across settings can help lead to development of supports that are tailored to each setting.
Executive functioning skills are critical for children to be able to regulate thoughts, emotions, and actions, and consequently adaptive and academic behaviors and outcomes. These skills consist of the abilities to store information in working memory, flexibly shift focus, and inhibit irrelevant responses and are often impaired in children with an autism spectrum disorder diagnosis (hereafter, autism) (Blair, 2016). Historically, executive function impairment has been thought of as a pivotal weakness explaining all of autism, and more recently as playing a role in cognitive and behavioral inflexibility (Geurts et al., 2009; Kenworthy et al., 2008). Executive function is a critical predictor of outcomes for autistic individuals, including play skills (Faja et al., 2016), language skills (Friedman & Sterling, 2019) academic readiness and functioning (Pellicano et al., 2017; St. John et al., 2018), adaptive functioning (Bertollo & Yerys, 2019; Gardiner & Iarocci, 2018; Pugliese et al., 2015), and independence and quality of life as an adult (Bishop-Fitzpatrick et al., 2016).
Executive function impairment in autistic children 1 is shown reliably regardless of whether you measure executive function in the lab with a performance-based test or through caregiver ratings of real-world performance (Lai et al., 2017; Wallace, Yerys et al., 2016). Accordingly, extant research has continued to use both performance-based and informant ratings to examine executive function impairment in autism. Furthermore, recent meta-analyses support executive function impairments in autism across multiple domains, such as working memory, flexibility, and planning (Demetriou et al., 2018; Lai et al., 2017).
Most of the studies on real-world executive function impairment have used caregiver informant measures, which capture behavior from the home setting and potentially other community settings (hereinafter, “home setting” for simplicity; Wallace, Kenworthy et al., 2016). These studies have noted a hallmark difficulty with shifting attention or deviating from routines (Granader et al., 2014; Rosenthal et al., 2013). For example, caregivers of autistic children may report that their child gets stuck on certain topics or activities (Gioia et al., 2000). While broad executive function impairments characterize autism spectrum disorder (ASD), shifting has been proposed as a promising clinical marker from caregiver informant data given its notable ability to differentiate autistic and typically developing (TD) individuals (Leung & Zakzanis, 2014). Moreover, caregiver informant studies report that real-world executive function challenges among autistic children increase with age and associate with adaptive behavior weaknesses across the range of intellectual functioning (Bertollo & Yerys, 2019; Gardiner & Iarocci, 2018 Pugliese et al., 2015; Rosenthal et al., 2013; Wallace, Kenworthy, et al., 2016). Related studies have also indicated that the association between executive function and adaptive behavior may be especially strong for autistic girls and be an important intervention target for this group (Torske, 2020; White et al., 2017).
While these caregiver ratings provide invaluable information about behavior in the home setting, they are unable to answer questions about or establish convergence with executive function in the school setting. Although a large sample of autistic children have performed worse than non-autistic children on performance-based measures of executive function given in the school setting (e.g. inhibition, working memory, and affective decision-making; Kouklari et al., 2018), information on informant measures of real-world executive function in the school setting is still preliminary. To our knowledge, only one prior study has examined real-world executive function for autistic children in the school setting, and it was limited by a small sample without a non-autistic comparison group (Freeman et al., 2017).
This is an important knowledge gap as some authors have critiqued performance-based measures as poor estimates of executive functioning in the real world, and informant report measures can provide useful complementary information when assessing executive function (Isquith et al., 2013). Qualifying convergence across executive function ratings from home and school settings is also needed as prior research shows poor inter-rater reliability between home and school ratings of child behavior (Achenbach et al., 2005; Azad & Mandell, 2016; Kaat & Lecavalier, 2015; Mitsis et al., 2000; Power et al., 1998; Stratis & Lecavalier, 2015). Furthermore, the school setting is unique due to a higher level of structure, which has the potential to either mitigate or exacerbate executive function impairments. Indeed, executive functioning relates to school readiness in autism (St. John et al., 2018), and school is a setting where evaluations and executive function interventions are already being successfully implemented (Kenworthy et al., 2014; Tamm et al., 2020).
Thus, we have examined informant ratings of executive function from both school and home settings for school-age autistic children to gain further insight into the stability of impairments and associations with age and adaptive behavior functioning.
This investigation addressed five key questions:
Do autistic and TD children differ on executive function ratings across home and school settings? Given robust differences identified in the home setting (Leung & Zakzanis, 2014; Wallace, Yerys, et al., 2016), we hypothesize this finding in both settings.
Do raters in the school setting identify shift skills as a peak impairment in autistic children? This is the case for caregiver ratings in the home setting (Granader et al., 2014; Rosenthal et al., 2013), and we hypothesize this finding will extend to the school setting.
Does real-world executive function impairment increase with age in the school setting? This age-associated change occurs in the home setting (Rosenthal et al., 2013), and we hypothesize it will also occur in the school setting.
Do executive function ratings in both settings explain concurrent adaptive behavior ratings in both settings? This has been found in the home setting previously (Bertollo & Yerys, 2019; Gardiner & Iarocci, 2018; Pugliese et al., 2015).
Are school and home ratings similar or different for autistic children across specific executive function skills (inter-rater reliability)? The Behavior Rating Inventory of Executive Function (BRIEF) manual and meta-analyses on other informant reports (e.g. Child Behavior Checklist (CBCL)) suggest modest inter-rater correlations (Gioia et al., 2000; Stratis & Lecavalier, 2015). Here, note that the term inter-rater reliability has been used to refer to rater agreement across settings.
In sum, the present investigation sought to extend previous findings on real-world executive functioning in autistic children from the home setting to the school setting and examine similarities and differences between settings. Importantly, such knowledge could advance identification of executive function impairment in autistic children and better inform setting-specific evaluation and interventions.
Method
Participants
A total of 337 participants (autism, N = 241 and TD, N = 96) between the ages of 6 and 18 years were included in this study, which enrolled children between 2009 and 2015. The inclusion criteria were children between the ages of 6–18 years and expert clinical judgment confirming that all children in the autism group met the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR) criteria for either autism, Asperger’s syndrome, or pervasive developmental disorder-not otherwise specified (American Psychiatric Association, 2000). Expert clinical best estimates were based on a DSM-IV-TR and the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-V) checklist and informed by the Autism Diagnostic Observation Schedule (ADOS), the Autism Diagnostic Interview–Revised (ADI-R; Lord et al., 1994, 2000), Vineland Adaptive Behavior Scales–Second Edition (VABS-II; Sparrow et al., 2005), and clinical impressions. Thus, it was possible for some children to not meet criteria on the ADOS but still be included in the ASD group. This method of diagnostic inclusion was used given that it parallels the broad autism diagnostic criteria used by the Collaborative Programs of Excellence in Autism network (Lainhart et al., 2006) and results in a more representative sample of the autistic population at large.
Exclusion criteria were assessed using caregiver report on a family and medical history form. For the autism group, exclusion criteria were extreme prematurity (gestational age <32 weeks), seizures, tic disorder, or an acquired head injury, and parent report of a genetic variation with known clinical significance (e.g. Fragile X and Down syndrome). In addition to the exclusion criteria above, children in the TD group were also excluded for taking psychoactive medication and elevated scores on the Attention Deficit Hyperactivity Disorder (ADHD) Rating Scale-IV (>85th percentile based on age- and sex-correction per the manual; DuPaul et al., 1998) or the Child and Adolescent Symptom Inventory–Fourth Edition Revised (>1.5 standard deviations (SDs) above the age- and sex-corrected mean per the manual; Gadow & Sprafkin, 2010).
A portion of the sample was reported previously in articles investigating association between parent ratings of executive functioning and adaptive behavior in autistic youth with lower intelligence quotient (IQ) (Bertollo & Yerys, 2019) and the factor structure of the parent BRIEF (Granader et al., 2014). Overall participant characteristics by diagnostic group can be found in Table 1.
Participant characteristics (M (SD); range).
SD: standard deviation; TD: typically developing; IQ: intelligence quotient; ADHD: attention deficit hyperactivity disorder; ADOS CSS: Autism Diagnostic Observation Schedule Calibrated Severity Score; ADI-R: Autism Diagnostic Interview–Revised.
n = 233.
n = 8 not reported.
n = 0 not reported.
n = 15 not reported.
n = 10 not reported.
n = 227.
n = 91.
n = 237.
n = 95.
n = 235.
n = 178.
n = 230.
p < 0.001.
Measures
Executive function
Ratings of executive function in the school and home setting were measured using the teacher and parent versions of the BRIEF, respectively (hereinafter, “School BRIEF” and “Home BRIEF”) (Gioia et al., 2000). For school, families chose the teacher or school professional who knew the child best as the rater, and for home, families chose the caregiver who knew the child best as the rater. The BRIEF captures executive function in everyday situations and is comprised of 86 items that fall into eight scales which are collapsed into an overall General Executive Composite (GEC), which has two underlying broad indices: the Behavioral Regulation Index (BRI) and the Metacognition Index (MI). The BRI includes Inhibit, Shift, and Emotional Control scales, and the MI includes the Initiate, Working Memory, Plan/Organize, Organization of Materials, and Monitor scales. Items have a 3-point Likert-type scale rating, including 1 (never), 2 (sometimes), and 3 (often). Higher raw and T-scores scores indicate greater impairment; T-scores
Adaptive behavior
Ratings of adaptive behavior in the school and home setting were measured using the teacher and parent versions of the VABS-II survey, respectively (hereinafter, “School VABS-II” and “Home VABS-II”) (Sparrow et al., 2005). The School VABS-II was added to the study after the School BRIEF once we learned that school professionals did not feel overburdened with the study; thus, the School VABS-II has a smaller available sample compared to the School BRIEF. Consistent with rater selection for the BRIEF, families chose the teacher or school professional who knew the child best as the school rater and the caregiver who knew the child as the home rater. Here, standard VABS-II scores (M = 100 and SD = 15) are reported for three domains: Communication, Daily Living, and Socialization. Lower standard scores represent worse adaptive skills, where a score between 71 and 85 (SD = 1.0–2.0) represents moderately low skill level and a score of 70 or lower (SD ⩾ 2.0) indicates low skill level.
Autism characteristics 2
The ADI-R is a semi-structured interview with caregivers of children and adults to assess the presence of ASDs (Lord et al., 1994). The caregiver is asked about their child’s current behavior in daily life, and scores are given according to the degree to which indicated behaviors are present. Scores are given for behaviors in reciprocal social interaction, non-verbal and verbal communication, and restricted and repetitive behaviors domains. For diagnostic criteria for autism to be met, behaviors in each domain must meet criteria and abnormality in at least one domain before 36 months of age must be present.
The ADOS is a semi-structured interview consisting of modules that assess social interaction, communication, play and imaginative use of materials to determine whether an individual meets the criteria for an autism diagnosis. Scores are given for Social Affect and Restricted and Repetitive Behavior domains, which are then totaled and converted into the Calibrated Severity Score (Gotham et al., 2009; Hus & Lord, 2014 CSS). CSS scores range from 1 to 10, with higher scores indicating more pronounced autism characteristics. The CSS from the ADOS was used as a covariate for autism characteristics in the present analyses.
ADHD characteristics
The total raw scores from the ADHD Rating Scale-IV: School Version and ADHD Rating Scale-IV: Home Version were used as covariates for ADHD characteristics in the present analyses (DuPaul et al., 1998). The ADHD Rating Scale-IV is a questionnaire based on the diagnostic criteria for ADHD where each of the nine inattention and nine hyperactivity/impulsivity behaviors are rated on a 4-point Likert-type scale. Caregivers and teachers/school personnel complete the form. Total raw scores can range from 0 to 54, with higher scores indicating more pronounced ADHD characteristics.
Cognitive ability
Cognitive ability was also used as a covariate in the present analyses and was assessed with the Differential Abilities Scale–Second Edition (DAS-II; Elliott, 2007) or the Wechsler Intelligence Scale for Children–Fourth Edition (WISC-IV; Wechsler, 2003). Each of the assessments has a mean standard score of 100 and an SD of 15 (M = 100 and SD = 15).
Procedure
Participants were recruited from our hospital’s primary care network, specialty clinics, and the community. They were all enrolled and tested at our hospital-based research center between the years of 2009 and 2015 as part of one or more case–control studies approved by the Institutional Review Board at The Children’s Hospital of Philadelphia. During these studies, participants completed various diagnostic, cognitive, and neuropsychological assessments, while the teacher or school professional and caregiver raters completed questionnaires.
The secondary analyses conducted in this study include all of the participants for whom a School BRIEF was completed. This was to ensure executive function ratings in the school setting could be analyzed. A minority of 38 ASD and 5 typically developing children (TDC) eligible participants across studies did not have a School BRIEF completed. For 25 of these ASD participants, IQ scores were lower than 75, and thus, study principal investigators (PIs) in consult with study clinicians may have elected not to send out Teacher BRIEF forms given that the BRIEF has not been validated in an intellectual disability population. All participants gave informed consent for the use of their data and this secondary analysis study was approved by the Institutional Review Board at The Children’s Hospital of Philadelphia. Community members were not included in the design of the present secondary analyses; however, community member involvement would be an important future step.
Analysis plan
To assess whether the autistic and TD groups differed on executive function ratings across home and school settings, we conducted multivariate analyses of covariance (MANCOVAs) using the School BRIEF and Home BRIEF ratings for the eight subscales while controlling for age, gender, IQ, and ADHD characteristics. Given robust differences between autistic and TD groups identified in the home setting, the a priori hypothesis was that findings would be replicated in both settings (Leung & Zakzanis, 2014; Wallace, Yerys et al., 2016).
Next, we moved onto analyses focused on the autism group. First, we conducted a one-way analysis of covariance (ANCOVA) within the autism group to examine peak impairment across the eight subscales from School BRIEF and Home BRIEF ratings while controlling for age, gender, IQ, autism characteristics, and ADHD characteristics. The a priori hypothesis was that the Shift scale would have significantly higher scores compared to other scales in autistic participants, consistent with previous observations using Home BRIEF ratings (Granader et al., 2014; Rosenthal et al., 2013).
Next, we examined the effect of age on School and Home BRIEF ratings within the autism group. The a priori hypothesis was that executive function would be rated as more impaired as age increased. Replicating the approach of Rosenthal et al. (2013), a mixed-model analysis of variance (ANOVA) with age group (5–7, 8–10, 11–13, and 14–18) as the between-subjects factor and BRIEF scales as the within-subjects factor was conducted. Due to the present sample only including children 6 years and older, the youngest age range was adjusted to include children between the ages of 6 and 7 years. Subsequent one-way ANOVAs and Tukey’s honestly significant difference (HSD) tests were then planned for post hoc analyses.
Then, to assess whether school setting ratings of executive function in specific areas were predictive of school setting ratings of adaptive behavior in autistic participants beyond participant age, gender, IQ, autism characteristics, gender and autism characteristics interactions, and ADHD characteristics, we conducted a series of linear regression models using School BRIEF and School VABS-II ratings. Parallel linear models were also conducted using Home BRIEF and Home VABS-II ratings. There were 215 autistic participants and 86 TD participants included in analyses requiring School VABS-II ratings. Two TD participants were excluded from analyses requiring Home VABS-II ratings
Finally, we examined inter-rater reliability of BRIEF ratings in the school and home settings within the autism group by computing intra-class correlation (3,1) coefficients and compared our results to parent-teacher inter-rater reliability from the BRIEF standardization sample using Fisher’s r-to-z transformations (Table 21; Gioia et al., 2000).
Prior studies in autistic children and adults have not removed participants from the autism or TD group for having an “Inconsistent” score on the inconsistency protocol of the BRIEF (Bertollo & Yerys, 2019; Freeman et al., 2017; Gioia et al., 2002; Granader et al., 2014; Isquith et al., 2013; Leung & Zakzanis, 2014; Pugliese et al., 2015; Rosenthal et al., 2013; Wallace, Kenworthy et al., 2016). We conducted a secondary analysis removing children with those scores on either the School BRIEF or Home BRIEF for sensitivity analyses (see Tables S3–S6). This excluded 15 autistic participants and 8 TD participants.
Results
Group characteristics
The autism and TD groups were matched on age, gender ratio, race ratio, and ethnicity ratio, but differed on full-scale IQ and ADHD characteristics (see Table 1). Specific data on socioeconomic status were not included.
Group differences in executive function ratings in school and home
The MANCOVAs controlling for age, gender, IQ, and ADHD characteristics revealed significant diagnostic group differences for both School BRIEF and Home BRIEF ratings. (Fs > 32, all p < 0.001) (see Table 2). All scales were available for the full sample size in each setting, thus indicating no missing items.
BRIEF ratings by diagnostic group and setting (t-scores (M (SD))).
BRIEF: Behavior Rating Inventory of Executive Function; SD: standard deviation; TD: typically developing; GEC: General Executive Composite; BRI: Behavioral Regulation Index; MI: Metacognition Index.
Peak impairment in autism executive function ratings in school and home
The one-way ANCOVA conducted within the autism group on the School BRIEF revealed a main effect of scale (F(7, 1568) = 27.53, p < 0.0001, η2 = 0.06) after controlling for age, gender, IQ, autism characteristics, and ADHD characteristics. Subsequent planned pairwise comparisons revealed that Shift was rated significantly higher (i.e. more impaired) than the other seven scales (all p < 0.01). Both Emotional Control and Monitor were found to be rated significantly higher than Inhibit, Plan/Organize, and Organization of Materials (all p < 0.05), and Initiate and Working Memory were found to be rated significantly higher than Inhibit and Organization of Materials (all p < 0.05).
The one-way ANCOVA conducted within the autism group on the Home BRIEF also revealed a main effect of scale (F(7, 1505) = 34.79, p < 0.0001, η2 = 0.07) after controlling for age, gender, IQ, autism characteristics, and ADHD characteristics. Subsequent planned pairwise comparisons revealed that the Shift scale was rated significantly higher than all other scales (all p < 0.05). The Working Memory scale was also found to be rated significantly higher than Emotional Control and Plan/Organize (all p < 0.05). In contrast, the Organization of Materials scale was rated significantly lower than all other scales (all p < 0.05).
Effect of age on autism executive function ratings in school and home
The mixed-model ANOVA with age group as the between-subjects factor and School BRIEF scale as the within-subjects factor indicated a main effect for age group (F(3, 237) = 4.82, p < 0.01) and a main effect for scale (F(7, 1659) = 32.36, p < 0.0001), as well as an interaction between age group and scale (F(21, 1659) = 4.03, p < 0.0001). In contrast, the Home BRIEF revealed a main effect for scale (F(7, 1582) = 38.14, p < 0.0001) and an interaction between age group and scale (F(21, 1582) = 2.20, p < 0.01). Examination of heteroskedasticity did not suggest any substantial issues given all F-max < 3.2.
Subsequent one-way ANOVAs using the School BRIEF revealed age effects for the following scales: Shift (F(3, 237) = 11.05, p < 0.0001), Plan/Organize (F(3, 237) = 6.48, p < 0.001), Organization of Materials (F(3, 237) = 5.48, p < 0.01), and Monitor (F(3, 237) = 3.39, p < 0.05) (see Table 3 for post hoc results).
Age differences in autism School BRIEF ratings.
BRIEF: Behavior Rating Inventory of Executive Function; NS: not significant.
p < 0.05; **p < 0.01; ***p < 0.001.
Subsequent one-way ANOVAs using the Home BRIEF revealed age effects only for the Organization of Materials scale (F(3, 226) = 2.73, p = 0.04) and post hoc analyses using Tukey’s test revealed significantly lower (i.e. less impaired) Organization of Materials scores in the 14–18 year olds compared to the 8–10 year olds (p = 0.04) (see Table S1 in Supporting Information).
Autism executive function and adaptive behavior ratings in school and home
The initial linear regression models conducted with the covariates of no interest (age, gender, IQ, and autism characteristics) significantly predicted each of the school-rated adaptive behavior domains: Communication, Daily Living Skills, and Socialization (overall models: Fs > 11, all p < 0.0001, all R2s > 0.21). Here, we use “covariate of no interest” to reflect extant knowledge of these covariates already accounting for substantial variance, and thus needing to be included in baseline models (Bertollo & Yerys, 2019). In addition, we included an interaction term between gender and autism characteristics given recent research indicating strong relationships between executive function and adaptive behavior for girls on the autism spectrum (Torske, 2020; White et al., 2017). There was a significant change in R2 (ΔR2) in each of the School VABS-II domains when School BRIEF scales were added into the models: Communication (F(8, 160) = 4.05, p < 0.001, ΔR2 = 0.08), Daily Living Skills (F(8, 158) = 5.49, p < 0.0001, ΔR2 = 0.12), and Socialization (F(8, 176) = 9.34, p < 0.0001, ΔR2 = 0.22) (see Table 4). When the baseline model also included ADHD characteristics (overall models: Fs > 20, all p < 0.0001, all R2s > 0.33), only the Socialization scale had a significant change in R2 (ΔR2) when School BRIEF scales were added into the models (F(8, 172) = 3.84, p < 0.001, ΔR2 = 0.10).
Linear regressions: autism participants.
VABS-II: Vineland Adaptive Behavior Scales–Second Edition; SE: standard error; IQ: intelligence quotient; ADOS CSS: Autism Diagnostic Observation Schedule Calibrated Severity Score; BRIEF: Behavior Rating Inventory of Executive Function.
The change in R2 is significant for Model 2 versus Model 1 in each adaptive behavior domain. Model 2 includes covariates and the School BRIEF.
p < 0.05; ***p < 0.001.
Similarly, the initial linear regression models conducted with the covariates of no interest (age, gender, IQ, and autism characteristics) and the gender and autism characteristics interaction term significantly predicted each of the home-rated adaptive behavior domains: Communication, Daily Living Skills, and Socialization (overall models: Fs > 12, all p < 0.0001, all R2s > 0.20). There was a significant ΔR2 in each of the Home VABS-II domains when Home BRIEF scales were added into the models: Communication (F(8, 203) = 8.11, p < 0.0001, ΔR2 = 0.13), Daily Living Skills (F(8, 205) = 2.50, p < 0.05, ΔR2 = 0.07), and Socialization (F(8, 205) = 10.72, p < 0.0001, ΔR2 = 0.23) (see Table S2 in Supporting Information). When the baseline model also included ADHD characteristics (overall models: Fs > 12, all p < 0.0001, all R2s > 0.24), there was a significant ΔR2 in each of the Home VABS-II domains when Home BRIEF scales were added into the models: Communication (F(8, 198) = 4.30, p < 0.0001, ΔR2 = 0.07), Daily Living Skills (F(8, 200) = 2.00, p < 0.05, ΔR2 = 0.06), and Socialization (F(8, 200) = 6.97, p < 0.0001, ΔR2 = 0.16).
Convergence of autism executive function ratings in school and home
For autistic participants, results from intraclass correlation coefficient (ICC) (3,1) computations indicated that there was a significant positive association between School and Home BRIEF ratings of GEC (ICC = 0.22, p < 0.001), BRI (ICC = 0.30, p < 0.0001), MI (ICC = 0.30, p < 0.0001), and seven of the eight underlying scales (no significant association was found for the Initiate scale): Inhibit (ICC = 0.37, p < 0.001); Shift (ICC = 0.22, p < 0.001); Emotional Control (ICC = 0.30, p < 0.0001); Initiate (ICC = 0.04, p = 0.27); Working Memory (ICC = 0.23, p < 0.001); Plan/Organize (ICC = 0.25, p < 0.0001); Organization of Materials (ICC = 0.32, p < 0.0001); and Monitor (ICC = 0.28, p < 0.001). In addition, Fisher’s r-to-z transformations revealed that there were no significant differences between the present correlations and inter-rater reliability correlations from the standardization data set in the BRIEF Manual (Table 21; Gioia et al., 2000) for autistic participants, with the exception of the Inhibit scale (p < 0.05), Initiate scale (p < 0.05), Organization of Materials scale (p < 0.01), and Monitor scale (p < 0.05) (see Table 5).
Home and school inter-rater correlations for BRIEF standardization sample and autism group.
BRIEF: Behavior Rating Inventory of Executive Function; CI: confidence interval.
p < 0.05.
Discussion
This investigation currently provides the most comprehensive assessment of real-world executive function in the school setting for children on the autism spectrum using a large pediatric sample. This is critical as school is a setting with a fairly rigid structure that may provide both unique executive function challenges and supports for autistic children. We addressed five key questions about real-world executive function using informant measures in the school setting, and thus enhanced prior knowledge from performance-based tests (Isquith et al., 2013). For Question 1 (diagnostic group differences), we found that all executive function ratings in the School BRIEF and Home BRIEF differed between autism and TD groups. For Question 2 (Shift as a peak impairment), we found the Shift scale to be rated as a peak impairment for the autism group in both the school and home settings. For Question 3 (executive function impairments increase with age), we found that individual executive function skills generally showed a widening gap between autism and TD groups across age in the school setting, but not the home setting. For Question 4 (association with adaptive behavior functioning), we found that School BRIEF and Home BRIEF ratings were predictive of adaptive behavior ratings in the corresponding setting for the autism group. For Question 5 (inter-rater reliability), we found significant correlations for all scales on the BRIEF but Initiate in the autism group. We also found no significant differences between present correlations in the autism group and inter-rater reliability correlations from the BRIEF standardization sample on Shift, Emotional Control, Working Memory, Plan/Organize, but not the Inhibit, Initiate, Organization of Materials, and Monitor scales. This finding may indicate that autistic children have more variable presentation across settings compared to the neurotypical population or that the latter four scales may yield less reliable ratings across settings than what is found in the neurotypical population. Thus, this study was able to extend findings of executive function impairments in autism from the home setting to the school setting while also revealing similarities and differences across settings.
Our findings that real-world executive function is impaired in the school setting for autistic children and that Shifting abilities are a peak impairment converge with prior studies in the home setting (Gioia et al., 2002; Granader et al., 2014; Rosenthal et al., 2013; Smithson et al., 2013; Yerys et al., 2009). In addition, our findings that executive function impairments increase in older age groups in the school setting also converge with prior findings from the home setting (Rosenthal et al., 2013).
However, Shift, Plan/Organize, Organization of Materials, and Monitor scales were rated as more impaired in older age groups in the school setting in this study whereas only the Initiate and Working Memory scales have previously been rated as more impaired in older age groups in the home setting (Rosenthal et al., 2013). Moreover, Rosenthal and colleagues’ (2013) study did not reveal the Organization of Materials scale to increase in older age groups in the home setting. These differences may be partially due to differing cognitive abilities across studies. Rosenthal and colleagues’ (2013) study only included autistic participants with an IQ score above 70 whereas this study included a broad IQ range from 30 to 158. Alternatively, the differences may be due to Rosenthal et al. (2013) using ratings from both clinic and research evaluations. It is possible caregivers may rate behaviors differently in a clinical setting prior to obtaining a diagnosis than in a research setting where diagnosis and care has already been established. Nonetheless, the present results from the school setting extend findings that some executive function abilities in autistic children may have greater separation from the standardization sample at older ages, consequently contributing to a more noticeable divergence with neurotypical peers during adolescence (Rosenthal et al., 2013).
The present finding that the autism group’s executive function ratings in the home and school settings predict corresponding adaptive behavior ratings in Communication, Daily Living Skills, and Socialization domains also extends prior research. This study is the first to show this relationship between executive functioning and adaptive behavior in the school setting, but both concurrent and longitudinal prediction studies have demonstrated this relationship in the home setting (Bertollo & Yerys, 2019; Gilotty et al., 2002; Gardiner & Iarocci, 2018; Pugliese et al., 2015). While prior studies have indicated that the relationship between executive function and adaptive behavior may be stronger for autistic females compared to autistic males, this study found executive function in specific areas to be predictive of school setting ratings of adaptive behavior in autistic participants beyond gender and a gender and autism characteristics interaction (Torske, 2020; White et al., 2017). However, the limited number of females and binary measure of gender in the current sample should be considered. Overall, these findings reveal that caregivers and school professionals are observing similar real-world executive function profiles and related impairments in autistic children.
However, the addition of ADHD characteristics as a covariate led to executive function ratings only predicting adaptive behavior ratings of socialization in the school setting and all adaptive behavior ratings in the home setting. This pattern of findings indicates that there may be an aspect of ADHD characteristics that is unique from executive function ratings and differentially associated with adaptive behavior weaknesses in the context of ASD. Understanding the relationship of ADHD and adaptive functioning in both autistic and non-autistic children could further inform differential diagnosis among ASD, ADHD, and co-occurring ASD and ADHD diagnoses. Furthermore, increased knowledge of distinct diagnostic profiles observed in the school setting may be useful for improving school-based evaluations. Future work is needed to understand what psychological, environmental, and biological factors influence this variance in adaptive behavior ratings across settings for youth on the autism spectrum.
Correspondence between executive function ratings in autism from the home and school settings was also supported by significant positive correlations between School and Home BRIEF ratings of the overall GEC, underlying BRI and MI, and seven of the eight scales. Furthermore, inter-rater reliability correlations from the standardization sample in the BRIEF Manual (Table 21; Gioia et al., 2000) only significantly differed from our correlations for autistic participants on the Inhibit, Initiate, Organization of Materials, and Monitor scales, indicating that reliability on the Shift, Emotional Control, Working Memory, and Plan/Organize scales aligns with what is reported from the standardization sample while the others may reflect a potential heightened difficulty of assessing executive function reliably across settings in autism.
This initial look at inter-rater reliability is an important step in demonstrating the utility of school ratings of the BRIEF for the autism population (see Shum et al., 2021 for similar work with a sample with increased likelihood for ADHD in China). It also contributes to an understanding of whether the BRIEF could be used in school settings for early identification of executive function impairment in autistic children and informing individualized, setting-specific interventions. Prior work indicates that examining executive functioning as a precursory step to autism diagnosis and intervention may be particularly useful for females on the autism spectrum (Torske, 2020; White et al., 2017). Furthermore, these individualized, setting-specific interventions may also be generally needed for autistic children due to a higher likelihood for their executive function impairments to vary across settings.
While this study was strengthened by a large sample size well-characterized for autism and having both caregiver and school professional ratings, it should be noted that there were several limitations. All findings are primarily relevant to non-Hispanic white males given their overrepresentation in the study sample. Furthermore, results may have been influenced by collinearity between autism criteria and the Shift scale, both of which involve ability to tolerate change, make transitions, and flexibly problem-solve. Findings regarding age-related changes are also limited by the absence of longitudinal data, and thus future longitudinal studies would be a needed next step.
Information regarding both caregiver and school professional raters was limited. While caregivers were largely parents that a child lived with more than 50% of the time and school professionals were typically the teacher for a child during their study participation, detailed information about raters’ interaction with a child was not collected. Future research would benefit from analyzing whether patterns of rater and child interaction are related to ratings of executive function impairment. This analysis may be particularly relevant to examine impact of time of school year and length of rater–child relationship on executive function ratings. Furthermore, such analysis could determine whether the age effects observed on the School BRIEF in this study are related to younger children spending more time in the same classroom and older children transitioning between classrooms throughout the day.
Furthermore, information regarding type of classroom (e.g. special education vs mainstream) and other reference points, such as number of other children with an autism diagnosis, was not analyzed. This information may need to be considered in future research to understand if there are systematic differences among various school professionals and between school professionals and caregivers with respect to their reference points in ratings. It may be that observed executive function impairments are related to environments that lack appropriate accommodations. In addition, this study did not explore whether co-occurring conditions other than ADHD, such as anxiety, depression, or oppositional defiant disorder influenced scores in each setting. Evaluating the effect of more co-occurring conditions would be important to explore in a future study to understand potential moderating effects.
The study also did not include performance-based or neural measures of executive function to examine convergence with informant ratings. Future research could benefit from exploring whether home or school ratings of executive function are more strongly associated with these other units of analysis of executive function in an autistic sample, as recent work indicates stronger relationships with school ratings of executive function for non-autistic children (Shum et al., 2021). In addition, future analyses would benefit from extending these findings into the BRIEF’s second edition (BRIEF-2) and evaluating measurement invariance in autism for school professional ratings. Examining these questions could further demonstrate the utility of using the BRIEF to measure real-world executive function in the autism population.
Conclusion
This study is the current most comprehensive examination of real-world executive function for autistic children in the school setting. These findings extend previous research on impaired executive function skills in school-age autistic children by showing that a similar but not identical profile is seen for real-world executive function skills in the school setting as at home. Our findings of diagnostic group differences in executive function and peak Shift impairments in both the school and home settings bolster the hypotheses that many autistic children have impaired executive function and Shifting as a primary area of weakness (Geurts et al., 2009). Moreover, our findings that individual executive functioning skills were worse in older age groups and that executive functioning skills predicted adaptive behavior in the school setting illustrate that executive function impairment in school is a crucial target for early identification and intervention efforts. The fact that nearly all scales on the BRIEF were found to be significantly correlated across rater type suggests that it may be a valid and useful informant measure for identifying executive function impairment in autistic children and informing individual interventions at school.
Taken together, these findings are even more critical because there is already a growing interest to intervene on executive function in the school setting due to its demands (Kenworthy et al., 2014; Tamm et al., 2020), but the evidence base of real-world executive function impairments in autism has not yet included ratings from school professionals for a large, controlled pediatric sample. Advancing our foundational knowledge of real-world executive functioning profiles across school and home settings can drive the field to develop more targeted prevention and intervention strategies that take setting into account.
Supplemental Material
sj-docx-1-aut-10.1177_13623613211041189 – Supplemental material for Real-world executive functioning for autistic children in school and home settings
Supplemental material, sj-docx-1-aut-10.1177_13623613211041189 for Real-world executive functioning for autistic children in school and home settings by Jessica E Tschida and Benjamin E Yerys in Autism
Supplemental Material
sj-docx-2-aut-10.1177_13623613211041189 – Supplemental material for Real-world executive functioning for autistic children in school and home settings
Supplemental material, sj-docx-2-aut-10.1177_13623613211041189 for Real-world executive functioning for autistic children in school and home settings by Jessica E Tschida and Benjamin E Yerys in Autism
Supplemental Material
sj-docx-3-aut-10.1177_13623613211041189 – Supplemental material for Real-world executive functioning for autistic children in school and home settings
Supplemental material, sj-docx-3-aut-10.1177_13623613211041189 for Real-world executive functioning for autistic children in school and home settings by Jessica E Tschida and Benjamin E Yerys in Autism
Footnotes
Acknowledgements
The authors acknowledge and thank the families for their participation in this study.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was sponsored by grants from the National Institute of Mental Health (K23MH086111; PI: B.E. Yerys, R21MH092615; PI: B.E. Yerys, RC1MH088791; R.T. Schultz), and a New Program Development Award to B.E. Yerys through the Intellectual and Developmental Disabilities Research Center funded by the National Institute of Child and Human Development (P30HD026979; PI: M. Yudkoff), a grant from the Philadelphia Foundation, a grant from the Pennsylvania Department of Health (SAP #4100042728) to R.T. Schultz, a grant from the Pennsylvania Department of Health (SAP #4100047863) to R.T. Schultz, a grant from Pfizer to R.T. Schultz, and a grant from the Robert Wood Johnson Foundation, #6672 to R.T. Schultz.
Supplemental material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
