Systematic Review of Acceptance and Commitment Therapy in Individuals with Neurodevelopmental Disorders,Caregivers,and Staff

Individuals with NDDs

Eight group design studies were completed with individuals with multiple NDDs (see Table 2). Three studies included participants with ADHD and comorbid disorders (Murrell et al., 2015; Vanzin, Crippa, et al., 2020; Vanzin, Mauri, et al., 2020). One study was completed with individuals with ASD (Maisel et al., 2019) and two further studies with individuals with ASD and comorbid disorders (Pahnke et al., 2014, 2019). The remaining two studies included individuals with LD (Rostami, 2014) and multiple IDs (García-Zambrano et al., 2019). Among the participants included in the eight studies, 38 were children (M = 11.2 years old), 73 were adolescents (M = 15.4 years old), and 67 were adults (M = 31.3 years old). In total, females accounted for 38 participants and males 153. Studies were conducted in diverse settings which included a research lab, clinics, rehab centers, schools, neuropsychiatric unit, and treatment programs. Six studies used pretest-posttest with control group designs, whereas the remaining three studies used a single group pretest-posttest design (see Table 2).

Seven studies included all six ACT processes during training (see Table 2). Two studies included defusion alone (Maisel et al., 2019) and one included defusion combined with SAC exercises (García-Zambrano et al., 2019). Regarding ACT processes evaluated in these papers, two studies assessed experiential avoidance (EA ¹ ; Murrell et al., 2015; Pahnke et al., 2019), two studies assessed cognitive fusion (Maisel et al., 2019; Pahnke et al., 2019), and one individual values (Murrell et al., 2015; see Table 2 for specific instruments used). In contrast, Garcia-Zambrano et al. (2019) directly measured participants’ verbal statements on self-as-process and SAC. The top three outcome measures implemented across all nine studies included anxiety, depression, and stress. Additionally, three studies assessed behavioral and emotional behaviors in individuals with ADHD (Murrell et al., 2015; Vanzin, Mauri, et al., 2020) and with ASD (Pahnke et al., 2014). ACT training ranged from five-minute defusion exercises (Maisel et al., 2019) to a 26-week training (Vanzin, Crippa, et al., 2020; Vanzin, Mauri, et al., 2020). None of the studies included treatment integrity measures, and only three studies assessed social validity (Maisel et al., 2019; Murrell et al., 2015; Pahnke et al., 2014).

Results indicated that social anxiety decreased in individuals with LD compared to the control group (Rostami et al., 2014). In addition, improvements were observed in stress level, hyperactivity, emotional symptoms, and increase in prosocial behaviors (Pahnke et al., 2014), and significant statistical changes in perceived stress, quality of life, depression levels, social impairment, EA, and cognitive fusion in individuals with ASD (Pahnke et al., 2019). Lastly, hyperactivity and oppositional behavior scores also decreased in individuals with ADHD (Murrell et al., 2015; Vanzin, Mauri, et al., 2020). In contrast, ACT was not effective in decreasing cognitive outcome measures such as executive functioning, focused and sustained attention, and inhibition in individuals with ADHD (Vanzin, Crippa, et al., 2020). Similarly, Garcia-Zambrano et al. (2019) did not identify significant differences in both groups in SAC statements, and Maisel et al. (2019) did not observe changes in thought believability and thought discomfort. Pahnke et al. (2019) reported anxiety and work and family-related impairments did not show significant changes.

Follow-up measures revealed that stress, hyperactivity, and emotional symptoms maintained in individuals with ASD at a 2-month follow-up (Pahnke et al., 2014). Likewise, a large effect size was observed in perceived stress, EA, and quality of life in a 3-month follow-up (Pahnke et al., 2019).

Parents of Individuals with NDDs

Seven studies were completed with parents (see Table 2). A total of 145 mothers and 29 fathers of children with ASD participated in the listed studies. Only one study included parents with adolescents with ASD (Lunsky et al., 2018). Studies were conducted at different locations, including an autism center, a research institute, a conference room at school, and online. Three studies used pretest-posttest with control groups (Corti et al., 2018; Hahs et al., 2019; Joekar et al., 2016) and the remaining four studies used within-subject repeated measures design (see Table 2).

During training, all but one study did not include all ACT six processes (Pennefather et al., 2018). Experiential avoidance was the most frequent assessed process followed by cognitive fusion and values (see Table 2). The top three outcome measures assessed were anxiety, depression, and stress (see Table 2 for additional measures). ACT training ranged from a 1.5-day training with a refresher 1 month later (Fung et al., 2018; Lunsky et al., 2018) to 12 ACT sessions every 2 weeks (Corti et al., 2018). All studies included didactic instruction and experiential training in a group format, and two studies included homework activities (Corti et al., 2018; Pennefather et al., 2018). Six studies conducted the training in situ and one via online (Pennefather et al., 2018). Treatment integrity and adherence measures were reported in two studies (Blackledge & Hayes, 2006; Corti et al., 2018), and social validity measures in three studies (see Table 2).

Results showed that parents in the ACT groups demonstrated significant improvements in shame, mindfulness, cognitive fusion, stress levels, depression, and EA when compared to their control groups (Corti et al., 2018; Hahs et al., 2019; Joekar et al., 2016). In the remaining four studies that did not have comparison groups, significant improvements were reported in EA, cognitive fusion, values, depression, stress level, social isolation, and physical health (Blackledge & Hayes, 2006; Fung et al., 2018; Lunsky et al., 2018; Pennefather et al., 2018) and indirectly increased child prosocial behaviors and decreased hyperactive behaviors (Pennefather et al., 2018). Notwithstanding, quality of life, anxiety, stress levels (Joekar et al., 2016), and cognitive fusion (Corti et al., 2018) did not exhibit significant differences in the ACT and control groups at posttest measures. Hahs et al. (2019) reported a small effect size in thought suppression and mindfulness scores in the ACT group.

Follow-up data revealed EA, cognitive fusion, valued living, along with parent health and well-being (Fung et al., 2018; Lunsky et al., 2018) were maintained at 1-month follow-up; though, decrease in EA and depression scores did not sustain at a 1-month follow-up (Joekar et al., 2016). Lower levels of distress and depression were also maintained at 3-months follow-up (Blackledge & Hayes, 2006).

Staff working with Individuals with NDDs

Five studies were completed with staff (see Table 2). They included teachers, nurses, family consultants, preschool staff, lead teachers and assistants, and social workers. A total of 134 females and 35 males participated in the studies. Only two studies reported the location of the training: a state-funded facility (Bethay et al., 2013) and a community day program (Noone & Hastings, 2010). Three group studies compared ACT with a wait-list group (see Table 2), one study compared ACT+ABA (applied behavior analysis) versus ABA alone (Bethay et al., 2013), and the remaining study used a single group pretest-posttest design (Noone & Hastings, 2010).

All five studies included the six ACT processes in their training. Three studies assessed multiple ACT measures, EA, cognitive fusion, and values (Bethay et al., 2013; Biglan et al., 2013; McConachie et al., 2014). Psychological distress, burnout, and stress level were the most common outcome measures assessed across all five studies (see Table 2 for additional measures used in these studies). ACT training sessions ranged from a 1.5 days session (Noone & Hastings, 2009, 2010) to a three-week training (Bethay et al., 2013). Training sessions were completed in a group format. None of the studies reported treatment integrity data, and two studies completed social validity measures (Bethay et al., 2013; Biglan et al., 2013).

Results showed that staff participants self-reported increases in feelings of efficacy, mindfulness awareness, and significant stress reduction (Biglan et al., 2013) and decreased believability in burnout-related thoughts in the groups that received ACT training (Bethay et al., 2013). In addition, cognitive fusion and valued living scores substantially changed at posttest measures (Bethay et al., 2013; Biglan et al., 2013). Correlational analyses revealed that EA and valued living were associated with lower levels of stress and depression and higher levels of personal accomplishment (Biglan et al., 2013). Three studies reported significant reduction in psychological distress (see Table 2). Lastly, McConachie et al. (2014) demonstrated significant reduction in thought suppression, but their results revealed a lack of positive impact on staff well-being. In contrast, no significant changes were observed in EA (Biglan et al., 2013; McConachie et al., 2014); whereas three studies indicated that stress levels did not decrease after the ACT intervention (McConachie et al., 2014; Noone & Hastings, 2009, 2010).

Follow-up data showed moderate increases in psychological distress at a 6-week measure (McConachie et al., 2014). In addition, lower psychological distress and stress levels were maintained at 3-month and 5-month follow-ups (Bethay et al., 2013; Biglan et al., 2013) respectively. Further, thought suppression scores decreased significantly at 6-month follow-up (McConachie et al., 2014); whereas, cognitive fusion scores remained low at 3-month follow-up (Bethay et al., 2013).

Individuals with NDDs

Four SCD studies were completed with individuals with ASD (see Table 2) and one with individuals with multiple IDs (Brazeau et al., 2017). Three studies included nine children (M = 6.6 years old), and the remaining three studies included nine young adults (M = 21.6 years old). There were three females and 15 males across the four studies. Studies were conducted in a vocational facility, clinical setting, university clinic, a participant’s home, and local gym.

Four studies used multiple baseline (MBD) across participants. Eilers and Hayes (2015) used a MBD and alternating treatment design in two different experiments. Two studies included all six ACT processes during training (Szabo, 2019; Szabo et al., 2019); whereas, one study incorporated four ACT skills processes (acceptance, defusion, present moment, SAC; Brazeau et al., 2017). An additional study combined present moment plus defusion (Hutchinson et al., 2019) and one defusion alone (Eilers & Hayes, 2015, Exp. 1 & 2). Training ranged from 30-second defusion exercises (Eilers & Hayes, 2015) to a 4-hour ACT training (Szabo, 2019). All sessions were carried out in an individual format. All studies reported social validity and treatment integrity data except for Eilers and Hayes (2015, Exp. 1) that did not include a social validity measure.

Results revealed that combining ACT with evidence-based behavior-analytic procedures such as behavioral skills training (BST), differential reinforcement, shaping, or prompting improved performance in different areas. In particular, interview skills performance improved for two participants (Brazeau et al., 2017) and three participants (Hutchinson et al., 2019) when BST was complemented with ACT. Eilers and Hayes (2015) reported that defusion combined with exposure therapy led to significantly lower levels of disruptive behaviors and an increase in play behavior in individuals with ASD. Furthermore, ACT alone decreased noncompliance, yelling, and mands for rule changes for all participants (Szabo, 2019) and improved rope climbing falls, tardy arrivals, and heart rate in two participants (Szabo et al., 2019). In contrast, one participant improved interview skills with BST alone (Brazeau et al., 2017), and one participant’s performance improved with direct contingency management alone (Szabo et al., 2019). Brazeau et al. (2017) reported that anxiety scores decreased in only two participants.

Follow-up measures revealed that all three target behaviors (i.e., rope climbing falls, tardy arrivals and heart rate) were maintained immediately after the intervention (Szabo et al., 2019). Furthermore, interview skills were maintained in two individuals 2 weeks after the conclusion of the study (Hutchinson et al., 2019) and above criterion for two participants 1 month later (Brazeau et al., 2017). Similarly, Szabo (2019) indicated compliance behaviors and manding for rule changes maintained at 4 and 8-weeks follow-up, and Eilers & Hayes (2015, Exp. 1) indicated problem behaviors maintained at a zero level for 3 months after intervention.

Parents of Individuals with NDDs

Only one SCD study with parents was identified (Gould et al., 2018). All three participants were mothers of children with ASD (M = 41 years old). A MBD across participants was used to examine the effect of ACT on values-based behaviors. Additionally, EA, self-compassion, and parents’ appraisal of having a child with a disability were assessed. All ACT processes were trained in six, 90-minute individual sessions. Social validity and treatment integrity data were not reported. Results revealed that two parents increased values-based behaviors during training and one parent exhibited an immediate increase during post-training sessions. All parents demonstrated reduction in EA, moderate decreases in parent’s appraisal of their child’s disability, and improvements in self-compassion scores. Follow-up data showed further gains in values-based behaviors in two parents at the 7-8-month follow-up.

Staff working with Individuals with NDDs

Four studies were completed with staff members (see Table 2). Ten female and one male participated across the four studies. Staff participants’ experience delivering ABA services to individuals with NDDs ranged from 3.2 months to 28 years. Studies were conducted in multiple sites, including a day treatment program, an ABA clinic, a residential home, and conference room.

Three studies implemented a MBD across staff members (see Table 2) and one study used a reversal design embedded with MBD across staff participants (Little et al., 2020). Pingo et al. (2020) included all ACT processes in their training; whereas, Little et al. (2020) taught present moment awareness, values, and committed actions. Two further studies taught values and committed actions (Castro et al., 2016) and mindfulness alone (Chancey et al., 2019). Experiential avoidance was assessed in three studies (Chancey et al., 2019; Little et al., 2020; Pingo et al., 2020) and stress and job satisfaction in one study (Pingo et al., 2020).

Training ranged from 5 to 15-minute ACT workshops (Chancey et al., 2019) to 1-hour ACT training (Little et al., 2020). Training sessions were conducted in individual (Little et al., 2020), group formats (Castro et al., 2016; Chancey et al., 2019), and face-to-face and computerized tutorial training sessions delivered in an individual format (Pingo et al., 2020). Three studies reported social validity measures (Castro et al., 2016; Chancey et al., 2019; Little et al., 2020), and two studies included treatment integrity measures (see Table 2). However, Pingo et al. (2020) reported treatment integrity during feedback sessions only. Chancey et al. (2019) did not report IOA scores for the dependent variables.

Results showed that staff members increased client engagement following the ACT intervention (Castro et al., 2016; Chancey et al., 2019); however, two out of the three staff participants required a second workshop to improve their performance (Castro et al., 2016). Little et al. (2020) observed that combining BST+ACT substantially increased performance training for junior staff members. Likewise, Pingo et al. (2020) indicated that work performance improved in three staff participants upon adding ACT to verbal and written feedback. The remaining two participants demonstrated higher work performance following verbal and written feedback only. Stress, job satisfaction, and EA scores did not exhibit significant changes at posttest in three studies (see Table 2). Follow-up measures demonstrated that level of responding maintained across all participants after ACT workshops (Chancey et al., 2019) and that training skills generalized across staff and clients (Little et al., 2020).

Evaluation of group designs

The third and sixth authors of this paper assessed the interrater agreement and quality of RCTs and NRCTs. The RoB2 was used to evaluate the RCTs. Inter-rater reliability was calculated using Cohen’s kappa for each category of risk of bias. The Cohen’s k indicated that inter-rater reliability between the two reviewers’ judgments was good, k = 0.80. Discrepancies were due to differences in interpretation of criteria and were discussed with the first author until a 100% agreement in coding was reached. Quality scores were expressed as low risk, some concern, or high risk. Interrater agreement for the ROBINS-I was also calculated using Cohen’s kappa for each category of risk of bias. The Cohen’s k showed good agreement between the two reviewers (k = 0.79). Quality scores were expressed as low risk, moderate risk, serious risk, or high risk.

RoB2

The methodological quality of the seven studies included in the systematic review is visually depicted in the right panel of Figure 2. Out of these, one study did not report a random sequence generation (McConachie et al., 2014). Blinding of outcome assessment was considered adequate in one study (Garcia-Zambrano et al., 2019), as direct verbal reports occurring in session were collected. Lastly, one research (Bethay et al., 2013) did not report the direction, magnitude, or statistical significance of the results. Overall, the risk of bias was rated as low risk in one study (Garcia-Zambrano et al., 2019), some concern in five studies (Bethay et al., 2013; Biglan et al., 2013; Hahs et al., 2019; Maisel et al., 2019; Rostami et al., 2014) and high risk in one study (McConachie et al., 2014).

OPEN IN VIEWER

Figure 2.

Quality assessment of randomized control trials (RoB2) (right panel) and quality assessment of nonrandomized control trials (ROBINS-I) (left panel).

ROBINS-I

A summary of the risk of bias in each study is presented in the left panel of Figure 2. Fourteen NRCTs were reviewed using the ROBINS-I. When using the ROBINS-I assessment, it is recommended to exclude papers with critical risk of bias from analyses; however, given the overall high risk of bias, all papers have been retained for the purpose of this review. Out of the domains evaluated, the most common source of moderate to serious risk of bias were confounding factors and measurement of the outcome. Confounding variables included baseline outcome differences, eligibility of the intervention, or comorbidity disorders. All papers presented moderate risk of bias in measurement of the outcome due to their reliance on subjective self-report measures, which may be influenced by knowledge of the intervention. The overall risk of bias was moderate risk (i.e., bias raises some doubts about the results) or serious risk (i.e., bias seriously weakens confidence in the results). Overall, two studies were identified as having serious risk (Murrell et al., 2015; Noone & Hastings, 2010). In particular, Murrell et al. (2015) presented significant confounding bias and missing outcome data; whereas, Noone and Hastings (2010) had significant bias with selection of participants. The remaining 12 studies showed some concerns in the measurement of the outcome domain. In all 10 studies, there were varying assessments of the subjectivity of different self-reported ACT and outcome measures.

Evaluation of single-case designs

Second and fourth authors independently assessed study quality for SCD studies based on the WWC standards. Interrater agreement for design and evidence standards were calculated by using point-by-point reliability method. Specifically, the number of agreements on the quality indicators was divided by the number of agreements plus the number of disagreements and multiplying by 100. The interrater agreement for the design standard indicators was 98%, and the evidence standards resulted in 96% agreement. Disagreements were resolved through consensus.

Design standards

It is important to clarify that ten SCD studies were assessed; however, 11 experiments were completed in all ten studies (see Table 2). Ten of 11 experiments (90%) provided descriptions of the baseline condition, the intervention, and the process for beginning intervention as shown in Table 1. One study (Gould et al., 2018) did not provide baseline information as the nature of the study required engagement in activities not previously practiced. All experiments (100%) were considered to provide evidence of manipulation of the independent variable. Nine of the 11 experiments reported collecting IOA in 20% of each condition with scores at or above the minimum agreement levels. However, one study failed to meet minimum thresholds for IOA (Chancey et al., 2019). All studies showed at least three demonstrations of effect. After evaluating all requirements for WWC SCD standards, two experiments Meet Standards (Eilers & Hayes, 2015, Exp. 2; Pingo et al., 2020). Eight experiments Meet Standards with Reservation (see Table 1). And one study did not Meet Standards (Chancey et al., 2019) as it failed to meet minimum thresholds for IOA.

OPEN IN VIEWER

Table 1.

Evaluation of Studies Using What Works Clearinghouse Guidelines.

	Systematic manipulation of IV	IOA for 20% of sessions	IOA at or above 80%	At least 3 Demonstr. of effect	3 data points per condition	5 data points per condition	WWC design standard	WWC evidence of effectiveness
Brazeau et al. (2017)	Y	Y	Y	Y	Y	N	MWR	Moderate
Castro et al. (2016)	Y	Y	Y	Y	Y	N	MWR	Moderate
Chancey et al. (2019)	Y	N	N	Y	Y	N	Does not meet	N/A
Eilers & Hayes (2015, Exp. 1)	Y	Y	Y	Y	Y	N	MWR	Moderate
Eilers & Hayes (2015, Exp. 2)	Y	Y	Y	Y	Y	Y	Meets	No Evidence
Gould et al. (2018)	Y	N	Y	Y	Y	N	MWR	Moderate
Hutchinson et al. (2019)	Y	Y	Y	Y	Y	N	MWR	Moderate
Little et al. (2020)	Y	Y	Y	Y	Y	N	MWR	Moderate
Pingo et al. (2020)	Y	Y	Y	Y	Y	Y	Meets	No Evidence
Szabo (2019)	Y	Y	Y	Y	Y	N	MWR	Moderate
Szabo et al. (2019)	Y	Y	Y	Y	Y	N	MWR	Moderate

Note. IOA = interobserver agreement; Y = yes; N = no; MWR = meets with reservations.

Evidence standards

After evaluating the basic design quality of each article, the quality of the evidence for each experiment within the remaining nine studies was evaluated based on visual analysis criteria developed by the WWC (2020) and adapted by Maggin et al. (2013). Visual analysis for two studies were rated as no evidence. First, the data path reported by Eilers and Hayes’ (2015, Exp. 2) revealed a lack of differentiation between conditions for problem behavior and play behavior. Although the defusion plus exposure condition outperformed the control condition for one participant, there was not enough data across all participants to demonstrate moderate evidence. Second, Pingo et al. (2020) demonstrated that two staff participants improved work performance without the addition of ACT training. Furthermore, data points in the adjacent phases showed a large degree of overlap and large data variance in intervention phases, and in some participants, data trended away from improvement direction. There was also no immediacy of basic effect between written and verbal feedback and ACT. The remaining eight studies demonstrated moderate evidence as they contained an insufficient number of data points (i.e., fewer than five per phase) to meet the criteria for strong evidence.