Abstract
Mindfulness-based programs are a promising intervention modality for reducing disruptive behavior, and Soles of the Feet (SOF) is one program that teaches internal awareness of personal events (e.g., unpleasant emotions) and a self-regulation strategy to decrease disruptive behaviors. This study conducted a meta-analysis of single-case research design (SCRD) studies that implemented SOF to decrease disruptive behaviors. Existing SOF studies were evaluated using high-quality SCRD standards, resulting in 15 studies included in the analysis (49 participants; mean age 23.12 years (SD = 15.87); highly heterogeneous backgrounds). Studies were analyzed to calculate effect sizes using Tau-U, an innovative non-parametric statistical approach for estimating effect sizes in SCRD studies. The aggregated weighted Tau-U effect size of SOF across all studies was −0.87. Moderator analyses indicated SOF’s effectiveness was robust across participant characteristics and delivery formats. This meta-analysis suggests that SOF is a moderately effective evidence-based practice for reducing disruptive behavior.
Disruptive behaviors are any behavior that significantly interrupts the routine of the acting individual or those in their immediate surroundings (e.g., physical aggression, off-task behavior). Disruptive behavior manifests in a wide range of individuals of differing ages and disability statuses, and occurs in a variety of different settings and ecological contexts. As an example, a small proportion of individuals with severe psychiatric disabilities exhibit physically aggressive behavior to overtly threaten or harm another person, which creates a safety concern for both the target individual and direct care providers (Potegal et al., 2010; Singh, Lancioni, Winton, Adkins, Wahler, et al., 2007). As another example, some school age children, with or without a disability, exhibit disruptive behaviors that have serious long-term educational, social, and mental health consequences (Felver et al., 2014), such as later academic difficulties (Kim-Cohen et al., 2005), adolescent delinquent behavior (Broidy et al., 2003; Tremblay et al., 1992), and elevated risk of adult psychopathology and related incarceration (Schaeffer et al., 2003).
Disruptive behaviors are not unique to one specific clinical population of individuals. These behaviors are often a byproduct of unpleasant emotional states (e.g., anger) and corresponding physiological arousal, which frequently has a response-activating function leading to various topographies of disruptive behavior (Bandura, 1973; Singh, Lancioni, Winton & Singh, 2011), Disruptive behaviors are a significant problem for a wide array of individuals, and effective interventions are needed to reduce the rates of disruptive behavior to promote long-term success.
Psychopharmacological and behavioral interventions are the mainstay treatments for individuals with high rates of disruptive behavior (Brosnan & Healy, 2011; Singh, Lancioni, Winton, Singh, Singh, et al., 2011). As noted by Singh, Lancioni, Winton, Singh, Singh, et al. (2011), the new generation of antipsychotics (e.g., risperidone, aripiprazole, and olanzapine) are utilized relatively frequently for individuals with both intellectual disabilities (ID) and severe behavioral management difficulties, especially in residential institutional settings. However, reviews of the current treatment literature suggest that the evidence supporting the effectiveness of these drugs for reducing aggression among individuals with ID is limited; for example there is a relative dearth of evidence supporting the use of antidepressants to minimize aggressive behaviors despite their frequent prescription (Singh, Lancioni, Winton, & Singh, 2011). With regards to children and adolescents, disruptive behavior that is impulsive or explosive appears to be medication responsive (Campbell et al., 1992). However, other forms of disruptive behaviors in children and adolescents, such as predatory or instrumental aggression, are largely not responsive to psychiatric medication (Singh, Lancioni, Winton, Adkins, Wahler, et al., 2007), thus necessitating a need for effective non-pharmacological interventions.
As a heterogeneous class of interventions, behavioral interventions have strong evidence base for their efficacy in decreasing disruptive behaviors (Lee & DiGiuseppe, 2018; Singh, Lancioni, Winton, Singh, Singh, et al., 2011). Further, the evidence supporting the implementation of behavioral interventions to decrease disruptive behaviors is robust for diverse populations, such as individuals with Autism Spectrum Disorder (ASD; Horner et al., 2002; Roth et al., 2014), children and adolescents with Oppositional Defiant Disorder or Conduct Disorder (Nitkowski et al., 2009), and typically developing youth in classroom settings (Barnes et al., 2014; Powell et al., 2011). In addition to effectively reducing frequency and intensity of disruptive behavior, behavioral interventions have the added benefit of teaching sustainable alternative behaviors that have the same function as disruptive behavior (e.g., escaping an aversive task), offering a unique advantage over pharmacological interventions that do not teach sustainable self-management behavior (Singh, Lancioni, Winton, Singh, Singh, et al., 2011). Although behavioral interventions are effective for reducing disruptive behaviors in heterogeneous populations, researchers have called for additional research into interventions derived from other theoretical traditions to identify additional effective intervention components (Lee & Giuseppe, 2018). Interventions with theoretical bases in mindfulness practices, collectively known as mindfulness-based programs (Crane et al., 2017), offer one such class of interventions that have, to-date, received less attention as a promising modality of treatment for disruptive behaviors.
Mindfulness-Based Programs for Disruptive Behavior
Mindfulness-based programs (MBPs) are among the armamentarium of interventions implemented with children, adolescents, and adults to address disruptive behavior. The earliest applications of mindfulness as a behavior change strategy to enhance well-being can be found over 2,500 years ago in Early Buddhism, with the classic example being the Buddha teaching mindful eating to the portly King Pasenadi (Anālayo, 2018). In the modern Western context, the most common definition of mindfulness is “Paying attention in a particular way: on purpose, in the present moment, and nonjudgmentally” (Kabat-Zinn, 1994, p. 4). The academic consensus operational definition of mindfulness provided by Bishop et al. (2004) defined it as, “[A] kind of nonelaborative, nonjudgmental, present-centered awareness in which each thought, feeling, or sensation that arises in the attentional field is acknowledged and accepted as it is” (p. 232). Although there many definitions of mindfulness (see Amaro & Singh, 2021), the key underlying feature is a focus on how to change one’s relationship to experiences that arise internally (e.g., cognitions and emotions).
The empirical evaluation of MBPs has greatly expanded in the recent years (Kallapiran et al., 2015; Khoury et al., 2015; Zoogman et al., 2015), with much of the research exploring how mindfulness may be an efficacious strategy to significantly reduce psychological distress, anxiety and depressive symptoms (Goldberg et al., 2018; Khoury et al., 2013). For instance, there is robust empirical support for the use of MBPs with adults to address psychosocial variables (e.g., stress, depression, and anxiety) with clinical and non-clinical samples, and from non-dominant cultural and/or marginalized backgrounds (Fuchs et al., 2013; Visted et al., 2015). The literature supporting the use of MBPs with youth is in the nascent stage, with even less attention devoted to the nature of aggregated treatment effects from MBP implemented with youth in the context of single-case research design (SCRD). However, a recent meta-analysis of single-case research indicates that MBPs with adolescents designed to decrease disruptive behaviors is associated with a medium effect size during treatment (g = 1.04) and maintenance phases (g = 1.41; Klingbeil et al., 2016).
Soles of the Feet (SOF; Felver & Singh, 2020; Singh et al., 2003) is an empirically supported manualized MBP that has been implemented with children, adolescents, and adults in multiple settings and among individuals both with and without cognitive disabilities, to specifically decrease disruptive behaviors. The purposes of the SOF intervention include (a) teaching individuals to recognize the antecedent variables that occasion their disruptive behavior, (b) having individuals disengage their attention from those precursors, (c) reorienting their attention to a neutral point on the body (i.e., somatic sensations of their feet), thereby discontinuing the escalation of the problematic behavior, and (d) returning calmly to the activity taking place. The name SOF is derived from the fact that the target stimulus for reorienting attention is the feet, including the bottoms, or soles, of the feet. The SOF intervention may address similar maintaining variables for escape-motivated disruptive behavior by temporarily reducing the aversive properties of antecedent triggers (i.e., an abolishing operation) and/or reinforcing competing responses (e.g., Carr et al., 1976; Sullivan et al., 2017). In addition, the SOF intervention encourages individuals to practice the skills taught in different contexts, thereby supporting generalization across settings once automaticity of the mindfulness routine has been achieved (Felver et al., 2014; Singh et al., 2003). Multiple studies have been conducted to empirically evaluate the utility of SOF for disruptive behavior, most of which employed SCRD methodology. However, to date, a synthesis of the existing research literature has yet to be conducted for this specific MBP.
Research Questions
The objective of this research is to analyze the effectiveness of the SOF intervention for decreasing disruptive behavior via a meta-analysis of the empirical SCRD literature. Among general calls for additional meta-analyses of specific MBP protocols (Felver et al., 2016), SOF is particularly ripe for meta-analysis as this program represents the largest area of MBP scientific inquiry utilizing SCRD, and to-date, there have been no SCRD meta-analyses conducted on any one specific MBP curriculum. Further, there have been recent advances in non-parametric statistical analyses that enable researchers to determine robust quantification of aggregated effect sizes from SCRD studies (Parker & Vannest, 2009; Parker et al., 2011), and these modern analytic approaches have not been applied to any meta-analysis of MBP to-date. Additionally, there are currently sufficient SOF SCRD peer-reviewed published studies, with enough participants, from multiple research teams, that meet the widely accepted standards for SCRD to conduct an acceptable meta-analysis (Horner et al., 2005; Kratochwill et al., 2013).
The purpose of this meta-analysis is to quantitively summarize the SCRD research on the effectiveness of the SOF intervention for disruptive behaviors using meta-analytic procedures, and specifically addressing (1) whether and to what degree the SOF intervention produces statistically significant changes in participants’ disruptive behaviors, and (2) whether participant, SOF protocol modifications, or study design features moderate the observed effects of SOF on disruptive behavior.
Method
Analytic Approach
The meta-analysis procedure included (1) article extraction and preliminary review, (2) a quality assessment by two independent raters, (3) an overall SOF intervention effect size estimation, and (4) analyses of moderating variables that may predict the effectiveness of the SOF intervention to decrease aggressive and disruptive behaviors. Relevant databases were searched to identify SCRD studies that implemented the SOF intervention and measured disruptive behaviors as a dependent outcome. Unpublished research studies (e.g., doctoral theses) were also sought. Articles were reviewed to assess inclusion criteria. Studies were analyzed to calculate effect size estimations using Tau-U, a non-parametric statistic approach for estimating effect size in SCRD studies (Parker et al., 2011). Finally, identified moderating variables were evaluated using Tau-U estimates at the individual participant level.
Initial Identification Criteria and Article Identification
Articles were required to meet the following criteria to be included in the meta-analysis. First, each study was required to have implemented a SCRD design, such as an ABAB withdrawal or a multiple-baseline design across participants (Franklin et al., 2014). Second, each study needed to clearly identify the SOF intervention as the independent variable. Third, each study needed to have quantitative, graphical data from two or more phases, depicting different discrete stages in the intervention process (e.g., baseline, SOF intervention, and follow-up) to allow the authors to differentially evaluate the disruptive behaviors in distinct phases of the study design. Fourth, each study needed to include some form of disruptive behavior (e.g., physical aggression, academic off-task behavior) as a dependent outcome. The current manuscript had an open inclusion criterion regarding participant characteristics; if the article met the aforementioned four criteria, the article was included with coded information of participants’ age, disability status, gender, and race. This decision was made to enable an examination of participant level characteristics that may moderate the effects of the SOF protocol.
In September 2020, the first author conducted searches of the following databases: PsycINFO, ERIC, Web of Science, and Social Sciences Full Text using the key words of “Soles of the Feet”, “SOF”, “mindfulness,” and “single-case design”. This process resulted in the identification of 31 total studies that implemented SOF, 17 of which met the aforementioned initial article identification criteria. The studies that were excluded from the current meta-analysis implemented the SOF intervention as an adjunct procedure (e.g., in conjunction with another intervention; Wilson et al., 2015), or utilized a research design other than SCRD that would introduce an incompatible estimate of effect size for meta-analytic comparison. For example, we excluded a group design clinical trial (Singh et al., 2013) and a descriptive feasibility trial (Roberts et al., 2020) that utilized SOF.
Quality Assessment for Study Inclusion
Following the initial identification stage, a quality assessment was conducted of the 17 articles to determine if the identified studies met high-quality SCRD research standards to be included in the meta-analysis. The quality assessment was based on the widely accepted standards for SCRD (Horner et al., 2005; Kratochwill et al., 2013), and utilized a quality assessment checklist and procedure developed for SCRD meta-analysis (see Tincani & De Mers, 2016). This 17-item checklist covered nine substantive domains of SCRD quality, including: appropriateness of research question; description of participant and setting; measurement of dependent variable; description of independent variable; SCRD procedures; reliability; and validity. Checklist items were scored following procedures detailed in the articles and rated as 0 = inadequate/unknown, 1 = partial/adequate, and 2 = adequate or not applicable. Following procedures detailed by Tincani and De Mers (2016), all identified articles needed to meet minimum criteria for high-quality research standards (Horner et al., 2005) on two critical checklist items: (1) a score of 2 on item 16 (minimum of three demonstrations of experimental control), and (2) a score of 1 or 2 on item 17 (at least one demonstration of replication of treatment effects across participants, settings, or materials). Quality indicator scores were calculated for each identified article by summing all 17 items to create a total score, dividing this score by the maximum possible score of 34, and multiplying this number by 100 to create a percentage value. Scores above 90% were considered “Strong,” scores between 75% and 89% were considered “Adequate,” and scores below 75% were considered “Inadequate” quality of evidence (Flippin et al., 2010; Tincani & De Mers, 2016).
All articles were coded by the third author, a doctoral graduate student in a school psychology training program with coursework experience in applied behavior analysis, and the first author, who held a doctorate in school psychology and had experience conducting SCRD. Prior to scoring quality indicators, the coders met to review the quality assessment checklist and operationally define scores for each of the 17-items. Coders then independently scored two randomly selected articles and met again to review their scores and resolve any instances of scoring disagreements. Next, coders independently scored all 17 identified articles on each of the 17-items of the quality assessment checklist. Interrater agreement between the two coders was calculated by dividing the total number of scoring agreements by the total number of items scored, and multiplying this number by 100 to create a percentage score. The average interrater agreement ranged between 38% and 100% across the 17-item with an overall mean agreement of 96%; half of all disagreements was due to ambiguity of what constituted “critical features” of physical settings. The two coders met to discuss and resolve each instance of coding disagreement to produce the final quality indicator score.
Each article was also inspected to record the following: participant characteristics (i.e., age, race/ethnicity, gender, disability status); SOF implementation characteristics; and specific details of the disruptive behavior. Two articles (Shababi-Shad, 2014; Singh et al., 2003) were excluded from the meta-analysis because of their score on the quality assessment checklist, yielding 15 articles that met final criteria for inclusion in the meta-analysis.
Effect Size Estimation
Data extraction
The GraphClick (Arizona Software Inc., 2010) for Mac platform was utilized to extract data from scanned and digitized graphs of included articles. Previous research indicates that the GraphClick software is a valid and reliable method of data extraction and allows for reliable extraction of data between raters (Boyle et al., 2013). Following a procedure similar to Tincani and De Mers (2016), the first data conversion step involved each graph being converted to a PDF file from the digital imaging files, and then individual data points were identified, assigned minimum and maximum values on the x and y axes, and converted into raw values. GraphClick software produced an output consisting of a list of values for all data points on the graph. The second data conversion step was completed to establish interobserver agreement (IOA). Specifically, data points from graphs of 44% of randomly selected articles were separately coded by the first and second authors. The second author was a doctoral candidate in a school psychology training program with coursework in applied behavior analysis experience and applied experience conducting SCRD clinical trials. IOA was calculated as point-by-point agreement for each data point with Agreements divided by total Agreements and Disagreements, multiplied by 100 to yield a percentage score. IOA was 90%. Next, the first and second authors met and examined and discussed identified coding disagreements. Following this meeting, first and last authors recoded all data and IOA was 100%.
Tau-U
The Tau-U index (Parker et al., 2011) was used to estimate the effect size for each of the included studies. Tau-U is a non-overlap statistic that is similar to non-overlap of all pairs (NAP; Parker & Vannest, 2009). While NAP has been used in recent meta-analyses (Roth et al., 2014; Walker & Snell, 2013), it is important to outline the strengths and weaknesses of using a non-overlap approach of effect size estimation. In terms of strengths, non-overlap analysis is considered a distribution free index in which it does not require interval-level measurement or a linear relationship between time and scores, is robust to the undue influence of outlier scores, and can at times be a better, more sensitive summary than mean or median difference (Huberty & Lowman, 2000; Parker et al., 2011). However, Parker et al. (2011) have outlined that the main limitation of non-overlap analysis is that it is insensitive to data trend, which is important to conclusion validity for two main reasons. First, non-overlap statistics does not capture positive trend in the intervention phase, which is an important measure of improvement. Second, non-overlap approaches are inadequate for identifying undesirable preexisting trends in the baseline phase, which suggests that the subject may have improved regardless of the intervention implementation. For these reasons, the current meta-analysis utilized the Tau-U index, which combines nonoverlap between phases with trend from within the intervention, while also controlling for baseline trend (Parker et al., 2011).
Tau-U is an effect size statistic that combines indices from nonparametric statistics of Kendall’s rank correlation, Tau, and Mann-Whitney U. Tau-U compares (1) phase A versus phase B nonoverlap, (2) nonoverlap and Phase B trend together, (3) nonoverlap with baseline trend controlled, and (4) nonoverlap and Phase B trend with baseline trend controlled. Overall, the Tau-U summary index is generally interpreted as “the percent of data that has improved over time considering both phase nonoverlap and Phase B trend, after control of Phase A trend” (Parker et al., 2011, p. 8). Additionally, a positive Tau-U score indicates that the dependent variable increased during the intervention phase, while a negative value suggests that the dependent variable decreased during the intervention phase (Vannest & Ninci, 2015). Tau-u has no well-received “rule of thumb” cutoff values to indicate significance. As Dart et al. (2014, p. 373) pointed out “Although guidelines for interpreting Tau-U are not currently available, tentative guidelines for a similar single-case effect size statistic, the non-overlap of all pairs (NAP), have been suggested (weak, 0.0–0.65; moderate, 0.66– 0.92; strong, 0.93–1.0; Parker & Vannest, 2009). It is important to note that Tau-U is a more conservative effect size estimator than NAP because it controls for baseline trend.” This meta-analysis used the Tau-U approach to estimate non-overlap between baseline (i.e., Phase A) and intervention (i.e., Phase B) phases, while also controlling for significant trends observed during the baseline.
Data Analysis
Phase contrasts and weighted Tau-U coefficients
Effect size estimates were derived for each study using a web-based calculator for determining Tau-U (Vannest et al., 2016). All of the studies included in the meta-analysis implemented a multiple baseline design. Therefore, the first step was to enter data sets for phase A (i.e., baseline) and phase B (i.e., intervention) into the online calculator. Following a procedure similar to Dart et al., (2014), we used a liberal significance value (p < .15) to detect baseline trend. This liberal significance value was utilized to maximize the benefit of using Tau-U as an effect size metric as many of the studies included a small number of baseline observations, which limits the power for detecting a significant trend. This procedure produced a phase contrast Tau-U metric for each participant of the studies included, comparing the baseline and intervention data for every subject. Second, we calculated a weighted average for each of the phase contrasts in a single study. This procedure resulted in a weighted Tau-U effect size metric for each study included in the meta-analysis.
Overall, studies included a range of disruptive behavior dependent variables and participant characteristics, and most studies reported more than one dependent outcome. For example, many of the studies included dependent measurement of both verbal and physical aggression. If a study measured both physical and verbal aggressive behavior, only physical aggressive behavior was included in analyses in order to have a similar topography of behavior across studies. Some studies measured maladaptive behavior (e.g., work disruption; Adkins et al., 2010) or problem behavior (e.g., academic off-task behavior; Felver et al., 2014, 2017), which although not labeled as “aggressive” behavior, were considered to be categorically “disruptive” given the direct effects of such behavior on an individual’s ability to function.
Aggregate Tau-U effect size
Once weighted Tau-U effect sizes were obtained for each study, the next step in the meta-analysis was to obtain an overall aggregate effect size estimate. Following the procedures outlined by Bowman-Perrott et al. (2013), we used the WinPepi (Abramson, 2012) program to obtain an aggregate Tau-U effect size to evaluate if the SOF intervention is an effective strategy to decrease aggressive and disruptive behaviors. Specifically, each Tau-U effect size was entered into the WinPepi program using the meta-analysis function, which aggregates the data and produces a single effect size for a given study. Identical to Bowman-Perrott et al. (2013), the following data analysis functions were selected: (a) Compare 2, (b) Meta-analysis; analysis of stratified data, (c) Others, or proportions or rates with effect sizes/CIs, and (d) Also enter standard error. Therefore, for each study in the meta-analysis, the weighted Tau-U effect size and the 95% confidence interval were entered into the WinPepi program to obtain an aggregated Tau-U effect size.
Results
Participant Characteristics
The 15 studies examined in this meta-analysis were published between 2007 and 2020 and included a total of 49 participants. Overall, the age range for participants was 9 to 72 years (M = 23.12, Median = 17. SD = 15.87), and 26 participants (53%) were younger than 18 years old. The majority of the participants included were male (n = 46; 94%). The most common reported race/ethnicity was White (n = 39), followed by Asian (n = 4), Black (n = 3), and Hispanic (n = 3). Participants’ diagnosis or disability classification was diverse across participants and are detailed in Table 1.
Studies Included in Meta-Analysis.
Note. QIS = Quality Indictor Score percentage value; ASD = Autism Spectrum Disorder.
The studies included in the meta-analysis reported a range of disruptive behaviors as dependent variables. The most common dependent variable was physical aggression (n = 36), followed by academic off-task academic behavior (n = 7), challenging behaviors (e.g., physical and verbal aggression, tantrums, and disruptive behavior; n = 3), bullying (n = 1), verbal aggression (n = 1), and work disruption (n = 1).
SOF Training Characteristics
Most participants were trained in SOF directly by a clinician who was experienced in delivering SOF as well as other MBPs (n = 37). Some participants were trained in SOF by a non-clinician without previous experience in mindfulness or SOF, including a participant’s parent (n = 6), teacher (n = 3), and school mental health counselor (n = 3).
Most participants were trained in SOF using the standard format of 5 days of 30 minutes training sessions (n = 28). Some participants were trained in SOF in a modified format that was tailored to the individual’s needs. These modified formats ranged in terms of duration of training (10 to 25 days) and training session time (15–60 minutes).
Study Design Characteristics
All of the studies included in the meta-analysis employed a multiple-baseline design across participants. Most studies relied on a third-party directly observing target behaviors (n = 14), however one study utilized self-report methods to measure target behaviors (n = 1). Further, 14 of the 15 studies collected IOA reliability data, with the IOA percentages all greater than 80%. SOF intervention fidelity data was reported in eight of the studies, with fidelity percentages all greater than 80%. Finally, SOF intervention acceptability was assessed in 5 of the 15 studies, with results suggesting that the SOF intervention is generally acceptable for participants with and without educational disabilities and among interventionists.
Overall Effect
The overall effect of the SOF intervention on aggressive and disruptive behavior was examined across all 15 studies, yielding an aggregated Tau-U effect size of −0.87, SE = 0.03, 95% CI [−0.93 to −0.81]. Figure 1 illustrates the weighted Tau-U effect sizes and 95% CI’s for all individual studies and the overall effect aggregated Tau-U effect size. There was no statistically significant correlation between individual study’s Quality Indicator Score and Tau-U effect size; rs(15) = −0.15, p = .60.

Forest plot for the weighted Tau-U effects and 95% confidence interval of the SOF intervention on disruptive behavior, and the overall aggregated Tau-U effect size estimation.
Tau-U, as a correlation measure of rank data (Parker et al., 2011), belongs to a family of Kendall’s Tau and typically takes a value between −1 and 1 (Brossart et al., 2018). Aggregated Tau-U effect size can be evaluated using standard interpretation of rank correlation. The 95% CI [−0.93 to −0.81] did not contain the value of −0.80, which indicated the global effect size Tau-U of −0.87 was statistically different from −0.80 (p < .05). The overall value of Tau-U of −0.87 indicates a strong correlational effect size in the discipline of psychology and a very strong effect size in medical research (Akoglu, 2018). In addition, using Kendall’s “Tau-r” formula, that is, r = sin (.5*π*Tau) in Walker (2003, p. 526), we converted the global effect size of −0.87 and obtained a Pearson’s r value of −.98, which also indicated a large effect size in Cohen’s (1988) recommended rules. Finally, we used the “Tau-d” formula Tau = 1 − (1 − d/3.464)2 in Bowman-Perrott et al. (2013, p. 46), and obtained a large Cohen’s d of 2.2 (>0.8, Cohen, 1988) from the absolute value (0.87) of the overall effect Tau-U.
Moderator Analyses
Moderator analyses were conducted following procedures outlined by Dart et al. (2014) to determine whether any participant or design characteristics were predictive of the effectiveness of SOF for disruptive behavior across studies. Prior to analysis, potential characteristic variables were examined in order to determine if there were viable sample sizes support interpretation of quantitative data.
At the participant level, age was included as a moderating characteristic, both as a continuous variable (years of age) and as a categorical variable (youth < 18 years of age, n = 26; adult ≥ 18 years of age, n = 23). Given the heterogeneity of participant’s diagnosis/disability, it was determined that it was not possible to meaningfully examine all the specific sub-categories of this variable; however there were comparable numbers of participants with/out intellectual disabilities, and subsequently this variable was collapsed into those with intellectual disabilities (n = 21) and those without intellectual disabilities (n = 28). The decision was made to not examine any other participant characteristic given the heterogeneity of these variables and correspondingly limited sample sizes.
SOF training characteristics were also examined for viable inclusion in moderating analyses. The majority of participants were trained in the SOF procedure by an experienced MBP and SOF mental health clinician (n = 37), with the remaining 12 participants trained by an individual without prior experience with any MBP (parent n = 6; teacher n = 3; school mental health counselor n = 3). SOF training was mostly delivered in the standard format of 5 days of 30 minutes training (n = 28); however, the other (n = 21) non-standardized SOF training formats were somewhat variable in terms of the number of days of training (range 10–15 days) and session length (range 15–60 minutes). The decision was made to examine the SOF training characteristics as dichotomous variables: experienced/inexperienced SOF trainer and standard/non-standard delivery format.
Participant-level phase contrast Tau-U effect size scores were the dependent variable for these moderation analyses. Given the small and, at times, unbalanced sample sizes used in these analyses, the a priori decision was made to utilize and report (in order) parametric and non-parametric statistics, respectively.
Age, Intellectual Disability, SOF Trainer, and SOF Format
As a continuous variable, age was not statistically significantly related to the effect of SOF on disruptive behavior: r(47) = 0.019, p = .90; rs(47) = .16, p = .27. As a categorical variable, there was no difference between youth and adult’s response to SOF for disruptive behavior; t(47) = 1.21, p = .23; U = 367, p = .17. SOF’s effect on disruptive behavior also did not differ depending on whether the participant had an intellectual disability; t(47) = 1.14, p = .17; U = 364, p = .156, who was delivering the curriculum; t(47) = 1.14, p = .26; U = 200, p = .61, or the format of delivery; t(47) = 0.54, p = .60; U = 252, p = .40.
Discussion
The meta-analytic synthesis of high-quality SCRD of SOF indicate that this procedure is a moderately effective strategy for reducing disruptive behaviors (Dart et al., 2014). The effectiveness of SOF was demonstrated by five independent research teams involving 49 participants and the effect sizes did not vary as a function of the quality of study methodology. Moderator analyses indicated that SOF’s effectiveness was robust across diverse participant characteristic (i.e., age and intellectual disability status) and intervention delivery formats. These results indicate that SOF is an evidence-based practice for disruptive behavior (Horner et al., 2005).
Limitations and Future Directions
This meta-analysis indicated a statistically significant functional relation between the SOF protocol and reductions in disruptive behavior; however, it did not explore the clinical significance of SOF for disruptive behavior. Clinical significance is defined as the applied value of an intervention and whether it makes a practical difference in the day-to-day life of the participant (Kazdin, 1998). Considering this definition, SOF does seem to yield clinically significant outcomes for the participants included in this review, as many of the disruptive behaviors were likely highly impairing for individuals and their caretakers, and following SOF these behaviors reduced to very low levels. For example, Singh, Lancioni, Manikam, et al. (2011) measured aggressive behaviors of adolescents with Autism Spectrum Disorder, with aggressive behavior operationalized as hitting, kicking, or biting family members. During baseline data collection, these adolescents were engaging in these behaviors approximately 15 times per week, and following SOF these behaviors reduced to ~0 to 1 times per month. The practical difference of living with a family member who engages in far lower rates of physical aggression is evident and underscores the pragmatic utility of SOF in this study. However, the clinical significance of SOF is ultimately determined relative to the specific context of the individuals under investigation, as the applied practical value of behavior change is subjective and unique to one’s circumstances and values. Future research should therefore carefully consider the idiographic value of SOF as a meaningful intervention to explore its clinical significance, for example by conducting qualitative interviews of participants post-intervention and by collecting acceptability data from relevant stakeholders.
Although the research included in this review was conducted by five independent research teams, the majority (67%) of the research was executed by a single investigator’s research team (Singh). Although unlikely given these replications across different laboratories, it is possible that the results obtained may have biased by unique characteristics of this single research team. Additional replication studies of SOF by independent research teams would be useful to further support the evidence-base of this protocol.
Of the studies included in this meta-analysis, only approximately half recorded and reported implementation fidelity. This represents a considerable limitation of the literature on the efficacy of the SOF intervention for reducing disruptive behavior. Without monitoring treatment integrity, researchers cannot be confident in their conclusions regarding the functional relation between the independent and dependent variable (Gresham et al., 1993). As Noell and Gresham (1993) pointed out, research concerning the degree and enhancement of intervention implementation has become increasingly prevalent in many applied fields (e.g., school psychology). Future researchers using the SOF intervention should take the steps necessary to ensure that implementation fidelity is monitored in order to more definitively conclude that the intervention, rather than extraneous factors, is responsible for observed changes in disruptive behavior.
The existing literature does not indicate if this program is effective for diverse functions of behavior. Existing conceptualizations of this procedure theoretically suggest that SOF may be most effective as an antecedent-based strategy (i.e., an abolishing operation) for the negative reinforcement of escape-maintained behavior (Felver & Singh, 2020); however, empirical data exploring this topic have not been published to date and future research should systematically evaluate the effectiveness of SOF across different functional classes of disruptive behavior.
This meta-analysis amalgamized SOF’s effects on disruptive behavior; however, in doing so this work was not able to explore the relative effectiveness of SOF across different subtypes of disruptive behavior (e.g., verbal aggression, instrumental aggression). Future research should consider evaluating the relative effects of SOF for different subtypes of disruptive behavior, as there may be differential responsiveness to intervention, and such research may potentially identify relative effectiveness of SOF for specific high-priority disruptive behaviors that are less responsive to common treatments (e.g., medication intervention for instrumental aggression; Singh, Lancioni, Winton, Adkins, Wahler, et al., 2007). In addition, the covert nature of the SOF behavioral routine makes it difficult to measure whether an individual is actually deploying the routine, and subsequently the existing literature has not explored this critical characteristic of the protocol. Future research should consider measuring this characteristic, such as by having the participant log their use of the procedure through ecological momentary assessment or daily journal entries.
The participants included in this review were highly diverse in terms of their categorical disability or diagnoses, however most of the participants were racially White and in some instances race was not reported directly in the manuscript. Future research should report detailed participant characteristics including race and ethnicity, and focus on evaluating SOF among historically marginalized groups with known health disparities related to disruptive behavior. Although our analyses revealed that the effectiveness of SOF does not vary as a function of participants’ disability status, further research should shed light on the extent to which SOF addresses disruptive behavior in general classroom settings. Disruptive behaviors in the classroom create barriers for effective instruction and present a serious challenge for educators (Ling et al., 2011). The SOF intervention may be an efficient tool that educators can implement to teach students to self-manage their disruptive behaviors in order to improve students’ ability to meet instructional goals. Although three studies included in our meta-analysis evaluated SOF for reducing disruptive behavior among students with disabilities (Felver et al., 2017; Singh, Chan, et al., 2017; Singh, Lancioni, Singh Joy, et al., 2007), only one study evaluated SOF within the context of a general education classroom (Felver et al., 2014). Therefore, future research is needed to validate the use of SOF for behavioral at-risk students at the targeted, rather than intensive, level of intervention.
This study contrasted participant behavior between baseline and intervention phases of each identified study. Given that SOF involves explicitly teaching both a new strategy (i.e., the SOF routine) and continuing to use this strategy during specific situation (i.e., settings that occasion disruptive behavior), there is not a clearly identifiable end point where the intervention phase ends and the maintenance phase begins. This fact is illustrated by intervention phase variability among these studies (mean = 39 weeks; SD = 33 weeks; IQR = 15–46 weeks). These data indicate a relative strength of SOF in that there appears to be evidence of maintenance for reduction to disruptive behavior for many participants lasting months (and in some cases years) after first being introduced to SOF. Conversely, intervention phase variability also reveals a limitation because it is difficult to definitively conclude how long such intervention maintenance may last. Future research may consider careful analysis of the stages of SOF skill acquisition, such as by applying the instructional hierarchy (Martens & Witt, 2004), to clearly identify the precise points individuals acquire (i.e., reach an accuracy criterion in the absence of prompting) or become fluent with (i.e., rapidly apply) the core SOF skills. Relatedly, the point at which individuals demonstrate maintenance in applying SOF skills would require close monitoring of reductions in instructional support (e.g., prompt fading or discontinued self-recording) whereas generalization to novel contexts would require data collection across setting during both baseline and intervention phases.
Conclusion
Taken as a whole, results suggest that SoF is an effective and robust protocol for disruptive behavior across heterogeneous contexts and participants. The evidence indicates that SOF should be considered as a protocol for individuals with disruptive behavior, particularly given the move in health care toward brief and time-limited therapies. To date, only one study has utilized an RCT design to explore the effects of SOF for disruptive behavior (Singh et al., 2013). Although this study indicated the suitability of SOF for group-design methodology, it utilized a wait-list control condition and thus strong inferences of the effectiveness of SOF relative to existing best practices cannot be strongly concluded. Given the effectiveness of SOF across the high-quality SCRD studies included in this meta-analysis, future researchers and funding entities should consider conducting large-scale group design RCT employing rigorous active control comparison conditions (e.g., cognitive-behavioral therapy) to study the scalable deployment of SOF across larger participant groups (e.g., many students across a school district; across a network of therapeutic group homes serving individuals with higher rates of disruptive behavior) and continue to scientifically explore the utility of this intervention.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Felver and Singh have published the school-based version of Soles of the Feet curriculum in Mindfulness in the Classroom (New Harbinger Publications, 2020) and receive royalties from the sale of this book. There are no other conflicts of interest to disclose for all authors.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
