Abstract
Selection of a special education evidence-based practice (EBP) requires developing an understanding of what interventions work as well as for whom they are effective. This review examined participant characteristics in the EBP literature for learners with autism spectrum disorders (ASD) identified by the National Professional Development Center on Autism Spectrum Disorders. Results indicated very limited representation of diverse participants in the entire body of research, and when reported, White youth represented a large majority of study participants. This work is an attempt to begin to better understand the extent that various contextual factors are reported in a body of literature used to identify EBPs. Implications for ASD research are discussed along with recommendations for future research.
People with autism spectrum disorder (ASD) vary substantially in their functioning levels. Some individuals with ASD are less affected and can more readily communicate and interact in ways consistent with social demands. Other individuals may be unable to speak, experience limited gains in communication skills, and exhibit frequent and intense behaviors. The heterogeneous nature of learners with ASD is linked to differences in how the disorder affects social, communication, and behavioral development. Accordingly, an understanding of an array of interventions to address the dynamic needs of these learners was and continues to be a priority of ASD researchers in special education and related disciplines (e.g., National Autism Center [NAC], 2009, 2015; Simpson, 2005; Wong et al., 2014).
Evidence-Based Practices (EBP)
The EBP movement in special education represents a collective value for advancing the field’s knowledge about how to best serve students with disabilities, including those with ASD. The Council for Exceptional Children (CEC, 2008) defines EBP as “a strategy or intervention designed for use by special educators and intended to support the education of individuals with exceptional learning needs” (p. 6). Others have stated that EBP is “professional wisdom supported by empirical research” (Whitehurst, 2002), and “practice that is informed by credible research” (Cook, Tankersley, Cook, & Landrum, 2008). Major turning points in identification of rigorous standards for conducting and evaluating empirical research in ASD include the special issue highlighting single subject research in Exceptional Children (Horner et al., 2005), the pilot criteria for evaluating single subject research by the What Works Clearinghouse (Kratochwill et al., 2010), and the standards for EBPs in special education developed by CEC (Cook et al., 2014). Numerous comprehensive methodological reviews have applied these criteria to the extant research literature, yielding a handful of leading publications that have brought increased clarity about effective practices for learners with ASD. These include the recommendations for helping professionals become better consumers of intervention methods for students with ASD reported by Simpson (2005), the National Standards Report by the NAC (2009, 2015), the EBPs review by Odom, Collet-Klingenberg, Rogers, and Hatton (2010), and the report by the Evidence-Based Practice Review Group at the University of North Carolina, Frank Porter Graham Child Development Institute (Wong et al., 2014).
Contextual Factors
These reviews increased confidence about intervention effects for learners with ASD while highlighting gaps in knowledge. One gap relates to contextual factors that moderate and mediate intervention effects. Contextual factors include but are not limited to race, gender, age, educational level, socioeconomic status, disability type, and severity. For example, the International Classification of Functioning, Disability, and Health explained these and other contextual factors as critical to a framework for measuring health and disability at both individual and population levels (World Health Organization, 2001). In this framework, disability is seen as a result of an interaction between a person (with a health condition) and the environmental and personal (i.e., contextual) factors.
The consideration of contextual factors is foundational to special education philosophical values regarding tailoring interventions and supports to the unique needs of learners while simultaneously capitalizing on student strengths, interests, and preferences, as well as incorporating parent (and student) concerns, values, and desired outcomes. Answering the question, “What works?” is important in the progression of evidence-based special education, but an additional and logical step is to understand what works for whom and under what conditions (Dyches, Wilder, Sudweeks, Obiakor, & Algozzine, 2004; Easton, 2014). Researchers must carefully consider and maintain close attention to contextual factors inherent to applied experiments because, although an important advancement, the menu of EBPs serve only as starting points for professionals and are not a panacea for students with ASD. Importantly, status as an EBP does not constitute guaranteed attainment of intended positive effects for every learner with ASD. “Unless the participants are clearly identified, it is impossible to connect an inter-vention with an individual who might benefit from it. This is especially important with ASD since no two individuals are alike, complicating the selection process” (West, McCollow, Umbarger, Kidwell, & Cote, 2013, p. 448).
Ascribing EBP status to a practice according to empirically sound methodology and positive results is helpful for developing repositories about what works. However, interventions may not always translate to expected benefit. Understanding how and to what extent various contextual factors, including the complex interplay between an individual’s unique history (i.e., experiences) and their current environment, affect research findings seems central to advancing the EBP movement. Identifying “what works for whom and under what conditions” means clarifying whether desired effects are obtained for participants with particular demographic profiles and understanding to what extent contextual factors diminish and intensify intervention response. Arranging environmental conditions consistent with learner needs is the essence of effective teaching. For example, explicit and systematic instruction is critical for achieving reading mastery (e.g., Pullen & Hallahan, 2015) as is organizing consistent activities, routines, and procedures with reinforcing consequences for social-behavioral improvement (Cihak & Ayres, 2010). This means special education professionals must develop and refine relationships with students and families in ways that enhance teacher ability to arrange effective instructional conditions (Odom & Wolery, 2003; Simpson, Mundschenk, & Heflin, 2011). Accordingly, special education and related professionals charged with educating students with ASD attempt to account for numerous proximal and distal contextual factors that influence learning. These factors can include family primary language, family orientation style, economic status, ethnicity, acculturation status, religious beliefs, access to community services and supports, food/nutrition status, homelessness, parent/caregiver mental illness or addiction status, parent educational attainment, parent disciplinary style and beliefs, and access to pediatric care. The role of personal factors, such as culture, language, ethnicity, socioeconomic status, biological and physiological factors, as well as strengths and weaknesses, preferences, and individual histories of reinforcement in intervention outcomes for students with ASD, cannot be discounted. Indeed, the contextual factors attached to participants have varying degrees of influence on the educational progress of each student with ASD.
Special educators must align curriculum, methods, strategies, and supports with the culture, primary language, ethnicity, socioeconomic status, biological and physiological factors, and individual histories of their students. They also must account for the ways these factors are constantly influencing and being influenced by educational environments. Professionals and researchers evaluate how various contextual factors affect practical and scientific results of an intervention. Relatedly, an EBP requires a scientific examination of the intensifying, attenuating, and (apparently) benign effects of various contextual factors on intervention effectiveness. “A variable may turn out to be relevant in contexts never suspected by the experimenter, or it may turn out to be minor and of minimal systematic importance” (Sidman, 1960, p. 40). Consistent with an EBP, only empirical research can advance our understanding of this issue. However, the extant literature indicates participants from a variety of racial and ethnic groups are not well-represented in ASD intervention research, and when diverse individuals volunteer to participate, details about contextual factors may be omitted or insufficiently described.
ASD, Diversity, and Intervention Research
Ethnically and linguistically diverse children with ASD and their families have long been underrepresented in ASD research (Hilton et al., 2010; Kistner & Robbins, 1986; Pierce et al., 2014; Tek & Landa, 2012). Wong et al. (2014) discussed the limited diversity of participants as a limitation of their review. Specifically, these authors pointed out they did not collect demographic information on the gender, race, and ethnicity of the participants in the studies. They explained, however, their belief (based on their reading of the studies) that most participants were White-Caucasian and that many researchers did not report participant race or ethnicity. Wong et al. said ASD researchers must begin recruiting and reporting ethnically diverse participants with ASD. Pierce et al. (2014) appeared to be the first to systematically examine a body of ASD literature for participant diversity. Pierce et al. examined the research published in three leading ASD-related journals in 2000, 2002, 2004, 2006, 2008, and 2010 for reports of participant ethnicity. They found 72% of articles did not include descriptors of race or ethnicity for participants. These researchers concluded that external validity of intervention research is hampered by limited participant diversity and that prospective replication and correlational research will be constrained by the omission of ethnic, racial, and cultural contextual details about participants.
Group Research Designs
Group research designs require extensive detail about participants because the methods used do not establish functional relations between independent variables and participant responding. The intention is to make causal inferences using hypothetico-deductive reasoning and statistical inference from representative samples to a population (Kennedy, 2005; Shadish, Cook, & Campbell, 2002). However, it is difficult to draw inferences without sufficient participant background data. Thus, extensive attention to participant and population characteristics is needed to ensure representativeness of the sample from which inferences will be made. Given that group researchers aim to generalize findings, participant details in group research are fundamental. However, individuals with ASD are a relatively small population who vary substantially in functioning due to unique experiences and impact of their disability. Characteristics of ASD can be manifested in numerous ways and also are amplified by individual preferences and interests, family dynamic and orientation, family education and economic opportunity, obvious and nuanced cultural differences, and various other factors associated with local, state, regional, national, and global contexts (e.g., cost of living, unemployment, immigration law, social services, racism, war).
Group research provides an average depiction of intervention effect and may prevent a full understanding as to who and why some participants responded very well or not at all. These heterogeneous and varied responses make group research of interventions for learners with ASD particularly difficult and costly. Representative sampling is resource-intensive and highly complicated and may be logistically impossible due to the low incidence of ASD. Homogeneity of groups also may discourage researchers from using group designs, especially because large groups are necessary for building sufficient statistical power. Although participants in a group study may have responded positively to an intervention, external validity of findings typically is constrained by the limited representativeness of participants (i.e., White, middle-class children with ASD who have educated, skilled, and resourceful parents as well as highly trained and willing teachers), population heterogeneity (e.g., ASD severity), and other contextual factors that command researcher consideration. These constraints are perhaps why a majority of intervention research for learners with ASD employs single subject research designs (Wong et al., 2014).
Single Subject Research Designs
Single subject research intends to establish causal relationships by manipulating independent variables (i.e., environment) to observe effects on individual behavior rather than inferring generality of findings to a population. External validity associated with group research is replaced by systematic replication of intervention (i.e., independent variable) effects on learner behavior (i.e., dependent variable). To that end, single subject research prioritizes establishment of a stable baseline and experimenter control of behavior over an in-depth description of participant characteristics to establish external validity (Wolery & Ezell, 1993). Nevertheless, describing participants with sufficient detail is an essential aspect of systematic replication associated with single subject research and modern meta-analytic techniques (Gast, 2010; Johnston & Pennypacker, 2009; Kennedy, 2005; Tawney & Gast, 1984). Despite this, confusion exists among special education researchers as to the extent to which participant characteristics are relevant to single subject research, and this confusion appears related to the influence of group research design in education (Wolery, 2013).
Horner et al. (2005) suggested researchers using single subject design to identify EBPs in special education include in their description “operational descriptions of the participants, setting, and the process by which participants were selected” (p. 166). They further explained specific disabilities of participants be reported along with diagnostic instruments used. Horner et al. did not include specific recommendations about participant demographics, but this could be reasonably inferred. The single subject designs technical document by Kratochwill et al. (2010) did not provide explicit guidance for participant characteristics to be included in research reports. According to CEC Standards for EBPs (Cook et al., 2014), researchers have to provide sufficient information about the participants, including gender, age, race/ethnicity, socioeconomic status, and language ability. In addition, the disability (e.g., ASD) or risk status of the participants (e.g., at risk for reading failure) and method for determining risk status (e.g., rating scale, standardized assessment) should be described as a piece of demographic information for the participants. Furthermore, the CEC indicators included requirements for reporting details about intervention agents, including whether they were teachers, researchers, paraprofessionals, parents, volunteers, peers, siblings, or a technological device. Importantly, the CEC standards were developed for application to single subject and group design research, whereas Horner et al. and Kratochwill et al. focused on single subject design.
Participant Diversity in EBPs
Although special education practices for learners with ASD have made substantial advances toward an evidence-based approach by conducting very large systematic reviews to identify what works, the extent to which diverse participants are represented in high-quality ASD intervention research remains unclear. If group studies include and sufficiently describe samples of participants with particular cultural, ethnic, linguistic, and economic backgrounds, then confidence in the generality of an EBP supported by such studies to similarly diverse students may be increased. Relatedly, if studies using single subject research designs incorporate detailed descriptions of diverse participants, then confidence in the potential replication of effect on other learners with similar profiles may be enhanced. Conversely, limited or no representativeness of diverse participants would indicate a need for researchers to pursue questions about the efficacy of EBPs for underrepresented groups while also advising practitioners to be particularly diligent in monitoring intervention effectiveness. To address a limitation of Wong et al. (2014), extend Pierce et al. (2014), and develop a better understanding of participant diversity in the EBP research for learners with ASD, we analyzed the research articles used by the National Professional Development Center on Autism Spectrum Disorder (NPDCASD) to qualify EBPs for participant demographics related to race, ethnicity, and nationality (REN). The following research questions guided our work:
Method
Data Collection and Analysis
Reviewers and review criteria
Review criteria were developed by identifying demographic categories associated with high-quality special education research (Cook et al., 2014; Gersten et al., 2005; Horner et al., 2005). All authors held faculty positions in special education across seven universities and had reported experience in data collection and analysis procedures. Each author served as a reviewer and examined studies from specific EBP categories based on interest and expertise in the intervention type. The reviewers retrieved the respective studies and examined them for the presence of participant information consistent with the aforementioned demographic categories. Each reviewer examined between 48 and 132 articles. Study participants whose REN was reported but did not have ASD were excluded.
Adjustments to data collection
Adjustments to the type of data to be collected were made via discussion with the team members when unanticipated issues regarding reporting occurred. Three specific issues arose. First, when parents were participants in a study that also included their child (e.g., a parent-implemented intervention), the authors discussed whether and how to include information in the database. Thus, the team decided to count each participating parent and child independently and included their respective demographic data, if reported. Second, the team did not anticipate international studies that did not explicitly describe the REN of the participants. For international studies, the team agreed to presume the nationality of participants according to where the study occurred when race or ethnicity was not explicitly stated. Last, the team did not foresee that some studies might report parent primary language, but not child participants. The team decided to presume child primary language only if parent language was reported and the study occurred somewhere other than North America or United Kingdom. No presumptions of language, race, or ethnicity were made when the study occurred in North America or United Kingdom. Instead, reviewers indicated the study did not adequately report participant REN. When necessary, individual reviewers revisited studies for consistency with the amended procedures. After all studies from Wong et al. (2014) were independently examined and the individual databases were completed, they were combined into a single database and interrater reliability procedures were initiated.
Interrater Reliability
To ensure the reliability of the collected data, we instituted a multistep process that included interrater training, independent data collection, results comparison, and scoring calculation. One author not involved in initial coding was provided with an empty database template, five pre-selected articles, and the instructions the other authors used for completing the table. Upon completing coding for the five articles, the interrater met with the first author and compared results of the five studies with previously collected data. The total number of agreements was divided by the total number of agreements plus total number of disagreements and multiplied by 100. This reliability training resulted in 100% agreement. Next, approximately 30% (n = 122) of articles were randomly selected from the pool of 408 included by Wong et al. (2014) and were provided to the reliability rater. The reliability rater then completed coding and provided results to the first author who then calculated reliability. To maintain a conservative reliability estimate, an agreement was obtained only when all data (i.e., race, ethnicity, and/or nationality) for the study collected by the interrater were identical to the data collected by the original reviewer (i.e., study-by-study reliability). If one datum (e.g., race) for a study differed between the two raters, a disagreement was scored for the entire study. The number of agreements was divided by the number of agreements plus number of disagreements and multiplied by 100%. Interrater agreement was 90.4%.
Results
Table 1 shows the number and percentage of studies reporting REN for all participants in studies that used group designs, single subject designs, and both methods combined. In this study, we analyzed all 408 studies included in Wong et al. (2014). Reporting of participant REN for group design studies varied by experimental design used and EBP. There were 39 group design studies associated with 12 EBPs. Fifteen EBPs did not have a group design study (Wong et al., 2014). Of the 12 EBPs supported by a group design study, five had only one study that reported REN for all participants (modeling, pivotal response training, prompting, technology-aided instruction and intervention, and video modeling). Two EBPs included more than one study that reported REN of all participants with the percentage ranging from 25% (parent-implemented intervention) to 57.1% (social skills training). There were five EBPs that comprised a total of 11 (28.2%) group design studies that did not adequately report REN for any participants (cognitive behavioral intervention, exercise, picture exchange communication system, scripting, and structured play group). In other words, none of those EBPs were associated with studies that sufficiently described REN of participants.
Number and Percentage of Studies Reporting REN for All Participants in Group Designs, Single Subject Designs, and Overall Total.
Note. REN = race, ethnicity, and nationality; EBP = evidence-based practice; ABI = antecedent-based intervention; CBI = cognitive behavioral intervention; DRA/I/O = differential reinforcement of alternative, incompatible, or other behavior; DTT = discrete trial teaching; ECE = exercise; EXT = extinction; FBA = functional behavior assessment; FCT = functional communication training; MD = modeling; NI = naturalistic intervention; PII = parent-implemented intervention; PMII = peer-mediated instruction and intervention; PECS = picture exchange communication system; PRT = pivotal response training; PP = prompting; R+ = reinforcement; RIR = response interruption/redirection; SC = scripting; SM = self-management; SN = social narratives; SST = social skills training; SPG = structured play group; TA = task analysis; TAII = technology-aided instruction and intervention; TD = time delay; VM = video modeling; VS = visual support.
Wong et al. (2014) included 369 studies that used single subject designs. All 27 EBPs were associated with at least one and up to 43 studies that used single subject designs. The number of single subject design studies that adequately reported REN totaled 62, or 16.8%. Every EBP except cognitive behavioral intervention included studies that did not adequately report participant REN. Cognitive behavioral intervention was associated with one single subject design study (Singh et al., 2011) that reported REN of all participants. Social narratives had the highest number of single subject studies that reported REN with nine studies or 52.9% of all studies on the topic (Wong et al., 2014). Video modeling and antecedent-based intervention each had seven studies that reported REN for all participants. Of the 10 single subject design studies of response interruption/redirection, five (50%) adequately reported REN. Of the 27 EBPs, six did not have any studies that adequately reported REN of all participants (discrete trial teaching, extinction, functional communication training, picture exchange communication system, scripting, and structured play group). Collectively, these six EBPs included 50 (13.5%) of the 369 single subject research studies identified by Wong et al.
Table 1 shows the combined number and percentage of studies that reported REN for each EBP. A total of 73 (17.9%) of 408 studies reported REN. Seven EBPs combined for a total of 31 studies that reported REN for at least 40% of participants: functional behavior assessment (40%), modeling (40%), naturalistic interventions (40%), pivotal response training (42.9%), response interruption/redirection (50%), social narratives (52.9%), and social skills training (40%). Six EBPs combined for a total of 55 studies, of which none reported REN of all participants: discrete trial teaching, extinction, functional communication training, picture exchange communication system, scripting, and structured play group. Other EBPs had relatively low percentages of studies with REN reported.
There were a total of 2,489 participants in the 408 studies included in Wong et al. (2014). Of the 2,489 participants, we found 770 (31%) had reported REN. Table 2 provides the number and percentage of participants from demographic categories for each EBP. We grouped REN into seven categories according to geographic regions for illustrative purposes: White, Asian, Black, Hispanic/Latino, Middle Eastern, multiracial, and Native American. Of the 770 total participants whose REN was reported, 489 (63.5%) were White. Multiracial participants comprised 159 (20.6%) of the total reported. Black participants and Asian participants were represented with 52 (6.8%) and 40 (5.2%), respectively. There were 18 (2.5%) Hispanic/Latino participants, 10 (1.3%) Middle Eastern participants, and 1 (0.1%) Native American participant in the 770 participants whose REN were reported.
Number and Percentage of Participants by Race for Each EBP.
Note. White comprised of participants described as White; European American; Euro-American, White, Caucasian; Caucasian; and White British, Polish, Dutch, Tasmanian, New Zealander. Asian comprised of participants described as Asian American, Asian, Chinese, Vietnamese, Korean, Taiwanese, Taiwan, and Chinese Canadian. Black (not Hispanic) is comprised of participants described as African American, African Caribbean, and Somali. Hispanic/Latino comprised of participants described as Hispanic/Latino, Hispanic American, Latino, Hispanic, Mexican American, and Spanish Speaking. Multiracial was comprised of participants described as not Caucasian, Non-White, minority, multiracial/Other, Other Mixed (one parent White/other non-White), Biracial, and African American/Latino. Middle Eastern was comprised of participants described as Middle Eastern, Indian, and Turkish. EBP = evidence-based practice; REN = races, ethnicities, and nationalities; ABI = antecedent-based intervention; CBI = cognitive behavioral intervention; DRA/I/O = differential reinforcement of alternative, incompatible, or other behavior; DTT = discrete trial teaching; ECE = exercise; EXT = extinction; FBA = functional behavior assessment; FCT = functional communication training; MD = modeling; NI = naturalistic intervention; PII = parent-implemented intervention; PMII = peer-mediated instruction and intervention; PECS = picture exchange communication system; PRT = pivotal response training; PP = prompting; R+ = reinforcement; RIR = response interruption/redirection; SC = scripting; SM = self-management; SN = social narratives; SST = social skills training; SPG = structured play group; TA = task analysis; TAII = technology-aided instruction and intervention; TD = time delay; VM = video modeling; VS = visual support.
Two EBPs accounted for a majority of the 770 participants. Parent-implemented intervention and social skills training accounted for 36.2% and 19.2% of the 770 participants. Of the 279 participants in parent-implemented intervention, 144 (51.6%) were White and 133 (48%) were identified as multiracial. Of the 148 participants in social skills training, 114 (77%) were White, 11 (7%) were Asian, and 10 (7%) were Black. The remaining 13 social skills training study participants were Hispanic/Latino (4%), Middle Eastern (2.0%), or multiracial (3%). Nineteen of the 27 EBPs had studies that adequately reported the REN for fewer than 15 participants, with six not reporting REN for any participants.
The number and percentage of reported participant gender is provided in Table 3. Of the 2,489 participants, 1,848 (74%) were male, 361 (15%) were female, and 280 (11%) were unreported. The representation of participants in studies that did and did not report REN is reported in Table 4. Of the seven aggregate racial categories, White participants were the largest percentage of participants overall at 19.6% of the 2,849 total participants. White participants also represented a significant majority of participants among those whose REN was reported at 63.5% or 489 of the 770 participants. Multiracial participants comprised about 6.4% of all participants in the studies used to qualify EBPs and were 20.6% or 159 of the 770 participants whose REN was reported. Black participants comprised only 2.1% of all participants and were 6.8% or 52 of the 770 participants whose REN was reported. Native American participants represented 0.1% or 1 of the 770 participants.
Gender of Participants in Studies by Each EBP.
Note. EBP = evidence-based practice; ABI = antecedent-based intervention; CBI = cognitive behavioral intervention; DRA/I/O = differential reinforcement of alternative, incompatible, or other behavior; DTT = discrete trial teaching; ECE = exercise; EXT = extinction; FBA = functional behavior assessment; FCT = functional communication training; MD = modeling; NI = naturalistic intervention; PII = parent-implemented intervention; PMII = peer-mediated instruction and intervention; PECS = picture exchange communication system; PRT = pivotal response training; PP = prompting; R+ = reinforcement; RIR = response interruption/redirection; SC = scripting; SM = self-management; SN = social narratives; SST = social skills training; SPG = structured play group; TA = task analysis; TAII = technology-aided instruction and intervention; TD = time delay; VM = video modeling; VS = visual support.
Representation of Participants in Studies That Reported and Did Not Report REN.
Note. REN = race, ethnicity, and nationality.
Analysis of the disaggregated categories indicates similar patterns. We obtained from the studies reporting race for all participants a total of 27 subgroups associated with broader racial or ethnic categories. One group, “White/European American/Caucasian,” comprised a significant majority of the research participants with 62.3% or 480 of the 770 participants whose REN was reported. None of the remaining 27 subgroups exceeded 9% of the reported racial demographics, with “Mixed, Mixed with one White and one Non-White Parent” comprising 8.8% and “Not Caucasian, Non-White” comprising 7.9% of the 770 participants with REN reported. The African American subgroup was 6.4% or 49 of the 770 participants and the “Hispanic/Latino; Hispanic American; Latino; or Hispanic” subgroup was 2.1% or 16 of the 770 participants with REN reported. The remaining 22 subgroups represented 2% or less of the participants whose demographics were reported.
Discussion
This review examined participant characteristics in the EBP literature for children and youth with ASD identified by the NPDCASD (Wong et al., 2014). We found low rates of reporting of participant REN overall as well as differences between EBPs. We also found that a large majority of participants whose REN was reported were White, with large concentrations in the parent-implemented intervention literature as well as the social skills training literature. This confirms Wong et al.’s suspicions and supports findings obtained by Pierce et al. (2014). Our main finding was that the EBP literature does not adequately include or report participants whose race or ethnicity is something other than White, Caucasian, or European American.
The Extent to Which the Studies Described REN
With regard to our first research question, we found that a relatively small percentage (17.9%) of studies included by Wong et al. (2014) reported REN of all participants. This seems to support findings obtained by Pierce et al. (2014) that indicated only 28% of articles adequately reported race or ethnicity of participants. Given that group research designs emphasize participant characteristics to inform inferences of findings to larger populations, it was not surprising that studies using group designs reported participant REN more frequently (28.2%) than studies that used single subject designs (16.8%). However, the percentage reporting REN in group studies is low, suggesting that REN may be perceived by researchers as having little bearing on generalized inferences drawn from study results. Similarly, single subject research emphasizes systematic replication of research findings to establish generality of findings, meaning that explicit details about participants is important for increasing confidence that an intervention will work for learners with similar profiles (Kazdin, 2010). Given that 90% of the 408 studies used to qualify EBPs utilized single subject design, and the fact that single subject research is widely used in special education research, the fact that few single subject design studies reported these basic demographics warrants attention from ASD intervention researchers and perhaps special education researchers in general. If researchers are unaware of the potential and actual differences between participants within and across studies, and if the participants in these research studies represent a small percentage of individuals affected by ASD, then it may be difficult for special education and related professionals to select and apply interventions for learners from non-White racial and ethnic groups.
Also related to our first question, we found that social narrative intervention had the highest number of studies that reported REN for all participants followed by both video modeling and antecedent-based intervention. An interesting finding relates to the six EBPs without any studies that adequately reported REN. Among them are four well-respected interventions rooted in applied behavior analysis that are widely considered to be fundamental components of ASD treatment programs: discrete trial teaching, extinction, functional communication training, and the picture exchange communication system. Given the apparent (or perceived) ubiquity of these long-standing interventions, we expected to find intervention studies in these EBP categories that sufficiently reported diverse participants.
Representation of Participants by REN
With regard to our second and third research questions related to representation of participants by REN, we found White participants (i.e., White, European American, Caucasian, White British, Polish, Dutch, Tasmanian, and New Zealander) represented 19.6% of the 2,489 participants included in all 408 EBP studies but were 63.5% of participants in the 73 studies that sufficiently reported REN. Participants reported as multiracial comprised 6.4% of overall number of participants and were 20.6% of participants in the 73 studies. Other groups were minimally represented among both the overall number of participants and the portion of participants whose race, ethnicity, and/or nationality was reported.
When REN was reported, White participants represented a significant majority in research studies that met the minimum design criteria. This may be due to a number of factors, including institutionalized privilege and the fact that White people historically have been the majority in many communities, states, and regions throughout the United States. Researchers, therefore, may not have had sufficient access to diverse participants and may have instead relied on convenience sampling rather than engaging in novel or replication research to better understand the efficacy of these interventions for students from diverse backgrounds. Another factor related to recent findings suggests ASD is diagnosed differently, with Whites generally being identified at rates significantly higher than Asian, Black, Hispanic/Latino, Middle Eastern, multiracial, and Native American populations (Travers, Krezmien, Mulcahy, & Tincani, 2014; Travers, Tincani, & Krezmien, 2013).
Limitations
Given this review is a direct extension of the work completed by Wong et al. (2014), main limitations relate to those detailed by those authors, including (a) only studies published between 1990 and 2011 were included, (b) age criteria was limited to studies that included participants from birth to age 22 years, and (c) studies that failed to produce effects (i.e., publication bias) or had deleterious effects were not included in their review. Another main limitation relates to our aggregation of REN. We combined subgroups/categories to produce seven broader racial categories to present and interpret findings. Although we used a conservative approach for calculating interrater reliability and obtained satisfactory reliability, our logic for combining groups may be considered a limitation because international study participants were combined with minority groups commonly used in research conducted in the United States. Other researchers may not use similar combinations and, therefore, may produce different findings. Our presentation of all reported races, ethnicities, and nationalities may sufficiently alleviate this concern. However, our presumption of nationality of some participants was based on countries where studies occurred (e.g., Taiwan). These cases were very rare and would not likely have produced meaningful changes in the results, but nonetheless, this remains a limitation worth considering when interpreting the findings. We did not examine participant age and the potential interaction with REN and gender variables. Future reviews could examine REN variables across EBPs and the potential impact on representations when broken down by age groups. If studies that included young children with ASD more often include REN data than those focusing on older adolescents, then this would indicate the need to better understand why this occurs as well as placing greater emphasis on including diverse adolescents with ASD who may respond differently to interventions.
Implications for Practice
First, practitioners must examine the degree to which the positive results reported for a narrow group of participants effectively translates to their students from traditionally underrepresented groups. Practitioners may lack confidence in the efficacy of EBPs for learners whose race or ethnicity is something other than White (i.e., Western European). It is imperative interventions are selected that rely on practitioner wisdom and a valuing of family and community priorities. Practitioners and families must choose the most appropriate interventions that meet the unique needs of the child and family. Choice is necessary and highly valued by families because proposed EBP treatments may conflict with a family’s beliefs, may have been tried and failed, or a family may know that a proposed treatment will not work for their child. This requires practitioners to have strong family, and school and community partnerships.
Second, practitioners should seek ways to deliver EBPs consistent with intervention protocols while incorporating student strengths, interests, and preferences, as well as accounting for the priorities of individualized education program (IEP) team members. This individualized and empirical approach to the provision of evidence-based special education service may significantly increase the potential effectiveness of EBPs among diverse learners with ASD. The studies reviewed represent the most current methodologically robust research in the field of ASD. The absence of rigorous studies with diverse participants may compromise confidence about the efficacy of these EBPs. Such skepticism need not be interpreted as reasons for outright rejection of EBPs or proof that “ABA doesn’t work.” Rather, this finding highlights why professionals should adopt a meticulous, data-based approach to intervention delivery and evaluation (CEC Interdivisional Research Group, 2014). Indeed, professionals who behave as scientist-practitioners may more effectively address the unique needs of their learners (Cook & Odom, 2013). We therefore suggest special education and related professionals be prepared to carefully outlay intervention procedures (as prescribed in the various practitioner guides offered by the NPDCASD) and utilize progress monitoring data to systematically inform intervention components.
Implications for Future Research
Although an uncommon topic in the literature, concerns about the narrow representation of diverse participants in special education intervention research have been previously expressed (Artiles, Trent, & Kuan, 1997; Bos & Fletcher, 1997; Kistner, & Robbins, 1986; Pierce et al., 2014). Researchers also have reported that diverse individuals are less likely to participate in ASD-related research (Hilton et al., 2010; Pierce et al., 2014). Our findings of extremely limited diversity among participants in the most rigorous intervention research further underscore these concerns. We echo Pierce et al. (2014) that research is needed to (a) better understand reluctance to participate among diverse members of society, (b) improve recruitment and retention of diverse participants in ASD intervention research, and (c) contribute to the development of standardized methods of collecting and storing detailed information about participants that are conducive to systematic reviews and meta-analyses of intervention efficacy for learners with distinct profiles. Indeed, researchers and agencies that fund ASD intervention research should take steps to encourage recruitment of participants with ASD from various races, ethnicities, ages, and socioeconomic statuses. However, previous calls for such efforts do not appear to have had the desired effect, at least not in the most rigorous studies. Thus, we propose funding competitions that specifically aim to (a) increase participant diversity by prioritizing diverse participants for proposed projects and (b) address problems that inhibit recruitment and retention of diverse participants in ASD intervention studies. Such incentives may motivate researchers to identify empirically valid ways to improve recruitment and retention of diverse participants while also paving the way for studies that provide clarity about intervention efficacy for diverse learners.
We were somewhat surprised to have found the best available evidence for ASD interventions to have not reported the REN of participants. The EBPs of discrete trial teaching, extinction, functional communication training, and picture exchange communication system are directly rooted in applied behavior analysis, the leading paradigm and state-of-the-art model for ASD treatment (Foxx, 2008). Despite this status, the absence of diverse participants in these bodies of research, as well as scripting and structured play group, may be construed to support legitimate (and perhaps illegitimate) points of criticism. Accordingly, researchers should conduct studies of these EBPs for learners with various racial, ethnic, and perhaps socioeconomic profiles.
Some evidence indicates diverse children are less likely to be identified with ASD than White children (Travers et al., 2013, 2014). Given the nuances associated with culture, language, and behavior, researchers may consider conducting qualitative research to obtain details about factors related to race, ethnicity, language, gender, and socioeconomic status that may affect intervention generality (La Roche & Christopher, 2009). Qualitative research can support evidence-based special education by providing ways to (a) substantiate promising practices, (b) understand the implementation of EBPs, and (c) determine how and why EBPs work for certain individuals or groups and not well for others (Odom et al., 2005). Such tools provide support for and promote the trustworthiness and effectiveness of EBPs (Steiner, 2007). Qualitative and mixed-methods research targeted at understanding why parents and caregivers from diverse backgrounds are reluctant to participate may help researchers design studies and interventions that encourage their participation. Also, a better understanding about the factors predictive of ASD identification, including individual (e.g., race, ethnicity, socioeconomic status) and system factors (e.g., per pupil funding, unemployment rate, median home value), may better guide federal agencies in funding requirements for ASD intervention research projects.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
