Social Validity Assessment in Behavior Interventions for Young Children: A Systematic Review

Abstract

We sought to identify, examine, and summarize empirical literature focused on early childhood behavior interventions examined using a single case research designs (SCD) and published between 2001 and 2018. Using systematic procedures, 28 studies that met established inclusion criteria were identified, reviewed, and compared with respect to general and social validity assessment characteristics of SCD studies on behavior interventions for young children with problem behavior. The findings of the current review suggest: (a) promoting implementation fidelity through implementation support to improve social validity outcomes, (b) providing guidelines for timing and frequency of social validity assessment, and (c) development of social validity assessment tools designed to assess each of the social validity dimensions (i.e., goals, procedures, and outcomes).

Keywords

social validity problem behavior young children behavior intervention fidelity

Young children who engage in behaviors that adults perceive as challenging, such as physical aggression, property destruction, tantrum, and prolonged crying, often experience negative developmental, social, and behavioral outcomes (Brauner & Stephens, 2006; Dunlap, Ester, Langhans, & Fox, 2006; Powell, Dunlap, & Fox, 2006). Because of the negative trajectory associated with these problem behaviors in young children, national organizations and researchers have emphasized providing early intervention using evidence-based practices in toddler and preschool years (Conroy, Dunlap, Clarke, & Alter, 2005; Fox, Dunlap, & Powell, 2002; Hemmeter, Snyder, Fox, & Algina, 2016; Wood, Blair, & Ferro, 2009). One factor affecting the use of evidence-based interventions is whether the interventions are socially valid (Horner et al., 2005). Many researchers have stressed that when the interventions are socially valid, they are feasible and result in long-lasting reductions of problem behavior, leading to improvement, generalization, and maintenance of child outcomes (Moes & Frea, 2002; Oono, Honey, & McConachie, 2013). Therefore, the field has emphasized the importance of promoting the use of socially valid interventions that are feasible, effective, and sustainable in typical natural settings (Gerow et al., 2018; Horner et al., 2005; Schwartz & Baer, 1991; Spear, Strickland-Cohen, Romer, & Albin, 2013).

Baer, Wolf, and Risley (1968), who introduced the concept of social validity as an essential feature of applied behavior analysis (ABA), emphasized the socially important goals and outcomes of intervention in discussing being “applied” and “effective” within the dimensions of ABA. The researchers indicated that behaviors targeted for intervention must be important to the person and society and that intervention should result in socially significant behavior change. In reaffirming the emphasis on social validity, Wolf (1978) discussed three dimensions of social validity: (a) social importance of goals (socially important dependent variable), (b) social acceptability of procedures (practical and cost-effective implementation of the independent variable; implementation of the independent variable over extended time period by typical intervention agents), and (c)social importance of outcomes (socially important magnitude of change in the dependent variable). Researchers have employed parts or all of these dimensions of social validity in designing, implementing, and evaluating interventions. These dimensions are essential to the decision-making process regarding what behavior should be changed, how it should be changed, how much the behavior should be changed, and how we will know it was effective (Gresham & Lopez, 1996). Researchers have involved consumers (i.e., teachers and parents) in deciding intervention target behaviors and routines, selecting or modifying the intervention procedures, and assessing the acceptability of the intervention process and outcomes (Strain, Barton, & Dunlap, 2012).

Despite the fact that efforts have been made to use social validity interventions and to expand methods for assessing social validity, there are still a limited number of studies reporting social validity (Ledford, Hall, Conder, & Lane, 2016; Snodgrass, Chung, Meadan, & Halle, 2018), and there is still a need for guidelines on how to assess social validity. For example, Ledford et al. (2016) reviewed 109 single case design (SCD) studies reported in 54 articles on social skills interventions for young children with autism spectrum disorder (ASD) published between 1994 and 2013 and found that less than half of the studies reported social validity data. Snodgrass et al. (2018) found that only 26.8% (n = 115) of 429 articles using SCD published in six special education journals between 2005 and 2016 reported a social validity assessment, and that of those articles with social validity assessment information, only 6.5% (n = 7) reported all three dimensions of social validity (i.e., goals, procedures, and outcomes).

These findings concerning the lack of social validity assessment were similar to those findings reported in earlier reviews of social validity (Carr, Austin, Britton, Kellum, & Bailey, 1999; Conroy et al., 2005; Hurley, 2012; Kennedy, 1992), indicating that the rate of studies reporting social validity has not increased. Ledford et al. (2016) discussed that the dearth of social validity assessments in SCD studies might be due to the lack of guidance and methods for conducing the assessments, cost and time to complete the assessments, and page limits set by journal policies. Ideally, social validity assessment results lead to the design and development of more feasible and effective interventions and inhibit the development of interventions that are likely to fail in real world (Schwartz & Baer, 1991).

Accordingly, researchers have suggested several methods of assessment to address all three dimensions of social validity, such as subjective assessment, objective assessment, and social or normative comparison. Subjective assessment typically involves gathering opinions about acceptability of and satisfaction with the intervention from consumers following the intervention (Hawkins, 1991; Turan & Meadan, 2011). Rating scales (Reimers, Wacker, & Cooper, 1991; Witt & Martens, 1983), questionnaires with open-ended questions (Gresham & Lopez, 1996), interviews (Spohn, Timko, & Sainato, 1999), and focus groups (Leko, 2014) have been used to gather consumer opinions. Given that assessments are conducted by asking consumers or implementers for their opinions about the intervention, these assessment methods are considered subjective in nature even though rating scales may utilize systematic assessment methods (Hurley, 2012).

Several researchers have suggested employing objective social validity assessments (Gresham & Lopez, 1996). For example, to assess the intervention acceptability, Hanley (2010) suggested directly assessing choices of intervention options with direct consumers (e.g., participating children), which could be assessed using a simultaneous treatment design. Additional objective social validity assessments include assessing consumers’ continuous use of interventions after intervention implementation support is removed, comparing the target child’s behavior change with that of typically developing peers (normative comparison), having masked raters who are unaware of the conditions being implemented, rating target behavior change and the extent to which the procedures implemented are beneficial, and examining maintenance of behavior change (Barton, Reichow, Schnitz, Smith, & Sherlock, 2015; Ennis, Jolivette, Fredrick, & Alberto, 2013). Yet, these objective methods are seldom used (Spear et al., 2013).

In addition, not only how but also when to assess the social validity has been discussed in the literature. Although social validity has sometimes been conceptualized as an outcome, it has also been conceptualized as a process (Finney, 1991; Schwartz & Baer, 1991), suggesting that assessing the three dimensions of social validity can occur at various points in time during development, implementation, and testing. Researchers also have assessed consumers’ perceived acceptability of the identified intervention goals and procedures prior to implementing the intervention or during the intervention phase to ensure that consumers perceive the intervention procedures to be feasible and easy to implement (Strain et al., 2012). This preintervention buy-in or acceptability of intervention procedures has been discussed as a potential factor that may affect implementation fidelity; that is, if interventions are acceptable, they are more likely to be implemented with fidelity (Lane, Beebe-Frankenberger, Lambros, & Pierson, 2001; Miramontes, Marchant, Allen-Heath, & Fischer, 2011; Witt & Elliott, 1985).

However, most researchers measure social validity of goals and procedures without natural change agents before or during intervention implementation (Spear et al., 2013). In fact, researchers have paid little attention to assessing the social significance of goals and social acceptability of procedures (Gresham & Lopez, 1996; Snodgrass et al., 2018). Spear et al. (2013) reported that of the 22 SCD studies on behavior interventions for students with emotional and behavioral disorders, none met all four social validity assessment quality indicators identified by Horner et al. (2005), which reflect Wolf’s (1978) three dimensions of social validity. Spear et al. also reported that only one study included practitioner input during the intervention design. Similarly, Snodgrass et al. (2018) found none of the subgroup of articles that addressed all three dimensions of social validity assessment (n = 28) conducted all six steps of the scientific method in employing social validity assessment. Snodgrass et al. suggested that social validity assessments should employ a scientifically rigorous process that includes all six steps of a scientific method: (a) research question, (b) literature review, (c) hypothesis, (d) data collection method, (e) hypothesis testing, and (f) analysis procedures.

Previous examinations of social validity provide valuable information regarding social validity measurement, including the overall prevalence, the frequency with which researchers use different methods, and the presence of quality indicators. However, current research does not clearly delineate the extent to which researchers or practitioners make adaptations to interventions based on social validity measurement. Furthermore, there are no clear guidelines regarding how social validity assessment results should be reported. In general, the social validity assessment results are reported using a brief summary statement (Snodgrass et al., 2018). We extended the literature on social validity by systematically reviewing SCD studies on early childhood behavioral interventions. Given that none of the systematic reviews on social validity focused on behavioral interventions for children with problem behavior, we aimed to examine the general and social validity assessment characteristics of SCD studies on behavioral interventions for this population. Specific research questions were: (a) what are the common characteristics of SCD studies on behavior interventions for young children with problem behavior, (b) how prevalent is it for the three dimensions (i.e., goals, procedures, and outcomes) of social validity to be assessed and to what extent are they assessed, and (c) what types of social validity assessment methods are commonly used?

Method

Article Search

Article search procedures were performed to identify articles published between 2001 and 2018 that focused on behavior interventions for young children. The initial search involved online databases: Academic Search Premier, PsycINFO, and Web of Science. First, a key word search of abstracts was conducted using the following key words: practice, intervention, treatment, support, strategy, therapy, program, procedure, and approach, in conjunction with such key words as problem behavior and challenging behavior. Second, the following key words were searched in the full text of articles: infant, toddler, preschooler, and young child, in conjunction with disability, disabilities, and delay. The initial search resulted in 698 articles (453 from Academic Search Premier, 161 from PsycINFO, and 163 from Web of Science, minus 79 duplicated studies). In the second search phase, 36 additional studies were identified. This two-phase article search resulted in a total of 734 studies as being potentially relevant.

Article Selection Procedures

To select articles for final review, we used a four-step screening process with the 734 articles using the following inclusion criteria: (a) published in peer-reviewed journals, (b) included child participants who have a diagnosed disability or developmental delay, or are at risk for disability in social-emotional development due to problem behavior, (c) included at least one child aged 6 or under, (d) employed an SCD, (e) implemented an intervention to address problem behavior, (f) reported qualitative or quantitative social validity assessment data, and (g) was written in English. To determine SCD studies for analysis, three SCD features suggested by the What Works Clearinghouse (WWC; 2017) were used. The features consisted of (a) an individual case (single participant or a cluster of participants) as the unit of intervention and unit of data analysis, (b) the individual case providing its own control for purposes of comparison, and (c) repeated measurement of outcome variable within and across different conditions or levels of the independent variables.

First, the titles of the identified 734 articles were reviewed, resulting in the elimination of 48 studies. Studies that contained explicitly unrelated terms in the title, such as meta-analysis, randomized trial, validation of scale, and cohort study, were excluded. Second, review of the article abstracts resulted in the elimination of an additional 509 articles, leaving 177 articles. Third, the Method sections of each of the remaining 177 studies were reviewed, resulting in the exclusion of 62 articles. Finally, the full text of the remaining 115 studies was reviewed, during which 87 studies were excluded for the following reasons: (a) did not include child participants aged between 0 and 6 (n = 14), (b) did not report any social validity assessment data (n = 56), (c) did not include a child with problem behavior (Galensky, Miltenberger, Stricker, & Garlinghouse, 2001; Healey, France, & Blampied, 2009; Jolstead et al., 2017), (d) included typically developing children whose status of at risk for social-emotional difficulties was not confirmed using a social-emotional screening or assessment tool to further determine the child’s needs for behavior intervention (Galensky et al., 2001; Healey et al., 2009; McLaren & Nelson, 2009; Murphy, Theodore, Aloiso, Alric-Edwards, & Hughes, 2007; Rispoli et al., 2015; Sawyer, Crosland, Miltenberger, & Rone, 2015), (e) did not employ an SCD, or (f) the unit of analysis was not individuals, but rather classroom (Benedict, Horner, & Squires, 2007; Carter & Van Norman, 2010; Hemmeter, Hardy, Schnitz, Adams, & Kinder, 2015; Hemmeter, Snyder, Kinder, & Artman, 2011; Stormont, Smith, & Lewis, 2007). The percentage of studies that addressed social validity was 51.4%. At the conclusion of this four-step screening process, 28 articles were identified for in-depth analysis (Figure 1).

Figure 1.

Flowchart of study selection process.

Coding Procedures

General characteristics of individual studies

To identify the general characteristics of the selected 28 studies, the authors coded the following variables: (a) author and year, (b) number of child participants, (c) child gender, (d) child age, (e) type of disability or developmental delay, (f) type of intervention, (g) intervention implementer, (h) design, (i) setting, (j) reporting of treatment fidelity, (k) dependent measure, and (l) intervention implementation support (i.e., frequency, duration, and method of implementation support). These variables were analyzed to gather information about the child participants’ characteristics, evidence-based or promising behavior interventions used for young children, the contexts under which the behavior intervention was implemented, and study design and behaviors targeted to evaluate the intervention outcomes.

Analysis of social validity assessments

Based on previous review studies on social validity (Hurley, 2012; Snodgrass et al., 2018; Strain et al., 2012), 10 variables were coded: (a) inclusion of a research question about social validity, (b) inclusion of literature review on social validity, (c) presence of measurement of three dimensions of social validity (goal, procedures, outcomes), (d) social validity assessment method (questionnaire–validated scale, questionnaire–author modification of validated scale, questionnaire–self-developed, questionnaire–self-developed with open-ended questions, interview, blind rating, normative comparison), (e) type of assessment tool (self-developed, validated tool, adapted from validated tool), (f) response method (verbal, paper, email, observation), (g) frequency of social validity assessment, (h) social validity assessment respondents (direct consumer, indirect consumer, immediate community member, extended community member), (i) data reporting method (summary statement only, raw data, descriptive statistics, parametric statistics, qualitative), and (j) presence of intervention revision based on feedback from social validity assessment participants.

Social validity assessment, coding categories were developed based on the categories discussed by Schwartz and Baer (1991). Direct consumers are individuals who are directly involved in the intervention, and indirect consumers are individuals who are strongly affected by the effects of intervention. Immediate community members are individuals who interact with direct and indirect consumers, and extended community members are individuals who may never have direct contact with intervention consumers.

Interrater Reliability

Interrater reliability was assessed during article screening and selection and during coding. During the fourth step of the selection process, review of full text, the authors applied the inclusion and exclusion criteria to the remaining 115 articles under consideration for inclusion to undergo in-depth data analysis. Interrater reliability was calculated as a percentage of agreements by dividing the number of agreements over the number of possible agreements. The interrater reliability in selecting the final 28 articles was 98.3%. The discrepancies were resolved through discussion and consensus for any discrepancies between coders to meet 100% agreement. The first author coded the selected 28 articles, and an undergraduate student in special education who was naïve to the purpose of the review and who received training on using the coding form independently coded 32.1% (n = 9) of the articles selected at random. The interrater reliability for coding was 92.3% (range = 88.9%-100%) across variables. The disagreements were resolved to 100% agreement.

Results

Characteristics of Individual Studies

Table 1 presents descriptive characteristics across studies. The total number of child participants were 74, including 47 boys and 21 girls; no gender information was provided for six children. Age range was 2 to 7 years. Of the 28 studies, 11 (39.3%) included children with ASD and nine studies (32.1%) included children who had a developmental delay in language, communication, or speech development. Four studies (14.3%) included children with developmental delays whose information concerning the developmental domains associated with the delays was not provided. Three studies (10.7%) included children with language disabilities. Five studies (17.9%) included children at risk for disabilities. The most commonly used intervention was positive behavior support (25%, n = 7), implemented at home or in the classroom. The second most frequently used interventions were functional communication training (18.8%, n = 5) and function-based intervention (14.3%, n = 4), followed by mindful parenting, self-management or self-monitoring, and visual support, each with 7.1% (n = 2). Other interventions (21.4%, n = 6) included antecedent-based intervention, Parent–Child Interaction Therapy, family-centered Prevent-Teach-Reinforce, and embedded self-determination practices. Most studies provided insufficient information on intervention frequency and duration, with only 35.7% (n = 10) reporting the total duration of intervention, which ranged from 4 days to 52 weeks.

Table 1.

Characteristics of Studies.

Author (year)	Participants			Intervention		Design	Setting	Fidelity report	Dependent measure	Implementation support
Author (year)	n	Age	Disability	Type	Interventionist	Design	Setting	Fidelity report	Dependent measure	Frequency and duration	Method
Baily and Blair (2015)	3	4, 5, 6	ASD, language delay	PTR	Parents	MBL	Home	X	PB, RB, parent fidelity	10–35 min per wk	In vivo coaching, modeling, rehearsal, verbal feedback
Blair, Liaupsin, Umbreit, and Kweon (2006)	3	NS (K)	ID	FBI	Teacher	MBL	Classroom	X	CB, AB	NR	NR
Blair, Fox, and Lentini (2010)	3	3, 3.5, 4	Language delay, ADHD, ASD	PBS	Teacher	MBL	Classroom	X	CB, E	NS (1–2 SSs)	Modeling, in vivo coaching
Blair, Lee, Cho, and Dunlap (2011)	3	4.5, 4.5, 5.5	ASD and ID, ASD and CP	PBS	Teacher and mother	MBL	Home and classroom	X	AB, PB, adult interactions	30 min per 1–2 weeks	In vivo coaching, verbal feedback
Brock and Beaman-Diglia (2018)	1	5	At risk	Multicomponent	Teacher	MBL	Classroom	X	CB, teacher fidelity	15 min	Graphical and verbal feedback
Chu (2015)	3	5, 6	ASD, ID, CP	PBS	Practicum student	MBL	Clinic	X	PB	NS (per 2 weeks)	Verbal feedback
Drogan and Kern (2014)	3	3, 4, 4	At risk	Turtle technique	Teacher	MBL	Classroom	X	PB, UT	NR	NR
Duda, Dunlap, Fox, Lentini, and Clarke (2004)	2	3, 3	Down syndrome, DD	PBS	Teacher	AB	Classroom	X	E, CB	5–10 min per SS	Coaching, modeling, verbal feedback
Duda, Clarke, Fox, and Dunlap (2008)	3	2.8, 5	Language delay	PBS	Mother	MBL	Home	X	E, CB	10 min per SS	Modeling, verbal feedback
Dufrene, Doggett, Henington, and Watson (2007)	3	5	At risk	FBI	Researcher and teacher	AB	Classroom	X	CB	NR	NR
Dunlap, Ester, Langhans, and Fox (2006)	2	2.5, 2.8	Language or speech delay	FCT	Mother	MBL	Home	X	CB, mother’s response	1 hr per SS (first few SSs)	Review of strategies, coaching
Fettig, Schultz, and Sreckovic (2015)	3	3.4, 3.8, 5.9	ASD, sensory integration disorder	FBI	Parents	MBL	Home	X	CB, parent use of strategies	NS	Verbal feedback, modeling
Fettig, Barton, Carter, and Eisenhower (2016)	3	2.5, 5, 7	ASD	FBI	EI provider	MBL	Home	X	Fidelity, CB	30–45 min per 1–2 weeks	Coaching, verbal feedback
Gibson, Pennington, Stenhoff, and Hopper (2010)	1	4	ASD	FCT	Teacher	AB	Classroom	X	CB, teacher satisfaction	NR	NR
Hancock, Kaiser, and Delaney (2002)	5	3.2–3.8	At risk	FCT	Parents	MBL	Classroom, conference room	X	Parent use of strategies, parent response, C, PB	30–45 min, 2 per week	Verbal feedback, role-play
Knowles, Blakely, Hansen, and Machalicek (2017)	2	2, 3	Language delay	Embedded self-determination	Mother	MBL	Home	X	CB, routine-based parenting behavior	NS (45 min)	In vivo coaching, modeling, verbal feedback
Lucyshyn et al. (2007)	1	5	ASD	PBS	Parents	MBL	Home and community	X	PB, activity patterns, SV, contextual fit	1 per 2–4 weeks	In vivo modeling, coaching, rehearsal, discussion, verbal feedback, booster training
Lui, Moore, and Anderson (2014)	1	5	ASD	Self-monitoring	Parents	MBL	Home		C, PB	NR	NR
Masse, McNeil, Wagner, and Quetsch (2016)	3	3, 4, 4	ASD	PCIT	Parents	MBL	Clinic	X	C, PB, AB	NR	Verbal feedback, modeling, role-play
McCoy, Morrison, Barnett, Kalra, and Donovan (2017)	3	3, 5	Speech disorder	Self-monitoring	Teacher	AT, MBL	Classroom	X	E, CB	NR	NR
McDaniel and Flower (2015)	3	5.5–7	Down syndrome	Visual support	Teacher, TA, counselor	MBL	Classroom	X	Disruption	Weekly	Review of data, verbal reminder
Moes and Frea (2002)	3	3.3–3.6	ASD	FCT	Family	MBL	Home		PB	NS (per 1–2 weeks)	Modeling, verbal feedback
Park and Scott (2009)	3	4–5	At risk	Antecedent-based	Teacher	AB	Classroom	X	PB, on-task	NR	NR
Singh et al. (2006)	3	4.4, 5.2, 6	ASD	Mindful parenting	Mother	MBL	Home		CB	2 hr per SS (3 SSs)	Instructions, practice
Singh et al. (2007)	4	4, 4.9, 5, 6	DD	Mindful parenting	Mother	MBL	Home		Aggression, SI, parental satisfaction and stress, fidelity	2 hr per SS (3 SSs)	Exercise, verbal reminder
Smith et al. (2011)	3	2.3–5	Speech/language disorder	PBS	Teacher	MBL	Classroom	X	E, CB, teacher use of strategies	NR	Verbal feedback
Strand and Eldevik (2018)	1	4	ASD	FCT	Instructor	AB	Home		PB, FCR, IR	NR	NR
Zimmerman, Ledford, and Barton (2017)	3	3.6, 4.3	SLD, DD	Visual support	Researcher, GS	AB	Classroom		E, CB	NR	NR

Note. ASD = autism spectrum disorders; ADHD = attention deficit hyperactivity disorder; PTR = Prevent-Teach-Reinforce; TA = teacher assistant; MBL = multiple baseline; PB = problem behavior; RB = replacement behavior; NS = not specified; ID = intellectual disability; FBI = function-based intervention; FCT = functional communication training; CB = challenging behavior; AB = appropriate behavior; NR = not reported; PBS = positive behavior support; CP = cerebral palsy; UT = use of turtle technique; DD = developmental delay; SS = session; EI = early intervention; SV = social validity; PCIT = parent-child interaction therapy; AT = alternating treatment; SI = social interaction; FCR = functional communication response; IR = independent request; SLD = speech/language disorder; GS = graduate student; K = kindergartners; FA = functional assessment; E = engagement; C = Compliance.

In the majority of the studies (50.0%, n = 14), classroom teachers were involved as implementer or co-implementer. Families (parents or other member) were involved as implementer in 12 studies (42.9%). Other implementers included practicum student, researcher, graduate student, instructor, early intervention provider, and other staff (paraprofessional and counselor). The most common design to evaluate the interventions was multiple baseline design (75.0%, n = 21), followed by withdrawal design (21.4%, n = 6). One study (3.6%) employed a combination of alternating treatments and multiple baseline designs. With the exception of two studies, all studies were conducted in the classroom (46.4%, n = 13), at home (family routines; 39.3%, n = 11), or both at home and in other settings (7.1%, n = 2; home and classroom or home and community). The other two studies were conducted at a clinic or an early intervention center within a university. Provision of ongoing implementation support to implementers during intervention after initial training varied across studies. In most studies (67.9%, n = 19), implementation support was provided 1 to 2 times during the intervention phase, on a weekly or biweekly basis, 5 to 45 min per session. However, nine studies (32.1%) did not provide specific information about the ongoing implementation support, only provided initial training before intervention implementation, or had the implementer subjectively self-monitor their implementation of the intervention procedures. The most commonly used implementation support methods were in vivo coaching and performance feedback delivered through verbal, graphical, or both verbal and graphical methods.

Assessment of Three Dimensions of Social Validity

Table 2 provides the analysis results on the prevalence and the extent to which the three dimensions of social validity were assessed in behavior intervention studies for young children with problem behavior. The results indicated that 11 studies (39.3%) included a research question on social validity, and five studies (17.9%) discussed social validity literature. Among the three dimensions, less than half of the studies (46.4%, n = 13) involved consumers in determining the intervention goals and procedures, and four studies (4.3%) only addressed goals before intervention implementation; this information has been omitted from Table 2 due to space limitations. In examining whether all three dimensions were formally assessed, it was found that only 25.0% (n = 7) of the studies provided the social validity assessment outcomes for goals, whereas 78.6% (n = 22) of the studies provided the assessment outcomes for acceptability of the intervention procedures and outcomes and 85.7% (n = 24) of the studies provided the assessment outcomes for acceptability of intervention outcomes.

Table 2.

Characteristics of Social Validity Assessment.

Author (year)	Research question on SV	Literature review on SV	Dimensions of SV			SV assessment method	SV assessment tool		Respondent	Report method	Frequency of assessment
Author (year)	Research question on SV	Literature review on SV	G	P	O	SV assessment method	Name	Response method	Respondent	Report method	Frequency of assessment
Bailey and Blair (2015)		X	X	X	X	Q-AMVS, interview, BR	Adapted TARF-R	Paper	Direct C, extended community member	DS, QR	1
Blair, Liaupsin, Umbreit, and Kweon (2006)				X	X	Q-SD	SD	Paper	Direct C	DS	1
Blair, Fox, and Lentini (2010)			X	X	X	Interview	SD	Verbal	Direct C, indirect C	QR	1
Blair, Lee, Cho, and Dunlap (2011)				X	X	Q-AMVS	Adapted TARF-R	Paper	Direct C, immediate community member	DS	1
Brock and Beaman-Diglia (2018)				X		OEQ	SD	Paper	Direct C	QR	1
Chu (2015)	X	X		X	X	Q-VS	TEI-SF	Paper	Indirect C	DS	2
Drogan and Kern (2014)	X			X	X	Q-VS	IRP-15	Paper	Direct C	DS	1
Duda, Dunlap, Fox, Lentini, and Clarke (2004)	X		X	X	X	Q-SD, BR	SD	Paper	Extended community member	DS	1
Duda, Clarke, Fox, and Dunlap (2008)			X	X	X	Q-SD, BR	SD	Paper	Extended community member	DS	1
Dufrene, Doggett, Henington, and Watson (2007)	X			X	X	Q-AMVS, Q-VS	Adapted ARP-R; IRP-15	Paper	Direct C	DS	1
Dunlap, Ester, Langhans, and Fox (2006)			X	X	X	Q-SD, BR	SD	Paper	Direct C, Extended community member	RD, DS	1–2
Fettig, Schultz, and Sreckovic (2015)				X	X	Q-SD + OEQ	SD	Paper	Direct C	DS	1
Fettig, Barton, Carter, and Eisenhower (2016)				X	X	Interview	SD	Verbal	Direct C	SS	1
Gibson, Pennington, Stenhoff, and Hopper (2010)				X	X	Q-SD + OEQ	BIRS-A	Paper	Direct C, Indirect C	RD, DS, QR	1
Hancock, Kaiser, and Delaney (2002)	X			X	X	Q-SD, interview	SD	Verbal, paper	Direct C	RD, DS	1
Knowles, Blakely, Hansen, and Machalicek (2017)	X			X		Q-SD	SD	Paper	Direct C	RD, DS	2
Lui, Moore, and Anderson (2014)				X	X	Q-VS	BIRS-A	Paper	Direct C	SS	3
Lucyshyn et al. (2007)	X	X		X	X	Q-SD	SD	Verbal, paper	Direct C	DS, QR	8
Masse, McNeil, Wagner, and Quetsch (2016)	X	X		X	X	Q-VS	TAI	Paper	Direct C	DS	2
McCoy, Morrison, Barnett, Kalra, and Donovan (2017)				X	X	Q-SD, interview	SD	Verbal, paper	Direct C, Indirect C	DS, QR	1
McDaniel and Flower (2015)	X		X	X	X	Q-SD + OEQ	SD	Paper	Direct C	SS, QR	1
Moes and Frea (2002)				X		Q-SD	SD	Paper	Direct C	DS	1
Park and Scott (2009)				X	X	Q-SD, interview	SD	Verbal, paper	Direct C	DS	1
Singh et al. (2006)	X			X		Q-VS	SUPS	Paper	Direct C	DS	3
Singh et al. (2007)	X	X			X	Q-VS	SUPS	Paper	Direct C	DS	3
Smith, Lewis, and Stormont (2011)			X	X	X	Q-SD	SD	Paper	Direct C	SS	1
Strand and Eldevik (2018)				X	X	Q-SD	SD	E-mail	Indirect C	RD, DS	1
Zimmerman, Ledford, and Barton (2017)					X	Normative comparison	NA	Observation	NA	SS	1

Note. SV = social validity; AMVS = Author Modification of Validity Scale; BR = blind rating; TARF-R = Treatment Acceptability Rating Form–Revised; DS = descriptive statistics; QR= qualitative report; SD = self-developed; OEQ = open-ended questionnaire; VS = validated scale; TEI-SF = Treatment Evaluation Inventory–Short Form; ARP-R = Assessment Rating Profile–Revised; IRP = Intervention Rating Profile; RD = raw data; SS = summary statement only; BIRS-A = Behavior Intervention Rating Scale–Adapted version; TAI = Therapy Attitude Inventory; SUPS = Subjective Units of Parenting Satisfaction; NA = not applicable; G = goal; P = procedure; O = outcome;Q = questionnaire; TAS = Treatment Acceptability Survey; C = consumer.

Commonly Used Social Validity Assessment Methods

The results indicated that subjective consumer questionnaires were the most frequently used social validity assessment method (89.3%, n = 25). Across studies, 48.0% (n = 12) used a self-developed questionnaire, 12.0% (n = 3) used a self-developed questionnaire with open-ended questions, 12.0% used a questionnaire with author modification of validated scale, 24.0% (n = 6) used a questionnaire with validated scale, and 4.0% (n = 1) used only open-ended questions without rating scale items (Brock & Beaman-Diglia, 2018). Six studies (21.4%) conducted an interview (Bailey & Blair, 2015; Blair, Fox, & Lentini, 2010; Fettig, Barton, Carter, & Eisenhower, 2016; Hancock, Kaiser, & Delaney, 2002; McCoy, Morrison, Barnett, Kalra, & Donovan, 2017; Park & Scott, 2009). Some researchers used objective measures such as: masked ratings (using videos with masked observers) or a normative comparison. Three studies used a masked ratings (Duda, Clarke, Fox, & Dunlap, 2008; Duda, Dunlap, Fox, Lentini, & Clarke, 2004; Dunlap et al., 2006), and one study used a normative comparison which involved comparing target child behavior with peer behavior (Zimmerman, Ledford, & Barton, 2017). One study used both subjective and objective methods such as a questionnaire and a masked ratings (3.6%; Bailey & Blair, 2015).

Self-developed survey tools were also common tools used to measure social validity (66.8%, n = 17). The majority of the studies used a validated tool (25.0%, n = 7) or a modified or adapted version of a validated tool (10.7%, n = 3), such as the Treatment Acceptability Rating Form–Revised (TARF-R; Reimers et al., 1991), Treatment Evaluation Inventory–Short Form (TEI-SF; Kelley, Heffer, Gresham, & Elliott, 1989), Therapy Attitude Inventory (TAI; Brestan, Jacobs, Rayfield, & Eyberg, 1999), Subjective Units of Parenting Satisfaction (Stanley & Averill, 1998), and Behavior Intervention Rating Scale–Adapted (BIRS-A; Elliott & Treuting, 1991). Using pencil and paper was the most common response method (85.7%, n = 24). Survey through email was found in one study (Strand & Eldevik, 2018). Observation (Zimmerman et al., 2017), verbal (Fettig et al., 2016), and audiotape recording (Blair et al., 2010) were also used. The frequency of social validity assessment ranged from 1 (67.9%, n = 19) to 8 (3.6%, n = 1). Three studies (10.7%) reported assessing the social validity at two different time points, and three studies (10.7%) reported assessing it at three different time points during the SCD study. One study reported assessing social validity at one or two time points, depending on the participant. In a 10-year longitudinal study (Lucyshyn et al., 2007), social validity was assessed at eight different time points over the course of the study.

Direct consumers participated in 79.3% of social validity assessments (n = 23). Direct consumer roles varied across studies and included parents, teachers, and the child. Indirect consumers participated in five studies (17.9%). Immediate community members such as early intervention providers participated in one study’s (3.6%) social validity assessment. Extended community members such as naïve observers participated in four studies (14.3%) for assessing social validity. Six studies (21.4%) included multiple evaluators in social validity measurement. Direct and indirect consumer assessment was identified in three studies, direct consumer and extended community member in two studies, and direct consumer and immediate community member in one study. The primary method used to report social validity results was descriptive statistics (71.4%, n = 20). Four studies (14.3%) provided descriptive statistics with qualitative information, and two studies (7.1%) reported descriptive statistics with raw data. Four studies (14.3%) reported only summary statements. Five studies (18.0%) reported the results using a qualitative method. No studies reported a modification or adjustment of the intervention procedures during intervention based on the social validity assessment results.

Discussion

This study examined the general and social validity assessment characteristics of SCD studies that addressed problem behavior in young children with disabilities, developmental delays, or at risk for disabilities. The focus was to provide recommendations for practices and future research in addressing social validity.

Major Findings and Implications

With regard to the first research question, common characteristics of SCD studies on behavior interventions for young children, the results revealed a relatively large number of SCD studies evaluated evidence-based or promising behavior interventions for young children with problem behavior. It was encouraging that the vast majority of behavior interventions were implemented in the home or early childhood classroom settings by parents (family members) or classroom teachers for young children with diverse needs. Due to space constraints, specifics on behavioral outcomes and implementation fidelity scores for the individual studies are not provided herein, but all reviewed studies reported positive outcomes for the participating children, and all studies with information on implementation fidelity reported high levels of fidelity. These findings support existing evidence that natural change agents can be effective in implementing research-based interventions to address problem behavior in young children (Conroy et al., 2005). Given that more than 90% of the reviewed studies reported positive social validity outcomes, the interventions evaluated in the studies appeared to be acceptable to the natural change agents and effective. Providing implementation support to natural change agents during intervention is imperative to ensure the interventions are being implemented as designed with fidelity and result in positive outcomes. However, the current review offered little evidence that the implementers (natural change agents) received effective implementation support after initial training prior to the intervention. Nine studies (32.1%) did not report the provision of implementation support during intervention implementation, and of these studies, most lacked clear information on the frequency and duration of the support.

With regard to the second research question, the prevalence and extent to which the three dimensions of social validity (goals, procedures, outcomes) were addressed in the behavior intervention literature for young children, the results confirm that the proportion of studies addressing all three dimensions remains low regardless of the target populations and interventions, as found in previous reviews on social validity (Hurley, 2012; Ledford et al, 2016; Snodgrass et al., 2018). Of the reviewed studies, only a limited number of studies addressed all three dimensions of social validity over the course of the study (Bailey & Blair, 2015; Blair et al., 2010; Duda et al., 2008; Duda et al., 2004; Dunlap et al., 2006; McDaniel & Flower, 2015; Smith, Lewis, & Stormont, 2011). The current study focused on examining whether the researchers not only socially validated the goals and procedures before or during intervention but also assessed them after intervention. It was found that among the three dimensions, the lowest was the reporting rate on assessment of intervention goals, both before (or during) and after intervention. Based on the results of the analysis, the probable reasons that all three dimensions of social validity are rarely assessed in the field might be due in part to the lack of validated social validity assessment tools that provide clear distinctions of the three dimensions.

Concerning the third research question, commonly used social validity assessment methods, although the majority of the reviewed studies used validated social validity assessment tools, none of the tools were designed to assess all three dimensions of social validity. For example, based on a factor analysis, Kelley et al. (1989) reported that the nine-item TEI-SF with a 5-point scale loaded on two factors, “acceptability” and ‘ethical issues/discomfort’; no items were designed to assess acceptability of “goal.” Similarly, the 10-item TAI, a parent-report scale, is designed to assess only two aspects of intervention: process and outcome (Brestan et al., 1999; Eyberg, Edwards, Boggs, & Foote, 1998); again, the third dimension, goal, is omitted. The Behavior Intervention Rating Scale (BIRS) is also designed to assess the intervention’s acceptability, effectiveness, and time to effect (Elliott & Treuting, 1991). BIRS does not consider the goal of the intervention. This suggests that developing social validity assessment tools designed to assess all three dimensions might be a way to promote addressing all three dimensions of social validity in designing and implementing effective and feasible interventions and conducting high-quality research.

Although the social validation of intervention goals and procedures can be conducted at the beginning, during, and after the intervention, the results of the study indicate that, as consistently found and discussed in the social validity literature (Hurley, 2012; Snodgrass et al., 2018), the researchers of behavior interventions for young children with problem behavior also tend to assess social validity after termination of intervention. This implies that the researchers and practitioners missed the opportunity to work with the consumers to shape or refine the goals and procedures throughout the intervention. Even though the nine studies (32.1%) in the current review measured social validity more than once, no study reported that the intervention procedures were modified based on consumer feedback during intervention. Given that many researchers in behavioral interventions have consistently argued that social validation of goals and procedures should be used for intervention modifications and implementer training to improve outcomes (Miltenberger, 1990; Schwartz & Baer, 1991; Strain et al., 2012), providing a guideline for timing and frequency of social validity assessment may promote researchers and practitioners to employ all three dimensions of social validity in working with natural change agents in designing and implementing interventions.

We also examined the people who participated in social validity assessments and the types of social validity assessment methods used in the early childhood problem behavior literature. It was found that most of the studies involved direct consumers (interventionists) to assess social validity, and only four studies (12.1%) involved naïve (blind) observers who could provide more objective information about the intervention outcomes than direct consumers. In addition, only one study used a normative comparison method, in the form of peer comparison (Zimmerman et al., 2017), and only two studies assessed social validity with target children. Given that young children with disabilities have limited developmental skills to respond to subjective social validity assessment, using a normative comparison might be useful in assessing social validity of interventions with target children, as suggested by Hanley (2010). The results of the study also indicate that the range of reporting methods for social validity assessment results varied from summary statement only to reporting the results for each item, which corroborate the findings from a previous systematic review on social validity (Snodgrass et al., 2018). The primary reporting method was using descriptive statistics to summarize the results. Although there were several studies (n = 7) that assessed all three dimensions of social validity, no studies provided separate assessment results for each dimension. Considering that simply describing the results with a brief summary statement or providing information on individual items without integrating the findings does not provide sufficient information for decision making regarding the social validity of an intervention process and outcomes, future researchers should contemplate providing sufficient information on the social validity assessment results for readers to judge the social validity of any intervention.

Limitations and Conclusion

The research studies reviewed in this article offer evidence that researchers have actively promoted implementation of socially valid evidence-based or promising behavior interventions in natural home and classroom environments for young children with problem behavior. However, a few limitations should be considered when interpreting the results and for future research. The reviewed studies were limited to studies that used SCDs, excluding group design studies. Furthermore, although databases and reference reviews were used extensively to select the studies, it is likely that the authors overlooked articles that should have been included in the analysis. In addition, due to the limited number of studies reviewed, more in-depth analysis was not conducted, which would have allowed the authors to examine additional variables that might moderate social validity outcomes.

Findings from the present review extend the contributions from previous comprehensive or systematic reviews of social validity assessment in SCD studies, which evaluated social competence interventions for preschool children (Hurley, 2012), social skills interventions for young children with ASD (Ledford et al., 2016), and intervention research in special education (Snodgrass et al., 2018). We focused on studies concerning behavior intervention for children with problem behavior published between 2001 and 2018 and provided specific characteristics of individual studies and social validity assessment procedures and outcomes reported in each study. Given that the use of socially valid interventions that are feasible, effective, and sustainable in natural settings is essential to address problem behavior in young children, assessing social validity on the front end may be beneficial. This will allow natural change agents to work with others to make adjustments to the intervention goals and procedures so that they are more meaningful and feasible, and the change agents receive ongoing implementation support as needed. Although providing clear guidelines on how to incorporate social validity in all aspects of intervention may help practitioners effectively work with the typical change agents, the current status of social validity assessment in the field suggests the need for developing a social validity assessment instrument designed to assess all three dimensions of social validity. Researchers and practitioners should address all three dimensions of social validity in designing and evaluating interventions and improving intervention quality and outcomes.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Baer

D. M.

Wolf

M. M.

Risley

T. R.

(1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97.

*Bailey

K. M.

Blair

K. S. C.

(2015). Feasibility and potential efficacy of the family-centered Prevent-Teach-Reinforce model with families of children with developmental disorders. Research in Developmental Disabilities, 47, 218–233.

Barton

E. E.

Reichow

Schnitz

Smith

I. C.

Sherlock

(2015). A systematic review of sensory-based treatments for children with disabilities. Research in Developmental Disabilities, 37, 64–80.

Benedict

E. A.

Horner

R. H.

Squires

J. K.

(2007). Assessment and implementation of positive behavior support in preschools. Topics in Early Childhood Special Education, 27, 174–192.

*Blair

K. C.

Fox

Lentini

(2010). Use of positive behavior support to address the challenging behavior of young children within a community early childhood program. Topics in Early Childhood Special Education, 30, 68–79.

*Blair

K. C.

Lee

I.-S.

Cho

S.-J.

Dunlap

(2011). Positive behavior support through family–school collaboration for young children with autism. Topics in Early Childhood Special Education, 31, 22–36.

*Blair

K. C.

Liaupsin

C. J.

Umbreit

Kweon

(2006). Function-based intervention to support the inclusive placements of young children in Korea. Education and Training in Developmental Disabilities, 41, 48–57.

Brauner

C. B.

Stephens

C. B.

(2006). Estimating the prevalence of early childhood serious emotional/behavioral disorders: Challenges and recommendations. Public Health Reports, 121, 303–310.

Brestan

E. V.

Jacobs

J. R.

Rayfield

A. D.

Eyberg

S. M.

(1999). A consumer satisfaction measure for parent-child treatments and its relation to measures of child behavior change. Behavior Therapy, 30, 17–30.

10.

*Brock

M. E.

Beaman-Diglia

L. E.

(2018). Efficacy of coaching preschool teachers to manage challenging behavior. Education and Treatment of Children, 41, 31–48.

11.

Carr

J. E.

Austin

J. L.

Britton

L. N.

Kellum

K. K.

Bailey

J. S.

(1999). An assessment of social validity trends in applied behavior analysis. Behavioral Interventions, 14, 223–231.

12.

Carter

D. R.

Van Norman

R. K.

(2010). Class-wide positive behavior support in preschool: Improving teacher implementation through consultation. Early Childhood Education Journal, 38, 279–288.

13.

*Chu

S. Y.

(2015). An investigation of the effectiveness of family-centred positive behaviour support of young children with disabilities. International Journal of Early Years Education, 23, 172–191.

14.

Conroy

M. A.

Dunlap

Clarke

Alter

P. J.

(2005). A descriptive analysis of positive behavioral intervention research with young children with challenging behavior. Topics in Early Childhood Special Education, 25, 157–166.

15.

*Drogan

R. R.

Kern

(2014). Examination of the mechanisms underlying effectiveness of the turtle technique. Topics in Early Childhood Special Education, 33, 237–248.

16.

*Duda

M. A.

Clarke

Fox

Dunlap

(2008). Implementation of Positive Behavior Support With a Sibling Set in a Home Environment. Journal of Early Intervention, 30, 213–236.

17.

*Duda

M. A.

Dunlap

Fox

Lentini

Clarke

(2004). An experimental evaluation of positive behavior support in a community preschool program. Topics in Early Childhood Special Education, 24, 143–155.

18.

*Dufrene

B. A.

Doggett

R. A.

Henington

Watson

T. S.

(2007). Functional assessment and intervention for disruptive classroom behaviors in preschool and head start classrooms. Journal of Behavioral Education, 16, 368–388.

19.

*Dunlap

Ester

Langhans

Fox

(2006). Functional communication training with toddlers in home environments. Journal of Early Intervention, 28, 81–96.

20.

Elliott

S. N.

Treuting

M. V. B.

(1991). The Behavior Intervention Rating Scale: Development and validation of a pretreatment acceptability and effectiveness measure. Journal of School Psychology, 29, 43–51.

21.

Ennis

R. P.

Jolivette

Fredrick

L. D.

Alberto

P. A.

(2013). Using comparison peers as an objective measure of social validity: Recommendations for researchers. Focus on Autism and Other Developmental Disabilities, 28, 195–201.

22.

Eyberg

S. M.

Edwards

Boggs

S. R.

Foote

(1998). Maintaining the treatment effects of parent training: The role of booster sessions and other maintenance strategies. Clinical Psychology, 5, 544–554.

23.

*Fettig

Barton

E. E.

Carter

A. S.

Eisenhower

A. S.

(2016). Using e-coaching to support an early intervention provider’s implementation of a functional assessment-based intervention. Infants & Young Children, 29, 130–147.

24.

*Fettig

Schultz

T. R.

Sreckovic

M. A.

(2015). Effects of coaching on the implementation of functional assessment–based parent intervention in reducing challenging behaviors. Journal of Positive Behavior Interventions, 17, 170–180.

25.

Finney

J. W.

(1991). On further development of the concept of social validity. Journal of Applied Behavior Analysis, 24, 245–249.

26.

Fox

Dunlap

Powell

(2002). Young children with challenging behavior: Issues and considerations for behavior support. Journal of Positive Behavior Interventions, 4, 208–217.

27.

Galensky

T. L.

Miltenberger

R. G.

Stricker

J. M.

Garlinghouse

M. A.

(2001). Functional assessment and treatment of mealtime behavior problems. Journal of Positive Behavior Interventions, 3, 211–224.

28.

Gerow

Hagan-Burke

Rispoli

Gregori

Mason

Ninci

(2018). A systematic review of parent-implemented functional communication training for children with ASD. Behavior Modification, 42, 335–363.

29.

*Gibson

J. L.

Pennington

R. C.

Stenhoff

D. M.

Hopper

J. S.

(2010). Using desktop videoconferencing to deliver interventions to a preschool student with autism. Topics in Early Childhood Special Education, 29, 214–225.

30.

Gresham

F. M.

Lopez

M. F.

(1996). Social validation: A unifying concept for school-based consultation research and practice. School Psychology Quarterly, 11, 204–227.

31.

*Hancock

T. B.

Kaiser

A. P.

Delaney

E. M.

(2002). Teaching parents of preschoolers at high risk: Strategies to support language and positive behavior. Topics in Early Childhood Special Education, 22, 191–212.

32.

Hanley

G. P.

(2010). Toward effective and preferred programming: A case for the objective measurement of social validity with recipients of behavior-change programs. Behavior Analysis in Practice, 3, 13–21.

33.

Hawkins

R. P.

(1991). Is social validity what we are interested in? Argument for a functional approach. Journal of Applied Behavior Analysis, 24, 205–213.

34.

Healey

France

K. G.

Blampied

N. M.

(2009). Treating sleep disturbance in infants: What generalizes? Behavioral Interventions, 24, 23–41.

35.

Hemmeter

M. L.

Hardy

J. K.

Schnitz

A. G.

Adams

J. M.

Kinder

K. A.

(2015). Effects of training and coaching with performance feedback on teachers’ use of Pyramid Model practices. Topics in Early Childhood Special Education, 35, 144–156.

36.

Hemmeter

M. L.

Snyder

Fox

Algina

(2016). The efficacy of the pyramid model: Effects on teachers, classrooms and children. Topics in Early Childhood Special Education, 36, 133–146.

37.

Hemmeter

M. L.

Snyder

Kinder

Artman

(2011). Impact of performance feedback delivered via electronic mail on preschool teachers’ use of descriptive praise. Early Childhood Research Quarterly, 26, 96–109.

38.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.

39.

Hurley

J. J.

(2012). Social validity assessment in social competence interventions for preschool children: A review. Topics in Early Childhood Special Education, 32, 164–174.

40.

Jolstead

K. A.

Caldarella

Hansen

Korth

B. B.

Williams

Kamps

(2017). Implementing positive behavior support in preschools: An exploratory study of CW-FIT Tier 1. Journal of Positive Behavior Interventions, 19, 48–60.

41.

Kelley

M. L.

Heffer

R. W.

Gresham

F. M.

Elliott

S. N.

(1989). Development of a modified treatment evaluation inventory. Journal of Psychopathology and Behavioral Assessment, 11, 235–247.

42.

Kennedy

C. H.

(1992). Trends in the measurement of social validity. The Behavior Analyst, 15, 147–156.

43.

*Knowles

Blakely

Hansen

Machalicek

(2017). Parents with intellectual disabilities experiencing challenging child routines: A pilot study using embedded self-determination practices. Journal of Applied Research in Intellectual Disabilities, 30, 433–444.

44.

Lane

K. L.

Beebe-Frankenberger

M. E.

Lambros

K. M.

Pierson

(2001). Designing effective interventions for children at-risk for antisocial behavior: An integrated model of components necessary for making valid inferences. Psychology in the Schools, 38, 365–379.

45.

Ledford

J. R.

Hall

Conder

Lane

J. D.

(2016). Research for young children with autism spectrum disorders: Evidence of social and ecological validity. Topics in Early Childhood Special Education, 35, 223–233.

46.

Leko

M. M.

(2014). The value of qualitative methods in social validity research. Remedial and Special Education, 35, 275–286.

47.

*Lucyshyn

J. M.

Albin

R. W.

Horner

R. H.

Mann

J. C.

Mann

J. A.

Wadsworth

(2007). Family implementation of positive behavior support for a child with autism: Longitudinal, single-case, experimental, and descriptive replication and extension. Journal of Positive Behavior Interventions, 9, 131–150.

48.

*Lui

C. M.

Moore

D. W.

Anderson

(2014). Using a self-management intervention to increase compliance in children with ASD. Child & Family Behavior Therapy, 36, 259–279.

49.

*Masse

J. J.

McNeil

C. B.

Wagner

Quetsch

L. B.

(2016). Examining the efficacy of parent–child interaction therapy with children on the autism spectrum. Journal of Child and Family Studies, 25, 2508–2525.

50.

*McCoy

D. M.

Morrison

J. Q.

Barnett

D. W.

Kalra

H. D.

Donovan

L. K.

(2017). Using iPad tablets for self-modeling with preschoolers: Videos versus photos. Psychology in the Schools, 54, 821–836.

51.

*McDaniel

S. C.

Flower

(2015). Use of a behavioral graphic organizer to reduce disruptive behavior. Education and Treatment of Children, 38, 505–522.

52.

McLaren

E. M.

Nelson

C. M.

(2009). Using functional behavior assessment to develop behavior interventions for students in Head Start. Journal of Positive Behavior Interventions, 11, 3–21.

53.

Miltenberger

R. G.

(1990). Assessment of treatment acceptability: A review of the literature. Topics in Early Childhood Special Education, 10, 24–38.

54.

Miramontes

N. Y.

Marchant

Allen-Heath

Fischer

(2011). Social validity of a positive behavior support model. Education and Treatment of Children, 34, 445–468.

55.

*Moes

D. R.

Frea

W. D.

(2002). Contextualized behavioral support in early intervention for children with autism and their families. Journal of Autism and Developmental Disorders, 32, 519–533.

56.

Murphy

K. A.

Theodore

L. A.

Aloiso

Alric-Edwards

J. M.

Hughes

T. L.

(2007). Interdependent group contingency and mystery motivators to reduce preschool disruptive behavior. Psychology in the Schools, 44, 53–63.

57.

Oono

I. P.

Honey

E. J.

McConachie

(2013). Parent-mediated early intervention for young children with autism spectrum disorders (ASD). Evidence-Based Child Health, 8, 2380–2479.

58.

*Park

K. L.

Scott

T. M.

(2009). Antecedent-based interventions for young children at risk for emotional and behavioral disorders. Behavioral Disorders, 34, 196–211.

59.

Powell

Dunlap

Fox

(2006). Prevention and intervention for the challenging behaviors of toddlers and preschoolers. Infants & Young Children, 19, 25–35.

60.

Reimers

T. M.

Wacker

D. P.

Cooper

L. J.

(1991). Evaluation of the acceptability of treatments for their children’s behavioral difficulties: Ratings by parents receiving services in an outpatient clinic. Child & Family Behavior Therapy, 13, 53–71.

61.

Rispoli

Burke

M. D.

Hatton

Ninci

Zaini

Sanchez

(2015). Training head start teachers to conduct trial-based functional analysis of challenging behavior. Journal of Positive Behavior Interventions, 17, 235–244.

62.

Sawyer

M. R.

Crosland

K. A.

Miltenberger

R. G.

Rone

A. B.

(2015). Using behavioral skills training to promote the generalization of parenting skills to problematic routines. Child & Family Behavior Therapy, 37, 261–284.

63.

Schwartz

I. S.

Baer

D. M.

(1991). Social validity assessments: Is current practice state of the art? Journal of Applied Behavior Analysis, 24, 189–204.

64.

*Singh

N. N.

Lancioni

G. E.

Winton

A. S.

Fisher

B. C.

Wahler

R. G.

Mcaleavey

. . . Sabaawi

(2006). Mindful parenting decreases aggression, noncompliance, and self-injury in children with autism. Journal of Emotional and Behavioral Disorders, 14, 169–177.

65.

*Singh

N. N.

Lancioni

G. E.

Winton

A. S.

Singh

Curtis

W. J.

Wahler

R. G.

McAleavey

K. M.

(2007). Mindful parenting decreases aggression and increases social behavior in children with developmental disabilities. Behavior Modification, 31, 749–771.

66.

*Smith

S. C.

Lewis

T. J.

Stormont

(2011). The effectiveness of two universal behavioral supports for children with externalizing behavior in Head Start classrooms. Journal of Positive Behavior Interventions, 13, 133–143.

67.

Snodgrass

M. R.

Chung

M. Y.

Meadan

Halle

J. W.

(2018). Social validity in single-case research: A systematic literature review of prevalence and application. Research in Developmental Disabilities, 74, 160–173.

68.

Spear

C. F.

Strickland-Cohen

M. K.

Romer

Albin

R. W.

(2013). An examination of social validity within single-case research with students with emotional and behavioral disorders. Remedial and Special Education, 34, 357–370.

69.

Spohn

J. R.

Timko

T. C.

Sainato

D. M.

(1999). Increasing the social interactions of preschool children with disabilities during mealtimes: The effects of an interactive placemat game. Education and Treatment of Children, 22, 1–18.

70.

Stanley

M. A.

Averill

P. M.

(1998). Psychosocial treatments for obsessive–compulsive disorder: Clinical applications. In Swinson

R. P.

Antony

M. M.

Rachman

Richter

M. A.

(Eds.), Obsessive–compulsive behavior: Theory, research and treatment (pp. 277–297). New York, NY: Guilford Press.

71.

Stormont

M. A.

Smith

S. C.

Lewis

T. J.

(2007). Teacher implementation of precorrection and praise statements in Head Start classrooms as a component of a program-wide system of positive behavior support. Journal of Behavioral Education, 16, 280–290.

72.

Strain

P. S.

Barton

E. E.

Dunlap

(2012). Lessons learned about the utility of social validity. Education and Treatment of Children, 35, 183–200.

73.

*Strand

R. C.

Eldevik

(2018). Improvements in problem behavior in a child with autism spectrum diagnosis through synthesized analysis and treatment: A replication in an EIBI home program. Behavioral Interventions, 33, 102–111.

74.

Turan

Meadan

(2011). Social validity assessment in early childhood special education. Young Exceptional Children, 14, 13–28.

75.

What Works Clearinghouse. (2017). What Works Clearinghouse procedures handbook (Version 4.0). Retrieved from https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_handbook_v4.pdf

76.

Witt

J. C.

Elliott

S. N.

(1985). Acceptability of classroom management strategies. In Kratochwill

T. R.

(Ed.), Advances in school psychology (Vol. 4, pp. 251–288). Hillsdale, NJ: Lawrence Erlbaum.

77.

Witt

J. C.

Martens

B. K.

(1983). Assessing the acceptability of behavioral interventions used in classrooms. Psychology in the Schools, 20, 510–517.

78.

Wolf

M. M.

(1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart 1. Journal of Applied Behavior Analysis, 11, 203–214.

79.

Wood

B. K.

Blair

K. S. C.

Ferro

J. B.

(2009). Young children with challenging behavior: Function-based assessment and intervention. Topics in Early Childhood Special Education, 29, 68–78.

80.

*Zimmerman

K. N.

Ledford

J. R.

Barton

E. E.

(2017). Using visual activity schedules for young children with challenging behavior. Journal of Early Intervention, 39, 339–358.