Exploratory factor analysis of the Adapted Skillstreaming Checklist for children with autism spectrum disorder

Abstract

The Adapted Skillstreaming Checklist measures social/social-communication skills and behavioral flexibility/regulation of children with autism spectrum disorder without intellectual disability. Prior studies provided support for the reliability and criterion-related validity of the Adapted Skillstreaming Checklist total score for these children; however, no studies have examined the Adapted Skillstreaming Checklist factor structure. This exploratory factor analysis examined the factor structure and internal consistency of parent ratings on the Adapted Skillstreaming Checklist for a sample of 331 children, ages 6–12 years, with autism spectrum disorder without intellectual disability. Results yielded a correlated three-factor solution. The individual factors and total score demonstrated very good internal consistency reliability. Findings supported the presence and interpretability of three subscales, as well as derivation of a total composite reflecting overall prosocial and adaptive skills and behaviors. Implications for assessment and research are discussed.

Keywords

Adapted Skillstreaming Checklist exploratory factor analysis parent ratings children with ASD without ID

Significant social/social-communication impairments and circumscribed and repetitive behaviors and interests define autism spectrum disorder (ASD; American Psychiatric Association, 2013). The multi-symptom nature of the disorder, along with significant heterogeneity in symptom expression and functional levels of those diagnosed, poses a major assessment challenge. Factors such as cognitive and language abilities and developmental level influence the manifestation of skills and symptoms and can affect the psychometric properties of measures (Koenig, De Los Reyes, Cicchetti, Scahill, & Klin, 2009; Lord, Corsello, & Grzadzinski, 2014). This suggests the need for development and evaluation of measures for more homogeneous (narrower) subgroups with ASD (Lord et al., 2014). Assessment of clinical features and performance of children with ASD also requires consideration of the manner in which the symptom, skill, and/or behavior is measured. For example, diagnostic observations yield accurate diagnoses; however, they often rely on dichotomous measurement of symptoms (absent or present) which provides little information on the degree to which the skill, symptom, or behavior is exhibited or degree of impairment (Achenbach, 2011; Davis & Carter, 2014).

Rating scales are also used to measure the clinical features and skills of children with ASD (Davis & Carter, 2014; Lopata et al., 2017b). In contrast to diagnostic observations which can be time and labor intensive and require extensive training (Norris & Lecavalier, 2010), rating scales are easily administered, brief, and can assess a range of skills and symptoms based on informants in authentic environments (Constantino & Gruber, 2012; Lord & Corsello, 2005; Norris & Lecavalier, 2010). Continuous scaling of most rating scales is useful as the skills and symptoms of these children are not dichotomous (absent or present) and they exist on a continuum (Ibanez, Stone, & Coonrod, 2014). As such, rating scales can provide important information on the extent, frequency, or severity of the trait (Achenbach, 2011). Continuous scaling is also useful in measuring treatment outcomes (Achenbach, 2011; Reynolds & Kamphaus, 2015) including for ASD studies (Constantino & Gruber, 2012). There is widespread recognition of the need for treatment sensitive measures in ASD intervention studies and the negative impact of this issue on efficacy determinations (Bellini, Gardner, & Markoff, 2014; Stichter, Herzog, Owens, & Malugen, 2016; White, Keonig, & Scahill, 2007). Poor alignment of scale items with treatment targets can reduce a scale’s sensitivity (Koenig et al., 2009; McMahon, Lerner, & Britton, 2013; Stichter et al., 2016) so developing interventions and scales that are keyed to the clinical features of ASD may improve sensitivity (White et al., 2007). Although researchers have developed study-specific measures to increase treatment sensitivity (e.g. DeRosier, Swick, Davis, McMillen, & Matthews, 2011), few have been rigorously tested for their psychometric properties. This led Lopata et al. (2017b) and White et al. (2007) to recommend that researcher-developed measures be tested for their psychometric properties (and ease of use and cost), especially those that exhibit good treatment sensitivity.

One segment of the ASD population that has increased is children with ASD without intellectual disability (ID); this subgroup currently comprises more than two thirds of those diagnosed (Christensen et al., 2016). The increase in prevalence among this subgroup indicates the need for measures that yield valid information on the skills and clinical features of these children, can be easily completed, and are treatment sensitive (Lopata et al., 2017b; McMahon et al., 2013). Assessing skills and performance on a continuum (continuous scaling) is particularly important for children with ASD without ID as there are few social/social-communication behaviors that are completely absent, which warrants a different type of scale item and assessment approach (Lord et al., 2014). Dichotomous measurement may also be limited as it fails to recognize that skills and symptoms can be observed in contradictory ways. For example, some children may exhibit limited social initiations or interactions, whereas others exhibit excessive, odd, or inappropriate initiations or interactions (Bellini et al., 2014; Davis & Carter, 2014). In addition to social functioning, measures should also assess behavioral performance related to circumscribed and repetitive behaviors and interests as these can interfere with the social and adaptive skills of children with ASD without ID (Bauminger-Zviely, 2014).

The Adapted Skillstreaming Checklist (ASC; Lopata, Thomeer, Volker, Nida, & Lee, 2008) is a rating scale specifically designed to assess the functioning of children with ASD without ID. In contrast to most measures that assess the absence of social-communication skills or behaviors and the presence of unusual interests or behaviors (Lord et al., 2014), the ASC assesses these two dimensions from an adaptive perspective (i.e. prosocial skills and behavioral flexibility and regulation). The ASC was originally developed as a study-specific measure to assess outcomes of a psychosocial treatment for children with ASD without ID, with the treatment targets keyed to the diagnostic elements (social/social-communication skills and circumscribed and repetitive behaviors and interests). Scale items measure prosocial skills and behaviors aligned with the treatment targets and diagnostic features. A number of psychosocial intervention studies for children with ASD without ID have found the ASC to be treatment sensitive (e.g. within-group pre-posttest effect sizes from medium-to-large for parent ratings; Lopata et al., 2017a; Lopata et al., 2008). Sample-specific psychometric data were only presented for two of the interventions studies; these indicated good internal consistency (0.94) and moderate-to-high correlations with related scales on established measures of adaptive and clinical functioning (Lopata et al., 2010; Lopata et al., 2008). Despite the initial support, the data were based on very small samples (i.e. N = 54 and N = 36).

Only one psychometric study tested the reliability and validity of ASC parent ratings for a large sample of children with ASD without ID (N = 275; Lopata et al., 2017b). Internal consistency was very good (0.92) and test–retest reliability was very good at 6 weeks (Pearson r = 0.81, ICC = 0.78) and good at 9 months (Pearson r = 0.63, ICC = 0.64). Strong negative correlations were found between the ASC total score and ratings of ASD symptom severity (r = −0.69; Constantino & Gruber, 2012). Criterion-related validity was also supported in significant positive correlations between the ASC total and ratings of adaptive skills (including social skills r = 0.64) and significant negative correlations with ratings of externalizing behavior problems (composite r = −0.45) on a broad clinical measure (Reynolds & Kamphaus, 2004, 2015). Based on these positive findings, the authors recommended exploratory factor analyses to assess the possible presence of subscales within the ASC. Given its treatment sensitivity, documenting the ASC factor structure may provide researchers with a more refined measure for testing efficacy.

This study assessed the factor structure of ASC parent ratings for a large sample of children with ASD without ID. It addressed the need for studies of standardized measures used to assess the skills and performance of these children, particularly those used to monitor changes over time or treatment outcomes (Davis & Carter, 2014; McMahon et al., 2013). It also addressed the need for studies of measures that assess skills on a continuous scale and testing for the presence of factors that parallel the primary symptom dimensions (Constantino et al., 2004; Fernandopulle, 2011). Finally, it met the need for studies using a well-characterized but narrowly defined subgroup with ASD (without ID) as cognitive and language abilities can affect a measure’s properties including its factor profile (Fernandopulle, 2011; Lord et al., 2014).

Method

Participants

Parent ratings of 331 children, ages 6–12 years, with ASD without ID were included in the analyses. All children had participated in one of multiple prior trials testing the effectiveness of various psychosocial treatments for this population, and they were recruited for those trials via school and public announcements. Each child had a prior clinical diagnosis of ASD (or autism, Asperger’s, or Pervasive Developmental Disorder–Not Otherwise Specified), Wechsler Intelligence Scale for Children–4th Edition (WISC-IV; Wechsler, 2003) short-form IQ > 70, and Comprehensive Assessment of Spoken Language (CASL; Carrow-Woolfolk, 1999) short-form expressive or receptive language score > 70. Each child also met criteria on the Autism Diagnostic Interview–Revised (Rutter, LeCouteur, & Lord, 2003) or Social Communication Questionnaire (Rutter, Bailey, & Lord, 2003) which was completed to confirm her or his diagnosis. The child sample was predominantly male (89%) and White (87%) and had a mean IQ and language level in the average range. Parents reported an average parent education level of 15.7 years (Table 1). Demographic data were compiled from the various treatment trial databases.

Table 1.

Demographic characteristics of child sample and parent raters.

Characteristic	Child participants (N = 331)
Characteristic	M (SD)
Age (years)	9.31 (1.65)
Parent education (years)	15.66 (2.24)
WISC-IV Short-Form IQ	104.91 (14.38)
CASL
Short-Form Expressive Language	99.84 (15.92)
Short-Form Receptive Language	105.15 (15.78)
ADI-R
Impairment in Social Interaction	18.51 (5.33)^a
Impairment in Communication	15.01 (4.31)^a
Restricted Repetitive Behavior	5.78 (2.09)^a
SCQ Total Score	21.54 (5.28)^b
	n (% of total)
Gender
Male	294 (88.8)
Female	37 (11.2)
Ethnicity
White	289 (87.3)
African American	8 (2.4)
Latino	5 (1.5)
Asian American	7 (2.1)
Mixed race/ethnicity	22 (6.6)

WISC-IV: Wechsler Intelligence Scale for Children-4th Edition; CASL: Comprehensive Assessment of Spoken Language; ADI-R: Autism Diagnostic Interview-Revised; SCQ: Social Communication Questionnaire.

The WISC-IV 4-subtest short-form consisted of the Block Design, Similarities, Vocabulary, and Matrix Reasoning subtests and the CASL 4-subtest short-form consisted of the Antonyms, Synonyms, Syntax Construction, and Paragraph Comprehension subtests.

ADI-R scores based on a sample size of n = 262.

SCQ Total Score based on a sample size of n = 69.

Measure

ASC

The ASC (Lopata et al., 2008) is a 38-item rating scale developed to measure the social/social-communication skills and behavioral and interest flexibility and regulation of children with ASD without ID. Each item measures a specific skill or behavior that is keyed to a clinical feature of ASD. As noted, the ASC items assess these skills from a prosocial and adaptive perspective (i.e. extent to which the skill or adaptive behavior is exhibited). Parents rate each item on a scale from 1 (almost never) to 5 (almost always). The ASC includes 30 items (including adapted items) from the Skillstreaming curriculum (Goldstein, McGinnis, Sprafkin, Gershaw, & Klein, 1997; McGinnis & Goldstein, 1997) and 8 researcher-created items. Individual item scores are summed to yield a total composite score, and higher scores indicate greater use of the prosocial and adaptive skill or behavior. (Data on the psychometric properties of the ASC were described in the introduction.)

Procedures

Institutional Review Board (IRB) approval was obtained for each of the treatment trials from which the cases were compiled, along with informed consent and assent (Canisius College IRB). For each treatment trial, parents completed a battery of baseline (pretreatment) measures that included the ASC. Upon completion and return, each protocol was immediately reviewed to ensure it was complete. Incomplete protocols or protocols containing errors (e.g. omitted items, multiple responses to an item, etc.) were immediately reviewed with the parent to correct the error(s). Each treatment trial also instituted a structured scoring and data entry protocol to ensure accuracy. Each ASC was scored independently by two research assistants, with any discrepancies in scoring resolved by a third scorer. Following a similar procedure, all demographic and protocol data were initially entered into the study database by a research assistant and independently checked by a second research assistant, with any discrepancy corrected by a third member of the team.

Data diagnostics and analysis plan

Exploratory factor analysis (EFA) was selected as no prior studies have tested for the presence of factors within the ASC. This exploratory analytic method is useful in examination of latent constructs in a set of items or measures in the absence of prior theory or research (Floyd & Widaman, 1995). Prior to conducting the EFA, data quality, completeness, and suitability for factor analysis were examined. Complete data were available for all 331 cases, with no out-of-range values. The sample of 331 was considered adequate for EFA based on the study goal of conducting the first structural study of the measure, the homogeneous sample, and preliminary analysis (item analysis and matrix tests including the Kaiser–Meyer–Olkin and Bartlett’s tests), as well as guidelines and empirical studies of sample size issues in the factor analysis literature. Individual item analysis was conducted to examine distributions of the items. Skewness, kurtosis, and item-total correlations were examined for all items; the range of skewness values was −0.38 to 0.51 and kurtosis values was −0.70 to 0.41, and the mean item-total correlation was 0.46 with a range of 0.27 to 0.63.

With regard to sample size guidelines for EFA, many recommendations have focused on total sample size or item/participant ratio, which may be set in study planning. However, the quality of the data also affects the quality of the analysis, knowable only once the data are obtained (Bandalos & Finney, 2010). In terms of guidelines, Tinsley and Tinsley (1987) recommended 5–10 participants per item up to samples of 300 (in the present study the ratio was 8.7 participants/item). Comrey (1988) recommended that a sample size of 200 is “reasonably good” (p. 759) for 40 or fewer variables (the present study included 38 with 331 participants). In summarizing the guidelines, DeVellis (2017) concluded that, while not capturing the full complexity of validity issues in factor analysis, the guidelines generally suffice in study planning. Costello and Osborne (2005) reviewed a wide array of guidelines and simulations that went beyond consideration of sample size and item/participant ratios. These simulations illustrated the impact of interactions between communality, sample size, item number, and factorial complexity on the accuracy of reproduced results. Costello and Osborne (2005) concluded that larger communality values in the context of relatively small numbers of factors will improve reproducibility of factor structures. In the present study, initial communality ranged from 0.265 to 0.714 with a mean of 0.450. Costello and Osborne (2005) also emphasized the importance of the exploratory context (not hypothesis testing or confirmatory analysis) in evaluating data for EFA. SPSS 25 (item and reliability analysis, EFA) and Stata 15.1 (parallel analysis) were used in the current analyses.

For the current data set, the Kaiser–Meyer–Olkin measure of sampling adequacy was 0.88, indicating that most of the variance in the data was attributable to underlying factors. Similarly, Bartlett’s test of sphericity indicated that the correlation matrix was suitable for structural analysis (p < 0.001). Given the goal of identification of latent structure and expectation of correlated factors, principal axis factoring with oblimin rotation was utilized. Following examination of the communalities, scree plot, and eigenvalues, an optimal initial solution was identified. Follow-up analyses examined alternative solutions. Parallel analysis was also used in determining the optimal number of factors. The pattern and structure coefficients were reviewed and reported to facilitate interpretation of the final solution (Bandalos & Finney, 2010).

Results

The scree plot with results of the parallel analysis is displayed in Figure 1. The break in the eigenvalues appears at approximately 2, following the third factor. The eigenvalues and percent of variance for the first three factors were 9.72 (25.6%), 2.88 (7.6%), and 2.57 (6.8%). The parallel analysis also supports the viability of a three-factor solution, with the parallel eigenvalues well below the first three factors. The next step limited the analysis to three factors, followed by oblimin rotation. The pattern and structure coefficients from this analysis are presented in Table 2. The values in Table 2 further support the three-factor solution as both simple and interpretable. All three factors are represented by substantial numbers of items (19 for Factor 1, 9 for Factor 2, and 10 for Factor 3). The coefficients for each factor are generally moderate, and the pattern and structure coefficients correspond well overall in terms of relative position and at the item level in terms of magnitude. Table 3 presents the factor intercorrelations which are low to moderate (0.20–0.39). Coefficient alpha reliabilities for the three factors are 0.90 (Factor 1), 0.80 (Factor 2), and 0.79 (Factor 3), and 0.92 for the full scale.

Figure 1.

Scree plot with parallel analysis.

Table 2.

Factor loadings for exploratory factor analysis of ASC items, principal axis with oblimin rotation.

Item	Pattern coefficients			Structure coefficients
Item	Factor 1	Factor 2	Factor 3	Factor 1	Factor 2	Factor 3
5. Does your child let others know that he or she is grateful for favors, etc.?^a	0.708			0.665
22. Does your child show understanding of another person’s feelings?^b	0.647			0.666	0.347
25. Does your child let others know that he or she cares about them?^a	0.640			0.587
18. Does your child make verbal or written apologies for things said or done?^b	0.640			0.605	0.314
20. Does your child let others know which emotions he or she is feeling?^a	0.637			0.649
7. Does your child tell others that he or she likes something they have done?^b	0.633			0.653		0.330
19. Does your child recognize which emotions he or she has at different times?^a	0.616			0.631
21. Does your child understand what other people are feeling?^a	0.610			0.669	0.437
15. Does your child give assistance to other children who might need or want it?^a	0.558			0.584		0.337
32. Does your child express an honest complement to others about how they played a game?^b	0.542			0.572
16. Does your child acknowledge and accept complements from others?^b	0.520			0.598		0.347
2. Does your child begin conversations with other people?^a	0.513	−0.374	0.336	0.543		0.463
6. Does your child become acquainted with new people on his or her own?^a	0.497	−0.386		0.506		0.406
11. Does your child give assistance to adults who might need some assistance?^b	0.493			0.526		0.316
14. Does your child take steps to become part of an ongoing activity or group?^a	0.451		0.342	0.511		0.464
8. Does your child request assistance when he or she is having difficulty?^a	0.422			0.468
17. Does your child offer to share what he or she has with others?^a	0.403			0.483		0.311
4. Does your child know how and when to ask questions of another person?^a	0.389		0.367	0.533		0.520
31. Does your child help arrive at a plan that satisfies both him/herself and others who have taken different positions (i.e. negotiates)?^a	0.314			0.457	0.378	0.364
29. Does your child stay out of situations that might get him or her in trouble?^a		0.622			0.636
26. Does your child exercise self-control under difficult circumstances?^b		0.555			0.609
30. Does your child accept the consequence of her or his behavior?^b		0.506		0.318	0.557
24. Does your child try to understand someone else’s anger without getting angry him/herself?^a	0.335	0.457		0.458	0.547
33. Does your child deal positively with being left out of some activity?^a		0.456			0.488
27. Does your child understand when permission is needed and the right person to ask for it?^b		0.435		0.367	0.513
28. Does your child deal in a constructive way with being teased?^a		0.417			0.469
36. Does your child express her or his thoughts and concerns without complaining or whining?^c		0.414			0.469
23. Does your child express anger without verbal or physical aggression?^b		0.334			0.369
38. Does your child have discussions without running on about a specific topic?^c			0.605			0.591
37. Does your child have discussions with others without sharing information that is unrelated to the topic at-hand?^c			0.571			0.544
3. Does your child talk to others about things of interest to both of them?^a			0.551	0.369		0.606
9. Does your child carry out instructions from others quickly and correctly?^a			0.483		0.318	0.528
13. Does your child end conversations before leaving or beginning a new topic?^c			0.470			0.452
35. Does your child wait his or her turn to talk (without interrupting)?^c			0.460		0.312	0.501
10. Does your child contribute to discussions occurring in the environment?^c			0.438	0.407		0.520
34. Does your child maintain eye contact when talking with others?^c			0.401	0.362		0.483
1. Does your child listen when you or others talk to him or her?^a			0.376	0.342	0.341	0.474
12. Does your child ignore distractions and remain focused on the task at hand?^c			0.337			0.359

Highest loadings per factor bolded. Coefficients < 0.30 are not displayed. Items reproduced with permission from Research Press and Lopata and Thomeer.

Exact item from Skillstreaming curriculum (Goldstein, McGinnis, Sprafkin, Gershaw, & Klein, 1997; McGinnis & Goldstein, 1997).

Item adapted from the Skillstreaming curriculum (Goldstein et al., 1997; McGinnis & Goldstein, 1997).

Item created by Lopata and Thomeer (Lopata, Thomeer, Volker, Nida, & Lee, 2008).

Table 3.

Factor correlations.

	Factor 1	Factor 2	Factor 3
Factor 1	1.00
Factor 2	0.27	1.00
Factor 3	0.39	0.20	1.00

The items that comprised the first factor were examined to determine the underlying construct (skill or behavioral feature). Factor 1 was labeled Social Communication Skills (SCS) as all 19 items were assessing prosocial interpersonal skills related to social-communication and social-cognition (social and emotion understanding and expression, initiating interactions, responding to and interacting with others, etc.). For example, Item 5 strongly loaded on this factor and it focuses on the communication of gratitude toward others. Items 22 and 25 also loaded strongly and describe empathic interaction skills (understanding and expressing emotions). There are eight items with pattern coefficient and seven items with structure coefficient loadings greater than 0.60. The lowest loadings were for Item 31, which assesses negotiation skills (pattern coefficient = 0.314, structure coefficient = 0.457). Overall, Factor 1 (SCS) accounted for approximately 26% of the total variance prior to rotation.

Based on the content of the items, Factor 2 was labeled Behavior Regulation Skills (BRS). The nine items on this factor comprise about 8% of the total variance and assess skills involving self-control and avoiding and responding appropriately to challenging situations. The item loading highest on this factor was Item 29 (avoiding trouble situations; pattern coefficient = 0.622, structure coefficient = 0.636) and the highest three items all had loadings that exceeded 0.50 for both pattern and structure coefficients. The item with the lowest loading was Item 23 (expressing anger without aggression; pattern coefficient = 0.334, structure coefficient = 0.369).

After reviewing the content of the items in the third factor, Factor 3 was labeled Interest Regulation during Discussions (IRD). The 10 items on this factor accounted for approximately 7% of the variance and they reflect the child’s skills in regulating her or his interests during discussions and the manner in which those interfere with social conversations and interactions with others. The highest loading items on this factor (Items 38, 37, and 3) had pattern and structure coefficients above 0.50 and these directly assess skills in refraining from running on about or sharing unrelated information about a circumscribed interest during discussions, and discussing topics of interest to others. The lowest loading item was Item 12 (ignoring distractions and remaining focused; pattern coefficient = 0.337, structure coefficient = 0.359).

With regard to cross-loading, pattern coefficients for Factor 1 included four items with some degree of cross-loading, though all of these coefficients were less than 0.40. The higher loadings of these items on Factor 1, as well as the content of the items, clearly indicate their inclusion on Factor 1 (SCS). Factor 2 had one item that cross-loaded with another factor (Item 24, pattern coefficient = 0.335); however, that item had a higher loading on Factor 2, and its content was clearly more aligned with the content of Factor 2 items. There were no cross-loading items for Factor 3 in the pattern coefficients. There were more cross-loaded items in the structure coefficients (the correlations of the item with the factor). Although the differences in magnitude of the structural coefficients and content of the individual items clearly supported their inclusion in the primary-assigned factor, the content of the cross-loaded items could be seen to represent overlap with the additional factor or factors. Given the relatively clear factor structure, importance of the items in terms of capturing important ASD-related features, and fact that this was the first test of the ASC factor structure, no items were dropped. Follow-up analyses examining two-, four-, and five-factor models showed that the three-factor model was superior in terms of both interpretability and in producing lower factor correlations.

Finally, because the ASC has been used to monitor treatment outcomes in several psychosocial intervention studies for children with ASD without ID, the relationship between age and each ASC item was examined. To assess the possibility of a correlation between age and each item, distribution statistics and plots were examined. Age was normally distributed (skewness = 0.28, kurtosis = −0.94). Next, 38 scatterplots with regression lines of the individual items with age were examined for evidence of unusual patterns (non-linearity, odd clustering, outliers). These analyses showed no evidence of unusual patterns that might influence correlations. Correlations of each item with age were then calculated. The mean correlation was −0.007 (SD = 0.060), median correlation was −0.008, and range was from −0.15 to 0.14. These analyses indicate that age was unrelated to ASC item ratings in these data. Lopata et al. (2017b) also reported no significant association between age and the ASC total score.

Discussion

Children with ASD without ID constitute a majority and increasing proportion of children with ASD. This subgroup is characterized by relative strengths in cognitive and language abilities which can affect both the expression of skills, behaviors, and symptoms and the properties of assessment instruments including its factor profile (Fernandopulle, 2011; Lord et al., 2014). As such, there is a need for development and testing of measures for narrower subgroups with ASD including those without ID. In addition, there is widespread recognition of the need for treatment sensitive measures (e.g. Bellini et al., 2014; Stichter et al., 2016), as well as measures that utilize continuous scaling which yields important information on the degree to which a trait is exhibited and/or responsive to treatment (Achenbach, 2011; Constantino & Gruber, 2012). Continuous scaling is also important as the skills and behaviors of children with ASD without ID exist on a continuum and there are few skills and behaviors that are completely absent (Lord et al., 2014). Given the problems with treatment sensitivity, White et al. (2007) suggested that this might be improved by aligning the measure items and treatment targets to common features of ASD.

The ASC (Lopata et al., 2008) is a rating scale developed to assess the social/social-communication skills and behavior and interest regulation and flexibility of children with ASD without ID. Prior studies provided strong support for the reliability, criterion-related validity, and treatment sensitivity of the ASC for these children; however, no studies were identified that examined its factor structure; this study examined the factor structure and reliability of the ASC for a large sample of children with ASD without ID. Results yielded a three-factor correlated solution. The correlations among the three factors were low-to-moderate supporting the derivation of a composite score reflecting overall prosocial and behavioral skills, in addition to the three separate factor (subscale) scores. Internal consistency estimates were high for the three individual factors (0.79 to 0.90) and total score. Internal consistency for the ASC total score in this study (0.92) is consistent with that reported by Lopata et al. (2017b) for children with ASD without ID.

The largest factor, Social Communication Skills (SCS), consisted of 19 items assessing a range of social-communication and social-cognitive skills (e.g. begins conversations, asks questions of another, understands another’s feelings, recognizes own emotions). The second factor, Behavior Regulation Skills (BRS) consisted of 9 items. This factor was comprised of items measuring behavioral self-control skills such as appropriately responding to teasing, accepting consequences, expressing anger without aggression, dealing appropriately with being left out, and so forth. The third factor, Interest Regulation during Discussions (IRD), included 10 items. While many of these items clearly depicted interest regulation skills during conversations (e.g. talking without oversharing, talking about topics of interest to others, remaining on a topic), several items appeared to be related to social skills associated with interest regulation skills. For example, a child’s skills in transitioning to a new conversational topic, ignoring distractions, and/or waiting her or his turn to talk would be affected by her or his ability to self-regulate her or his own interest and engage with/follow the interest(s) of others. The correlations among the ASC factors provide some additional support for the link between interest regulation and social competencies as the association was highest between the SCS and IRD factors. This association was also reported by McDonald et al. (2015) who found circumscribed and repetitive interests and behaviors were significantly associated with adaptive social skills. Bauminger-Zviely (2014) similarly noted that restricted and repetitive interest and behaviors negatively impact social and adaptive functioning.

Overall, results suggest that the ASC items are measuring the skill areas identified by Lopata et al. (2017b, 2008); however, the prior descriptions identified two broad categories (i.e. social/social-communication skills and behavioral and interest regulation). The broad single area of behavioral and interest regulation skills described by Lopata et al. (2017b, 2008) appeared to consist of two factors in the current study, with BRS reflecting appropriate behavioral regulation and responses to negative events and IRD reflecting a separate skill area involving effectively managing intrusive circumscribed interests, especially during discussions, and their associated impact on some social skills.

Despite this being the first study to examine the factor structure of the ASC, the findings may have some clinical implications. For example, the prior intervention studies that used the ASC consisted of cognitive-behavioral treatments targeting social-communication and social-cognitive skills, as well as instructional techniques commonly used for children with ASD without ID in clinical and school settings (i.e. direct instruction, modeling, role-play/rehearsal, and performance feedback; McMahon et al., 2013; Reichow, Steiner, & Volkmar, 2012). Given the increasing use of cognitive-behavioral treatments (Ho, Stephenson, & Carter, 2018) and the common use of these individual instructional techniques in social interventions for children with ASD without ID, the ASC may provide researchers with a treatment-sensitive and psychometrically sound outcome measure. Findings of a correlated three-factor solution might also allow researchers testing interventions to examine treatment effects at a subscale level, as well as the overall ASC composite score. This might help more precisely measure treatment effects on specific areas of prosocial and adaptive functioning associated with ASD. Increased use of the ASC as part of social intervention studies for children with ASD without ID is needed to further assess its treatment sensitivity.

Although this study was the first to provide information on the ASC factor structure and it had a number of strengths (e.g. rigorous screening procedure, relatively large sample of children with ASD without ID, testing of a treatment sensitive measure, etc.), several limitations warrant mention. One limitation involved the relatively homogeneous and narrowly defined group of children in the sample (ASD without ID). While this helped minimize confounding of results (as child IQ, language, and developmental level can affect the properties of a measure), it limits the generalizability to others with ASD outside the inclusion parameters. The sample was also largely White and male, which further restricts the generalizability of findings. The current results were also limited to only parent ratings. Teachers are considered a critical source of information on the skills and symptoms of children with ASD (Norris & Lecavalier, 2010) due to their advanced knowledge of typical and atypical child development and observations of the children in educational settings (Constantino & Gruber, 2012; Mayes & Lockridge, 2018). Furthermore, because schools are the principal settings where psychosocial interventions are provided to these children (Kasari & Smith, 2013), teachers are often used to assess the children’s treatment responsiveness. Another limitation involved the fact that neither the current study nor the initial ASC study by the scale developer (Lopata et al., 2017b) conducted or reported any interviewing of the informants’ understanding of the items. A final cautionary note appears warranted regarding Item 34 that assesses eye contact during discussions. Although absent or reduced eye contact is a common clinical feature of ASD (APA, 2013), the expectation of eye contact may be culturally oriented toward White Western cultures and not necessarily expected or appropriate in all cultures. Given these limitations, future studies should consider testing the ASC with older and younger youth with ASD without ID, as well as with youth with ASD and ID to assess the potential impact of functional level on the scale’s properties. Studies should also seek to test the ASC properties in more racially and ethnically diverse samples, as well as for other informants (e.g. teachers) and clinical groups. In addition, future studies would benefit from interviews to clarify informants’ understanding of all the items; this includes studies with ASD and non-ASD samples. Such interviews will provide valuable information on the consistency with which informants interpret the items for children with ASD, as well as possible differences for non-ASD groups. For example, informants for typically developing children or children with other clinical diagnoses may interpret the items on the IRD factor as involving general conversational management skills that are not related to a circumscribed (i.e. special) interest. This may be in contrast to the core circumscribed and repetitive interests captured by informants’ ratings of children with ASD without ID.

The current results, along with prior psychometric testing, suggest that the ASC yields reliable and valid information on the skills and behaviors of children with ASD without ID. It also appears to be treatment sensitive to social interventions which are commonly used to develop the social and social-cognitive skills of these children. A unique aspect of the ASC is its assessment of ASD-related features (dimensions) from a prosocial and adaptive perspective using continuous scaling; this yields valuable information on the extent to which the skill or behavior is exhibited, which is important when tracking performance over time. This approach is also considered useful as the skills and behaviors of these children exist on a continuum with few being non-existent. Ongoing testing and replication studies of the ASC are clearly warranted as the field moves toward psychometrically sound measures that can be completed quickly and efficiently and that are cost-effective (Murray, Mayes, & Smith, 2011).

Supplemental Material

AUT868639_Lay_Abstract – Supplemental material for Exploratory factor analysis of the Adapted Skillstreaming Checklist for children with autism spectrum disorder

Supplemental material, AUT868639_Lay_Abstract for Exploratory factor analysis of the Adapted Skillstreaming Checklist for children with autism spectrum disorder by Christopher Lopata, James P Donnelly, Marcus L Thomeer, Jonathan D Rodgers, Martin A Volker and Adam J Booth in Autism

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Department of Education, Institute of Education Sciences (grants R324A080136 and R324A130216), US Department of Defense (grant W81XWH-15-1-0195), and a research grant from the Peter and Elizabeth C. Tower Foundation. Findings and conclusions are those of the authors and do not necessarily reflect the views of the funding agencies.

ORCID iDs

Christopher Lopata

Jonathan D Rodgers

References

Achenbach

T. M.

(2011). Commentary: Definitely more than measurement error: But how should we understand and deal with informant discrepancies? Journal of Clinical Child & Adolescent Psychology, 40, 80–86. doi:10.1080/15374416.2011.533416

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Bandalos

D. L.

Finney

S. J.

(2010). Exploratory and confirmatory factor analysis. In Hancock

G. R.

Mueller

R. O.

(Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 93–114). New York, NY: Routledge.

Bauminger-Zviely

(2014). School-age children with ASD. In Volkmar

F. R.

Rogers

S. J.

Paul

Pelphrey

K. A.

(Eds.), Handbook of autism and pervasive developmental disorders: Vol. 1. Diagnosis, development, and brain mechanisms (4th ed., pp. 148–175). Hoboken, NJ: John Wiley.

Bellini

Gardner

Markoff

(2014). Social skill interventions. In Volkmar

F. R.

Rogers

S. J.

Paul

Pelphrey

K. A.

(Eds.), Handbook of autism and pervasive developmental disorders: Vol. 2. Assessment, interventions, and policy (4th ed., pp. 887–906). Hoboken, NJ: John Wiley.

Carrow-Woolfolk

(1999). Comprehensive assessment of spoken language. Circle Pines, MN: American Guidance Service.

Christensen

D. L.

Baio

Braun

K. V. N.

Bilder

Charles

Constantino

J. N.

. . . Yeargin-Allsopp

(2016). Prevalence and characteristics of autism spectrum disorder among children aged 8 years–Autism and developmental disabilities monitoring network, 11 sites, United States, 2012. MMWR, 65, 1–23.

Comrey

A. L.

(1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56, 754–761. doi:10.1037/0022-006X.56.5.754

Constantino

J. N.

Gruber

C. P.

(2012). Social Responsiveness Scale, Second Edition (SRS-2). Torrance, CA: Western Psychological Services.

10.

Constantino

J. N.

Gruber

C. P.

Davis

Hayes

Passanante

Przybeck

(2004). The factor structure of autistic traits. Journal of Child Psychology and Psychiatry, 45, 719–726.

11.

Costello

A. B.

Osborne

J. W.

(2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your exploratory factor analysis. Practical Assessment, Research & Evaluation, 10. Retrieved from http://pareonline.net/getvn.asp?v=10&n=7

12.

Davis

N. O.

Carter

A. S.

(2014). Social development in autism. In Volkmar

F. R.

Rogers

S. J.

Paul

Pelphrey

K. A.

(Eds.), Handbook of autism and pervasive developmental disorders: Vol. 1. Diagnosis, development, and brain mechanisms (4th ed., pp. 212–229). Hoboken, NJ: John Wiley.

13.

DeRosier

M. E.

Swick

D. C.

Davis

N. O.

McMillen

J. S.

Matthews

(2011). The efficacy of a social skills group intervention for improving social behaviors in children with high functioning autism spectrum disorders. Journal of Autism and Developmental Disorders, 41, 1033–1043. doi:10.1007/s10803-010-1128-2

14.

DeVellis

R. F.

(2017). Scale development (4th ed.). Los Angeles, CA: SAGE.

15.

Fernandopulle

(2011). Measurement of autism: A review of four screening measures. Indian Journal of Psychological Medicine, 33, 5–10.

16.

Floyd

F. J.

Widaman

K. F.

(1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286–299. doi:10.1037/1040-3590.7.3.286

17.

Goldstein

A. P.

McGinnis

Sprafkin

R. P.

Gershaw

N. J.

Klein

(1997). Skillstreaming the adolescent: New strategies and perspectives for teaching prosocial skills (Rev. ed.). Champaign, IL: Research Press.

18.

B. P. V.

Stephenson

Carter

(2018). Cognitive-behavioral approaches for children with autism spectrum disorder: A trend analysis. Research in Autism Spectrum Disorders, 45, 27–41. doi:10.1016/j.rasd.2017.10.003

19.

Ibanez

L. V.

Stone

W. L.

Coonrod

E. E.

(2014). Screening for autism in young children. In Volkmar

F. R.

Rogers

S. J.

Paul

Pelphrey

K. A.

(Eds.), Handbook of autism and pervasive developmental disorders: Vol. 2. Assessment, intervention, and policy (4th ed., pp. 585–608). Hoboken, NJ: John Wiley.

20.

Kasari

Smith

(2013). Interventions in schools for children with autism spectrum disorder: Methods and recommendations. Autism, 17, 254–267. doi:10.1177/1362361312470496

21.

Koenig

De Los Reyes

Cicchetti

Scahill

Klin

(2009). Group intervention to promote social skills in school-age children with pervasive developmental disorders: Reconsidering efficacy. Journal of Autism and Developmental Disorders, 39, 1163–1172. doi:10.1007/s10803-009-0728-1

22.

Lopata

Lipinski

A. M.

Thomeer

M. L.

Rodgers

J. D.

Donnelly

J. P.

McDonald

C. A.

Volker

M. A.

(2017a). Open-trial pilot study of a comprehensive outpatient psychosocial treatment for high-functioning children with ASD. Autism, 21, 108–116. doi:10.1177/1362361316630201

23.

Lopata

Rodgers

J. D.

Donnelly

J. P.

Thomeer

M. L.

McDonald

C. A.

Volker

M. A.

(2017b). Psychometric properties of the Adapted Skillstreaming Checklist for high-functioning children with ASD. Journal of Autism and Developmental Disorders, 47, 2723–2732. doi:10.1007/s10803-017-3189-y

24.

Lopata

Thomeer

LVolker

M. A.

Toomey

J. A.

Nida

R. E.

Lee

G. K.

. . . Rodgers

J. D.

(2010). RCT of a manualized social treatment for high-functioning autism spectrum disorders. Journal of Autism and Developmental Disorders, 40, 1297–1310. doi:10.1007/s10803-010-0989-8

25.

Lopata

Thomeer

M. L.

Volker

M. A.

Nida

R. E.

Lee

G. K.

(2008). Effectiveness of a manualized summer social treatment program for high-functioning children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 38, 890–904. doi:10.1007/s10803-007-0460-7

26.

Lord

Corsello

(2005). Diagnostic instruments in autistic spectrum disorders. In Volkmar

F. R.

Paul

Klin

Cohen

(Eds.), Handbook of autism and pervasive developmental disorders: Vol. 2. Assessment, interventions, and policy (3rd ed., pp. 730–771). Hoboken, NJ: John Wiley.

27.

Lord

Corsello

Grzadzinski

(2014). Diagnostic instruments in autistic spectrum disorders. In Volkmar

F. R.

Rogers

S. J.

Paul

Pelphrey

K. A.

(Eds.), Handbook of autism and pervasive developmental disorders: Vol. 2. Assessment, interventions, and policy (4th ed., pp. 609–660). Hoboken, NJ: John Wiley.

28.

Mayes

S. D.

Lockridge

(2018). Brief report: How accurate is teacher report of autism symptoms compared to parent report? Journal of Autism and Developmental Disorders, 48, 1833–1840. doi:10.1007/s10803-017-3325-8

29.

McDonald

C. A.

Thomeer

M. L.

Lopata

Fox

J. D.

Donnelly

J. P.

Tang

Rodgers

J. D.

(2015). VABS-II ratings and predictors of adaptive behavior in children with HFASD. Journal of Developmental and Physical Disabilities, 27, 235–247. doi:10.1007/s10882-014-9411-3

30.

McGinnis

Goldstein

A. P.

(1997). Skillstreaming the elementary school child: New strategies and perspectives for teaching prosocial skills (Rev. ed.). Champaign, IL: Research Press.

31.

McMahon

C. M.

Lerner

M. D.

Britton

(2013). Group-based social skills interventions for adolescents with higher-functioning autism spectrum disorder: A review and looking to the future. Adolescent Health, Medicine and Therapeutics, 4, 23–38. doi:10.2147/AHMT.S25402

32.

Murray

M. J.

Mayes

S. D.

Smith

L. A.

(2011). Brief report: Excellent agreement between two brief autism scales (Checklist for Autism Spectrum Disorder and Social Responsiveness Scale) completed independently by parents and the Autism Diagnostic Interview-Revised. Journal of Autism and Developmental Disorders, 41, 1586–1590. doi:10.1007/s10803-011-1178-0

33.

Norris

Lecavalier

(2010). Screening accuracy of level 2 Autism Spectrum Disorder Rating Scales: A review of selected instruments. Autism, 14, 263–284. doi:10.1177/1362361309348071

34.

Reichow

Steiner

A. M.

Volkmar

(2012). Social skills groups for people aged 6 to 21 with autism spectrum disorders (ASD). Cochrane Database of Systematic Reviews, 7, CD008511. doi:10.1002/14651858.CD008511.pub2

35.

Reynolds

C. R.

Kamphaus

R. W.

(2004). Behavior assessment system for children (2nd ed.). Circle Pines, MN: AGS Publishing.

36.

Reynolds

C. R.

Kamphaus

R. W.

(2015). Behavior assessment system for children (3rd ed.). Bloomington, MN: Pearson.

37.

Rutter

Bailey

Lord

(2003). Social communication questionnaire. Los Angeles, CA: Western Psychological Services.

38.

Rutter

LeCouteur

Lord

(2003). Autism diagnostic interview-revised. Los Angeles, CA: Western Psychological Services.

39.

Stichter

J. P.

Herzog

M. J.

Owens

S. A.

Malugen

(2016). Manualization, feasibility, and effectiveness of the school-based Social Competence Intervention for Adolescents (SCI-A). Psychology in the Schools, 53, 583–600. doi:10.1002/pits.21928

40.

Tinsley

H. E. A.

Tinsley

D. J.

(1987). Uses of factor analysis in counseling psychology research. Journal of Counseling Psychology, 34, 414–424. doi:10.1037/0022-0167.34.4.414

41.

Wechsler

(2003). Wechsler Intelligence Scale for Children (4th ed.). San Antonio, TX: The Psychological Corporation.

42.

White

S. W.

Keonig

Scahill

(2007). Social skills development in children with autism spectrum disorders: A review of the intervention research. Journal of Autism and Developmental Disorders, 37, 1858–1868. doi:10.1007/s10803-006-0320-x

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB