Open-trial pilot study of a comprehensive outpatient psychosocial treatment for children with high-functioning autism spectrum disorder

Abstract

This study examined the feasibility and initial outcomes of a comprehensive outpatient psychosocial treatment (MAXout) for children aged 7–12 years with high-functioning autism spectrum disorder. The 18-week treatment, two 90-minute sessions per week, included instruction and therapeutic activities targeting social/social communication skills, facial emotion recognition, non-literal language skills, and interest expansion. A behavioral system was implemented to reduce autism spectrum disorder symptoms and problem behaviors and increase skills acquisition and maintenance. Feasibility was supported via high levels of treatment fidelity and parent, child, and staff satisfaction. Significant post-treatment improvements were found for the children’s non-literal language skills and facial emotion recognition skills, and parent and staff clinician ratings of targeted social/social communication skills, broad social skills, autism spectrum disorder symptoms, and problem behaviors. Results suggested that MAXout was feasible and may yield positive outcomes for children with high-functioning autism spectrum disorder.

Keywords

comprehensive outpatient psychosocial treatment high-functioning autism spectrum disorder manualized treatment

Introduction

The most recent autism spectrum disorder (ASD) estimates indicated a significant increase, with the greatest change among those affected being the increase in children with average-or-higher cognitive ability (i.e. nearly half of children with ASD; Center for Disease Control and Prevention (CDC), 2014). Children with high-functioning autism spectrum disorder (HFASD) exhibit the core symptoms of social impairments and circumscribed and repetitive interests and behaviors, but are differentiated based on relative strengths in cognitive and language abilities (American Psychiatric Association (APA), 2013). For individuals with HFASD, social impairments involve deficits in rudimentary social/social communication skills and behaviors, as well as more complex social-cognitive understanding/processing (APA, 2013; Bauminger-Zviely, 2013). Other social communication problems can include impaired ability to interpret emotions in facial expressions and understand non-literal and abstract language (APA, 2013; Harms et al., 2010; Safran et al., 2003). Restricted and repetitive behaviors/interests are also problematic and have been associated with lower adaptive social skills in children with HFASD (McDonald et al., 2015).

Developing effective treatments is challenging given the complex and multi-symptom nature of the disorder. Significant heterogeneity in functional levels also indicates a need for treatments designed to meet the unique needs of subgroups with ASD including HFASD. Social skills groups are one of the most common psychosocial treatments for children with HFASD. These groups typically meet one time per week (60–90 min) and utilize techniques such as explicit instruction, modeling, rehearsal/role-play, feedback, reinforcement, repeated practice, and/or parent training (Reichow et al., 2012) to promote social competencies (e.g. social understanding and knowledge, social skills/behaviors). Recent reviews of social skills studies have led some to conclude that they are an evidence-based practice (e.g. Wong et al., 2013), whereas others have characterized them as promising (due to study limitations) for children with HFASD (e.g. Reichow et al., 2012). Despite their promise, the gains reported have been modest on average, inconsistent in magnitude and across studies, and frequently narrow in scope (e.g. Frankel et al., 2010; Koenig et al., 2010). One reason for the variable findings might involve the intensity of the treatments (Reichow et al., 2012), with the one session per week format lacking the intensity needed to produce larger and consistent effects (Frankel et al., 2010).

Additionally, social skills groups are narrow in focus and do not treat the full range of symptoms and/or broader development. As a result, comprehensive treatment models (CTMs) have been advocated as they are designed to address targeted skills, ASD symptoms, broader functioning, and the needs of individual children (Odom et al., 2010; Smith et al., 2007). They are administered intensively for an extended period of time and/or at a high level of engagement, and they include multiple treatment elements targeting multiple domains including skills, ASD symptoms, and broader functioning (Odom et al., 2010; Smith et al., 2007). Many CTMs include social skills instruction and utilize similar techniques (direct instruction, modeling, role-play, reinforcement, etc.), yet few have been designed and evaluated specifically for children with HFASD (Stichter et al., 2012). In one notable exception, Stichter et al. (2012) tested a multi-component outpatient treatment for 20 children with HFASD. The 10-week protocol (two 60-min sessions per week) was delivered to groups of four to seven children with HFASD; utilized explicit instruction, modeling, practice, and reinforcement; and targeted multiple areas of impairment associated with HFASD. Child testing results (pre vs post) were non-significant for emotion recognition and total problem-solving, and mixed for perspective taking. Parent ratings reflected a significant increase in social skills and problem-solving (small-to-medium effects). Variability in findings led the authors to suggest that the treatment may have been too short in duration to bring about consistent gains across the target areas. The study did not report fidelity data, apply statistical corrections for the many comparisons, or indicate the number of facilitators per group. Furthermore, the treatment did not address the restricted and repetitive behaviors that characterize HFASD.

The limited data suggest that the dosage (duration and/or intensity) of comprehensive outpatient psychosocial treatments needs to be increased to yield more robust and broader improvements. Lopata, Thomeer, and colleagues documented the efficacy of an intensive comprehensive summer psychosocial treatment, summerMAX, for children with HFASD. The 5-week (5 days per week) treatment included five 70-min daily treatment cycles targeting social/social communication skills, facial emotion recognition skills, non-literal language skills, and interest expansion. Individual treatment cycles included 20 min of skills instruction followed by a 50-min therapeutic activity to practice the skills. Each group included six children with HFASD and three staff clinicians (undergraduate and graduate students). Skills were taught using direct instruction, modeling, role-play/rehearsal, and feedback, and a behavioral reinforcement system was used to promote and reinforce skills and decrease ASD symptoms and challenging behaviors. In addition to several non-randomized trials, summerMAX was tested in two randomized controlled trials (RCTs) (Lopata et al., 2010; Thomeer et al., 2012). Results of the trials indicated that at posttest, children who received summerMAX (compared to controls) demonstrated significantly better social-cognition and social communication skills (i.e. knowledge of target skills and non-literal language skills) and had significantly better parent and staff clinician ratings of targeted and broad skills and ASD symptoms. Effect sizes were predominantly in the medium and large ranges on scales demonstrating significant changes.

The evidence has clearly supported the efficacy of summerMAX for children with HFASD; however, the intensive full-day summer format may not be feasible or cost-effective for clinical practitioners (Freitag et al., 2013). As such, an outpatient adaptation is an appealing alternative and could provide practitioners with flexibility in selecting a treatment delivery format (i.e. summer and/or outpatient) based on needs and resources. Prior to investing significant resources in a large-scale RCT, it is first important to develop and test the feasibility of the treatment protocol (Odom et al., 2010; Smith et al., 2007). Feasibility trials are conducted to develop the treatment manual and fidelity measures and identify sensitive and valid outcome measures, as well as to test the protocol’s feasibility (fidelity of implementation and satisfaction with the treatment; Smith et al., 2007). Despite the need for such trials, few studies have systematically evaluated the feasibility of psychosocial treatments for HFASD (White et al., 2007). These studies can also generate preliminary data to determine whether exposure to the treatment was associated with positive gains (Lopata et al., 2013).

This study was conducted to test the feasibility and initial outcomes of a comprehensive outpatient psychosocial treatment, MAXout, for children with HFASD. The MAXout protocol was adapted from the summerMAX treatment previously found to be effective for children with HFASD. Feasibility was operationalized as fidelity of implementation (i.e. adherence) and satisfaction (parent, child, and staff), and initial child outcomes were assessed using direct child testing, parent ratings, and staff clinician ratings.

Method

Participants

A total of 44 children aged 7–12 years with HFASD were recruited using public announcements and participated in this study. Inclusion criteria were a prior clinical diagnosis of autism, Asperger’s, or pervasive developmental disorder–not otherwise specified; Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV; Wechsler, 2003) short-form intelligent quotient (IQ) >70 (and Verbal Comprehension Index or Perceptual Reasoning Index score ⩾80); and Comprehensive Assessment of Spoken Language (CASL; Carrow-Woolfolk, 1999) short-form expressive or receptive language score ⩾80. The WISC-IV short-form consisted of the Block Design, Similarities, Vocabulary, and Matrix Reasoning subtests, and the CASL short-form consisted of the Antonyms, Synonyms, Syntax Construction, and Paragraph Comprehension subtests. In addition, all met criteria on either the Autism Diagnostic Interview–Revised (ADI-R) (n = 33; Rutter et al., 2003) or Social Communication Questionnaire (n = 11; Rutter et al., 2003), completed to verify diagnosis. A total of 55 cases were screened for inclusion; three were rejected due to an IQ or language score below the minimum, four due to significant aggression problems, and four due to failing to meet criteria on the ADI-R. The children were randomly placed into groups of similar-aged participants (i.e. 7–9 year olds and 10–12 year olds). The majority was male and Caucasian, with an average parent education level of 16.46 years (see Table 1).

Table 1.

Demographic characteristics.

Characteristic	Participants (n = 44)
	Mean (SD)
Age (years)	9.09 (1.70)
Parent education (years)	16.46 (2.00)
WISC-IV
Short-form full scale	108.61 (14.54)
Short-form VCI	108.39 (15.33)
Short-form PRI	106.94 (17.62)
CASL
Short-form expressive language	104.69 (18.26)
Short-form receptive language	107.72 (16.20)
ADI-R
Social	18.61 (5.28)
Communication	15.76 (3.98)
Restricted and repetitive behavior	5.55 (2.37)
Social Communication Questionnaire	22.46 (6.41)
	n (% of total)
Gender
Male	41 (93.2)
Female	3 (6.8)
Ethnicity
Caucasian	39 (88.6)
African American	1 (2.3)
Latino	1 (2.3)
Asian	1 (2.3)
Other	2 (4.5)

WISC-IV: Wechsler Intelligence Scale for Children, Fourth Edition; VCI: Verbal Comprehension Index; PRI: Perceptual Reasoning Index; CASL: Comprehensive Assessment of Spoken Language; ADI-R: Autism Diagnostic Interview–Revised; SD: standard deviation.

Measures

When assessing the outcomes of comprehensive psychosocial treatments, a multi-dimensional and multi-method approach is warranted. Per recommendations to improve ASD outcome assessments, the measures assessed skills directly targeted by the treatment, as well as ASD features and broader skills and behaviors (Lord et al., 2005). The measures were selected based primarily on their use in prior trials of the summerMAX treatment for children with HFASD. All screening and direct child outcome tests were administered at the treatment site by graduate students and doctoral-level psychologists with advanced training in the specific measures.

CASL Idiomatic Language subtest

The CASL (Carrow-Woolfolk, 1999) Idiomatic Language subtest consists of 49 items in which the examinee is read an incomplete idiom and prompted to verbally provide the remaining word(s). Norms and psychometric information are provided only for individuals aged ⩾11 years. Internal consistency reliability reported in the test manual was 0.90. As the majority of children in the study were younger than 11 years, raw scores were used in the analyses. This subtest was selected as no other measure was identified that contained a standardized sample of idioms in ascending order of difficulty.

Cambridge Mindreading Face-Voice Battery for Children

The Cambridge Mindreading Face-Voice Battery for Children (CAM-C) measures emotion recognition for 15 emotion concepts using facial expression video clips and speech audio clips. The measure assesses recognition of six basic and nine complex emotions, with higher scores indicating greater accuracy. Children view or listen to a clip and select one of four emotion words that reflect the emotion of the person in the clip. In this study, the children completed the CAM-C on laptop computers and no time limits were imposed for completion of the test. The evaluators set up the CAM-C on each laptop and supervised the administration, but the children entered their own item responses and completed the test. The CAM-C consists of two subtests which yield scores for recognition of emotions in facial expressions (Faces total) and speech segments (Voices total). Given the focus of the treatment protocol, only the Faces score was used. The Faces subtest consists of 45 items, with the total score indicating the number of items correct. Test–retest reliability (10- to 15-week interval) was reportedly 0.79 for the Faces scale. The CAM-C effectively discriminates between children with HFASD and typical children, particularly the complex emotions (O Golan, 10 June 2009, personal communication). In a recent psychometric study of the CAM-C complex emotion items, Golan et al. (2015) found that the Faces total score accurately differentiated children with HFASD from typically developing children and test–retest reliability (10- to 15-week interval) for an HFASD sub-sample was 0.74. For the full sample, the Faces total score was negatively correlated with ASD symptom ratings. The CAM-C was also selected as it has been found to be treatment sensitive in prior emotion recognition intervention studies with children with HFASD (e.g. LaCava et al., 2010).

Adapted Skillstreaming Checklist

The Adapted Skillstreaming Checklist (ASC) assesses skills taught in the summerMAX program (Lopata et al., 2008). The 38-item scale contains 34 adapted items from the Skillstreaming curriculum (Goldstein et al., 1997; McGinnis and Goldstein, 1997) and four researcher-developed items assessing HFASD diagnostic features and problem behaviors. Parents and clinical staff rate the child’s skills on a scale ranging from 1 (almost never) to 5 (almost always), with higher total scores indicating greater use of the skills. The ASC composite yielded an internal consistency reliability of 0.94 and correlated 0.66 with the Behavior Assessment System for Children, Second Edition–Parent Rating Scale (BASC-2-PRS) (Reynolds and Kamphaus, 2004) Social Skills and 0.79 with the BASC-2-PRS Adaptive Skills Composite (see Lopata et al., 2010). These results reflect basic evidence of reliability and criterion-related validity.

BASC-2-PRS and BASC-2–Teacher Rating Scales

The BASC-2-PRS and BASC-2–Teacher Rating Scales (TRS) (Reynolds and Kamphaus, 2004) assess behaviors across a wide variety of domains. Informants rate items on a scale ranging from 0 (never) to 3 (almost always). This study used the Social Skills scale and Behavior Symptoms Index (BSI) as these assess broader functioning in areas problematic in HFASD. The Social Skills scale measures interpersonal aspects of social adaptation and skills needed for successful interaction, with higher scores indicating greater use of the skills. The BSI is a broad clinical composite that subsumes items from the Hyperactivity, Aggression, Anxiety, Depression, Atypicality, and Attention Problems scales, with lower scores indicating fewer problem behaviors. Internal consistency reliabilities for the Social Skills scale were reportedly 0.84–0.88 (PRS) and 0.91–0.93 (TRS) and for the BSI were reportedly 0.94–0.95 (PRS) and 0.96–0.97 (TRS). Moderate correlations have been established between these scales and comparable scales on other well-known rating scales (Reynolds and Kamphaus, 2004).

Social Responsiveness Scale, Second Edition, School Age Form

The Social Responsiveness Scale, Second Edition (SRS-2; Constantino and Gruber, 2012) is a 65-item objective measure of ASD-related symptoms and impairments including social communication deficits and circumscribed and repetitive behaviors and interests. Informants rate the frequency of behaviors on a scale of 1 (not true) to 4 (almost always true), with higher scores indicating greater symptoms and severity. Psychometric studies of the scale’s items have consistently documented a single-factor structure representing a unitary construct underlying ASD symptom severity. The Total score has internal consistency estimates of 0.92–0.97. Data in the manual indicate moderate-to-high correlations with other ASD diagnostic and screening measures, and that the test accurately discriminates between individuals with ASD, and individuals with other disorders and typically developing individuals.

Parent, child, and staff satisfaction surveys

Satisfaction surveys (Lopata et al., 2010; Thomeer et al., 2012) were administered to parents (10 items), children (8 items), and staff (7 items) following completion of treatment. Parents rated their satisfaction with staff understanding of child needs, cooperation and communication, effectiveness of the treatment, child progress, enjoyment, services, parenting classes, overall satisfaction, and comparison of this program with other programs. Children rated their satisfaction that activities were fun, counselors helped them make friends and learn friendship-making skills, they learned about new topics, better understood facial expressions, and made at least one new friend, overall satisfaction, and comparison of this program with other programs. Parent and child survey items were rated on a scale from 1 (completely dissatisfied) to 7 (completely satisfied). Staff rated their satisfaction with the clarity of duties, utility of the required readings, utility of the manual, effectiveness of training, effectiveness of feedback in increasing performance, timeliness of supervisor responses, and recommendation of the position to others. Staff rated items on a scale from 1 (strongly disagree) to 7 (strongly agree).

Procedures

The feasibility and initial outcomes of the comprehensive outpatient treatment, MAXout, were assessed in an open trial. The study was approved by the Institutional Review Board at Canisius College and conducted according to the approved protocol, including attainment of written parental consent and child assent. Pretest child measures and parent ratings were conducted immediately prior to treatment, and posttest measures were conducted immediately following completion of treatment. Staff pretest ratings were completed following 10 treatment sessions (to allow for staff familiarity with each child’s behaviors and skill levels), and posttest ratings were completed immediately following completion of treatment.

MAXout was created using the same basic treatment elements and structure as the summer program (summerMAX), only adapted into an outpatient model. The manualized outpatient protocol was administered after school on a college campus (in small group rooms) during two 90-min sessions per week over 18 weeks. Each 90-min session was divided into two 45-min treatment cycles (total of 72 treatment cycles), with the initial 15 min of every treatment cycle consisting of an intensive skills instruction period (skills group) followed by a 30-min therapeutic activity designed to practice the skills learned in the skills groups. To increase the intensity and active engagement of the treatment, each treatment group comprised four children with HFASD and two staff clinicians (undergraduate and graduate students). This small group size ensured that each child was a primary treatment recipient (primary actor or co-actor) in every 15-min skills group. The small group size also allowed for high rates of practice trials and feedback (reinforcement) during the skills groups and therapeutic activities. The treatment protocol was highly structured and targeted social/social communication skills, facial emotion recognition skills, non-literal language skills, and interest expansion using direct instruction, modeling, role-play/rehearsal, feedback/reinforcement, and repeated practice.

As noted, the initial 15 min of each treatment cycle consisted of an intensive skills group. Of the 72 skills groups, 60 were used to target 30 specific social/social communication skills (each covered twice). These social/social communication skill groups were conducted using the nine-step instructional sequence delineated in Skillstreaming (i.e. (1) define the skill, (2) model the skill, (3) establish trainee skill need, (4) select role-player, (5) set up the role-play, (6) conduct the role-play, (7) provide performance feedback, (8) assign skill homework, and (9) select next role-player; Goldstein et al., 1997), with skills taught in a progression from basic to more complex. Each of the 30 skills was selected to address a specific ASD feature (see Lopata et al., 2008). Six skills groups were used to teach decoding of facial emotions and six to teach interpretation of non-literal language. The facial emotion recognition groups included direct instruction and repeated practice in identifying facial features and positions that represent different emotions and recognizing physiological reactions indicative of the emotions. More specifically, this instruction included a description of various facial expressions and associated emotional states, in vivo examination of expressions by staff clinicians and child participants, and labeling of emotions and the physiological experience and behavioral reactions associated with these emotions. Non-literal language groups consisted of direct instruction and repeated practice in identifying and interpreting the multiple meanings of common language including idioms. The manualized protocol required that each child complete at least one active treatment sequence in which she or he was the primary treatment recipient (primary actor or co-actor) in every 15-min skills group, in addition to being an active observer and providing feedback to peers.

The last 30 min of each treatment cycle consisted of a therapeutic activity designed to practice and reinforce the social interaction, facial emotion recognition, and non-literal language skills targeted in the skills groups, as well as expand the children’s interests and behaviors. These activities were conducted using direct instruction, modeling, repeated practice, performance feedback, and transfer of learning. Because all of the activities are cooperative in nature and require interpersonal interaction and collaborative problem-solving among the children, they either directly or indirectly involved practice of social/social communication skills. Facial emotion recognition activities progressed from identification of basic expressions in pictures, to expressions in a video (live action children’s movie), to in vivo examination of other’s expressions during activities. These activities also focused on the link between expressions and physiological reactions and on how to recognize and use expressions in social situations. The non-literal language therapeutic activities consisted of practice in decoding non-literal elements of communication. In general, the non-literal language therapeutic activities required that the children determine whether a statement was meant to be interpreted literally, and if not, they had to identify alternative interpretations based on other contextual clues. The children then determined the best interpretation and received performance feedback. Finally, the children participated in interest expansion therapeutic activities designed to increase their engagement in topics and activities beyond their own circumscribed interests. These activities required exploration of the interests of others in the group and novel topics outside each child’s narrow interest(s). The children worked together on non-self-selected topics, shared newly learned information with their group members, discussed possible social scenarios where the new information might be useful, and described the social skills they used to complete the project.

A response-cost system and individual daily note (IDN) were also implemented to increase and maintain prosocial skills and reduce ASD symptoms and problem behaviors. In the response-cost system, points were verbally awarded or withdrawn immediately after the occurrence of target skills/behaviors. They were awarded for following operationally defined rules or using any previously taught skill (i.e. social/social communication skill, facial emotion recognition skill, and non-literal language skill) and lost for violating a group rule, not following instructions, or engaging in operationally defined ASD-associated behavior/negative social behavior (i.e. poor eye contact, sharing irrelevant information, run-on communication, etc.). Points were recorded on a standardized point sheet by one of the staff clinicians in each group, and brief point-based feedback was provided every 15 min throughout the treatment session describing each child’s performance during that 15-min interval. Because the point system addressed a common set of operationally defined skills and ASD symptoms that characterize these children, as well as program rules, the response-cost system was universally applied with all the participants. Each child also had an IDN containing two or three individualized skills/behavioral targets. IDN targets and baseline rates were identified using behavioral observations and tracking during the first 2 weeks of treatment. A standardized IDN template was used which included the child’s individualized skills/behavioral targets and target performance levels. IDN feedback was provided twice (at the middle and end of) each session. A small reward (i.e. reinforcer; edible snack) was provided, on site, at the end of each treatment session for each child achieving her or his IDN target level and parents provided a reinforcer at home for reaching the IDN target level. Parents also provided a reinforcer at home for each child reaching her or his weekly response-cost point target. These two behavioral systems allowed for targeting of a common set of HFASD clinical features and the needs of individual children (manualized and individualized; Smith et al., 2007).

Finally, parents participated in six 90-min group parent training sessions over the 18-week treatment. These sessions were led by a doctoral-level psychologist at the treatment site while the children were in treatment sessions. Each session included delivery of specific instructional content, as well as discussion of how parents could apply the information to improve the daily performance of their children. Specifically, parents were educated on components of the program and strategies for promoting skills and generalization, reducing ASD symptoms, and managing problem behaviors.

Staff clinician training and treatment integrity

The staff clinicians (undergraduate and graduate students) were recruited using public announcements disseminated by several local colleges and universities. A comprehensive staff training and fidelity monitoring plan was instituted to establish clinician competence and ensure adherence to the protocol (Smith et al., 2007). Staff had to achieve a score of 100% on a written exam assessing mastery of the treatment manual at the beginning of the training week. They then completed 5 days of classroom training and applied practice exercises led by a doctoral-level psychologist and pre-doctoral candidate in classrooms at the treatment site (college campus). Standardized fidelity checklists were used during applied practice exercises, and all staff were required to demonstrate ⩾ 90% fidelity (accuracy) administering the protocol. The fidelity checklists were also used to measure fidelity throughout the 18-week treatment. Fidelity observations were conducted by research assistants not involved with treatment delivery. These observations were done through one-way mirrors such that staff clinicians were unaware of when they were occurring. The research assistants established inter-observer agreement (IOA; ⩾ 90%) using the fidelity sheets prior to the study. Fidelity was assessed in 776 (49%) randomly selected component sessions and was 95% for skills groups and 94% for therapeutic activities. A total of 113 (15%) secondary observations were conducted simultaneously by a second observer to confirm the reliability of the primary fidelity evaluators. Average per session agreement was 97% (IOA was calculated as the percentage of agreement between the two raters at the individual element item level across the total items).

Results

Data analyses

Feasibility (fidelity and satisfaction) was assessed using descriptive data. Treatment outcomes (pre vs post) were assessed using paired-samples t-tests. Effect size estimates were calculated for all mean differences using Cohen’s d (Cohen, 1988), along with 95% confidence intervals. Outcome measures consisted of child tests, parent rating scales, and staff clinician rating scales. Experiment-wise alpha was maintained at 0.05 by adjusting the per-comparison alpha to < 0.005 (.05/10 comparisons). (Note that all results were in the hypothesized direction.)

Feasibility

Parent, child, and staff ratings reflected high levels of satisfaction. From a total of 70 possible points, parent total score ratings averaged 62.4 (item M = 6.2 of a maximum = 7). Average child satisfaction total was 42.3 of a possible 56 (item M = 5.3 of a maximum = 7). For staff, the average total score was 43.2 of a possible 49 (item M = 6.2 of a maximum = 7). As previously noted, fidelity data indicated a very high level of implementation accuracy (⩾ 94%).

Treatment outcomes

Ten comparisons were conducted, and all yielded significant improvements (Table 2). Child testing indicated significant increases on the Idiomatic Language (p < 0.001) and CAM-C Faces (p = 0.0045) scales, with the effect sizes being 0.32 and 0.25, respectively. Parent ratings indicated significant increases for the ASC and BASC-2 Social Skills and decreases for the SRS-2 and BASC-2 BSI scores (ps < 0.001). Effect sizes on these ratings ranged from 0.44 (BASC-2 Social Skills) to 1.35 (ASC). Staff clinician ratings reflected significant increases for the ASC and BASC-2 Social Skills and decreases for the SRS-2 and BASC-2 BSI scores (ps < 0.001). The effect sizes for these ratings ranged from 0.84 (BASC-2 BSI) to 1.80 (ASC).

Table 2.

Treatment outcomes (child tests, parent ratings, and staff clinician ratings), pretest and posttest scores, tests of significance, effect sizes, and confidence intervals.

	Pretest, mean (SD)	Posttest, mean (SD)	t(43)	p	Cohen’s d (95% CI)
Child testing
CASL Idiomatic Language	10.20 (7.68)	12.69 (7.69)	5.66	<0.001*	0.32 (0.19, 0.45)
CAM-C Faces	27.96 (7.42)	29.77 (7.40)	2.74	0.0045*	0.25 (0.06, 0.43)
Parent ratings
ASC	100.34 (17.89)	123.52 (16.48)	9.70	<0.001*	1.35 (0.95, 1.74)
BASC-2
Social Skills	36.57 (10.10)	40.89 (9.48)	4.96	<0.001*	0.44 (0.24, 0.64)
BSI	69.61 (10.89)	62.00 (8.71)	−7.81	<0.001*	−0.77 (−1.02, −0.52)
SRS-2	76.82 (9.90)	67.30 (7.59)	−7.43	<0.001*	−1.06 (−1.44, −0.71)
Staff clinician ratings
ASC	98.16 (21.04)	135.32 (20.19)	10.10	<0.001*	1.80 (1.28, 2.31)
BASC-2
Social skills	40.64 (8.71)	48.59 (7.63)	5.39	<0.001*	0.97 (0.56, 1.38)
BSI	65.30 (9.31)	58.09 (7.76)	−4.89	<0.001*	−0.84 (−1.22, −0.46)
SRS-2	75.05 (7.70)	59.36 (15.07)	−6.60	<0.001*	−1.31 (−1.78, −0.83)

CASL: Comprehensive Assessment of Spoken Language; CAM-C: Cambridge Mindreading Face-Voice Battery for Children; ASC: Adapted Skillstreaming Checklist; BASC-2: Behavior Assessment System for Children, Second Edition; BSI: Behavior Symptoms Index; SRS-2: Social Responsiveness Scale, Second Edition, School Age Form; CI: confidence interval; SD: standard deviation.

Sample size n = 44 for all comparisons. p-values were calculated as one-tailed tests. Standards for Cohen’s d: small = 0.20, medium = 0.50, and large = 0.80 (Cohen, 1988).

Comparisons significant at the adjusted critical p < 0.005.

Discussion

Children with HFASD constitute a growing subgroup and require treatments designed to meet their unique needs. Developing effective treatments is a major challenge as the treatments must be comprehensive and intensive enough to yield broad and robust gains. There is a dearth of comprehensive outpatient psychosocial treatments designed to address the multiple symptoms and unique features of children with HFASD (Stichter et al., 2012). A critical step in the development process is establishing the feasibility of a treatment protocol, as well as its promise, prior to conducting an RCT (Lopata et al., 2013; Smith et al., 2007).

In an effort to overcome limitations in prior treatments for children with HFASD (non-comprehensive, inadequate intensity, insufficient duration), a comprehensive outpatient psychosocial treatment, MAXout, was adapted from a prior summer treatment (summerMAX) for children with HFASD. The outpatient treatment included multiple components that targeted multiple skills and symptom areas associated with HFASD. The intensive 36-session protocol represents a substantial increase in dosage compared to other outpatient psychosocial treatments for children with HFASD. To test the feasibility of the treatment, data were collected on fidelity of implementation and satisfaction by critical stakeholders. Fidelity was assessed in nearly half of the component sessions and was very high (⩾ 94%), and the high IOA among the observers (97%) provided support for the reliability of the standardized fidelity sheets. High levels of satisfaction were reported by parents, children, and staff clinicians, and the somewhat lower child satisfaction was expected given that the treatment was targeting areas known to be challenging for children with HFASD. These results suggest that MAXout can be implemented with a high level of accuracy, and key stakeholders found the protocol quite acceptable. In addition, these feasibility results (fidelity and satisfaction) were consistent with prior studies of the summer program (summerMAX; Lopata et al., 2010; Thomeer et al., 2012) from which the MAXout treatment was adapted.

Multiple child tests and parent and staff clinician ratings were used to assess whether completion of the MAXout treatment was associated with child improvements. Following treatment, the children demonstrated significantly better understanding of idiomatic language and emotions in facial expressions compared to baseline. The finding of significantly improved idiomatic language is consistent with results of the prior summerMAX treatment studies. This study used a different measure for facial emotion recognition testing (CAM-C) and it demonstrated treatment sensitivity. This is a promising development as questions have been raised about the ability of emotion recognition tests to detect treatment gains associated with psychosocial treatments (Lopata et al., 2015). Although the facial emotion recognition results were promising and reflected significant improvement in the children’s skills, they were based on an average gain of approximately two points. Parent and staff clinician ratings were consistent and suggested that exposure to the treatment yielded significant improvements in the children’s social/social communication skills (both targeted and broader skills), problem behaviors, and ASD symptoms. These improvements were consistent with, and on some measures exceeded, those of prior studies of the summerMAX program (Lopata et al., 2008, 2010; Thomeer et al., 2012), and they suggest similar gains may be derived from the outpatient (MAXout) treatment.

Results of this pilot study were somewhat unique (compared to other outpatient treatments) in finding significant gains across the measures, with most being medium-to-large in magnitude. The MAXout treatment was longer in duration and more intensive (i.e. two 90-min sessions per week over 18 weeks; groups of four children with HFASD and two staff clinicians) than prior outpatient programs which may account for the differences in patterns of findings (e.g. Frankel et al., 2010; Koenig et al., 2010; Stichter et al., 2012). The current findings suggest that increasing the dosage (duration and intensity) may be a critical factor in developing comprehensive outpatient psychosocial treatments for children with HFASD. Interestingly, increasing the dosage did not appear to negatively affect the feasibility of the treatment protocol. A final note is needed regarding the instructional techniques used in the protocol. Although MAXout is unique as a comprehensive outpatient model, it utilized instructional techniques common in other psychosocial treatments for youth with HFASD (e.g. direct instruction, modeling, role-play/rehearsal, reinforcement; Reichow et al., 2012), and the current results appear to support their ongoing inclusion in psychosocial treatments for this population.

This study had a number of strengths including a well-characterized sample, manualized protocol, structured fidelity monitoring system, use of multiple methods and sources for feasibility and outcome assessments, and statistical correction to control experiment-wise error. MAXout also represents one of the few comprehensive outpatient psychosocial treatments specifically for children with HFASD. Despite these strengths, the pre–post findings should be considered preliminary due to limitations inherent in a one-group pre–post trial including several threats to internal validity (e.g. statistical regression, maturation). While it appears unlikely that time alone could have brought about the improvements given the long-term and pervasive impairments of individuals with HFASD, the absence of a no-treatment control group precludes causal attributions. Additionally, no follow-up assessment of maintenance was done. Another limitation involved the use of raters (parents and clinicians) who were aware the children were receiving treatment; therefore, rater bias cannot be ruled out for the rating scale measures. This is a common limitation in studies for HFASD as the treatment is delivered by staff clinicians and parents sometimes participate in the treatment (e.g. parent education; Reichow et al., 2012). This limitation should also be considered in light of the improvements observed on the direct child tests. Finally, the study lacked direct behavioral observations by naïve raters. Although often not feasible in larger clinical studies for children with HFASD (White et al., 2007), behavioral observations by naïve raters would minimize potential rater bias.

Results clearly demonstrated the feasibility of the MAXout protocol and suggested that completion of the treatment was associated with significant improvements for children with HFASD. Given the initial indications of positive outcomes and methodological limitations in this study, a large-scale RCT appears warranted. That RCT could be further strengthened by including a more diverse (gender and ethnic) group of participants, direct behavioral observations by naïve raters, long-term follow-up assessment, and/or the Autism Diagnostic Observation Schedule (ADOS) as part of the screening battery. Such a study will allow for clearer conclusions concerning the efficacy of MAXout for children with HFASD. Finally, future studies might also consider the use of an alternative design such as a sequential multiple assignment randomized trial (SMART) design to determine the minimally sufficient dosage. Validating the efficacy of programs such as MAXout for children with HFASD is critical as the field needs effective treatment programs that are easily exportable. Exportability may be greatly enhanced when the program is manualized, with clearly delineated content and operationalized instructional procedures (such as MAXout). The ability to provide such treatments in group format (vs individually) is also appealing and may increase their feasibility in outpatient clinical settings.

Footnotes

Acknowledgements

The authors would like to acknowledge Gregory A Fabiano, PhD, for his input on methodological considerations in this trial.

Funding

This work was supported by the Peter and Elizabeth C Tower Foundation. Findings and conclusions are those of the authors and do not necessarily reflect the views of the funding agency.

References

American Psychiatric Association (APA) (2013) Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: APA.

Bauminger-Zviely

(2013) Cognitive-behavioral-ecological intervention to facilitate social-emotional understanding and social interaction in youth with high-functioning ASD. In: Scarpa

White

Attwood

(eds) CBT for Children and Adolescents with High-Functioning Autism Spectrum Disorders. New York: Guilford Press, pp. 226–255.

Carrow-Woolfolk

(1999) Comprehensive Assessment of Spoken Language. Circle Pines, MN: American Guidance Service.

Center for Disease Control and Prevention (CDC) (2014) Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 states, United States, 2010. MMWR 63(SS02): 1–21.

Cohen

(1988) Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum.

Constantino

Gruber

(2012) Social Responsiveness Scale, Second Edition (SRS-2). Torrance, CA: Western Psychological Services.

Frankel

Myatt

Sugar

. (2010) A randomized controlled study of parent-assisted children’s friendship training with children having autism spectrum disorders. Journal of Autism and Developmental Disorders 40(7): 827–842.

Freitag

Cholemkery

Elsuni

. (2013) The group-based social skills training SOSTA-FRA in children and adolescents with high functioning autism spectrum disorder—the study protocol of the randomised, multi-centre controlled SOSTA—net trial. Trials 14(6): 1–12. Available at: http://www.trialsjournal.com/content/14/1/6

Golan

Sinai-Gavrilov

Baron-Cohen

(2015) The Cambridge Mindreading Face-Voice Battery for Children (CAM-C): complex emotion recognition in children with and without autism spectrum conditions. Molecular Autism 6: 22.

10.

Goldstein

McGinnis

Sprafkin

. (1997) Skillstreaming the Adolescent: New Strategies and Perspectives for Teaching Prosocial Skills (Revised edition). Champaign, IL: Research Press.

11.

Harms

Martin

Wallace

(2010) Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychology Review 20(3): 290–322.

12.

Koenig

White

Pachler

. (2010) Promoting social skill development in children with pervasive developmental disorders: a feasibility and efficacy study. Journal of Autism and Developmental Disorders 40(10): 1209–1218.

13.

LaCava

Rankin

Mahlios

. (2010) A single case design evaluation of a software and tutor intervention addressing emotion recognition and social interaction in four boys with ASD. Autism 14(3): 161–178.

14.

Lopata

Thomeer

Lipinski

. (2015) RCT examining the effect of treatment intensity for a psychosocial treatment for high-functioning children with ASD. Research in Autism Spectrum Disorders 17: 52–63.

15.

Lopata

Thomeer

Volker

. (2008) Effectiveness of a manualized summer social treatment program for high-functioning children with autism spectrum disorders. Journal of Autism and Developmental Disorders 38(5): 890–904.

16.

Lopata

Thomeer

Volker

. (2010) RCT of a manualized social treatment for high-functioning autism spectrum disorders. Journal of Autism and Developmental Disorders 40(11): 1297–1310.

17.

Lopata

Thomeer

Volker

. (2013) A manualized summer program for social skills in children with high-functioning ASD. In: Scarpa

White

Attwood

(eds) CBT for Children and Adolescents with High-Functioning Autism Spectrum Disorders. New York: Guilford Press, pp. 199–225.

18.

Lord

Wagner

Rogers

. (2005) Challenges in evaluating psychosocial interventions for autistic spectrum disorders. Journal of Autism and Developmental Disorders 35(6): 695–708.

19.

McDonald

Thomeer

Lopata

. (2015) VABS-II ratings and predictors of adaptive behavior in children with HFASD. Journal of Developmental and Physical Disabilities 27(2): 235–247.

20.

McGinnis

Goldstein

(1997) Skillstreaming the Elementary School Child: New Strategies and Perspectives for Teaching Prosocial Skills (Revised edition). Champaign, IL: Research Press.

21.

Odom

Boyd

Hall

. (2010) Evaluation of comprehensive treatment models for individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders 40(4): 425–436.

22.

Reichow

Steiner

Volkmar

(2012) Social skills groups for people aged 6 to 21 with autism spectrum disorders (ASD). Cochrane Database of Systematic Reviews 7: CD008511.

23.

Reynolds

Kamphaus

(2004) Behavior Assessment System for Children. 2nd ed. Circle Pines, MN: AGS.

24.

Rutter

Bailey

Lord

(2003) Social Communication Questionnaire. Los Angeles, CA: Western Psychological Services.

25.

Rutter

LeCouteur

Lord

(2003) Autism Diagnostic Interview-Revised. Los Angeles, CA: Western Psychological Services.

26.

Safran

Ellis

(2003) Intervention ABCs for children with Asperger syndrome. Topics in Language Disorders 23(2): 154–165.

27.

Smith

Scahill

Dawson

. (2007) Designing research studies on psychosocial interventions in autism. Journal of Autism and Developmental Disorders 37(2): 354–366.

28.

Stichter

O’Connor

Herzog

. (2012) Social competence intervention for students with Aspergers syndrome and high functioning autism. Journal of Autism and Developmental Disorders 42(3): 354–366.

29.

Thomeer

Lopata

Volker

. (2012) Randomized clinical trial replication of a psychosocial treatment for children with high-functioning autism spectrum disorders. Psychology in the Schools 49(10): 942–954.

30.

Wechsler

(2003) Wechsler Intelligence Scale for Children. 4th ed. San Antonio, TX: The Psychological Corporation.

31.

White

Koenig

Scahill

(2007) Social skills development in children with autism spectrum disorders: a review of the intervention research. Journal of Autism and Developmental Disorders 37(10): 1858–1868.

32.

Wong

Odom

Hume

. (2013) Evidence-Based Practices for Children, Youth, and Young Adults with Autism Spectrum Disorder. Chapel Hill, NC: Autism Evidence-Based Practice Review Group, Frank Porter Graham Child Development Institute, The University of North Carolina.