Abstract
Anxiety disorders frequently co-occur in youth with autism spectrum disorders. In addition to developing efficacious treatments for anxiety in children with autism spectrum disorders, it is important to examine the transportability of these treatments to real-world settings. Study aims were to (a) train clinicians to deliver Facing Your Fears: Group Therapy for Managing Anxiety in Children with High-Functioning Autism Spectrum Disorders to fidelity and (b) examine feasibility of the program for novel settings. A secondary aim was to examine preliminary youth treatment outcome. Results indicated that clinicians obtained excellent fidelity following a workshop and ongoing consultation. Acceptability ratings indicated that Facing Your Fears Therapy was viewed favorably, and critiques were incorporated into program revisions. Meaningful reductions in anxiety were reported posttreatment for 53% of children. Results support the initial effectiveness and transportability of Facing Your Fears Therapy in new clinical settings.
Psychiatric disorders frequently co-occur in children with autism spectrum disorders (ASD) and markedly impede their treatment progress, signaling the need for accessible, individualized, and efficacious interventions (Dawson and Burner, 2011; Levy et al., 2010). Anxiety disorders are among the most commonly co-occurring conditions (Leyfer et al., 2006) and can be debilitating for youth with ASD, significantly interfering with participation in home, school, and community settings. Furthermore, anxiety severity is associated with increased aggression, reduced participation in social activities, and poor relationships for youth with ASD (Selles and Storch, 2012).
Cognitive-behavioral treatments (CBTs) are well-established, highly researched, evidenced-based treatments that are considered the gold standard psychosocial interventions for typically developing youth with anxiety (Silverman et al., 2008; Walkup et al., 2008). CBT has been extended to youth with ASD, with the completion of at least seven randomized controlled trials yielding positive results (Chalfant et al., 2007; Reaven et al., 2012; Sofronoff et al., 2005; Storch et al., 2013; Sung et al., 2011; White et al., 2012; Wood et al., 2009). Treatment has been delivered individually (Wood et al., 2009), in small groups (Chalfant et al., 2007; Reaven et al., 2012; Sofronoff et al., 2005), and in combined individual and small groups (White et al., 2012). Treatment studies that have been conducted thus far for children with ASD and anxiety have occurred primarily in tightly controlled university-based research clinics. Thus, there have been limited efforts to deliver the interventions in real-world clinical settings using practitioners who have minimal experience using CBT with youth with ASD. To date, clinician qualifications have varied as some studies have explicitly required clinicians to have therapeutic experience in ASD and anxiety (Storch et al., 2013), while in other studies, prior training in CBT has not been a prerequisite for participation (Sofronoff et al., 2005). A range of clinical settings, including pediatric clinics and community mental health centers, can play a crucial role in the treatment of co-occurring psychiatric disorders in youth with ASD; however, lack of specialized training for providers in these settings may limit the usefulness of these frontline clinicians (Brookman-Frazee et al., 2012).
Although it has been critically important to develop treatments that address the serious co-occurring anxiety symptoms present in so many children with ASD, it is equally important to facilitate the portability of these treatments beyond the lab setting, into the hands of clinicians across settings (Schoenwald and Hoagwood, 2001). Although enhancing the portability of efficacious treatments outside of clinical research environments is not easy, introducing new treatments into clinical settings early in the development process may inform protocol development, increase acceptability of the intervention, and maximize success for eventual clinical practice (Beidas et al., 2011; Weisz et al., 2004).
Introducing evidence-based interventions to clinicians
Dissemination research is the “systematic study of processes and factors that lead to widespread use of an evidence-based intervention” (Rabin et al., 2008: 119). One aspect of dissemination research is transportability or the examination of the transfer of potentially efficacious interventions to usual-care settings (Schoenwald and Hoagwood, 2001). Important questions for transportability research to answer may include defining (a) the intervention, (b) the intended target population, (c) who can conduct the intervention, and (d) under what circumstances the intervention may be delivered (Schoenwald and Hoagwood, 2001). Implementation research seeks to identify the processes that lead to the successful integration of an intervention into a novel setting (Rabin et al., 2008). In other words, implementation research identifies whether the core components of an intervention were delivered as originally intended (e.g. treatment fidelity) and the extent to which the intervention is feasible in a given context. Feasibility examines the acceptability of an intervention by the targeted population, as well as by the clinicians who deliver the treatment program. Factors such as participant recruitment, attendance, and treatment completion may also help determine feasibility of an intervention (Proctor et al., 2011).
Clinicians who are inexperienced with a new intervention may seek professional development opportunities that typically include studying a manual and/or attending a workshop, although this process is often seen as insufficient for true skill attainment (Herschell et al., 2010). Evidence suggests that behaviorally oriented training techniques such as direct feedback, behavioral rehearsal, and role-play are key workshop components of skill transfer (Beidas and Kendall, 2010; Herschell et al., 2010). Furthermore, the transportability of new treatments is often hindered by a lack of critical feedback and input from practitioners during the development process (Hatgis et al., 2001). Pairing ongoing expert consultation and supervision with intensive trainings to targeted providers, and allowing for bidirectional input from these same providers to the treatment developers, may be important for the successful adoption of new interventions. Allowing for bidirectional input likely creates active and engaged providers rather than passive recipients of training (Beidas et al., 2011). Funneling provider input directly into program revisions is an invaluable step toward enhancing the portability and development of acceptable treatments (Weisz et al., 2004). Moreover, careful attention and planning in the transportability of a new intervention may be a necessary precursor for wide-scale dissemination efforts (Schoenwald and Hoagwood, 2001).
This article describes the initial steps in the implementation of a group CBT program for managing anxiety in youth with ASD (Facing Your Fears: Group Therapy for Managing Anxiety in Children with High-Functioning Autism Spectrum Disorders (FYF); Reaven et al., 2011) in a novel clinical setting. The FYF program followed a multi-stage model essential for the development and implementation of evidence-based practice, including the completion of pilot testing, feasibility trials, and the development of treatment manuals and fidelity measures (Beidas et al., 2011; Reaven et al., 2009, 2012; Smith et al., 2007). However, rather than move toward large multi-site studies to further assess efficacy, FYF was introduced into a new clinical setting with clinicians new to the FYF program to further inform protocol development and enhance adoptability of the intervention.
The two primary aims of this study were to (a) train mental health clinicians outside of our university research program to deliver FYF with fidelity and (b) examine feasibility of the program (e.g. acceptability/satisfaction data, attendance, and treatment completion) to inform further development of the intervention. A secondary aim was to examine preliminary youth anxiety treatment outcomes. It was hypothesized that clinicians in a pediatric hospital setting inexperienced in the FYF intervention would deliver the treatment program to fidelity, following participation in a workshop with ongoing supervision. Additionally, preliminary treatment outcomes were hypothesized to parallel findings obtained from previous FYF treatment trials (Reaven et al., 2009, 2012), although treatment outcomes may not be as robust as in previous trials, consistent with dissemination research (Weisz et al., 2013).
Method
Participants
Two groups of participants entered the study: clinicians and treatment participants (i.e. 8- to 14-year-olds with ASD and clinical anxiety, and at least one parent). Participants were recruited in Halifax, Nova Scotia, through Research Ethics Board (REB)–approved study announcements, clinician referral, and word of mouth. Informed consent and assent was obtained for all participants prior to collecting any data.
Clinicians
A total of 13 clinicians participated in the study. Inclusion criteria were as follows: (a) graduate student in a doctoral program in clinical psychology (Dalhousie University) or a PhD-level clinical psychologist and (b) working in the outpatient service for children with ASD at a regional pediatric hospital (IWK Health Centre in Halifax, Nova Scotia). Just over 40% of the clinicians held a doctorate degree in psychology. On average, the clinicians had 3.6 years (range 1–10 years) of experience implementing CBT therapies, but were relatively inexperienced in implementing CBT with youth with ASD (M = 1.1 years of experience; range = 0–5 years) (see Table 1).
Clinician characteristics.
SD: standard deviation; ASD: autism spectrum disorder; CBT: cognitive-behavioral treatment.
All clinicians were graduate students in a doctoral program in clinical psychology or PhD-level clinical psychologists.
Treatment participants
A total of 16 children and their parents participated in the study. Children were included with the following criteria: (a) chronological age of 8–14 years and living with someone who could give informed consent to participate; (b) diagnosis of ASD, defined by Autism Diagnostic Observation Schedule (ADOS; Lord et al., 1999) score above spectrum cutoff and clinical diagnosis of an ASD as determined by a review of history and current clinical presentation by a clinical psychologist; (c) estimated Verbal IQ of 85 or above, determined through standardized cognitive testing using the Wechsler Abbreviated Scales of Intelligence (WASI; Wechsler, 2002), or an equivalent measure of intelligence administered within the past 2 years; (d) ability to read at a mid-second-grade level, determined by the Letter-Word Identification and Reading Comprehension subtests of the Woodcock–Johnson Achievement Tests–Third Edition (WJ-III; Woodcock et al., 2001); and (e) clinically significant symptoms of anxiety, defined as a score above the clinical significance cutoff on separation (Separation Anxiety Disorder (SEP)), social (Social Anxiety Disorder (SOC)), and/or generalized anxiety (Generalized Anxiety Disorder (GAD)) subscales of the Screen for Child Anxiety and Related Emotional Disorders (SCARED)–Parent Version (Birmaher et al., 1999). These three diagnoses are commonly treated together because of the considerable overlap in the therapeutic approach for these disorders (Barrett et al., 2004; Kendall and Hedtke, 2006).
Children were excluded if (a) their family was not fluent in English; (b) their family could not commit to attending 12 of 14 sessions within the intervention period; (c) they presented with psychosis, severe aggressive behavior, or other severe clinical symptoms that required more intensive treatment such as day treatment or hospitalization; (d) their family intended to seek additional psychosocial treatment for anxiety during the 4-month study period; and (e) they were not able to separate from their parent for a minimum of 30 minutes during the pre-intervention assessment visits. If a child was not eligible to participate in the study, the family was referred to other therapeutic services. (See Table 2 for family characteristics.)
Family characteristics.
SD: Standard deviation.
Setting
All group treatment sessions occurred at the IWK Health Centre in Halifax, Nova Scotia. The IWK serves as the tertiary pediatric health center for the Maritime region of Canada. Virtually all children with ASD in metropolitan Halifax and the surrounding area (population 370,000) are diagnosed at this center. All the trainings, recruitment, qualifying assessments, and treatment occurred on-site at the IWK Health Centre, and the sample of participants was generally representative of the Halifax community.
Procedure
This study was completed in compliance with the Colorado Multiple Institutional Review Board (University of Colorado Anschutz Medical Campus) and the REB of the IWK Health Centre.
FYF intervention
The FYF program is delivered in multi-family groups over 14 weekly sessions, each lasting 1½ hours. Large group time, separate parent and child group meetings, and parent–child dyads comprise the weekly sessions. Three clinicians were assigned to each group cohort to provide coverage when the parent and child groups met simultaneously. Three treatment manuals were used: a parent workbook, a child workbook, and a facilitator manual (Reaven et al., 2011). (The FYF workbooks were not published until after participant feedback was incorporated into revised manuals.)
Core CBT components for the treatment of childhood anxiety disorders are included in FYF (e.g. graded exposure, relaxation, deep breathing, emotion regulation strategies, and use of cognitive self-control; Kendall, 1994; Silverman et al., 2008). Given the social, emotional, communicative, and cognitive challenges of children with ASD, enhancements to the delivery of the core concepts were made, including the provision of written worksheets and multiple-choice lists; an emphasis on creative and varied outlets for expression; and a focus on strengths, talents, and expanding areas of interest. Video modeling is included to support generalization of skills across settings. Because parent participation is integral to this intervention, instructions and activities for parents are interwoven throughout each group session. An in-depth description of FYF including a detailed parent curriculum is available elsewhere (see Reaven et al., 2011).
Aim 1—Training clinicians and assessing treatment fidelity
Eligible clinicians were identified and asked to participate in the study. Consenting clinicians participated in a 2½-day on-site training workshop delivered by two of the study authors (J.A.R. and A.B.-S.). Clinicians completed an assessment of CBT knowledge both pre- and post-workshop and then facilitated the 14-week FYF intervention program within 2 months of the workshops. Four treatment groups were completed over the course of the study. Clinicians received twice monthly phone consultation from the treatment developers (J.A.R. and A.B.-S.) throughout the duration of the intervention. All treatment sessions were videotaped and adherence to the treatment was coded by the FYF developers.
Training workshop
All participating clinicians were required to attend the training workshop. The workshop was comprised of a series of didactic presentations, small group activities, videotaped examples, role-playing exercises, and session-by-session review of the FYF intervention. Several clinicians joined the project mid-way through the study (after two treatment groups had been completed). They were asked to view a videotape of the original workshop and attend a day-long teleconference reviewing the critical elements of FYF. All clinicians completed a workshop evaluation questionnaire after participating in the training.
Phone consultation
Bi-monthly, hour-long phone consultation was provided by two of the authors (J.A.R. and A.B.-S.) during each cohort for the duration of the study. All FYF group sessions were digitally video-recorded and sent as encrypted files to the research site in Colorado. Sessions were viewed prior to phone consultation. The consultation format was consistent across the project and included the following components: (a) question-and-answer period regarding the FYF sessions that occurred since the last phone consultation; (b) feedback regarding strengths of facilitation, missing elements, and suggestions for delivery of session content; and (c) plan for the upcoming FYF sessions.
Aim 2—Examining feasibility
Acceptability measures were collected from all participants (i.e. parents, children, and clinicians) following each session. Likert ratings were completed for each core activity, and participants were also asked to provide narrative comments about the helpfulness (or lack of helpfulness) of each activity.
Participant attendance was also recorded at each session.
Secondary aim—examining preliminary youth anxiety treatment outcome
Interested families contacted the IWK Health Centre and a research assistant explained the goals and procedures of the study. If the parent was interested, a research assistant conducted a brief telephone screen for eligibility. If the child was a potentially eligible participant, the family was invited to the clinic, where they completed the ASD diagnostic measures, standardized cognitive testing (if unavailable either through previous research visits or clinical testing), and the SCARED. If a child met diagnostic criteria for an ASD, cognitive criteria and obtained scores above the clinical significance cutoff on SEP, SOC, and/or GAD subscales of the SCARED, the parent was invited to complete the Anxiety Disorders Interview Schedule for Children–Parent Version (ADIS-P; Silverman and Albano, 1996). Parents completed the ADIS-P with a trained clinician (“clinician examiner” (CE), a PhD-level psychologist), who provided clinical severity ratings (CSRs) across a range of psychiatric disorders of childhood. The CE reviewed the parent’s report of the child’s psychiatric symptoms and determined which Axis I diagnosis was “primary” (i.e. yielded the highest CSR on the ADIS-P) and which diagnoses were “secondary.”
Parents were asked to report the child’s medications and other treatments received prior to and throughout the study period. Children currently taking medications were eligible for the study; however, they were required to be on a stable dose of psychiatric medication (e.g. at least 1 month at the same dosage prior to the qualifying assessment), and maintain the same dosage of medications over the course of the study. Medication usage was tracked throughout the study.
Within 2–6 weeks of the last treatment session, the ADIS-P was readministered to the parent. The CE did not facilitate the FYF group treatment at any point, nor receive information about children’s progress in therapy.
Measures
Aim 1—Training clinicians and assessing treatment fidelity
Clinician measures
Demographic questionnaire
A brief questionnaire was developed to track clinicians’ years of experience in the field of psychology, specific training with CBT, and experience working with children with ASD.
Assessment of CBT knowledge
A 20-item multiple-choice test similar to assessments used in other studies (Rees and Gillam, 2001) was developed to evaluate clinicians’ knowledge of CBT. Three similar but nonidentical versions of the assessment were developed and were administered three times: (a) prior to, (b) following the training workshop, and (c) following the completion of the first group cohort.
Treatment fidelity measure
A treatment fidelity measure was developed to assess the facilitators’ adherence to protocol. The measure utilized a checklist format to assess the presence or absence of core treatment components on a session-by-session basis. In addition, a global rating of quality was assigned at the end of every modality for every session (e.g. parent group, child group, parent/child dyads, and large group activity). Ratings were based on a Likert scale ranging from poor (1), to adequate (3), to excellent (5) quality. The quality rating was intended to be a global score encompassing therapeutic alliance, therapist expertise, and competence in the delivery of core treatment content for each modality (adapted from the Yale Adherence and Competence Scale (YACSII) Guidelines; Nuro et al., 2005). All treatment sessions were video-recorded. Two of the authors (J.A.R. and A.B.-S.) viewed the first group cohort together and agreed on parameters for scoring absence or presence of core components, as well as for assigning quality ratings. For the remaining three cohorts, each rater viewed the sessions separately to rate the core components and quality of each treatment session. Because phone consultations occurred on a bi-monthly basis, adherence to protocol was not assessed as an independent entity, but rather as a part of an ongoing feedback loop (e.g. constructive comments from the consultants were provided to the clinicians, clinicians conducted a group session, consultants viewed the session, and again provided feedback, etc.).
Aim 2—Examining feasibility
All participants
Acceptability measures
All participants provided weekly written feedback regarding the FYF treatment, including ease of administration (clinicians only), content relevance, and helpfulness of the activities. A 1–5 Likert scale was used to rate each item (children used a 5-point pictorial scale to rate the helpfulness of each session’s activities). Participants also provided written comments regarding the most and least helpful aspects of each session. These written comments were reviewed and grouped thematically to aid interpretation. Acceptability data were collected to inform both the training workshop as well as the FYF intervention.
Group attendance and treatment completion data were obtained.
Secondary aim: examining preliminary youth treatment outcome
Child eligibility measures
Diagnosis of ASD
Diagnostic status was determined by expert clinical review of the ADOS (Lord et al., 1999) and other clinical information documented using a symptom checklist based on the Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition, Text Revision (DSM-IV-TR, American Psychiatric Association, 2000). The ADOS is a semi-structured, play-based direct assessment of social and communicative behaviors indicative of ASD. It is considered the “gold standard” for diagnosing ASD (Lord et al., 1994). Clinicians administering the ADOS were trained to 80% reliability.
Screening for cognitive functioning and reading abilities
All potential participants lacking standardized cognitive assessment within the past 2 years were administered the WASI (Wechsler, 2002) and the reading subtests from the WJ-III (Woodcock et al., 2001).
Screening for anxiety symptoms
The SCARED–Parent version (Birmaher et al., 1999) is a 41-item inventory comprising five anxiety subscales (Panic, GAD, SEP, SOC, and School Anxiety) and a total score. Parents reported on symptoms over the past month. The SCARED demonstrates excellent psychometric properties in typically developing youth populations (Birmaher et al., 1997, 1999; Hale et al., 2011). In youth with ASD, the extended 71-item version of the SCARED demonstrates strong internal consistency (α > 0.9) and moderate convergent validity with the ADIS (van Steensel et al., 2012). Results from another study confirm the 41-item measure’s five-factor structure and suggest good sensitivity (.71) and specificity (.67) among parents of youth with ASD (Gadgil et al., submitted).
Child outcome measures
ADIS-P
The ADIS-P was the primary outcome measure and was administered pre- and post-intervention. While the ADIS-P and C (parent and child versions) are both typically administered for general pediatric populations (Walkup et al., 2008), the additive value of the ADIS-C in the ASD population has been questioned. Youth with ASD tend to under-report anxiety symptoms and, as such, clinicians tend to rely more upon parent report of symptoms on the ADIS than on child report (Storch et al., 2013). Thus, only the parent version was used in this study. The ADIS-P is a semi-structured psychiatric interview that assesses the presence of anxiety disorders as well as other co-occurring mental health conditions. The ADIS demonstrates strong psychometric properties, including good test–retest reliability for diagnoses and symptom scales (Silverman et al., 2001), as well as convergent validity with other anxiety measures (Lyneham and Rapee, 2005; Wood et al., 2002). The CE conducted the ADIS-P with the parent pre- and post-intervention, determined diagnostic classifications across DSM-IV categories, and provided summary codes of severity and interference, called “Clinician Severity Ratings” (CSRs: 0 = no symptoms, 8 = very severe impairment).
ADIS-P administration training (of I.M.S. by A.B.-S.) involved scoring above 80% reliability on clinical diagnoses and CSRs on three videotaped administrations of the ADIS-P and three live administrations (serving as the interviewer). The Clinical Global Impressions Scale–Improvement (CGIS-I) ratings (National Institute of Mental Health, 1970) were compiled by the CE after the participants had completed all study activities. These ratings were derived in a manner similar to the methods described in previous studies (Walkup et al., 2008; Wood et al., 2009). The CE reviewed the ADIS-P and the SCARED data obtained pre- and posttreatment, and assigned a rating of 1–7 concerning the overall impression of improvement (1 = “very much improved”; 4 = “no change”; 7 = “very much worse”). Children who obtained a CGIS-I of 1 or 2 were considered to be positive treatment responders. Reliability data on these diagnoses and CGIS-I ratings were collected from the University of Colorado.
Analysis plan
Aim 1
The effectiveness of the clinician training and their subsequent ability to deliver the intervention was assessed in several ways. First, descriptive statistics were used to assess participant satisfaction with the training. Second, the trainees’ knowledge of CBT was compared before and after training with paired-sample t-tests, and again with a follow-up test several months later to check for loss of knowledge. Finally, fidelity of implementation was assessed in three ways: (a) by calculating the percentage of the core treatment components that were adhered to across all sessions, (b) by calculating the inter-rater reliability for treatment fidelity over the last three group cohorts using intraclass correlations (ICC), and (c) by calculating descriptive statistics of the global quality rating for each treatment modality.
Aim 2
Feasibility of the FYF program was assessed by calculating descriptive statistics of the acceptability ratings from parents, children, and clinicians, as well as by reviewing comments by the same individuals. Attendance rates were also calculated from all participants.
Secondary aim
Although treatment outcomes are not the primary focus of this study, they are nonetheless important to assess. We first characterized the sample of participants by determining their demographics, medication status, complexity of their condition, and their primary anxiety diagnosis. The participants’ anxiety was assessed prior to treatment and after completion of the FYF program, and these measurements were the basis for determining their outcomes. Paired-sample t-tests were calculated to compare their baseline anxiety with their posttreatment anxiety. Note that while multiple comparisons were calculated, we did not adjust alpha for inflation of Type I error. Given that this study was primarily about feasibility rather than outcomes and we are not trying to make strong claims about the effectiveness of the FYF program, all alphas were set at .05 (O’Keefe, 2003; Perneger, 1998; Tutzauer, 2003). As such, some caution is warranted while interpreting these findings.
Results
Aim 1—Training clinicians and assessing treatment fidelity
Clinician outcomes
Evaluation of training workshop
Clinicians responded to a series of statements (e.g. “I would recommend this training to a colleague”) using a 6-point scale (1 = “strongly disagree,” 6 = “strongly agree”). Overall, clinicians expressed high satisfaction with the training provided (M = 5.34, standard deviation (SD) = 0.17, range = 5.18–5.55).
Assessment of CBT knowledge
Clinicians demonstrated significant improvements on the assessment of CBT knowledge after attending the training. Results of a paired-sample t-test indicated significant improvements in clinicians’ CBT knowledge from pre-training (M = 81% correct, SD = 7%, range = 70%–90%) to post-training (M = 90% correct, SD = 7%, range = 75%–95%), t(9) = 2.41, p = .039. These improvements were maintained at follow-up, after clinicians had facilitated treatment groups (M = 92% correct, SD = 4%, range = 85%–95%), with no significant difference between post and follow-up scores, t(4) = 0.74, p = .501.
Treatment fidelity
Treatment fidelity was calculated based on the percentage of core components that the facilitators implemented across the entire 14-week program. Treatment fidelity percentages ranged from 87% to 95% (M = 92%) across all four group cohorts (Group 1—94%; Group 2—95%; Group 3—87%; and Group 4—92%). All groups exceeded the minimum standard for acceptable treatment fidelity (80%). Inter-rater reliability for treatment fidelity in the last three group cohorts was excellent, ranging from 93% to 96% (ICC = .99, p = .005).
In addition to collecting data regarding absence/presence of core components, global ratings of quality for each modality were obtained. Average quality ratings (Likert 1–5) for each modality were as follows: large group (4.75), parent group (4.60), child group (4.54), and parent/child dyadic work (4.55). Ratings were similar across modalities and across all group cohorts, indicating that the clinicians delivered the FYF intervention with consistently high quality. Inter-rater reliability of quality was defined as agreement within 1 point on a 5-point Likert Scale. Percent agreement between the two independent raters across the last three cohorts was excellent and ranged from 88% to 100%.
Aim 2—Feasibility
Acceptability of the FYF intervention
Acceptability was high across all four treatment groups (M = 4.15, SD = .50), although ratings dipped slightly for Group 2 (M = 3.87, SD = .64). Acceptability ratings were highest among parents (M = 4.35, SD = .33) and clinicians who facilitated the parent group (M = 4.33, SD = .36). Positive ratings were also given by clinicians who facilitated the child group (M = 4.08, SD = .37) and by the children themselves (M = 3.84, SD = .66) (see Table 3 for session-by-session acceptability ratings).
Average acceptability ratings from children, parents, and clinicians for each FYF session.
FYF: Facing Your Fears: Group Therapy for Managing Anxiety in Children with High-Functioning Autism Spectrum Disorders
ns indicate the range of the number of participants who completed acceptability forms at the end of each session. For most sessions, acceptability forms were completed by four children, five parents, and three clinicians.
Narrative comments were reviewed and incorporated into an updated version of the treatment manual. Revisions occurred at several points during the study, with the majority of revisions occurring after the completion of the first two cohorts. It should be noted that the revisions did not entail substantial changes in core content, but rather were primarily changes in delivery of the material. For example, based on feedback from clinicians, instructions were modified to clarify certain activities; additional worksheets and “cheat sheets” were generated to augment existing materials; videos and worksheets were updated to be more developmentally appropriate; instructional caveats were included in the facilitator manual to accommodate specific child needs; the word “autism” was removed from the title of the parent and child manuals to minimize potential stress around this label; and greater flexibility was integrated into the treatment schedule so that sessions felt less “rushed.”
Positive comments that parents, children, and clinicians made were also documented. For example, parents responded with the following comments in response when asked what was most helpful: “seeing the anxieties that he recognizes in himself”; “reminders that talking about worries helps”; “doing the actual fear facing and seeing him go through it till he was calm”; and “great course, skill building with lots of everyday uses … was very glad to take it, my child definitely learned some skills.” Child participants made positive comments as well and these included “I liked how a peer expressed an opinion that was different from mine but he did it respectfully”; “I really liked the worry bug activity, it was funny!”; and “very fun! I faced my fear and finished the movie.” Parents tended to find exploration and discussion-related activities most helpful, while children found the video activities to be most helpful; clinicians reported that discussion and skill-building activities were most helpful.
Attendance
A total of 16 children (and their parents) participated in the FYF program. One parent–child pair dropped out early in the treatment process, citing the lack of fit between the child’s needs and the treatment program. The completion rate was 94% and no child missed more than three sessions. When a family was absent, they were asked to arrive early the following week to review missed content.
Secondary aim: examining preliminary youth treatment outcome
Medications
The majority of child participants were taking medications prior to the intervention (69%; 11/16 participants). Families were asked to report any changes in medications during the study. Of the 11 participants, 8 made no changes in their medications. Of the three participants who made changes, one participant was taken off medication altogether, a second experienced a decrease in dosage, and a third added a new medication for Attention Deficit Hyperactivity Disorder (ADHD). Data were included for these three participants in the final analysis because none of the changes involved an increase in medication targeting anxiety symptoms.
Psychiatric complexity of child participants
Results from the ADIS-P administered pre-intervention indicated that the number of diagnoses (in addition to ASD) ranged from 2 to 7 (M = 4.0; SD = 1.15). Overall, 15 children (93.7%) met criteria for more than one anxiety diagnosis, 12 met criteria for ADHD (75.0%), and 1 (6.3%) met criteria for Oppositional Defiant Disorder (ODD). None met criteria for a co-occurring mood disorder (dysthymia/major depression). The sample appears to be comparable to other treatment studies of anxiety in ASD (Storch et al., 2013). Data collected at baseline indicated that six children (40%) were receiving some form of treatment for anxiety prior to entering the study.
A primary anxiety diagnosis was determined for each of the participants by examining the diagnosis with the highest clinician-assigned CSR. GAD was the most common primary diagnosis (56.3%), followed by SOC (25.0%) and SEP (18.8%). When two diagnoses had the same CSR, primary diagnosis was determined by having the parent designate the most concerning set of symptoms.
Treatment outcome
Intent-to-treat analyses were conducted for two outcome variables (n = 16): CSRs generated for the three principal anxiety diagnoses and diagnostic status of the principal anxiety diagnoses per ADIS-P criteria. One participant did not complete treatment, and the last observation was carried forward for these analyses.
CSRs
Paired-sample t-tests were used to examine pre–post differences in anxiety severity and interference. Significant decreases in severity were found for the three primary anxiety diagnoses post-intervention: SOC, t(15) = 2.43, p = .03, d = .61; SEP, t(15) = 2.13, p = .05, d = .53; and GAD, t(15) = 3.03, p = .008, d = .76.
Diagnostic status of anxiety disorders
Paired-sample t-tests were used to assess differences in the number of anxiety disorders post-intervention. Child participants demonstrated a significant reduction in overall number of anxiety disorders, t(15) = 2.78, p = .014, d = .70, post-intervention. In all, 40% of participants lost their principal anxiety diagnosis after the completion of the intervention program.
CGIS-I Improvement ratings
CGIS-I scores of 1 or 2 reflected “a clinically meaningful improvement in anxiety severity,” consistent with previous studies (Reaven et al., 2012; Wood et al., 2009). Of the 15 youth that completed the study, over half (8/15) met criteria for a clinically meaningful outcome (53.3%), while 20% (3/15) somewhat improved, and 26.7% (4/15) experienced no change in severity of symptoms, post-intervention. None of the children experienced a worsening of symptoms.
Discussion
The main purpose of the study was to train clinicians new to FYF to deliver the program to fidelity and to assess the feasibility of the intervention for clinical practice. A secondary purpose of the study was to examine preliminary youth anxiety treatment outcome. Clinicians demonstrated significant gains in CBT knowledge following a training workshop and these gains were maintained after completion of the first treatment cohort. Examination of adherence to protocol indicated that clinicians did deliver the program as intended and with high quality across all four group cohorts.
Group attendance and treatment completion rates were high. In addition, acceptability was high across all participant groups and group cohorts. In the 2 years following the end of the research project, the study authors remained in contact with the clinician facilitators. The group facilitators were asked whether they were continuing to deliver FYF, and if so, how many youth had participated following the conclusion of the study. The facilitators stated that they had continued to deliver FYF, and had treated an additional 26 youth with ASD and anxiety. These results may reflect the potential sustainability of FYF and suggest that “knowledge transfer” may have occurred (Beidas et al., 2011). Treatment fidelity was not assessed for these youth, so the extent to which they delivered FYF to fidelity is unknown, thus representing an area for further research.
Preliminary youth anxiety outcome was examined via (a) change in anxiety severity and interference related to principal anxiety diagnoses, (b) change in diagnostic status of principal diagnoses; and (c) global ratings of improvement in anxiety. Youth participants evidenced significant reductions in anxiety symptoms after participating in the FYF intervention. Significant decreases in severity were apparent for all three primary anxiety diagnoses (GAD, SOC, and SEP). Furthermore, youth participants had fewer anxiety diagnoses after completing the FYF program, and 40% of participants no longer met diagnostic criteria for their primary anxiety diagnosis posttreatment. Finally, over half of youth participants met criteria for a clinically meaningful improvement in anxiety symptoms following the intervention and 20% demonstrated modest improvements. These preliminary results are highly consistent with the results obtained from previous treatment trials (Reaven et al., 2012).
Overall, these results may signal the positive potential for dissemination and implementation of the FYF treatment for youth with ASD and anxiety in new clinical settings. The combination of an interactive training workshop paired with ongoing phone consultation was sufficient for clinicians to demonstrate excellent quality and adherence to the treatment protocol. Furthermore, the high acceptability ratings, excellent attendance, and low drop-out rate, in addition to the positive, although preliminary, youth outcomes, suggest that the FYF program may be a feasible and effective treatment for reducing anxiety symptoms in youth with ASD and anxiety in clinical practice settings.
Perhaps the ease with which clinicians in a pediatric hospital setting who were unfamiliar with FYF were able to learn the intervention and to deliver it to fidelity may be partly due, in part, to the user-friendly qualities of the treatment manual (Reaven et al., 2011). For example, detailed descriptions of specific activities, video examples of core concepts, a “helpful hints” section highlighting potential problems and solutions, and in-session worksheets likely strengthen the usability of the manual (Najavits et al., 2000). Obtaining specific feedback from the group facilitators as well as youth and family participants regarding the treatment program, and incorporating this feedback directly into a revised manual, may further enhance the accessibility of the FYF program for providers in clinical practice (Beidas et al., 2011). The integration of acceptability data into program development reflects a bidirectional approach to treatment implementation and dissemination, in which critical insights from practitioners and participants can support the efficacy and long-term sustainability of new programs (Hatgis et al., 2001). This may be a critical step in bridging the research-to-practice gap (Dingfelder and Mandell, 2011).
Limitations and future directions
The small sample and the quasi-experimental design are notable limitations to the current study. The extent to which the improved outcomes were a placebo effect related to involvement in FYF is unknown and speaks to the importance of conducting more rigorously designed studies. Furthermore, the sample was primarily Caucasian; although the racial backgrounds of the sample were generally representative of the geographic area, this does limit the generalizability of the findings to other populations. Additionally, because the primary anxiety diagnoses targeted by FYF are SEP, SOC, and GAD, the extent to which the results would generalize to youth presenting with other primary diagnoses such as Obsessive–Compulsive Disorder is unknown.
The assessment of anxiety symptoms in this sample was determined through the SCARED (Birmaher et al., 1999) and then through the ADIS-P (Silverman and Albano, 1996). It is important to note that neither instrument was specifically designed for youth with ASD. Although there is support for the co-occurrence of anxiety disorders in youth with ASD, agreement about how to diagnose anxiety symptoms in individuals with ASD must occur. The psychometric validation of anxiety measures originally developed for the typical population for individuals with ASD is an important next step (Kerns and Kendall, 2012).
Clinician participants varied in their years of experience with CBT but were all working in a pediatric hospital setting for youth with ASD (although several clinicians were graduate students enrolled in a university PhD program). The extent to which clinicians new to ASD (e.g. community mental health center clinicians) would obtain similar fidelity ratings is unknown and represents a direction for future research. Furthermore, the FYF program requires a minimum of three clinician facilitators to deliver the group treatment. Although there are a number of advantages to group therapies (e.g. management of long waiting lists, normalization of experiences, and decreased isolation for participants; Reaven et al., 2009), not all clinical settings have the capacity to provide three facilitators to run groups, representing a potential limitation in the feasibility of offering group therapy in some environments. In addition, although all group cohorts met the threshold for excellent treatment fidelity, the extent to which good treatment fidelity is necessary for positive treatment outcome is unknown, and could represent an additional area for further research.
Finally, future research may also include the systematic comparison of clinician training methods for evidence-based interventions for youth with ASD. For example, the use of modular trainings rather than a session-by-session approach may better equip some clinicians. Similarly, the extent to which ongoing consultation for clinicians inexperienced with a particular treatment program is necessary to achieve positive treatment outcomes for a new intervention needs to be determined. The results of additional research may provide much-needed guidelines for training of community practitioners stationed at the front lines of autism treatment and may enhance dissemination of treatment programs for youth with ASD.
Footnotes
Acknowledgements
The authors gratefully acknowledge the assistance of Mary Hetrick, Angela Turner, and Claire Smith, as well as the families who participated in this study. Additional support came in part from core grants awarded to JFK Partners, the University Center for Excellence in Developmental Disabilities at the University of Colorado Anschutz Medical Campus from the U.S. Department of Health and Human Services, through the Administration on Developmental Disabilities, Grant #90DD0561. J.R., A.B.-S., and S.H. receive royalties from the Facing Your Fears Program, currently published by Paul Brookes Publishing (Reaven et al., 2011).
Funding
This research received funding from NIH - 1R21MH089291-01.