Abstract
Given the wide heterogeneity in the autism population, one challenge for intervention studies is to identify outcome measures that have similar meaning across individuals. This is particularly pronounced in intervention studies of adults with autism spectrum disorder, where outcomes such as employment, independence, or community living are common targets. However, these outcomes can be more challenging to obtain for those who have greater support needs, for example, and therefore may be more or less salient depending on the sample under study. Goal attainment scaling can help address this issue as it allows individuals to identify personally meaningful goals and track progress toward these goals. While goal attainment scaling is gaining popularity in the autism field as an outcome measure, most intervention studies that use goal attainment scaling have been situated in clinical or school settings. Generating reliable and scalable goals outside of these settings can be challenging. In this article, we describe the promise of goal attainment scaling for tailoring individualized outcomes among youth and adults with autism spectrum disorder and discuss the challenges of current goal attainment scaling methods in community-based intervention research. We then describe a new goal attainment scaling approach that is rigorous, practical, and can be used across research settings to measure individualized outcomes.
Lay abstract
Among people with autism—all who have the same diagnosis—there are major differences on a nearly limitless number of areas, such as language, daily living skills, intellectual ability, sensory difficulties, and physical and mental health diagnoses. Despite these many differences, the targeted outcomes of intervention studies are often measured the same way across autistic adults, including outcomes such as getting a job, achieving greater independence, or getting more services. People have different goals and abilities, and it is important to have a way for intervention studies to measure outcomes in a way that is more personal to each individual. To address this issue, we developed a new approach—called “Goal Attainment Scaling—Community-based” or GAS-CB—to measure individualized outcomes across different research settings. In this article, we describe the need for individualized outcomes in autism intervention research and current approaches to gathering these outcomes, with our discussion focused on a method called goal attainment scaling. We then describe reasons why current goal attainment scaling approaches might not be useful in intervention research that takes place in the community. Finally, we discuss a new goal attainment scaling approach (GAS-CB) that can be flexibly used for research participants with very different characteristics.
Keywords
Over the past 10 years, the challenges and needs of adults with autism spectrum disorder (ASD) have received increasing attention from researchers, practitioners, and policymakers (Interagency Autism Coordinating Committee [IACC], 2017). Studies have shown that many autistic adults struggle and do not receive needed supports across multiple areas of life including postsecondary education, employment, health care access, social life, and community living (Howlin & Magiati, 2017; Roux et al., 2015; Wehman et al., 2014). In response, a number of interventions and programs have been developed to improve life outcomes for youth and adults with ASD (e.g. Baker-Ericzén et al., 2018; DaWalt et al., 2018; Hedley et al., 2017; Laugeson et al., 2015; Ruble et al., 2018; Smith et al., 2014; Taylor et al., 2017; Wehman et al., 2013; White et al., 2017).
One challenge across these intervention studies is how to account for the heterogeneity of ASD presentation in outcome measurement (Masi et al., 2017; Taylor, 2017). Among individuals with ASD, there is wide variability on a nearly limitless number of characteristics, including intellectual functioning, adaptive skills, language skills, co-occurring physical and mental health conditions, and sensory challenges/abilities (Lord et al., 2020). A common way to handle this diversity in intervention studies is by restricting eligibility criteria to include participants, for example, who are verbal (vs non-verbal) or whose intelligence quotient (IQ) scores exceed a pre-determined threshold. Often, there is a strong rationale for restricting eligibility, such as when an intervention is targeted toward a subsample of the autism population (e.g. those without an intellectual disability) or a specific co-occurring condition (e.g. anxiety). However, there are also times when an intervention may be applicable to individuals with varying strengths and support needs. In studies with broad eligibility criteria, choosing an outcome that is meaningful and obtainable for all participants is critical for understanding the efficacy of the intervention.
This issue is particularly relevant for intervention studies focused on adults with ASD. In many of these studies, outcomes are defined functionally by measuring changes in domains such as employment, level of independence, or service access (for a review, see Hedley et al., 2017). However, given the substantial heterogeneity among adults with ASD, functional or service-related outcomes vary greatly within this group. Outcomes such as employment or level of independence may be more difficult to obtain for individuals with ASD who have an intellectual disability (ID), as they often require more intense supports to achieve the same outcome as individuals with ASD without ID. Furthermore, measures of service access are influenced by availability of services, which differ based on place of residence and an individual’s support needs (Burke et al., 2019). It is also unclear to what extent functional and service-related outcomes reflect the priorities, values, and dreams of the individuals with ASD. How can researchers consider different abilities, contexts, and priorities of individuals on the autism spectrum while still measuring outcomes in a rigorous and consistent way?
One promising approach to address many of these issues is goal attainment scaling (GAS). GAS is a hybrid of standardized and personalized approaches to measure outcomes, and can be relevant to diverse groups by centering outcome definition and assessment on each individual’s goals and perspectives (Kiresuk & Sherman, 1968; Schlosser, 2004). It provides a framework for both acknowledging a person’s unique needs and abilities and quantifying personalized outcomes in a standardized way so that researchers can compare one person’s outcomes to others. Although GAS was first developed to measure goal attainment in community mental health interventions (Kiresuk & Sherman, 1968), it has since been used in a variety of settings including psychiatric in-patient clinics and school-based research.
The GAS process includes three steps: (1) identifying individualized goals, (2) developing a scaling rubric to define possible outcomes for each goal, and (3) rating goal attainment at a later point. Personalized goals can be developed by professionals (e.g. clinicians and teachers) who know the individuals well, by the individuals themselves, and/or by their families. The identified goals need to be of sufficient quality (i.e. Specific, Measurable, Attainable, Relevant, and Times-specific or SMART; Doran, 1981) so that they can be effectively scaled in the second step. Scaling rubrics, created for each goal, reflect a range of outcomes related to that goal, from the least to the most favorable outcome. These GAS rubrics allows for varying goals to be standardized to the same metric, facilitating comparisons across goals and individuals. The most common way to scale goals is on a 5-point scale from −2 to 2 (Kiresuk & Sherman, 1968). Using this scale, −2 indicates much less than expected attainment, −1 indicates baseline performance (and somewhat less than expected attainment), 0 indicates meeting the goal (i.e. expected level of outcome), and 1 and 2 both indicate exceeding the goal.
After a pre-determined time has passed, goal attainment is rated based on the created GAS rubric. Attainment data can be gathered via direct observation (e.g. video demonstration or work samples) of specific behaviors or via self-, parent-, or professional-report. Although the three steps of GAS are the same across studies, the process can look very different depending on who sets the goals, who scales the goals, who rates attainment and how, and the ways to ensure reliability and validity (for further detail, see Ruble et al., 2012; Shogren et al., 2021).
GAS applications in autism intervention research
Because of GAS’s ability to generate personalized goals and rate attainment on a standardized metric, there has been increasing interest in using GAS as an outcome measure in autism intervention studies. Three approaches have been most used: drawing from goals developed through school programming (e.g. Ruble et al., 2018); parent- and professional-driven goals and scaling (e.g. Duncan et al., 2018); and teaching youth to set, scale, and monitor their own goals (e.g. Shogren et al., 2012). Information regarding these different GAS applications for ASD intervention studies is detailed below and presented in Figure 1.
Previous GAS application in ASD intervention research and the new GAS-CB approach.
One of the more common approaches to GAS in autism research is drawing from goals developed by teachers for measuring outcomes of Individualized Education Programs (IEPs; Ruble et al., 2018, 2019). Teachers set the IEP annual goals as they typically do in the process of educational programming, and then they scale these goals to rate attainment based on the parameters of a project and the focus of evaluation. Aligned with IEP progress, GAS rubrics are often developed by teachers with supervision from the research team. As teachers are required to develop high-quality goals and scaling in this application of GAS, ongoing trainings and coaching are offered from researchers. After a set time, goal attainment is assessed by reviewing video demonstrations, work samples, and/or data collected by teachers.
This is a promising method for interventions in which youth are still in school (such as the Collaborative Model for Promoting Competence and Success; Ruble et al., 2018). Since the goal setting and attainment processes are already part of the students’ IEPs, GAS can be easily incorporated into school-based interventions (Oren & Ogletree, 2000). Variations of this approach can also be used for adults who have left the school setting (and do not have available IEP goals to scale) by having a trained facilitator conduct a thorough ecological assessment or person-centered planning process (e.g. PATH or MAP) to identify and develop personalized goals. However, the process of performing a lengthy (3–4 h) person-centered planning meeting with each participant can be prohibitive for many studies interested in GAS as an outcome measure. Furthermore, incorporating person-centered planning could change the intervention and its effects, making it challenging to parse out what outcomes changed due to person-centered planning, the intervention, or the combination of both (Shogren et al., 2021).
A second approach to GAS relies on parents (with the guidance of a research team) or trained community professionals to set and scale goals, with attainment data collected via parent-report (see Duncan et al., 2018 for application in the Surviving and Thriving in the Real World intervention; also Pfeiffer et al., 2011; Siu et al., 2019). Typically, in this approach, participants (e.g. parents) complete a clinical interview with a standard series of questions to generate goals, and then the GAS rubrics are developed by the trained research team. To ensure high-quality goals and scaling, goal setting and goal attainment assessments are conducted by blind researchers who are trained in clinical observation and the use of GAS. This approach addresses some of the potential challenges related to utility of the school-based method, as it can more easily be used outside of settings where goals are already collected. However, a disadvantage is that the goals may not be truly person-centered as the individuals with ASD themselves are not involved in the process.
The third approach focuses on youth with ASD setting their own goals, modifying them as needed, and monitoring progress themselves with support from teachers or social workers (see Shogren et al., 2012 for use in the Self-Determined Model of Instruction; also Levin & Kramer, 2015). In this approach, training sessions on goal setting and goal scaling are offered to youth, typically as part of an intervention. Then, with the support of teachers, the youth develop scaling rubrics for their goals based on a GAS template (which often includes the traditional scaling points of −2 to +2). In this approach, instead of professionals or parents setting and scaling the goals, the resulting goals are truly person-centered and reflective of the youth’s own opinions and ideas. Goal attainment is commonly rated based on both teacher and youth reports.
On its face, this method would seem to be a promising application of GAS to intervention studies with autistic adults; training in goal setting and monitoring could happen either within or independent of existing structures such as schools, and the generated goals are truly personalized. However, a major limitation is that extensive training for the individual is often necessary to develop goals that are sufficiently rigorous to support reliable and valid scaling, and to learn to track their own goal progress (Shogren et al., 2012). This training may not be feasible for all studies. Furthermore, this method can change—rather than simply measure—the impact of an intervention. Studies that generate goals this way are essentially testing the effect of the targeted intervention plus the training to set and monitor goals, without being able to determine if the intervention’s effects will endure when the goal training component is removed (Shogren et al., 2021). Thus, for intervention studies in which GAS is purely an outcome variable and not a core component of the program, teaching youth to set and monitor their own goals may not be recommended.
Need for a new GAS framework
Although these existing GAS approaches have proven useful in autism research, there are aspects of these approaches that may pose difficulties for implementation in community-based intervention studies. In particular, developing goals through youth’s collaboration with teachers, person-centered planning with a trained facilitator, or by the youth themselves (with training)—although likely to generate high-quality, highly personalized goals—may be challenging or even prohibitive for studies interested in GAS as an outcome variable. Thus, there is need for a GAS approach that (1) does not require extensive GAS preparation and training for participants to generate goals and (2) integrates input from the individuals with ASD themselves. To address this gap, we developed a new GAS framework for setting, scaling, and rating goals that is maximally flexible, and allows for personalized goals while standardizing ratings of attainment. This framework, called GAS—Community-Based (GAS-CB), has the potential to be useful as an add-on to many studies that aim to measure individualized outcomes in community-based research across heterogeneous groups.
GAS-CB
In this section, we discuss GAS-CB, a new framework for implementing the three “steps” of GAS (goal setting, scaling, and attainment rating) with individuals with ASD and families during community-based interventions. We describe the guidelines for implementing each step, while highlighting the challenges of existing GAS frameworks when engaging in community-based research, using a parent advocacy intervention as an example.
It is important to note that in all applications in which GAS is used as an outcome measure in research—including in GAS-CB—it is critical for goals to be psychometrically valid (Ruble et al., 2012; Shogren et al., 2021). After describing the GAS steps, we discuss procedures used in our research to ensure the validity and reliability of the goals, scaling, and attainment ratings.
Development of the GAS-CB approach
GAS-CB was developed in the context of evaluating ASSIST, a 12-week parent training program that teaches parents about the adult service system and how to most effectively advocate for those services on behalf of their son or daughter with ASD (Taylor et al., 2017; Taylor et al., in press). Although this program targets parents, the ultimate goal is to improve the transition to adulthood for youth with ASD by increasing service access and community participation (e.g. employment, postsecondary education, and community living). In addition to these “objective” outcomes, we were interested in measuring whether parents were using information from the training in a way that aligns with their youth’s plans and desires. Although GAS seemed like a logical tool to measure these more “personalized” outcomes, we encountered several challenges in applying existing GAS approaches to this study, which prompted the development of the GAS-CB framework.
Step one: goal setting
Challenges
The first challenge was determining how to collect person-centered goals without extensively training youth or conducting a lengthy person-centered meeting with the youth and their family. The ASSIST study includes families of youth who are in high school and those who have already left school, so using IEP goals was not feasible as there would not be available goals for the whole sample. We considered relying on the youth with ASD and/or their families to develop goals; however, as previously mentioned, this often requires an additional intervention to teach them how to set measurable goals (Levin & Kramer, 2015; Shogren et al., 2012). Trying to gather personalized goals directly from youth or families without teaching goal setting skills runs the risk of creating goals that are not sufficiently rigorous (i.e. SMART) to support reliable and valid scaling, rendering them unusable (Shogren et al., 2021). Incorporating such GAS training into our intervention design was beyond the scope of the study, and it would have changed the focus and impact of the intervention. Similarly, we determined that conducting person-centered planning using well-validated methods (e.g. PATH or MAP) to generate goals was not feasible given the burden of an additional 3-h research activity on families’ time, which could also impact the intervention by adding planning/assessment to the process.
Given that we would be developing goals “from scratch” (and not relying on pre-written goals, such as from IEPs) without training research participants, a related concern was how our protocol could generate personalized goals without overwhelming participants with different possibilities. When goals are set within the school system, goal domains are typically targeted to specific outcomes based on the curriculum or priorities identified by an IEP team. However, community-based interventions, such as ASSIST, may include a broader array of transition outcomes (e.g. employment, postsecondary education, community living, leisure, and social activities) which take place in diverse settings. Given that youth with ASD have a wide range of abilities and interests, youth and families may prioritize different areas. One individual may prioritize finding full-time competitive employment, while another may look for social opportunities. Therefore, as an interventionist attempting to identify meaningful goals that reflect the youth’s priorities (with many potential domains and countless possibilities within each domain), how can one gather goals in a way that is time-efficient without overwhelming youth or families?
GAS-CB approach
To address the concerns above, we incorporated the following activities into the first step of the GAS process: (1) generating a goal bank; (2) working with parents to select and customize a smaller set of goals from the goal bank; and (3) working with youth to choose preferred goals from the parent’s shorter list and customize them further. We began with parents, as they could provide valuable information about the youth’s performance and interests (Morningstar et al., 1995). Starting the process with the parent also allowed us to develop a smaller pool of goal options to present to the youth, which we expected to be less demanding than asking youth to choose from a large set of possibilities or come up with goals from scratch (without any training in goal setting).
To guide the goal setting process, we created a goal bank with over 80 transition-related goals in five key transition domains: employment and postsecondary education, daily living, healthy living, social and spiritual, and safety and security. The goal bank gave us a starting point that allowed us to reduce the time to complete a full GAS session. From this bank, parents chose five “bare” goals (i.e. pre-customized goals, such as, “Youth completes housekeeping activities”) based on the youth’s interests and priorities. During the semi-structured customization interview, trained research staff asked questions to understand the youth’s current performance in relation to the goal, strengths and challenges, and supports that could be helpful. Based on the parent’s responses, research staff worked with families to customize the bare goals to be maximally rigorous (i.e. meeting criteria for a SMART goal) and relevant to their youth. Staff also solicited feedback from the parent in terms of feasibility and potential appeal to the youth. Often, bare goals changed significantly by the end of the interview—not only by adding a timeframe, needed supports, and scaling criteria, but also by accounting for the youth’s interests and the resources available to the family. For instance, after integrating the parent’s feedback, the bare goal of “Youth explores career interests” transformed into “Within a year, youth speaks to members of the workforce in three different fields of interest (e.g. computer graphics, culinary art, and auto mechanics) to learn about their job experiences.”
The same research staff then conducted a goal selection interview with the youth. To maintain the youth’s privacy and to ensure that parental presence did not influence the youth’s responses, staff spoke with the youth without the parent in the room. However, parents could stay if the youth’s communication abilities made goal selection impossible without parental support, or if the youth preferred to have the parent present. Research staff read each of the five customized goals to the youth, recorded their reactions, and prompted the youth to choose three of those goals by asking questions such as “Is this something you’d be interested in working on?,” “Is this goal thumbs up, thumbs down, or thumbs in the middle?,” or “Which goal was your favorite?” As part of the protocol, research staff also supported youth in creating new goals if they did not feel that the parent-chosen goals were a good fit. In these cases, staff guided youth to choose the life domain they were most interested in. Research staff also suggested directions for new goals based on the parent interview and what the youth shared about their interests. In the ASSIST study, about 10% of youth created new goals using this procedure.
Once the youth picked their top three goals, research staff worked to further customize the goals to their own specifications. For youth with complex communication needs, the interviewer would rephrase the goals into simplified language and offer visual supports so that youth could indicate their selections by pointing, giving a thumbs up/down, or nodding/shaking their head. This process enabled us to work with families to efficiently and effectively set personalized goals within 30–45 min (including both the parent and youth interviews).
Step two: goal scaling development
Challenges
After goals are set, a GAS rubric is created for each goal to allow comparable, standardized ratings across goals. When developing a scaling rubric, progress toward a goal can be defined in different ways, such as by measuring frequency (e.g. How often does the youth do this activity?) or level of independence (e.g. How much help does the youth need to do this activity?). Like goal setting, GAS scaling should be individualized to generate meaningful outcomes for each youth (Shogren et al., 2021). Specifically, GAS rubrics are personalized based on one’s present level of performance to be reasonable and realistic, while also being measurable and equidistant (i.e. same amount of change between levels of possible attainment; Ruble et al., 2012).
Although existing methods for scaling (such as the common 5-point scale; Kiresuk et al., 1994; Ruble et al., 2012 and Figure 2) are applicable to the GAS-CB approach, we encountered a challenge in choosing the best scale points. We were developing annual goals, so we would be evaluating progress after a 12-month window. Given the many steps that might occur over an entire year, we were concerned that this 5-point scaling approach may not capture incremental progress (Cytrynbaum et al., 1979). This concern can be illustrated through the example of an annual goal about applying for a job. What if, over the course of the year, the youth researches part-time jobs and reviews their requirements but has not yet achieved the set goal of applying for a job? The traditional approach would rate this attainment as “less than expected—present level of performance” since the youth did not achieve the annual goal. However, that rating overlooks the steps that the youth has taken. Because we wanted to capture incremental progress toward a goal over the year, the original scaling framework of −2 to +2 was not entirely compatible with our purposes for using GAS.
Comparing traditional GAS versus GAS-CB scaling approaches.
GAS-CB approach
Although GAS rubrics are not completed until after the baseline interview in the GAS-CB approach, the scaling process influenced the entire goal setting protocol. To ensure that scaling was relevant to the youth, research staff asked for both the parent’s and youth’s input on which aspect of the goal was most meaningful to them (e.g. frequency of the behavior, developmental sequence of skills or activities, greater independence, or generalization into different contexts) to determine the best outcomes to use for scaling the goal. Research staff talked to the parent about the present level of performance relative to that goal so that the GAS rubric could reflect realistic expectations, setting the bar high but still within reach over a 12-month period. While goal scaling in other GAS applications sometimes evaluates more than one aspect of change (e.g. assessing changes in both frequency and independence), we opted for simplified unidimensional scaling rubrics. Examining only one “scaling option” maximizes participants’ abilities to provide an accurate report and researchers’ ability to document progress precisely (Krasny-Pacini et al., 2016).
After confirming the three selected goals in Step 1 and identifying desired scaling options for each goal from the parent and youth, research staff developed GAS rubrics for each goal. We chose a 6-point scale from −1 (much less than expected) to +4 (exceeds annual goal; see Figure 2) to account for a range of achievement between baseline and the ideal level of attainment (Beidel et al., 1983). This 6-point rating scale is similar to the more traditional 5-point scale (Kiresuk et al., 1994) in that it accounts for both regression (performance lower than baseline) and for exceeding the annual goal. The primary difference is that the 6-point scale has two levels (+1 and +2) dedicated to progress between baseline (0) and the annual goal (+3). In contrast, the 5-point does not have any scale points to describe this amount of change. Thus, for our application of GAS-CB, the 6-point scale accommodated our desire to focus on the different increments of attainment between baseline and meeting the goal, although other research teams may have different needs related to documenting these incremental changes.
Step three: goal attainment rating
Challenge
To rate goal attainment based on GAS rubrics, research staff may collect external observations (e.g. document review or behavioral observations), ratings from the youth themselves, or ratings from parents or teachers/professionals who have direct interaction with the youth. Depending on the study, it may not be feasible for community-based interventions to collect objective observational data from participants. Even in studies where direct observation is possible, behaviors related to goals and GAS rubrics may be challenging to consistently observe and rate during a restricted time period (Shogren et al., 2021). This is particularly true when there is a broad range of transition goals that are targeted to different contexts such as home, school, work, or the community. Determining the best opportunity to directly observe these activities for multiple goals for each participant could be well beyond the capabilities of a research team.
GAS-CB approach
Although direct observation of attainment is ideal, it is not always feasible. Thus, we developed a rigorous process for collecting attainment data using parent and youth reports. At the end of the interval for which the goals were developed (in our case, 12 months), interviews were conducted with both youth and parents, asking in detail about observable and measurable characteristics of the goals directly linked to the scaling options included in the GAS rubrics (e.g. how many times per week the parent sees the youth completing the goal activity). Furthermore, collecting attainment data from both parties provides opportunities to compare perspectives on goal attainment, as existing research suggests that different reporters rate attainment differently (Shogren et al., 2020).
Maintaining validity and reliability when using the GAS-CB approach
GAS can only be considered a valid, standardized measure for group design studies when goals can be scaled and rated in ways that are psychometrically valid and enable equivalence in GAS rubrics across groups in measurability, level of difficulty, and benchmarks (Ruble et al., 2012). In our application of GAS-CB, we were guided by the following well-established steps to promote reliability and validity (Ruble et al., 2012): establishing clear GAS rubrics linked to the goal, practicing writing goals and associated scaling rubrics, and testing the equivalence of GAS rubrics between groups. To facilitate reliability, we developed a semi-structured interview that used a systematic approach in gathering information for writing goals to be scaled for GAS (e.g. current performance, available supports, environment, and timeline). We determined that in the context of the ASSIST intervention, annual goals would be the most relevant to test intervention effectiveness. During the goal setting interview, we emphasized that the goals should involve activities that could last for a year; if the goal could be obtained within 6 months, we re-directed parents and youth to develop a longer-term goal.
We developed a training protocol with detailed instructions for the GAS-CB process and facilitated a 1-day training session for research staff at each of our study sites. Once the training was complete, research team members who would be conducting the semi-structured interviews to establish goals and GAS rubrics had the opportunity to practice writing goals and creating scaling rubrics by filling out training templates based on example scenarios. These training templates were evaluated by the trainers for their measurability, unidimensionality, difficulty, and equidistance (for further descriptions, see Ruble et al., 2012). When research staff reached 90% agreement with the trainer, they were cleared to begin GAS-CB interviews. If agreement was lower than 90%, staff received additional training.
During data collection, the GAS-CB trainers reviewed the first five goals and GAS rubrics from each team member, and then they reviewed 20% of each research staff’s remaining goals and rubrics to ensure that scaling rubrics were equivalent (by assessing measurability, unidimensional, difficulty, and equidistance; Ruble et al., 2012). If non-equivalence was identified, we worked with the research staff to revise the goals and scaling rubrics, and notified participants of the changes. We also increased the proportion of goals and GAS rubrics reviewed by the GAS-CB trainers as needed. Beyond trainings, ongoing coaching and multi-site case discussions were available to ensure team members were administering GAS the same way across sites.
The GAS-CB protocol also includes steps to promote reliability and validity of the GAS ratings. Research staff who collected GAS outcome data were blinded to treatment group assignment. Instead of presenting the scaling rubric directly to participants and asking them to rate their attainment based on that scale, research staff read each goal and asked a general open-ended question: “Over the last year, how have you been doing with this goal?” Refraining from sharing the GAS rubric with participants can help minimize the possibilities of over- or under-reporting attainment. After the attainment interviews, a second independent researcher rated attainment by listening to the audio recordings, providing a measure of inter-rater reliability.
Although researchers may choose different specific activities to ensure the goals and GAS rubrics are psychometrically valid and are reliably obtained, ensuring validity and reliability is essential in any application of GAS-CB so that goal attainment based on GAS can be compared across individuals and groups. Having an established protocol is key to ensuring that comparisons are feasible (Shogren et al., 2021).
Community involvement
Incorporating the feedback of youth with ASD and their families was integral during the development of GAS-CB. Research staff who were family members of adults with ASD led the goal bank generation, and additional family members were consulted to ensure there were sufficient options for youth at various ability levels. The full GAS-CB procedure was piloted with two parent/youth dyads and modifications were made based on youth and parent feedback.
Conclusion
Everyone has unique dreams and goals for their lives. To judge whether an intervention is maximally beneficial, researchers should—along with objective measures—seek to assess personalized outcomes that are important to individuals with ASD. Not only will this approach allow researchers to measure whether an intervention is supporting individuals in attaining their own goals, but it will also allow for personalized outcome measurement that takes into account the wide heterogeneity in functioning, impairments, and strengths within the ASD population.
GAS can be used as a complement to more objective measures such as service access, employment, or community integration (Schlosser, 2004) in a wide array of community-based intervention research projects. For example, when targeting service access outcomes, researchers can both document quantifiable indicators of access (e.g. types of services and hours of services) as well as use GAS to determine if greater service access is aligned with youth attaining their desired goals. Studies of mental health interventions might also benefit from incorporating GAS, examining whether decreases in psychological distress allow youth to more effectively attain their personal goals. Although GAS does not take the place of outcomes such as service access or psychological symptoms, it can be a useful complement to ensure that the priorities of the individual with ASD are measured and included in establishing the impacts of interventions.
The new GAS-CB framework focuses on overcoming challenges of current GAS applications, which are often best suited to certain settings (such as school-based or clinic-based settings), rely on the availability of professionals to conduct (typically extensive) personalized assessments and planning processes, or depend on extensive training to engage individuals with ASD and/or their families in the GAS process. These methods of identifying goals and creating GAS rubrics may not be feasible for some studies, and may even influence the intervention as delivered by adding additional components (Shogren et al., 2021). In contrast, GAS-CB can: generate psychometrically valid, individualized goals and scaling rubrics that are relevant across research settings and projects; directly incorporate the perspective of the youth with ASD (even those with more significant support needs) and their families; be used without requiring extensive training of research participants; and identify goals through a relatively short semi-structured interview process. It is our hope that GAS-CB can be used across many community-based intervention studies, allowing researchers to examine whether interventions support individuals with ASD in attaining their own goals.
In general, to generate scores that could be comparable across studies, researchers may want to lean toward the more traditional 5-point scaling when possible (see Figure 2). Although we chose a 6-point scale in our application of GAS-CB to focus on incremental progress toward a long-term goal, the scale that researchers use will likely depend on their specific research questions and the purposes of the GAS. If fine-grained change is of interest, for example, researchers may want to consider the traditional 5-point scale but include “half levels” of 0.5 for a more precise score (Ruble et al., 2019).
It is important to note that there are significant trade-offs to consider when choosing GAS-CB instead of existing GAS methods (e.g. Duncan et al., 2018; Ruble et al., 2018; Shogren et al., 2012). To maximize usability of the GAS-CB framework, some rigor was sacrificed. The most rigorous way to judge attainment, for example, would be to gather objective assessment through observation, permanent projects, or other means that are less subject to reporter bias (Krasny-Pacini et al., 2016; Schlosser, 2004). Ideally, the person with ASD would be the main informant in the goal setting, scaling, and attainment process (Shogren et al., 2021). Beginning with parents and using the starting point of a goal bank in GAS-CB could make it more difficult to develop highly individualized, high-quality goals and scaling than in-depth person-centered planning processes.
More stringent protocols that incorporate “best practices” for GAS, however, have the potential to lose some impact if they preclude wide-spread use of GAS among studies interested in personalized outcomes. Thus, protocols such as this one that provide additional flexibility to make GAS feasible—while also offering guidance to conduct the protocol reliably and validly—play an important role in making methods for personalized outcomes accessible to studies that do not easily lend themselves to more traditional GAS methods.
Supplemental Material
sj-pdf-1-aut-10.1177_13623613211024492 – Supplemental material for Goal attainment scaling—community-based: A method to incorporate personalized outcomes into intervention research with youth and adults on the autism spectrum
Supplemental material, sj-pdf-1-aut-10.1177_13623613211024492 for Goal attainment scaling—community-based: A method to incorporate personalized outcomes into intervention research with youth and adults on the autism spectrum by Chung Eun Lee, Karrie A Shogren, Jordan Segal, Florencia Pezzimenti, Janeth Aleman-Tovar and Julie Lounds Taylor in Autism
Footnotes
Author note
Chung Eun Lee is now at Chonnam National University.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the National Institute of Mental Health (R01 MH116058-01A1, J.L.T., PI), with core support from the National Institute of Child Health and Human Development (1P50HD103537-01, PI: Neul). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Mental Health.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
