A Test Review: Gilliam,J. E. (2014). Gilliam Autism Rating Scale–Third Edition (GARS-3)

Administration

Because the examiner helps to collect diagnostic information from a variety of sources, assimilate data, and qualify individuals for services, examiners should be individuals who have training and experience working with individuals with autism (Gilliam, 2014). Potential examiners may include school psychologists, speech and language pathologists, educational diagnosticians, autism specialists, or other trained professionals who know how to interpret and use data to inform autism diagnoses. In addition, examiners are responsible for selecting appropriate raters to complete the GARS-3 (Gilliam, 2014).

Most often, raters on the GARS-3 are classroom teachers, parents, or other caregivers who have sustained contact with the individual being assessed. The manual recommends that a potential rater have at least 2 weeks of sustained contact with an individual prior to completing the GARS-3 (Gilliam, 2014). Raters should also understand how the GARS-3 helps to diagnose autism and should read all questions thoroughly. Examiners should make themselves available to answer any questions that the rater may have while completing the GARS-3. It is unnecessary for raters to compare the individual being rated with other people of the same age because normative data on the GARS-3 have already taken age of the individual being rated into account. Instead, each behavior should be evaluated only how accurately the item describes a person’s behavior (Gilliam, 2014).

The GARS-3 can be completed either as a questionnaire or a structured interview (Gilliam, 2014). Each rater will need a copy of the summary/response form and a writing utensil. Pertinent information including the name of the person being rated and the data of the rating should be included on the front page of the GARS-3. Raters are then asked to complete 58 Likert-type items pertaining to observable measurable behaviors related to the person being rated. The Likert-type items are each rated on a scale of zero to three with zero being not at all like the individual, one being not much like the individual, two being somewhat like the individual, and three being very much like the individual. Each item on the GARS-3 must receive a rating. If the rater is uncertain of how to rate an item, they should observe the individual for more time prior to completing the rating or extrapolate data from observations that have been conducted (Gilliam, 2014).

The Centers for Disease Control and Prevention (2012) reported that roughly 40% of children with autism are mute. Furthermore, the GARS-3 manual indicates that nearly 25% of the students who were rated during the norming of the instrument were mute. Therefore, it is reasonable to conclude that some individuals who are referred for an evaluation will be unable to communicate. In these cases, the authors of the GARS-3 recommend omitting the Maladaptive Speech and Cognitive Style subscales from the test (Gilliam, 2014).

Scoring/Interpretation

After completion of the GARS-3, the examiner will first total raw scores for each of the six subscales or four subscales if the individual being rated is mute (Gilliam, 2014). These raw scores are then converted into percentile ranks and scaled scores using Appendix A in the back of the Examiner’s Manual. Each percentile rank indicates the percentage of the normative sample that is equal to or below the percentile. Scaled scores for subscales were developed by applying a direct linear transformation to raw scores to obtain a distribution with a mean of 10 and a standard deviation of 3. Higher scaled scores on a subscale represent increasingly severe autistic behavior. The GARS-3 manual notes that scaled scores are ideal for comparing an individual’s performance with the normative sample and that these scores can be manipulated statistically making them ideal for use in research (Gilliam, 2014).

After scaled scores have been calculated for each subscale, the scaled scores of all subscales should be totaled to calculate an overall composite score known as the Autism Index (Gilliam, 2014). Examiners should consult Appendix B in the Examiner’s Manual to obtain corresponding Autism Index scores and percentile ranks from the sum of scaled scores. The Autism Index composite is a different kind of standardized score with a mean of 100 and a standard deviation of 15. The greater an Autism Index score is, the higher the probability the individual being rated has autism and the more severe the autistic behavior. According to Gilliam, this score is the best score for identifying an individual with autism, as it is composed of the sum of scaled scores. In addition, Gilliam notes that this score is also the most reliable score among standard scores on the GARS-3. It is further noted that another Autism Index exists for individuals who are nonverbal by summing the scores of the four subscales administered. Both the six-subscale and four-subscale versions of the Autism Index have been found to be reliable, valid, and discriminative. Because all the persons in the normative sample had autism, the norms of the GARS-3 depict how certain scores relate to scores of other individuals with autism. On the bottom of the front page of summary/response form is an Interpretation Guide to help indicate the likelihood of autism, severity level of autism, and descriptor for recommended support level (Gilliam, 2014).

Development and Standardization

When selecting a normative sample for the GARS-3, authors of the measure contacted parents, teachers, educational diagnosticians, psychologists, speech clinicians, teacher assistants, and other educational professionals (mainly through electronic solicitation) and asked them to complete the GARS-3 on individuals diagnosed with autism (Gilliam, 2014). To participate in the research, individuals being rated had to have received a diagnosis of autism, be between 3 and 22 years old, and reside in the United States. The majority of raters in the normative sample were parents (23.2%), teachers (33.4%), and speech clinicians (13.0%). In addition, the majority of raters held either a bachelor’s (24.9%) or an advanced degree (58.6%). Most of the raters in the normative sample claimed to have either studied autism in school or considered themselves very knowledgeable on the topic (87.3%); furthermore, a large portion of the raters claimed to have known the person being rated for 4 or more years (80.3%; Gilliam, 2014).

The total normative sample consisted of 1,859 children and young adults with diagnoses of autism (Gilliam, 2014). The majority of this sample was reported to only have an ASD (61.3%). This normative sample was drawn from a diverse geographic population that included individuals from 48 different states. Age of individuals being rated was fairly evenly spread between 3 and 19 years, with somewhat smaller sample sizes being reported for individuals aged between 20 and 22 years (approximately 60%-75% decline). The normative sample was most males (77%), Whites (80%) and Black/African Americans (12%). Specifically, only a very small portion of the normative sample was composed of Asian/Pacific Islanders (5%) and American Indian/Eskimo/Aleuts (<1%), indicating that more research may need to be done to assess accuracy of this measure with regard to these races (Gilliam, 2014).

Reliability

The internal consistency of each individual subtest on the GARS-3 as well as the two overall indexes were measured using Cronbach’s alpha (Gilliam, 2014). Overall, the GARS-3 demonstrated superior internal consistency across various age groups. Average Cronbach’s alphas of .94 and .93 were recorded for the Autism Index 4 and Autism Index 6, respectively, (Gilliam, 2014).

A study in which raters completed the GARS-3 twice within in a 2-week period on 122 individuals was conducted to determine test–retest reliability (Gilliam, 2014). Reasonable correlations were recorded for the various subscales with average correlation coefficients ranging from .76 to .87. In addition, superior test–retest correlations of .90 and .90 were recorded for the Autism Index 4 and Autism Index 6, respectively. To measure interrater reliability, an additional study was conducted in which 232 raters rated 116 individuals with autism using the GARS-3. Intraclass correlation coefficients (ICC) were then used to calculate interrater reliability. ICCs are often used to measure interrater reliability when data are considered to be interval data and are often preferred over Pearson’s r (Lecavalier, 2005). Average ICCs were within the acceptable range (.71-.85) for all subscales and were considered to be good for the Autism Index 4 (.84) and Autism Index 6 (.84; Gilliam, 2014).

Validity

Authors of the GARS-3 present validity information for content-description validity, criterion-prediction validity, and construct-identification validity in the Examiner’s Manual. To obtain content-description validity, the authors of the GARS-3 reviewed diagnostic criteria from Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; APA, 2000), DSM-5 (APA, 2013), International Classification of Diseases, Tenth Edition (ICD-10) (World Health Organization, 1992), and the Autism Society (2012) concerning behaviors pertinent for the diagnosis of autism (Gilliam, 2014). From this information, a checklist was created with 120 items that was sent to various parents, university professors, and clinicians for review (these individuals were asked to rate the importance of each item). This, along with asking various parents and professionals to submit electronic ratings using a GARS-3 prototype, was used to collect data that eventually were analyzed by factor analysis leading to a final product with 58 items. In addition, item discrimination was investigated by examining each item on each subscale with the total score for that subscale. All point-biserial correlations were within the acceptable range (.57-.86) indicating that the GARS-3 has good item discrimination (Gilliam, 2014).

To assess the criterion-prediction validity, four independent studies were conducted correlating scores from the GARS-3 with other assessments used in screening for autism that included the Autism Behavior Checklist from the Autism Screening Instrument for Educational Planning–Third Edition (ASIEP-3; Krug, Arick, & Almond, 2008), the Autism Diagnostic Observation Scale (ADOS; Lord, Rutter, DiLavore, & Risi, 1999), the Carolina Autism Rating Scale–Second Edition (CARS-2; Schopler, Van Bourgondien, Wellman, & Love, 2010), and the Gilliam Asperger’s Disorder Scale (GADS; Gilliam, 2001). In each of these studies, both parent and teacher forms were utilized. On average, scores from the Autism Index 6 correlated with more than four tests at an acceptable .72 with a range of .68 to .77 across the four different correlations (Gilliam, 2014). Similarly, the Autism Index 4 averaged a .76 correlation with the other four instruments and had a range of .72 to .83. To further measure criterion-prediction validity, the sensitivity and specificity of the GARS-3 were measured using binary classification and receiver operating characteristic/area under the curve (ROC/AUC) analyses. Results of these analyses found the Autism Indexes to be excellent in sensitivity (individuals with ASD testing positive) and fair to excellent for specificity (individuals without ASD testing negative; Gilliam, 2014).

Construct-identification was established by using the GARS-3 on various populations, including children with intellectual disability (n = 15), attention-deficit hyperactivity disorder (n = 73), emotional disturbance/behavioral disorder (n = 58), learning disability (n = 163), speech and language impairments (n = 54), and nondisabled typically functioning individuals (n = 130; Gilliam, 2014). Results indicated that the GARS-3 was able to differentiate children with autism from children with other disabilities or typically developing children. It was noted that children with intellectual disability scored higher on the GARS-3 than all other diagnostic groups being rated. This anomaly is most likely due to the small number of participants in the intellectual disability group or the possibility that some of the children within this group were inaccurately classified. To further establish construct-identification validity, an exploratory factor analysis was conducted on GARS-3 items. Results of the analysis revealed six factors that were consistent with the structure of the GARS-3 and autism domains in the DSM-5 (Gilliam, 2014).