Abstract
Screening children to determine risk for Autism Spectrum Disorders has become more common, although some question the advisability of such a strategy. The purpose of this systematic review is to identify autism screening tools that have been adapted for use in cultures different from that in which they were developed, evaluate the cultural adaptation process, report on the psychometric properties of the adapted instruments, and describe the implications for further research and clinical practice. A total of 21 articles met criteria for inclusion, reporting on the cultural adaptation of autism screening in 19 countries and in 10 languages. The cultural adaptation process was not always clearly outlined and often did not include the recommended guidelines. Cultural/linguistic modifications to the translated tools tended to increase with the rigor of the adaptation process. Differences between the psychometric properties of the original and adapted versions were common, indicating the need to obtain normative data on populations to increase the utility of the translated tool.
Prevalence estimates of Autism Spectrum Disorder (ASD) have increased over the last decade; however, epidemiological studies report variation in these estimates across geographical areas and ethnic and racial groups (Elsabbagh et al., 2012). Rather than differences in actual prevalence, these findings may instead reflect a lack of awareness of autism, effects of stigma, differing methods used to investigate prevalence rates, identify cases, and evaluate cases, and different samples across studies. Inadequate psychometric properties as well as lack of appropriate cultural adaptations of screening and diagnostic tools may further contribute to varying prevalence estimates.
Screening is frequently proposed as a key step to identifying children at risk for ASD and to expedite early behavioral and educational interventions to improve outcomes (National Research Council (US) Committee on Educational Interventions for Children with Autism, 2001; Szatmari et al., 2003), although some question a population-wide strategy (e.g. Al-Qabandi et al., 2011; Le Couteur, 2003). As currently defined, an autism screening tool is a formalized brief questionnaire completed by a parent or provider before an in-depth diagnostic evaluation to identify a child at risk of autism (Filipek et al., 2000; Plauché Johnson et al., 2007). Administration of the tool may take place before a concern is identified, as in the case with whole-population-based screening, or after a parent or provider raises a concern. Current guidance from the American Academy of Pediatrics (Plauché Johnson et al., 2007) and others (Filipek et al., 2000) recommend formal screening in pediatric primary care beginning at 18 months. While significant improvements have been made in the development, validation, and implementation of autism screening tools in several countries (Ehlers et al., 1999; Kamp-Becker et al., 2005; Nordenbæk et al., 2011; Song et al., 2009; Swinkels et al., 2006), most tools have been developed in the United States or United Kingdom and are increasingly being used in cultures other than those in which they were created (Dixon et al., 2011).
Appropriate use of existing tools in other cultural and linguistic environments goes beyond translation to include a thorough process of identifying potential incongruities in language and concepts and then modifying the tool so that it is understood by the target population. The goal of cultural adaptation is to establish functional equivalence with the original version (Banville et al., 2000; Beaton et al., 2000). Moreover, one cannot assume that psychometric properties are retained within the new version of a screening tool simply because language and concept equivalence have been established, and therefore psychometric analysis of the adapted tool is also recommended (Beaton et al., 2000).
The purpose of this systematic review is to (1) identify ASD screening tools that have been culturally adapted across cultures and countries, (2) evaluate the extent to which the adaptation process adhered to recommended cultural adaptation guidelines, (3) report on the psychometric properties of the adapted tools, and (4) describe the implications of these findings for further research and practice.
Method
Search strategy
Based on the predetermined scope of the review, a medical research librarian conducted searches of applicable databases, including MEDLINE, CINAHL, PsycINFO, Web of Science, ERIC, Embase, OVID, Mental Measurements of Tests in Print, PubMED, HaPI, World Health Organization (WHO), and Google Scholar. Search terms were grouped into six categories: all variants of ASD (e.g. autism, ASD, pervasive developmental disorder (PDD)), screening tools, cultural adaptation, language considerations, child, and psychometrics. The search included studies published in languages other than English.
Study selection
The search produced 477 studies. Selection for final inclusion occurred in two steps as depicted in the Flow Chart in Figure 1. First, in the initial review, the 477 articles were screened by two coauthors on the basis of title and abstract. Selection criteria were relatively broad to avoid omitting relevant studies. Articles were excluded if the studies were unrelated to ASD screening or cultural adaptation, conducted prior to the publication of Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition (DSM-IV), or focused on adult populations. From this initial review, 86 articles were selected for full review, using criteria designed to capture only those studies involving adaptation of a screener to a different culture/language than that in which it was developed. These articles were read by at least two coauthors; disagreement or uncertainty regarding selection between the readers was brought to the full group for discussion. An article was excluded if the study (1) did not involve cultural adaptation of an autism screening tool, (2) examined a diagnostic rather than a screening instrument, (3) was solely an investigation of ASD prevalence, or (4) was a commentary or a review. Based on these criteria, 68 articles were eliminated, leaving 21 articles for inclusion. A final search was conducted to identify additional studies through reference lists, however, no new studies were identified.
Flowchart illustrating the inclusion and exclusion of studies at each step of the review.
Assessment of cultural adaptations and reporting of psychometric properties
At least two reviewers independently rated the rigor of the adaptation process for each study based on established guidelines for achieving linguistic and cultural equivalency (Beaton et al., 2000; Guillemin et al., 1993). Recommended components of adaptation (forward-translation, back-translation, review by committee, and field pretesting) were rated using a 4-point scale: all guidelines met, guidelines partially met, mentioned but not described, and no information reported (Table 1). These guidelines were used to rate the adaptation process of the reviewed studies. Differences between reviewer ratings were discussed and reconciled, sometimes after assessment by a third reviewer. Psychometric properties of the adapted screening tools were extracted from validation studies, focusing on measures of reliability, validity and discrimination, and differences from the original tool noted by the authors.
Guidelines for assessing the extent to which reported cross-cultural adaptation of ASD screening instruments adhered to guidelines adapted from Guillemin et al. (1993).
ASD: autism spectrum disorder.
Results
Table 2 summarizes characteristics of the screen and sample used in each of the adaptation studies. Table 3 provides ratings of the adaptation process based on the guidelines from Table 1 and the reported classification measures (sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV)) of the screen for identifying children with ASD from those with typical development (TD) or other clinical conditions. No attempts were made to calculate missing classification measures and only the psychometric properties that were reported in the articles have been provided in Table 3. Information from multiple articles reporting the results from the same population is combined in the tables. The 21 studies reported the results of the adaptation of nine different screening tools in 10 different languages for children from 12 months to 18 years. Only one of the tools was originally developed in a language other than English (Ehlers et al., 1999). Seven studies adapted the Modified Checklist for Autism in Toddlers (M-CHAT), spanning six languages and 15 countries, a number that is likely to increase because, according to the developer, the M-CHAT has been translated from English into more than 25 languages (http://www2.gsu.edu/~psydlr/M-CHAT/Official_M-CHAT_Website.html). More than half of the studies focused on screeners developed for young children. Studies with older children adapted screeners that had been developed to identify Asperger’s syndrome (AS) and other higher functioning ASDs. Screeners were validated with a wide variation of samples: total populations, subsamples combining unaffected and affected children, and cohorts of children diagnosed with ASD and other disorders. Below we review the studies organized by the nine screening tools adapted.
Screen and sample characteristics of selected studies.
ADHD: Attention Deficit Hyperactivity Disorder, AS: Asperger’s syndrome, ASD: autism spectrum disorder, COS: Childhood Onset Schizophrenia, DD: Developmental Delay, HFA: high functioning autism, NR: Not Reported, PDD: Pervasive Developmental Disorder, PDD-NOS: PDD not otherwise specified, TD: Typically Developing.
Cultural adaptation ratings and classification measures of selected studies.
ASD: Autism Spectrum Disorder; TD: Typically Developing; NR = Not Reported.
See Table 1 for key.
The Autism Detection in Early Childhood (ADEC) was originally developed in Australia. It is a 16-item tool completed by health professionals for children in the age group of 12–24 months designed to distinguish ASD from other developmental disorders (Young, 2007). In a study by Hedley et al. (2010) in Mexico, the ADEC was translated by the bilingual research team into Spanish and checked by a bilingual psychologist. A small field test of the translation was conducted to identify misinterpretation, although these results were not reported. Health professionals screened two cohorts including children with TD and those with autism or other developmental disorders (Cohort 1), and those referred for evaluation of a developmental delay (Cohort 2). Acceptable internal consistency (α = 0.73) and high inter-rater reliability between the adapted and original versions (r = 0.96) were reported from analyses of Cohorts 1 and 2. A toddler subsample from Cohort 2, consisting of children with and without a diagnosis of PDD, was also assessed using the ADEC to determine cut-off scores for the adapted tool. The recommended cut-off score from the original version was congruent with that of the toddler subsample. Sensitivity, specificity, PPV, and NPV were adequate to excellent in this mixed sample. This version of the ADEC demonstrated high concurrent validity with the Childhood Autism Rating Scale (CARS; Schopler et al., 1980) and the Autism Diagnostic Interview–Revised (ADI-R; Le Couteur et al., 2003) and distinguished between children with autism/PDD–not otherwise specified (PDD-NOS), non-PDD diagnosis, and TD.
The Autism Screening Questionnaire (Berument et al., 1999), an earlier version of the Social Communication Questionnaire (SCQ; Rutter et al., 2003), is a 40-question yes/no, caregiver completed screening tool developed in the United Kingdom with versions for children below and above the age of 6 years. Sato et al. (2009) created the Brazilian version through translation into Portuguese and back-translation. Review of these versions by a multidisciplinary committee resulted in only minor semantic changes to two items. Researchers completed the questionnaire via telephone with parents of children previously diagnosed with ASD, Down syndrome, and other psychiatric disorders. Internal consistency (α = 0.90) and test–retest reliability were high. Using the same cut-off score as the original version of the tool, sensitivity and specificity were also high.
The Autism Spectrum Screening Questionnaire (ASSQ: Ehlers and Gillberg, 1993; Ehlers et al., 1999) is a 27-item, 3-point questionnaire developed in Sweden and intended to screen for AS and other higher functioning ASDs in children in the age group of 7–16 months. Items assess social problems, communication problems, repetitive and restrictive behavior, motor clumsiness, and tics. The ASSQ has been translated into English (Ehlers and Gillberg, 1993), Lithuanian (Lesinskiene, 2000), Norwegian (Posserud et al., 2006), Finnish (Mattila et al., 2009), and Mandarin Chinese (Guo et al., 2011). Validation studies have been published for the latter three versions.
Finnish studies (Mattila et al., 2009, 2012) described an adaptation of the ASSQ consisting of translation by two psychologists, back-translation by an official Swedish translator, and comparison of the two versions (translation vs back-translation) by a panel of pediatric neurologists. The wording of the highest alternative on the scale was changed from “fits definitely” to “fits” because clinical experience suggested that Finnish parents would be reluctant to select the former alternative. Screening comprised ASSQ ratings by teachers and parents of all 8-year-old children, mainly of Finnish descent, living in the catchment area of one hospital area in one county and attending 304 schools. All children in the screened sample who met criteria for ASD through diagnostic evaluation with the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 1999) and ADI-R or hospital records were identified. Parent and teacher ASSQ scores showed a low correlation, but the means and medians of both ratings were significantly higher in 7–12-year-old children with high functioning autism (HFA)/AS in outpatient treatment than in the total sample. On summation, parent and teacher scores showed the best discrimination between children with and without an ASD diagnosis in the analysis of the total sample; sensitivity, specificity, and NPV were extremely high (Mattila et al., 2012). Mattila et al. (2009) also evaluated cut-offs in the same sample; however, sensitivity and specificity values were slightly different for the same optimal cut-off because all subjects with ASD had not yet been identified. Of note, the cut-off score recommendations differed from those reported in the original Swedish and Mandarin Chinese studies (see below).
The ASSQ was also translated into Norwegian (Posserud et al., 2006) for the Norwegian Bergen Child Study (Heiervang et al., 2007). In a large sample of 7–9-year-olds, the adapted instrument was found to have good internal consistency (0.86 for teachers and 0.89 for parents) and a stable three-factor structure (Posserud et al., 2008). All children who screened positive and a sample of those who screened negative participated in a full diagnostic evaluation. Receiver Operating Characteristic (ROC) analyses, assessing the ability of the ASSQ to distinguish ASD from non-ASD cases, revealed excellent discriminant ability for parent and teacher ratings with even more optimal screening properties for the score combining ratings from both respondents (Posserud et al., 2009).
Guo et al. (2011) evaluated the validity of a Mandarin Chinese translation of the ASSQ (CH-ASSQ). The ASSQ was translated by two of the native-speaking authors and further refined through back-translation. After review by four native-speaking experts and two English-speaking authors, minor changes were made to make the translation more culturally appropriate, while maintaining the clinical meaning of the items. The CH-ASSQ was completed by parents of 165 children recruited from the Institute of Mental Health, Peking University, who had diagnoses of ASD, Attention Deficit Hyperactivity Disorder (ADHD), or Childhood Onset Schizophrenia, and by parents of 120 unaffected children recruited from the public school system. Analyses were performed on the entire sample as well as a subsample of those children between the ages of 6 and 17 years (66% of the ASD group and 59% of the unaffected group), the same age range used in the validation of the original Swedish ASSQ. An ROC analysis of children with ASD versus unaffected children revealed excellent discrimination using an empirically determined cut-off score, somewhat lower than that determined in the Swedish version, with an overall diagnostic accuracy of 88.3% for the total sample and 92.6% for 6–17-year-olds. A similar analysis comparing children with ASD with the other clinical groups determined a higher optimal cut-off score with slightly reduced but still strong discrimination.
The Autism Spectrum Quotient–Child Form (AQ-Child Form) is a 50-item parent-report questionnaire for children in the age group of 4–11 years developed in the United Kingdom (Auyeung et al., 2008). Wakabayashi et al. (2007) adapted the AQ-Child Form for use in Japan and included forward-translation by one of the Japanese authors, comparison of the two versions by a bilingual psychologist, and backward translation by native English speakers. Pilot testing revealed parental confusion over items tapping imagination (e.g. “When s/he is reading a story, s/he can easily imagine what the characters might look like?”); therefore a “Don’t Know” response option was included. The study used a sample of 475 children with normal intelligence, consisting of those with AS, HFA, or PDD-NOS, as well as unaffected students recruited from primary and secondary schools. The adapted tool demonstrated good internal consistency across domains (α = 0.84) but somewhat weaker reliability within subdomains (α = 0.69–0.84). Group comparisons showed that the Japanese AQ significantly discriminated youth diagnosed with HFA and AS from children with TD. The best-discriminating cut-off score and overall scores were lower than in the original sample, a difference, the authors suggested, that may have resulted from the addition of the “Don’t Know” option or from a reduced recognition of social problems. Other psychometric properties were not reported.
The First Year Inventory (FYI) is a 63-item parent questionnaire developed in the United States for 12-month-olds to assess behaviors in two domains that indicate a risk for ASD: social-communication and sensory–regulatory (Baranek et al., 2003; Reznick et al., 2007). Ben-Sasson and Carter (2012) produced an Israeli version with back-translation. A linguistic and cultural review by a bilingual, interdisciplinary panel of experts was conducted, although no description of the results was provided. Parents of 12-month-old children drawn from private and publicly subsidized day care centers completed the FYI. Internal reliability within the social subdomain (α = 0.68) and sensory subdomain (α = 0.53) was questionable to poor. Construct validity was demonstrated with the Autism Observation Scale for Infants (AOSI; Bryson et al., 2008) and with the Early Learning Composite scores of the Mullen Scales of Early Learning (MSEL; Mullen, 1995). In this sample, lower socioeconomic status (SES) was correlated with higher FYI risk scores. Additionally, the Israeli sample presented with higher scores on the sensory–regulatory domain than the original study. The authors suggested that this may reflect increased rates of dysregulation problems and sensory symptoms found in other studies of Israeli children (Neuman et al., 2004; Tirosh et al., 2003) but note that methodological differences prevent direct cross-country comparison. Sensitivity, specificity, PPV, and NPV were not reported.
The M-CHAT, created in the United States, is a 23-item yes/no parent-report checklist designed for universal screening for ASD in toddlers in the age group of 16–30 months (Kleinman et al., 2008; Robins et al., 2001). Development of a Mexican Spanish language version of the M-CHAT (MM-CHAT) included translation into Spanish with cultural adjustments of some items (Albores-Gallo et al., 2012). For example, because Mexican mothers reportedly had no specific name for the game of “peek-a-boo,” a description was provided. Nevertheless, parents had difficulty understanding some of the translated items. Psychometric properties were investigated with a case control study that included 456 children in the age group of 18–72 months, either with TD or an ASD diagnosis. Reliability was acceptable (α = 0.76) for the total score and for six critical items (α = 0.70). Concurrent validity estimates were in the moderate range using the Child Behavior Checklist (CBCL; Achenbach and Rescorla, 2000) but showed more mixed results with the ADI-R. The MM-CHAT scores were significantly greater in children with ASD than those with TD; however, the critical items distinguishing the two groups were not the same as in the original validation sample. The authors suggested that variation in the endorsed critical items might reflect cultural differences in parenting and social behavioral styles. Other psychometric properties were not reported.
The M-CHAT was adapted for use in Spain with parents of 4535 toddlers in two population-based validation studies (Canal-Bedia et al., 2011). All components of the recommended cultural adaptation process were included, although some were not described fully. Two translations by bilingual individuals with child development expertise were carried out and a back-translation by a native English speaker included a comparison with the original version by the authors and the developers. Following pretesting with 622 children, six items were modified to clarify their meaning. For instance, examples were added to items that referred to the use of specific toys. The adapted M-CHAT discriminated children with ASD from those with TD but not from those with a developmental disorder other than ASD. Sensitivity, specificity, and NPV were high. However, PPVs were low in both studies conducted for this adaptation, a finding attributed to low prevalence rates in the samples.
Kamio and Inada (2006) constructed a Japanese version of the M-CHAT (M-CHAT-JV) through translation by the research team and back-translation by a bilingual, professional translator. A pilot study with the parents of 659, 18-month-old children at well-child, pediatric visits showed adequate discrimination with critical items similar to those of the original instrument but prompted the addition of illustrations to four items to enhance caregiver recognition of delayed typical development (i.e. negative symptoms). Inada et al. (2011) reported psychometric properties of the M-CHAT-JV administered to volunteer parents of children from local day care centers and of toddlers referred to the authors for developmental concerns. The adapted screening tool had strong inter-rater (mothers and fathers) and test–retest reliability and was highly correlated (r = 0.58) with the CARS-Tokyo Version (Kurita et al., 1989). Ratings of 18-month-old children participating in well-child visits differed significantly between children diagnosed with autism or PDD-NOS at 3 years of age and those without a diagnosis. Sensitivity, specificity, and NPV were moderate to high, though PPV was low. Of note, a proposed short version with nine items demonstrated comparable validity, specificity, PPV, and NPV to the full version.
In the full version of the M-CHAT-JV, sensitivity and specificity were improved by using a cut-off of two positive responses (rather than three as originally recommended) because children who were later diagnosed with ASD passed the screen at a higher rate than children in the original US sample. The authors suggested that this difference could have resulted from sampling differences or a culturally based reluctance to endorse “yes” in the yes/no format of the M-CHAT. In addition, a critical item from the original M-CHAT (“interest in other children”) was not useful, as all toddlers later diagnosed with ASD received passing scores on this item. The investigators suggested that Japanese parents might have interpreted their children’s behavior as shyness or modesty.
Losapio and Pondé (2008) adapted the M-CHAT with a Brazilian sample of 40 parents of children from pediatric outpatient clinics. All of the recommended cultural adaptation steps were reported. A Brazilian psychiatrist translated the M-CHAT to Portuguese and a native English-speaking professional translator without prior knowledge of the instrument back-translated the screening tool. A bilingual rater evaluated the equivalence of the original and back-translated versions. Pretesting involved interviews and follow-up probe questions with the parents of two groups of 20 toddlers seen in pediatric clinics: those with autism and those without a diagnosis. Two autism experts evaluated the results and a variety of semantic and grammatical changes were made to items to maintain referential and conceptual meaning. Psychometric analyses were not reported.
Nygren et al. (2012) assessed the psychometric properties of a Swedish version of the M-CHAT through population-based screening over 12 months in Swedish health centers (N = 3999). A few minor, unspecified adjustments of the Swedish language version were made following translation and back-translation to ensure the integrity of the instrument. Little detail was provided about the translation process, and pretesting was not reported. Reliability and validity were not reported but sensitivity and PPV were moderate to high. Only three of the six critical items from the original M-CHAT overlapped with those showing the strongest prediction of ASD diagnosis. One of the original six critical items, “Does your child respond to his/her name when you call?” had the lowest endorsement by parents whose children were later confirmed to have ASD.
Seif Eldin et al. (2008) translated the M-CHAT for use in nine countries in the Gulf and eastern Mediterranean regions. In each country, small numbers of children from child development centers and general pediatric clinics, including a subsample of children with ASD, were recruited for assessment of diagnostic accuracy. Members of the Eastern Mediterranean Association of Child and Adolescent Psychiatry from five countries carried out forward- and back-translation and compared the two versions. Changes resulting from this process were not reported. To address the unique language needs of each country, the authors altered classical Arabic with country-specific dialects for some words. Reliability and validity data were not provided, but the translated version yielded strong sensitivity, specificity, and PPV.
Wong et al. (2004) translated the M-CHAT into Chinese and combined it with the professional-completed observational section B from the original Checklist for Autism in Toddlers (Baron-Cohen et al., 2000) to develop the CHAT-23. Reported cultural and linguistic adaptation procedures included forward- and back-translation, although no description of the process was provided. In pretesting, parents reported difficulty with the forced choice “yes/no” format, preferring qualitative words to describe children’s symptoms. Thus, a graded score was employed: never, seldom, usually, and often. The mixed sample in the validation study included children with AS, non-ASD with developmental delay, and TD. The CHAT-23 successfully discriminated Chinese toddlers between 18 and 24 months, with diagnosed autism from toddlers with and without global developmental or language delays. ROC analysis revealed moderate to high sensitivity and specificity using three cut-off criteria: (1) failing 2 of 7 critical questions, (2) failing any 6 of the 23 questions, or (3) failing any 2 of the 4 items in the part B observational section. The inter-rater reliability for scoring section B was high (r = 0.95).
The Social Communication Disorders Checklist (SCDC) is a 12-item screening tool developed in the United Kingdom and completed by parents of children in the age group of 5–17 years (Skuse et al., 1997, 2005). Bölte et al. (2011) validated an adapted German version of the SCDC with a mixed sample composed of children with ASD, a clinical (behavioral or psychiatric) diagnosis, or TD. The tool was translated, back-translated, and adjusted by a native English speaker. The adapted version was pretested for understandability, although details or resulting alterations were not reported. Cronbach’s alphas for children with ASD, other psychiatric diagnoses, and TD were 0.78, 0.91, and 0.80, respectively. ROC analysis yielded excellent discrimination between children with ASD and TD but less so between children with ASD and other clinical diagnoses. Sensitivity for detecting ASD in populations including TD and clinical children were quite high but specificity values varied widely, with low values distinguishing ASD and clinical groups. Convergent validity demonstrated significant correlations with the ADI-R and the ADOS, as well as the SCQ-21. However, correlations were modest with the ADOS.
The SCQ is a 40-item, yes/no parent-report questionnaire for children over the age of 4 developed in the United Kingdom (Eaves et al., 2006; Rutter et al., 2003). Gau et al. (2011) collaborated with child psychiatrists and psychologists to create a Chinese version and validated it with children with ASD and non-affected siblings (N = 922). Translation and back-translation were conducted but no details were provided. Culture-relevant colloquial expressions were substituted, although the nature of these changes was not reported. Similar to the English language version, exploratory and confirmatory factor analyses yielded a 3-factor structure (social interaction, repetitive behaviors, and communication). However, differences with the UK version were evident in the specific items that cross-loaded on each factor. For example, some items that loaded on the communication subscale in the English version either loaded on the social interaction subscale or cross-loaded on to the communication and social interaction subscales in the Chinese version. The authors suggest that Taiwanese parents may consider gesturing (e.g. nodding, head shaking, pointing, imitation) and conversing behaviors (chit-chat) to be socially interactive rather than communicative. Additionally, since repetition and sameness are stressed in Chinese language, the verbal ritual item may have been interpreted as communicative rather than as a repetitive and stereotyped behavior. Test–retest correlations were moderate (ICC = 0.77–0.78) and internal consistency was acceptable to excellent (α = 0.73–0.91) for all three subscales. Concurrent validity between the subscales of the SCQ and the ADI-R demonstrated correlations ranging from slight to high (r = 0.11–0.65). The SCQ discriminated between children with and without ASD but not between autism and AS.
The Social Responsiveness Scale (SRS) is a measure of autistic traits in social reciprocal behavior developed in the United States, employing a 65-item parent or teacher rating scale for children in the age group of 4–18 months (Constantino and Gruber, 2002; Constantino et al., 2003). Bölte et al. (2008) assessed a German version of the SRS but offered little description of the adaptation procedures. Screening was applied to a large population of German youth who were typically developing or referred from clinics with ASD or other psychiatric diagnoses. Overall SRS scores for the entire normative sample and for children with ASD were lower in the normative German sample than in comparable US samples. Reliability of ratings for typically developing and clinic-referred children was greater than 0.91, and correlations between the SRS and different components of the ADOS, ADI-R, and SCQ indicated moderate to good convergent validity, ranging from 0.35 to 0.58. The authors also collected SRS data on siblings of youth with autism and found increased SRS scores in male siblings, providing what they described as the first cross-cultural validation of familial aggregation of autistic traits as measured by the SRS. ROC analyses of the total sample revealed steep declines in sensitivity and steep increases in specificity as the cut-off-score increased (from 56 to 100). The clinical and TD samples in the Bölte et al. (2011) study of the SCDC (see above) also completed the SRS. In analyses of only the ASD and TD groups, cut-offs of 75 and 85 yielded sensitivities of 0.80 and 0.74, respectively, and specificities were 1.0. For ASD versus the group diagnosed with other psychiatric disorders, cut-offs of 75 and 85 yielded sensitivities of 0.80 and 0.74, respectively, and specificities of 0.69 and 0.79, respectively.
Discussion
The goal of screening for autism is to identify children at risk, facilitate conversations between providers and parents, and further evaluation, leading to earlier identification and improved outcomes. Although there are advantages favoring the use of screening tools developed in the specific culture and language in which they will be implemented, the construction of such measures requires extensive resources and effort. As a result, the adaptation and use of tools created in other cultural/language contexts is increasing. In this systematic review of adaptations of autism screening tools, all of the studies reviewed were published since 2004, with the majority appearing between 2009 and 2012. The results revealed a wide variation in the rigor of the reported adaptation across studies, although the details of the methods were rarely reported. More comprehensive cultural adaptation procedures tended to result in more extensive modifications.
Forward-translation procedures meeting full guidelines were reported in only four of the 21 studies and almost half failed to provide information on this process at all. Three studies failed to include back-translation in their adaptation process, six indicated that it was done but provided no description, and only one study followed full guidelines. Review of the translation and/or back-translation and comparison with the original source was mentioned in 10 of the 21 articles, although, in four of these adaptations, only one person carried out the review. All six of the studies employing a committee review involved experts but the composition of the group was multidisciplinary in only two cases. Involvement of individuals from the general population was never mentioned. Pretesting was reported in only seven studies, although details (e.g. sample, recruitment, and analytic procedures) were provided in only two instances.
In 12 studies, adaptation led to changes in the tool, varying from minor alterations of a few items to more widespread adjustments to improve clarity and comprehension, retain conceptual equivalence, and conform to the language and culture. In general, the more rigorous/well-described the adaption process, the more likely that modifications were made to the tools. No modifications were reported for five studies and in these studies, forward-translation was never described, back-translation either did not occur or was not described, committee review was not discussed, and pretesting was only mentioned once, though the process was not described. In six of the 21 studies, minor modifications of the screening tool were made, such as substituting equivalents for hard-to-translate idiomatic and colloquial expressions or providing culturally relevant examples. These studies were more likely to report recommended components of adaptation; the forward-translation process was described (four studies), back-translation was mentioned (five studies) and described (three studies), and four studies submitted the translations for review. Major adaptations or changes in format were made in six studies and, except for one, translation or back-translation met recommended guidelines. Review was carried out in four cases and pretesting occurred in five cases. Besides changes in wording, these more extensive cultural adaptations revealed the need to construct items in a way that could be understood or deemed appropriate by the respondents, such as altering response alternatives or adding illustrations. However, even with such adaptations, we have found it necessary to use a trained bicultural/bilingual individual to administer the Spanish M-CHAT to produce adequate understanding (Linas et al., 2013). This is an issue that will need to be addressed in future studies involving culturally or linguistically distinct subgroups, or persons with lower or no literacy.
The studies reviewed varied greatly in the extent of psychometric analyses described. Nine studies reported indices of internal consistency, and few reported assessments of test–retest reliability and factor structure, where appropriate. Evidence for convergent or concurrent validity of the adapted tool was provided in only seven cases. Ten studies included comparisons of children and youth screening positive or diagnosed with ASD with unaffected individuals, with all reporting higher scores for the diagnosed or at-risk children. A small number of studies compared ASD samples to other conditions (e.g. other developmental disorders, behavioral health problems) or within the ASD spectrum (e.g. autism vs AS). In general, the tools differentiated among these conditions less well.
A total of eleven studies provided analysis of the diagnostic accuracy of the adapted instruments, although the samples differed markedly. Only four studies from Scandinavia and Spain examined the accuracy of adapted tools (ASSQ and M-CHAT) with population-based samples. In most cases, sensitivity and specificity of the instruments were determined in populations that included a mixture of clinical and population-based samples; however, such estimates may be unstable (Camp, 2006). In general, estimates of sensitivity and specificity, when provided, were sufficient to discriminate ASD children from typically developing children. The PPV value was often lower than the sensitivity estimate; however, a low PPV is to be expected in a screening instrument that assesses a disorder with low prevalence, such as ASD. Sensitivity and specificity are not dependent on the prevalence of a disorder because sensitivity is calculated using affected individuals and specificity is calculated using unaffected individuals. If a sample includes a higher proportion of people with autism (e.g. Hedley et al., 2010; Seif Eldin et al., 2008; Wong et al., 2004), then the PPV tends to be higher than studies using samples with a lower proportion of people with autism.
Most of the studies reporting psychometric characteristics noted differences between the adapted and original versions. The nature of the differences depended on the analytic strategies used. A handful of studies reported differences in mean scores (Bölte et al., 2011; Wakabayashi et al., 2007) and factor loadings (Gau et al., 2011). In six cases, the optimal cut-off score for identifying children with ASD varied from that suggested in the original tool. For instance, there was broad variation of ASSQ cut-off scores across Swedish, Finnish, Norwegian, and Chinese versions. Five adaptations reported differences from the original in the frequency with which particular items were endorsed, mostly in terms of lack of overlap of critical items on the M-CHAT with those identified in the US sample. Various reasons were proposed to account for the differences in psychometric properties of the adapted and original tools, including shared community experience or cultural differences in the meaning and perceived norms of certain behaviors, particularly social interactions, linguistic factors, and in parenting attitudes. However, evidence was not provided to confirm these possible explanations.
Although screening for ASDs is promoted by different organizations (mostly in the United States) and the adaptation of screening tools is increasing, there are concerns over the psychometric properties of autism screening tools (e.g. the possibility of missing cases and low “hit rates” in population-based studies). In addition, within-culture factors may limit the validity of a tool, such as variations in education levels, SES, literacy, knowledge of ASD, and experience of stigma, as suggested by some of the studies covered in this review. As a result, some argue that justification for the use of ASD screening tools for whole population-based screening is lacking (Le Couteur, 2003; Williams and Brayne, 2006). While there has been an increase in the number of published autism screening tools, there have not been equal efforts to validate the tools (Dixon et al., 2011). Rather than whole-population screening for autism, some suggest regular opportunities for “surveillance” by professionals to detect and respond rapidly to any developmental concerns at pre-school and school age. Given this approach, some suggest that autism scales should be geared to identify children with autism from an at-risk population, rather than the total population (Matson et al., 2011).
Ten studies used a diagnostic tool (e.g. ADOS, ADI-, R) to validate the adapted screening tool (Albores-Gallo et al., 2012; Bölte et al., 2008, 2011; Canal-Bedia et al., 2011; Gau et al., 2011; Mattila et al., 2009, 2012; Nygren et al., 2012; Posserud et al., 2009; Wong et al., 2004).Of these, six studies mentioned the use of a translated diagnostic tool (Bölte et al., 2008, 2011; Canal-Bedia et al., 2011; Gau et al., 2011; Mattila et al., 2009, 2012). The rationale for developing culturally appropriate screening tools extends to the development of culturally appropriate diagnostic tools. The use of culturally adapted diagnostic tools is necessary for reducing false positive and false negative diagnoses. Diagnostic tools that have been culturally adapted are also necessary to the process of validating screening tools to avoid conclusions based on circular reasoning—if the diagnostic tool used to validate a screening tool is not culturally valid, the cultural validity of the screening tool is also compromised.
The major limitation of this review is that evaluation of the cultural adaptation process depended on information provided in the published articles. Our review suggested that the more the adaptation process employed recommended procedures, the more likely that changes related to cultural/linguistic differences were made. However, it is possible that components or details of the cultural adaptation were conducted but not reported. Second, our focus in this systematic review was to identify studies where the adaptation of the screening tool occurred from one country to another. We did not review articles that reported adaptation of a screening tool for different groups (e.g. ethnic/racial, dominant language, SES) within a country.
Conclusion
This review points to the importance of a reasonable adaptation process of a tool in a new cultural and linguistic setting. Failure to adapt the existing screening tool could result in the unintended consequence of over- or under-identification of children at risk of autism. Working with culturally and linguistically diverse populations requires efforts beyond translation. The adaptations reviewed modified screening tools in the following ways: (1) added specific culturally relevant indicators of ASD, thus reducing the possibility that socially appropriate and reinforced behaviors within a particular culture could be misinterpreted as risk factors for ASD; (2) changed wording to avoid misinterpretation because of cultural norms related to non-verbal and social communication; (3) added examples to avoid confusion over item meaning; and (4) altered the format of the tools to align with response styles of respondents. A pretest or cognitive interview process almost always led to important changes in the translated measure, suggesting that these components of adaptation may be necessary to increase the validity of the tool. Many of the studies also reported differences in the psychometric properties between the original and translated tool. These results suggest that screening tools used in research, clinical, and educational settings should be adapted to the specific culture and language of the population screened.
Footnotes
Funding
This work was supported by the U.S. Department of Health and Human Services, Health Resources and Services Administration, Maternal and Child Health Research Program (R40 MC 20171).