The diagnosis of autism and autism spectrum disorder in low- and middle-income countries: Experience from Jamaica

Abstract

The administration requirements of the Autism Diagnostic Observation Schedule and the Autism Diagnostic Interview–Revised, widely used in high-income countries, make them less feasible for diagnosis of autism spectrum disorder in low- and middle-income countries. The flexible administration requirements of the Childhood Autism Rating Scale have resulted in its use in both high-income countries and low- and middle-income countries. This study examines the agreement between assessments using the Childhood Autism Rating Scale with those using the Autism Diagnostic Observation Schedule or Autism Diagnostic Observation Schedule, Second Edition and Autism Diagnostic Interview–Revised in Jamaica. Children aged 2–8 years (n = 149) diagnosed with autism by an experienced clinician using the Childhood Autism Rating Scale were re-evaluated using the Autism Diagnostic Observation Schedule and Autism Diagnostic Interview–Revised. The proportion diagnosed with autism spectrum disorder using the Autism Diagnostic Observation Schedule, Autism Diagnostic Observation Schedule, Second Edition, and Autism Diagnostic Interview–Revised was determined and mean domain scores compared using analysis of variance (ANOVA). The mean age was 64.4 (standard deviation = 21.6) months; the male:female ratio was 6:1. The diagnostic agreement of the Childhood Autism Rating Scale with the Autism Diagnostic Observation Schedule and Autism Diagnostic Observation Schedule, Second Edition was 100.0% and 98.0%, respectively. Agreement with the Autism Diagnostic Interview–Revised was 94.6%. Domain scores were highest for children with more severe symptoms (p < 0.01). Despite a high level of agreement of the Childhood Autism Rating Scale with the Autism Diagnostic Observation Schedule, Autism Diagnostic Observation Schedule, Second Edition, and Autism Diagnostic Interview–Revised, the Childhood Autism Rating Scale should be evaluated further with a broader range of autism spectrum disorder symptomatology, and by clinicians with varying experience before recommendation for use in low- and middle-income countries.

Keywords

Autism Diagnostic Interview–Revised Autism Diagnostic Observation Schedule autism spectrum disorder Childhood Autism Rating Scale Jamaica

Introduction

Autism spectrum disorder (ASD) is a lifelong neurodevelopmental disorder manifesting in early life, characterized by impairments in social communication and social interaction; and restrictive, repetitive patterns of behavior, interests or activities, as indicated in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5) (American Psychiatric Association, 2013). The DSM-5 definition of ASD encompasses and replaces Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR) definitions of autistic disorder or autism, Asperger’s disorder, and pervasive developmental disorder-not otherwise specified (PDD-NOS) (American Psychiatric Association, 2000). ASD is a serious public health concern with a high prevalence and major familial and societal economic impact; currently 1 in 68 children is affected in the United States (Christensen et al., 2016). This growing problem has resulted in intense interest in the development of valid and reliable instruments to facilitate early diagnosis and treatment.

In 2002, two assessment instruments were recommended by the National Institutes of Health (NIH) autism working group: the Autism Diagnostic Interview–Revised (ADI-R) (Lord et al., 1997) and the Autism Diagnostic Observation Schedule (ADOS) (Lord et al., 2000). The ADI-R is a 93-item, standardized, semi-structured interview administered to parents, which generates scores in four areas: A (Social Impairment), B (Communication for Verbal or Non-Verbal subjects), C (Restricted, Repetitive, and Stereotyped Patterns of Behavior), and D (Abnormality of Development Evident at or before 36 months) (Lord et al., 1997). The ADOS is a semi-structured observational assessment comprising activities that elicit behaviors consistent with autism. The ADOS is scored using an algorithm based on two domains, Social Interaction and Communication (Lord et al., 2000). The ADOS has four modules, the module chosen for administration is based on the verbal level of the individual being assessed. The 2012 Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) revision includes a new Toddler module for children 12–30 months, updated administration guidelines, a new algorithm, and new domains of Social Affect (Communication and Reciprocal Social Interaction) and Restricted and Repetitive Behaviors (Lord et al., 2012).

ADOS and ADOS-2 administration and scoring result in classification in one of three categories—Autism, or the milder autism spectrum classification, are assigned when an individual’s scores meet or exceed threshold cut-points in each domain as well as a summation of the two domain scores; non-spectrum is assigned when these criteria are not met (Lord et al., 2000). ADI-R administration and scoring result in classification in one of two categories: autism or non-autism (Lord et al., 1997). A classification of autism requires meeting or exceeding threshold cut-points in all domains.

The ADOS and ADI-R require substantial time and costs for training and certification, as well as administration. Administration times are 40–60 min and 90–150 min for the ADOS and ADI-R, respectively (Lord et al., 2012; Rutter et al., 2003). Test kits and booklets for administration to 100 children for either instrument cost over US$2000. These instruments are recognized as the gold standards to facilitate diagnosis of ASD in high-income countries (HICs).

Early diagnosis and treatment of ASD are also of concern to professionals in low- and middle-income countries (LMICs), particularly because there are limited resources to address the long-term disability that can occur without early intervention. Access to diagnostic instruments has been identified as a contributing factor to the diagnostic delay of 2–3 years that occurs in LMICs (Samms-Vaughan, 2014). The ADOS and ADI-R may not be feasible assessment instruments in LMIC because of high administration costs, few trained specialists, and large numbers of children awaiting assessment on long waiting lists. Nevertheless, it is important that ASD be diagnosed accurately in LMICs so that children with and without ASD can receive appropriate intervention.

The original Childhood Autism Rating Scale (CARS) (Schopler et al., 1980) is a diagnostic behavioral assessment scale with good psychometric properties (Schopler et al., 1988), for use in children 2 years and above. The CARS second edition (CARS-2) was developed to more accurately identify persons with higher cognitive functioning and has two separate forms, one for persons with estimated IQs equal to and below 79 and one for those with IQs equal to or above 80 (Schopler et al., 2010).

Both the CARS and CARS-2 have 15 domains: relating to people; imitation; emotional response; body use; object use; adaptation to change; visual response; listening response; taste, smell, and touch response; fear or nervousness; verbal communication; non-verbal communication; activity level; level and consistency of intellectual response and general impressions (Chlebowski et al., 2010). Historical reports and direct observations of children’s behavior are used to rate each domain on a scale from 1 to 4, with 1 indicating the behavior of typically developing children and 4 indicating the greatest impairment associated with autism (Schopler et al., 1980). A CARS total score of ⩾30 is consistent with a DSM-IV diagnosis of autism, with 30–36.5 rated mild to moderate autism and ⩾37 rated severe autism. Although the PDD-NOS classification was not included when the original CARS was developed, subsequent research identified the cutoff score to distinguish autism from the milder PDD-NOS to be 32 for 2 year olds and 30 for 4 year olds, while that to distinguish autism and PDD-NOS from other developmental disorders and typical development was 25 for both ages (Chlebowski et al., 2010).

The CARS can be administered by professionals with prior licensing, certification, and experience in assessment of children and ASD; its administration and scoring time is 5–10 min (Schopler et al., 2010) following a typical history taking exercise. Test kits and booklets for administration to 100 children cost less than US$350.

The CARS has therefore become a tool of choice for both HICs and LMICs, including Saudi Arabia (Blaurock-Busch et al., 2012), Australia (Williams et al., 2013), Indonesia (Winarni et al., 2013), the Republic of Korea (Yim et al., 2013), and the United States (Geier et al., 2012). Recently, the CARS-2 (Schopler et al., 2010) was chosen for use in a low resource setting in Tanzania (Harrison et al., 2014).

Previous studies have evaluated agreement between the CARS and the ADOS and ADI-R. In a study of 83 individuals 2–34 years old in Israel, agreement between the CARS and the ADI-R was 85.7% (Pilowsky et al., 1998). In a clinical sample of children 22–114 months in Iceland, the CARS identified 36 of 54 children with autism (66.7%), while the recommended ADI-R three behavior domain cutoff identified only 18 of the 36 (Saemundsen et al., 2003). Observed agreement between the CARS and ADI-R was 66.7% when the threshold was met or exceeded for all three behavior domains, but was 83.3% and 94.4% when the threshold was met or exceeded for any two or any one of the behavior domains, respectively. Children diagnosed by both the CARS and the ADI-R had more severe behavior symptoms and cognitive deficits (Saemundsen et al., 2003). In a clinical sample of young children 18–32 months in the United States, there was significant agreement between ADOS classifications and clinical judgment (κ = 0.593, p < 0.001), and between ADOS and CARS classifications (κ = 0.619, p < 0.001), but not between ADI-R classifications and clinical judgment (κ = 0.153, p = 0.176) or between ADI-R and CARS classifications (κ = 0.095, p = 0.486) (Ventola et al., 2006). It was suggested that very young children may not readily manifest the repetitive and stereotyped behaviors that form one of the three behavior domains required for an autism classification by the ADI-R (Ventola et al., 2006). These studies have been primarily conducted in HICs.

The ease of administration of the CARS also resulted in its use at Jamaica’s main university-based referral center for the diagnosis of ASD; however, it has never been evaluated for its diagnostic accuracy or feasibility in Jamaica. The language of the ADI-R could potentially have terms that are culturally insensitive or difficult for some Jamaican parents to understand, and the specific social situations selected for use in the ADOS may not be culturally relevant. The purpose of this article is to determine the accuracy and feasibility of the CARS for diagnosis of ASD in Jamaica, a LMIC, by examining the agreement between diagnostic classifications made by the CARS and those made by the ADI-R, ADOS, and ADOS-2. As this study was the first to objectively evaluate these instruments in Jamaica, and was not designed for the evaluation of cultural adaptation in Jamaica, only language changes were made, where culturally relevant, to ensure that the concepts being examined were understood by the Jamaican population. No changes were made to ADOS/ADOS-2 administration procedures. Translation and cultural adaptation of instruments often build on the experience obtained through prior use in a population and involve data collection and analytical procedures specific to validation (Smith et al., 2016).

Methods

The Epidemiological Research on Autism in Jamaica (ERAJ) project investigated whether environmental exposures to heavy metals play a role in ASD through an age- and sex-matched case–control study that enrolled children between December 2009 and May 2012. The criteria for enrollment in ERAJ as a case were Jamaican birth, 2–8 years of age at enrollment, and a clinical diagnosis of autism. The cases were recruited from the University of the West Indies’ (UWI) Jamaica Autism Database (JAD) and comprised a clinical sample of over 400 referred children, who were assessed as having autism by a developmental pediatrician with over 20 years’ experience, using the CARS. Some 261 children from the JAD met criteria for the ERAJ study and were invited to participate. The children reported on in this article were those identified as cases in the ERAJ study. Details regarding recruitment and screening have been reported (Rahbar et al., 2012, 2013). All participating parents provided written informed consent. The study was approved by the Institutional Review Boards of the University of Texas Health Science Center at Houston and the UWI.

The ADOS and the ADI-R were administered by a trained senior psychologist. Similar to the study by Ventola et al. (2006), the psychologist was not blinded and was aware of the experimental study being undertaken (Ventola et al., 2006). Because the ADOS test items have not changed with the ADOS-2, ADOS administration (modules 1 or 2) was used to score the algorithms for both the ADOS and ADOS-2 (Lord et al., 2000). Established cutoff points were utilized for the ADOS, ADOS-2, and ADI-R (Lord et al., 1997). Total ADI-R scores were calculated by summation of all four areas scored; previous researchers have used summed domain scores for correlation analysis (Pilowsky et al., 1998). The ADOS and ADOS-2 domain scores were also summed.

Statistical analysis

Of 261 eligible children, 149 agreed to participate in the ERAJ study, resulting in a refusal rate of 42.9%. Refusals were primarily due to the blood samples required by ERAJ, the length of time required to complete assessments (4–5 h), and the absence of an incentive as children already had a diagnosis. Descriptive analyses were conducted to assess the demographic characteristics of the 149 participating children. CARS scores were available for 143 children; clinical records were not located for six children. Demographic characteristics for the sample refusing participation were also determined.

We assessed the proportion of children for whom there was diagnostic agreement using the ADOS, ADOS-2, and ADI-R. Correlation coefficients were determined for total CARS and ADOS, ADOS-2, and ADI-R scores. Using ANOVA, mean differences in CARS, ADI-R, and ADOS scores of those children who were classified as having autism versus no autism by the ADI-R and autism, autism spectrum or non-spectrum by the ADOS, and ADOS-2 algorithms were determined. All statistical tests were performed at a 5% level of significance using SAS version 9.4 (SAS Institute Inc., 2013).

Results

Of 149 children, 128 (85.9%) were male with a male:female ratio of approximately 6:1; 92.6% of children were Afro-Caribbean. Children’s ages ranged from 25 to 107 months with a mean of 64.4 (standard deviation (SD) = 21.6) months. Further demographic information on participants is given in Table 1. Respondents providing historical information that facilitated CARS diagnosis were 47.5% mothers, 6.8% fathers, and 42.4% both parents. Respondents for the ADI-R were 77.2% mothers, 6.7% fathers, and 13.4% both parents, with other relatives being the additional responders. The mean total CARS score was 36.9 (SD = 3.7), with a median of 36.5 and a range of 30–48.5. A total of 75 children (52.4%) were classified as having mild to moderate autism with CARS scores of 30–36.5; 68 children (47.6%) had CARS scores of ⩾37 and were classified as having severe autism. The mean age of the sample refusing participation was 68.4 (SD = 19.2) months, the male:female ratio was 5:1, and the mean total CARS score was 35.7 (SD = 4.2).

Table 1.

Characteristics of children and their parents (N = 149).

Variables	Categories	ASD case No. (%)
Child’s sex	Male	128 (85.9)
Child’s sex	Female	21 (14.1)
Age of child (months)	24 ⩽ age < 48	43 (28.9)
	48 ⩽ age < 72	58 (38.9)
	72 ⩽ age < 107	48 (32.2)
Maternal education^a	Up to high school	74 (49.7)
Maternal education^a	Beyond high school	75 (50.3)
Paternal education^a	Up to high school	50 (34.2)
Paternal education^a	Beyond high school	96 (65.8)

ASD: autism spectrum disorder.

At child’s birth.

The mean time between CARS and ADOS/ADI-R assessments was 17.4 (SD = 20.1) months with a range of 0–77 months. All 149 children (100%) assessed as having autism by the CARS were also assessed as having autism or autism spectrum by the ADOS, 98.0% (146/149) were assessed as having autism or autism spectrum by the ADOS-2, and 94.6% (141/149) were assessed as having autism by the ADI-R. We found significant (p < 0.005) but moderate correlation coefficients between the CARS and ADOS (r = 0.34), ADOS-2 (r = 0.42), and ADI-R total scores (r = 0.35). The correlation coefficient was highest between the CARS and ADOS-2.

Of the 10 children for whom there was no diagnostic agreement between the CARS and the ADI-R or ADOS-2, seven did not reach the threshold in at least two domains of the ADI-R, two did not reach the threshold for the ADOS-2, and one did not reach the threshold for both the ADOS-2 and ADI-R. The ADI-R domains that were most frequently below threshold were social impairment (n = 8) and restrictive, repetitive behaviors (n = 8). Four children did not reach threshold for verbal communication. Restrictive, repetitive, and stereotyped behavior was the only ADOS-2 domain below threshold.

All 141/149 (94.6%) children classified with autism by the ADI-R were also classified as having autism or autism spectrum by the ADOS. On the ADOS-2, 139/141 (98.6%) were assessed as having autism or autism spectrum, with 133/139 (95.7%) having autism. Table 2 shows the ADOS, ADOS-2, and ADI-R classification of children diagnosed with autism on the CARS.

Table 2.

ADOS, ADOS-2, and ADI-R classification of children diagnosed with autism on the CARS (n = 149).

ADOS	ADI-R				Total
	Autism		No autism
	N	%	N	%
ADOS
Autism	125	83.9	3	2.0	128
Autism spectrum	16	10.7	5	3.4	21
Non-spectrum	0	0	0	0	0
ADOS-2
Autism	133	89.3	6	4.0	139
Autism spectrum	6	4.0	1	0.7	7
Non-spectrum	2	1.3	1	0.7	3
Total	141	94.6	8	5.4	149

ADOS: Autism Diagnostic Observation Schedule; ADOS-2: Autism Diagnostic Observation Schedule, Second Edition; ADI-R: Autism Diagnostic Interview–Revised; CARS: Childhood Autism Rating Scale.

Mean scores for ADI-R, ADOS, and ADOS-2 domains by ADOS and ADOS-2 classifications are displayed in Table 3. There were significant differences in mean ADI-R domain scores between children classified with autism and those classified with autism spectrum by the ADOS (Table 3). Using the ADOS-2 classification, significance was attained in the Social Impairment (p < 0.01) and Non-Verbal Communication (p < 0.01) ADI-R domains. Total ADI-R score, ADOS, and ADOS-2 domain scores were significantly higher for children classified as having autism by the ADOS compared to those classified as having autism spectrum (p < 0.01) and non-spectrum (p < 0.01).

Table 3.

Comparison of ADI-R, ADOS, and CARS mean total and domain scores by ADOS and ADOS-2 classification.

Instruments and domains	ADOS					ADOS-2
	Autism (n = 128)		Autism spectrum (n = 21)		p	Autism (n = 139)		Autism spectrum (n = 7)		Non-spectrum (n = 3)		p
	M	SD	M	SD	p	M	SD	M	SD	M	SD	p
ADI-R domains
Social impairment	18.96	5.16	11.71	4.41	<0.01	18.29	5.51	15.00	5.26	8.67	3.05	<0.01
Communication (verbal)	5.02	7.47	11.48	4.54	<0.01	5.55	7.46	11.29	6.75	11.00	1.00	0.07
Communication (non-verbal)	8.15	6.10	0.33	1.53	<0.01	7.47	6.25	1.57	4.16	0	0	<0.01*
Restrictive/repetitive/stereotyped behavior	3.38	1.52	1.95	1.96	<0.01	3.22	1.61	2.71	1.38	2.33	4.04	0.50
Abnormality evident ⩽ 36 months	4.39	0.81	3.76	1.14	<0.01	4.34	0.88	3.86	0.38	3.67	1.53	0.17
ADI-R total	39.89	7.88	29.24	7.87	<0.01	38.86	8.49	34.43	9.64	25.67	4.73	0.02
ADOS domains
Communication	5.94	1.53	3.62	0.80	<0.01	5.71	1.64	4.57	1.72	3.67	0.58	0.02
Social interaction (SI)	10.06	1.94	5.86	1.06	<0.01	9.76	2.14	5.86	1.35	4.67	0.58	<0.01
Communication + SI total	15.98	2.49	9.48	1.08	<0.01	15.45	2.98	10.43	2.70	8.33	0.58	<0.01
Play	2.79	1.07	1.14	0.73	<0.01	2.65	1.14	1.57	1.13	0.67	0.58	<0.01
Stereotyped behaviors/restricted interests	2.92	1.41	1.24	0.94	<0.01	2.81	1.43	1.43	0.79	0	0	<0.01*
ADOS-2 domains
Social affect (SA)	14.95	2.56	8.71	1.79	<0.01	14.53	2.87	8.29	2.21	6.67	0.58	<0.01
Restricted/repetitive behavior (RRB)	3.74	1.68	1.43	1.03	<0.01	3.58	1.74	1.57	0.98	0.33	0.57	<0.01
SA + RB total	18.70	3.42	10.14	2.52	<0.01	18.10	3.87	9.86	3.18	7.00	0	<0.01*
	Autism (n = 123)		No autism (n = 20)		p	Autism (n = 134)		Autism spectrum (n = 6)		Non-spectrum (n = 3)		p
	M	SD	M	SD	p	M	SD	M	SD	M	SD	p
CARS domains
Relating to people	2.75	0.55	2.65	0.36	0.43	2.74	0.53	2.67	0.61	2.67	0.29	0.92
Emotional response	2.61	0.50	2.28	0.70	<0.01	2.58	0.57	2.33	0.75	2.33	0.58	0.50
Imitation	2.75	0.61	2.45	0.76	<0.05	2.72	0.62	2.58	0.74	2.50	1.32	0.76
Body use	2.65	0.58	2.28	0.70	<0.01	2.63	0.57	2.58	0.49	1.00	0.00	<0.01
Object use	2.32	0.82	2.08	0.91	0.22	2.29	0.84	2.67	0.41	1.33	0.58	0.07
Adaptation to change	2.04	0.86	1.70	0.95	0.11	2.02	0.88	1.83	0.82	1.17	2.89	0.23
Listening response	2.23	0.70	2.15	0.71	0.63	2.24	0.69	1.58	0.66	2.50	0.50	0.06
Taste/smell/touch response	2.30	0.74	2.14	0.81	0.40	2.26	0.76	2.58	0.58	2.27	0.25	0.58
Visual response	2.80	0.43	2.65	0.61	0.17	2.80	0.45	2.33	0.61	3.00	0.00	<0.04
Fear/nervousness	1.62	0.79	1.62	0.76	0.97	1.62	0.78	1.67	0.88	1.50	0.87	0.96
Verbal communication	3.17	0.34	3.00	0.28	0.04	3.15	0.34	3.00	0.00	3.17	2.89	0.56
Activity level	2.33	0.68	2.52	0.57	0.22	2.36	0.67	2.42	0.49	2.00	1.00	0.64
Non-verbal communication	2.37	0.58	1.95	0.48	<0.01	2.34	0.58	1.83	0.26	1.83	0.29	<0.04
Level/consistency of intellectual response	2.40	0.42	3.52	4.36	<0.01	2.55	0.42	2.50	0.32	2.50	0.00	1.00
General impression	3.00	0.45	2.78	0.38	<0.05	2.97	0.45	2.75	0.27	3.00	0.00	0.48
Total CARS score	37.52	3.61	34.75	3.42	0.004	37.06	3.65	35.33	3.72	33.17	2.56	0.11

ADOS: Autism Diagnostic Observation Schedule; ADOS-2: Autism Diagnostic Observation Schedule, Second Edition; ADI-R: Autism Diagnostic Interview–Revised; CARS: Childhood Autism Rating Scale; M: mean; SD: standard deviation.

Since standard deviation in the Non-spectrum (n = 3) group is equal to zero, we calculated the p-value by comparing the means in the other two groups (i.e. Autism and Autism spectrum groups).

Similarly, mean scores for ADI-R, ADOS, and ADOS-2 domains by ADI-R classifications of children having autism or no autism are presented in Table 4. On the ADI-R, mean domain scores were significantly higher for children classified with autism compared to those without autism with respect to social impairment (p < 0.01), non-verbal communication (p = 0.02), and repetitive behaviors (p < 0.01). ADOS and ADOS-2 mean domain scores were higher for children classified as having autism by the ADI-R compared to those who were not so classified. Most, but not all, mean domain score differences achieved statistical significance. Mean total ADI-R, ADOS, and ADOS-2 scores were significantly higher for children classified as having autism by the ADI-R (p < 0.01).

Table 4.

Comparison of mean total and domain ADI-R, ADOS, and ADOS-2 scores by ADI-R classification (n = 149).

	ADI-R				p
	Autism (n = 141)		No autism (n = 8)
	M	SD	M	SD
ADI-R domains
Social impairment	18.55	5.16	7.25	1.67	<0.01
Communication (verbal)	5.96	7.62	5.25	4.33	0.79
Communication (non-verbal)	7.34	6.31	1.88	3.48	0.02
Restrictive/repetitive/stereotyped behavior	3.29	1.62	1.13	0.64	<0.01
Abnormality evident ⩽ 36 months	4.32	0.88	4.00	0.92	0.32
ADI-R total	39.46	7.60	19.50	3.38	<0.01
ADOS domains
Communication	5.69	1.66	4.25	1.04	0.02
Social interaction (SI)	9.58	2.30	7.50	2.51	0.02
Communication + SI total	15.26	3.17	11.75	3.28	<0.01
Play	2.60	1.17	1.88	0.99	0.09
Stereotyped behaviors/restricted interests	2.74	1.48	1.63	0.92	0.03
ADOS-2 domains
Social affect (SA)	14.21	3.22	11.63	3.62	0.03
Restricted/repetitive behavior (RB)	3.51	1.79	1.75	1.04	<0.01
SA + RB total	17.72	4.36	13.38	4.37	<0.01

ADOS: Autism Diagnostic Observation Schedule; ADOS-2: Autism Diagnostic Observation Schedule, Second Edition; ADI-R: Autism Diagnostic Interview–Revised; M: mean; SD: standard deviation.

Discussion

All Jamaican children who were classified as having autism using the CARS were also classified as having autism or autism spectrum using the ADOS. Using the ADOS-2 algorithm, 98.0% were assessed as having autism or autism spectrum, and using the ADI-R, 94.6% were assessed as having autism. As the CARS did not include the older DSM-IV category of PDD-NOS (considered to reflect the milder end of the autism spectrum) as a diagnostic category, our sample did not include children with milder symptoms. However, a strength of the Jamaican sample is that there is fairly equal representation of age categories (2–3, 4–5, 6–8 years), socioeconomic status (SES) as measured by levels of maternal education (up to high school, beyond high school), and autism severity as defined by the CARS (mild to moderate, severe).

Agreement of the CARS with other instruments, rather than concordance, was evaluated; assessment of concordance typically includes children with diagnoses of non-spectrum, autism spectrum, and autism as on the ADOS/ADOS-2. Sample selection likely explains the high agreement between the ADOS and ADI-R (94.6%), and between the ADOS-2 and the ADI-R (93.9%). In a clinical sample of 5- to 8-year-old children with intellectual disability in the Netherlands, the agreement between the ADOS and the ADI-R was 83.4% for children with autism and 81.0% for ASD (PDD) (De Bildt et al., 2004).

Despite sample limitations, some of our findings follow patterns observed in other studies. More children were classified with autism or autism spectrum by the ADOS than with autism by the ADI-R. The study by Ventola et al. (2006) documented earlier indicated that the ADOS, CARS, and clinical judgment were in agreement with each other for toddlers, but were not in agreement with the ADI-R (Ventola et al., 2006). Both the CARS cutoff of 30 and stringent ADI-R criteria (meeting threshold in all three behavior domains) allow only a diagnosis of autism and exclude some children who would be classified as the milder autism spectrum or the previously used classification of PDD-NOS (De Bildt et al., 2004). The difference in agreement may also be explained by the ADI-R relying on parents’ recollection of behaviors of children, while the CARS integrates history and direct observation.

We also found that the ADI-R, ADOS, and ADOS-2 mean scores for many domains were higher for children classified as having autism or autism spectrum versus no autism. These results are consistent with those reported by Saemundsen et al. (2003) who showed that ADI-R mean domain and total scores were higher for those classified as having autism than not having autism using the CARS and ADI-R (Saemundsen et al., 2003).

There are some specific features to the administration of the CARS, ADOS, and ADI-R in Jamaica that may have impacted our results. In this study, the two domains of the ADI-R most frequently below threshold for diagnosis were the age and social interaction criteria. Notably, the majority of children not meeting the age threshold (80%) were more than 5 years old, and 60% were in the age range of just below 6 years to 8 years at ADI-R administration. The ADI-R has a specific focus on behavior at 4–5 years; a greater proportion of these items contribute to the social interaction domain. Jamaican parents may not have accurately recalled the age of first symptoms or social interaction symptoms at 4–5 years, given the age at which the ADI-R was administered. This finding has practical implications for the use of the ADI-R in Jamaica for older children, especially given the recognized delay between parental concern and diagnosis (Samms-Vaughan, 2014).

Recent analysis from the Study to Explore Early Development (SEED) found that ADI-R and ADOS diagnoses had 77% agreement (438/584). SEED ASD criteria, which included ADOS criteria in association with more relaxed ADI-R criteria, increased the agreement to 86% (500/584), but diagnosed 62 fewer children than clinical judgment (Wiggins et al., 2015). The relaxed criteria excluded the age domain and required either the social or communication domain to be met along with sub-threshold scores in the other domain or the behavior domain. Application of the SEED criteria would have resulted in 3 of the 10 children below threshold being classified as ASD.

Many children whose original autism diagnosis on the CARS was not in agreement with the ADOS and ADI-R had higher language functioning. ADOS and ADI-R questions display several language and cultural nuances specific to the United States or other developed countries. Research has shown reduced diagnosis of ASD in certain racial and ethnic groups (Mandell et al., 2009). Researchers have suggested methods for more culturally sensitive assessment for detection of ASD in racial and ethnic groups that experience disparities (El-Ghoroury and Krackow, 2012). In our experience, the language and social situations included in the ADI-R sometimes had to be adjusted to be more culturally relevant for Jamaicans. For example, the additional questions enquiring about change in personal routine include switching from one pair of mittens to another or from winter to summer clothing. In the ADOS, Jamaican children of lower SES who may have limited experience with birthday parties may have been adversely scored in that section. The CARS is less structured and more flexible and does not require children to be placed in specific social situations, but integrates history and observation of the child’s typical social experiences.

According to Robertson, the ADOS and ADI-R measure different manifestations of ASD symptomatology using differing methods; therefore, these instruments should be used together (Robertson et al., 1999). The recommendation for both instruments to be used may be problematic in Jamaica. Apart from cost, the time for administration of both may be a disincentive to parents; administration time was one of the reasons for reduced participation in the ERAJ study. Moreover, instruments are meant to facilitate diagnosis, but expert clinical observation is considered more detailed and thorough. This was supported by the SEED study, which reported that diagnostic instruments alone cannot replace informed clinical judgment in diagnosing ASD (Wiggins et al., 2015).

This study indicates that a classification of autism using the CARS, administered by an experienced professional, is highly substantiated by the ADOS and the ADI-R. In limited resource settings, as occurs in LMICs, there are typically few diagnostic centers operated by the few highly trained and experienced clinicians present in a country. This lack of resources precludes access to diagnosis, early intervention, and treatment for the majority of children. Use of the CARS with its shorter training and administration time, lower costs, and fewer cultural concerns may allow more professionals across LMICs to be trained in the diagnosis of ASD and may facilitate early diagnosis and treatment.

Our study had some limitations. First, our sample was limited to children assessed with autism by the CARS and did not include the full spectrum of symptomatology; findings and conclusions are therefore restricted to this group. Specifically those with milder symptomatology, who may have had CARS scores of 25–29.5, were not included, and the usefulness of the CARS for diagnosis of this group is unknown. The absence of a non-ASD comparison group also precluded determination of specificity and concordance across instruments. Additionally, the CARS was administered by a single highly experienced professional. Instrument-related limitations included the absence of scientifically, culturally-adapted CARS, ADOS, and ADI-R instruments for the population. Culturally adapted instruments might have had different norms and cutoff points and influenced diagnostic classifications and the agreement reported (Norbury and Sparks, 2013). Finally, there was a prolonged time between administration of the CARS and other instruments (mean of 17 months); responses to interventions received could have impacted scoring on other instruments. However, the CARS classification has been shown to be stable over time (Kleinman et al., 2008), and as with many LMICs, few children receive comprehensive intervention services in Jamaica.

Conclusion

In this study, we have shown that for a very high proportion of children, a diagnosis of autism on the CARS was in agreement with assessment by the ADOS and ADI-R. This provides preliminary support for the use of this easily administered, cost- and time-efficient instrument in LMIC settings. The use of the CARS by clinicians of varying levels of experience with ASD and with a broader range of ASD symptomatology, particularly milder symptoms, should be further evaluated before recommendations for its use in LMICs are made. If the ADOS and ADI-R are to be used in LMIC, culturally valid adjustments should be considered.

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is co-funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and the National Institutes of Health Fogarty International Center (NIH-FIC) by a grant [R21HD057808], as well as National Institute of Environmental Health Sciences (NIEHS) by a grant [R01ES022165] awarded to University of Texas Health Science Center at Houston. We also acknowledge the support provided by the Center for Clinical and Translational Sciences (CCTS), which is mainly funded by the NIH Centers for Translational Science Award (NIH CTSA) grant [UL1 RR024148], awarded to University of Texas Health Science Center at Houston in 2006 by the National Center for Research Resources (NCRR) and its renewal [UL1 TR000371] by the National Center for Advancing Translational Sciences (NCATS). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NICHD or the NIH-FIC or NIEHS or the NCRR or the NCATS.

References

American Psychiatric Association (2000) Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR). 4th ed. Text revision. Washington, DC: American Psychiatric Publishing.

American Psychiatric Association (2013) Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Publishing.

Blaurock-Busch

Amin

Dessoki

. (2012) Toxic metals and essential elements in hair and severity of symptoms among children with autism. Maedica 7(1): 38–48.

Chlebowski

Green

Barton

. (2010) Using the childhood autism rating scale to diagnose autism spectrum disorders. Journal of Autism and Developmental Disorders 40(7): 787–799.

Christensen

Baio

Van Naarden

.; Centers For Disease Control and Prevention (CDC) (2016) Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2012. MMWR Surveillance Summaries 65(3): 1–23.

De Bildt

Sytema

Ketelaars

. (2004) Interrelationship between Autism Diagnostic Observation Schedule-Generic (ADOS-G), Autism Diagnostic Interview-Revised (ADI-R), and the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) classification in children and adolescents with mental retardation. Journal of Autism and Developmental Disorders 34(2): 129–137.

El-Ghoroury

Krackow

(2012) Enhancing the identification of autism spectrum disorders via a model of culturally sensitive childhood assessment. Professional Psychology: Research and Practice 43(3): 249–255.

Geier

Kern

King

. (2012) Hair toxic metal concentrations and autism spectrum disorder severity in young children. International Journal of Environmental Research and Public Health 9(12): 4486–4497.

Harrison

Zimak

Sheinkopf

. (2014) Observation-centered approach to ASD assessment in Tanzania. Intellectual and Developmental Disabilities 52(5): 330–347.

10.

Kleinman

Robins

Ventola

. (2008) The modified checklist for autism in toddlers: a follow-up study investigating the early detection of autism spectrum disorders. Journal of Autism and Developmental Disorders 38(5): 827–839.

11.

Lord

Pickles

Mclennan

. (1997) Diagnosing autism: analyses of data from the Autism Diagnostic Interview. Journal of Autism Developmental and Disorders 27(5): 501–517.

12.

Lord

Risi

Lambrecht

. (2000) The autism diagnostic observation schedule-generic: a standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders 30(3): 205–223.

13.

Lord

Rutter

Dilavore

. (2012) Autism Diagnostic Observation Schedule (ADOS-2). 2nd ed. Torrance, CA, USA: Western Psychological Services.

14.

Mandell

Wiggins

Carpenter

. (2009) Racial/ethnic disparities in the identification of children with autism spectrum disorders. American Journal of Public Health 99(3): 493–498.

15.

Norbury

Sparks

(2013) Difference or disorder? Cultural issues in understanding neurodevelopmental disorders. Developmental Psychology 49(1): 45–58.

16.

Pilowsky

Yirmiya

Shulman

. (1998) The Autism Diagnostic Interview-Revised and the Childhood Autism Rating Scale: differences between diagnostic systems and comparison between genders. Journal of Autism and Developmental Disorders 28(2): 143–151.

17.

Rahbar

Samms-Vaughan

Ardjomand-Hessabi

. (2012) The role of drinking water sources, consumption of vegetables and seafood in relation to blood arsenic concentrations of Jamaican children with and without autism spectrum disorders. Science of the Total Environment 433: 362–370.

18.

Rahbar

Samms-Vaughan

Loveland

. (2013) Seafood consumption and blood mercury concentrations in Jamaican children with and without autism spectrum disorders. Neurotoxicity Research 23(1): 22–38.

19.

Robertson

Tanguay

L’Ecuyer

. (1999) Domains of social communication handicap in autism spectrum disorder. Journal of American Academy of Child and Adolescent Psychiatry 38(6): 738–745.

20.

Rutter

Lord

(2003) Autism Diagnostic Interview-Revised (ADI-R). Los Angeles, CA: Western Psychological Services.

21.

Saemundsen

Magnusson

Smari

. (2003) Autism Diagnostic Interview-Revised and the Childhood Autism Rating Scale: convergence and discrepancy in diagnosing autism. Journal of Autism and Developmental Disorders 33(3): 319–328.

22.

Samms-Vaughan

(2014) The status of early identification and early intervention in autism spectrum disorders in lower- and middle-income countries. International Journal of Speech-Language Pathology 16(1): 30–35.

23.

SAS Institute Inc. (2013) SAS® 9.4. Cary, NC: SAS Institute Inc.

24.

Schopler

Reichler

Devellis

. (1980) Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS). Journal of Autism and Developmental Disorders 10(1): 91–103.

25.

Schopler

Reichler

Rochen Renner

(1988) The Childhood Autism Rating Scale. Los Angeles, CA: Western Psychological Services.

26.

Schopler

Van Bourgondien

Wellman

. (2010) Childhood Autism Rating Scale. Second edition (CARS2): Manual. Los Angeles, CA: Western Psychological Services.

27.

Smith

Malcolm-Smith

De Vries

(2016) Translation and cultural appropriateness of the Autism Diagnostic Observation Schedule-2 in Afrikaans. Autism. Epub ahead of print 25 May. DOI: 10.1177/1362361316648469.

28.

Ventola

Kleinman

Pandey

. (2006) Agreement among four diagnostic instruments for autism spectrum disorders in toddlers. Journal of Autism and Developmental Disorders 36(7): 839–847.

29.

Wiggins

Reynolds

Rice

. (2015) Using standardized diagnostic instruments to classify children with autism in the study to explore early development. Journal of Autism and Developmental Disorders 45(5): 1271–1280.

30.

Williams

Perkins

Wheeler

. (2013) Can questions about social interaction correctly identify preschool aged children with autism? Journal of Paediatrics and Child Health 49(2): E167–E174.

31.

Winarni

Utari

Mundhofir

. (2013) Fragile X syndrome: clinical, cytogenetics and molecular screening among autism spectrum disorder children in Indonesia. Clinical Genetics 84(6): 577–580.

32.

Yim

Kim

Park

. (2013) Assessment of the correlation between TIMP4 SNPs and schizophrenia and autism spectrum disorders. Molecular Medicine Reports 7(2): 489–494.