Development and psychometric evaluation of a psychosocial quality-of-life questionnaire for individuals with autism and related developmental disorders

Abstract

This study investigated the psychometric properties of the Child and Family Quality of Life scale, a measure of psychosocial quality of life in those with autism and related developmental disorders. Parents of 212 children suspected of autism spectrum disorder completed the Child and Family Quality of Life prior to a diagnostic evaluation. Results indicated that the Child and Family Quality of Life measured six unique quality-of-life constructs (child, family/caregiver, financial, external support, partner relationship, and coping), had good reliability across score ranges and exhibited expected patterns of convergent validity. Caregivers of autism spectrum disorder–affected children reported reduced family quality of life prior to the time of diagnosis relative to caregivers of children with other developmental disabilities. The Child and Family Quality of Life is a brief, reliable measure for assessing psychosocial quality of life in families affected by developmental disability. This study is the first to demonstrate impairments in family quality of life early in the developmental course of autism spectrum disorder, prior to formal diagnosis. In addition to traditional child-focused intervention strategies, families with autism spectrum disorder–affected children require early, broad intervention strategies that positively impact the whole family.

Keywords

autism behavior problems child functioning developmental disabilities family functioning quality of life

Introduction

Autism spectrum disorder (ASD) and related developmental disorders (DDs)—including intellectual disability, attention-deficit/hyperactivity disorder (ADHD), speech/language disorder, specific learning disability, and developmental coordinator disorder—are prevalent and impairing (Boulet et al., 2009; Boyle et al., 1994). Recent estimates of ASD prevalence suggest that nearly 2% of children meet symptom criteria, with a dramatic increase in prevalence over the last two decades (Centers for Disease Control and Prevention, 2014). Prevalence rates for the broader DD group have increased slightly over time (Boyle et al., 2011), but the impact on society remains substantial. Approximately one in six children aged 3–17 months are diagnosed with a DD, and many children show substantial functional impairment (Boulet et al., 2009). Not surprisingly, functional impairments and behavior problems in children with ASD and other DDs often result in broader psychosocial impact, reducing the quality of life (QoL) of the entire family.

Over the last two decades, attention has grown toward understanding the broader consequences of ASD/DD, including measurement of child and family QoL in ASD/DD populations (Hu et al., 2011; Lucas-Carrasco and Salvador-Carulla, 2012). Existing literature has identified impairments in several aspects of QoL (e.g. relationships with others, coping skills, and sibling QoL) in individuals and families affected by DD (Gardiner and Iarocci, 2012; Hoefman et al., 2014; Lee et al., 2008; Rana and Mishra, 2015). QoL appears to be more strongly negatively impacted in individuals, caregivers, and families affected by ASD relative to typical children (De Vries and Geurts, 2015; Suzumura, 2015) and other forms of DDs (Abbeduto et al., 2004; Lee et al., 2008; Vohra et al., 2014), with persistence into adulthood for even high-functioning ASD-affected individuals (Lin, 2014). It is not yet known what aspects of psychosocial QoL are most strongly differentially affected in ASD/DD or at what point in the child’s life QoL is first impacted. From a societal perspective, understanding the impact of ASD on psychosocial QoL is particularly important given increases in ASD prevalence over the last two decades (Centers for Disease Control and Prevention, 2014) and data suggesting less than ideal outcomes in older adolescents and adults with ASD (Eaves and Ho, 2008; Orsmond et al., 2013; Shattuck et al., 2012). This study focused on developing and psychometrically evaluating a measure of psychosocial QoL in individuals with ASD/DD. A reliable and valid measure of QoL has the potential to both enrich our understanding of QoL within ASD/DD and advance future research and clinical practice.

Defining QoL

QoL is defined by the World Health Organization (WHO) as the:

individual’s perception of their position in life in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards and concerns, ranging from the person’s physical heath, psychological state, level of independence, social relationships, personal beliefs, and their relationship to salient features of the environment. (The WHOQOL Group (WHO), 1995: p.1)

There are two important aspects of this definition that influence measurement of QoL in ASD/DD populations. The first is that multiple levels of the ecological system are relevant to child and family QoL (Bronfenbrenner, 1989), including the child, their condition/disability, siblings, parents, extended family, and other aspects of the meso- and exo-system. Each unit of the system interacts with other units to alter QoL over time. Child and family QoL also influences the functioning of the system—creating a dynamic reciprocal relationship. For example, difficulties with the child’s behavior can generate stress in the parents which, in turn, lead to parental conflict or conflict with external supports which, in turn, decrease child and family QoL. Decreases in QoL can then precipitate further stress on the parents and family, diminishing their ability to manage their child’s behavior.

The second important aspect of defining QoL is to distinguish between broad conceptualizations of QoL (e.g. general happiness) and specific facets (e.g. physical or family QoL). In the context of ASD/DD, broad conceptualizations—while potentially useful to track as global outcomes—are less helpful in a clinical context for directing referrals, providing recommendations, or guiding treatment approaches. Measurement of specific aspects of psychosocial QoL is crucial for identifying strengths that can be bolstered and weak areas that might be targets for clinical intervention. Several studies have shown that specific family and caregiver qualities contribute to functioning and outcome in children with ASD, including social support, perceived control, hardiness, and optimism (Cappe et al., 2011). Low overall QoL, increased parent stress, and difficulties with coping have been associated with aggression, self-injury, and comorbid behavior problems in ASD (Kamio et al., 2012). In turn, parent stress can impact the mental and physical health of the child, as well as overall family functioning (Hall and Graff, 2010, 2011). Parents of children with ASD report higher levels of stress/lower QoL relative to parents of children without ASD, including parents of children with Down syndrome (Abbeduto et al., 2004). Understanding parental stress, coping, external support, and other psychosocial aspects of QoL is vital for designing the most appropriate clinical recommendations and engaging the larger family system.

This study focused on developing and evaluating a brief, multidimensional caregiver-report questionnaire assessing the parent/caregiver’s perception of specific aspects of their child’s psychosocial QoL, the Child and Family Quality of Life (CFQL) measure. The CFQL accounts for the interactive, multisystems nature of QoL by including information about the child, caregiver, family, and external support system. The measure focuses primarily on psychosocial QoL because physical/health and environmental/socioeconomic aspects of QoL can often be accurately assessed by the clinician using existing methods (e.g. physical examination and Pediatric Quality of Life Inventory (PEDS-QL)) or via careful probing of sociodemographic information. The CFQL obtains the primary caregiver/parent’s perceptions of each aspect of QoL because caregiver report is frequently available, is often central to developing the clinical picture (Head and Abbeduto, 2007; McStay et al., 2014), and QoL is difficult to measure in young or severely affected children with ASD/DD via their own report.

Comparison with existing pediatric QoL measures

It is reasonable to ask why a new measure of QoL is needed given the availability of several pediatric QoL measures, including the PEDS-QL (Varni et al., 1999), Child Health Questionnaire (CHQ; Ng et al., 2005), Revised Children Quality of Life Questionnaire (KINDL-R; Hullmann et al., 2012), and Quality of My Life Questionnaire (QOML; Gong et al., 2007). These measures tend to focus on health/physical illness-related QoL or broad psychosocial aspects of QoL rather than specific aspects of psychosocial QoL needed to inform clinical practice. Additionally, none of the existing measures have explicitly included children with ASD/DD in the development and validation of the instrument. Diagnosis-specific measures have been developed as an alternative to broad pediatric QoL measures. These measures have clear utility in disorder-specific studies, as they can readily capture unique aspects of each illness (e.g. The Attention-Deficit/Hyperactivity Disorder (ADHD) Impact Module – Child (AIM-C); Landgraf et al., 2002). However, specificity to each disorder makes them inappropriate for use in the full range of DD and prohibits their clinical application in settings where a range of diagnostic conditions and functional levels are evaluated or treated. The Family Quality of Life survey (Park et al., 2003) has also been used in ASD/DD populations, but this measure was not explicitly linked to clinical practice (Perry and Isaacs, 2015), and the factor structure differs from a priori scoring (Hu et al., 2012; Isaacs et al., 2012). Thus, the CFQL fills a gap by capturing multiple aspects of psychosocial functioning in children and families affected by ASD/DD without including features so specific that they would only apply to one disorder or a narrow range of functional levels. As a result, the CFQL is expected to assist in (1) understanding developmental patterns of QoL, including the reciprocal relationship between ASD/DD and QoL, (2) developing care models for the broader family system beyond individual child treatment approaches, and (3) tracking ecologically valid outcomes from behavioral and biological interventions (Head and Abbeduto, 2007).

The present study

The primary aim of this study was to conduct a comprehensive psychometric evaluation of the CFQL scale. The CFQL was designed to evaluate seven domains of psychosocial QoL relevant to children with ASD/DD, their families, and primary caregivers. These domains include (1) child, (2) family, (3) caregiver, (4) partner relationship, (5) external support, (6) coping, and (7) financial aspects of QoL. Each domain was chosen to provide links to specific clinical actions when low QoL is observed. The secondary aim of this study was to compare psychosocial QoL immediately prior to the time of diagnosis in children and families affected by ASD relative to children and families affected by other DDs. In an effort to further inform clinical utility of the instrument, this investigation also explored whether unique patterns of QoL could be empirically identified.

Methods

Sample

The sample consisted of consecutive referrals for evaluation of possible ASD to a specialty diagnostic evaluation clinic. Parents of 212 children (aged 13–86 months) completed the CFQL as part of their packet of questionnaires before their first clinic visit. All data from multidisciplinary evaluation visits, including rating scales, test scores, and clinical diagnoses were entered into a patient registry database. Caregivers provided consent for the use of de-identified medical records and the institutional review board of the Cleveland Clinic reviewed and approved use of the patient registry.

Measures

CFQL

The CFQL is a caregiver-completed, 32-item questionnaire designed to evaluate clinically relevant aspects of psychosocial QoL in individuals at risk of or with an existing DD diagnosis. It includes seven a priori-derived scales: Child, Family, Caregiver, Financial, External Support, Partner Relationship, and Coping. Low scores on each scale were linked to specific clinical recommendations. For example, low Partner Relationship QoL would alert clinicians to the possible need for referral to couples counseling. Each a priori scale is measured by four or five items with each item using a 5-point Likert scale representing the level of agreement with the statement (1 = Strongly Disagree, 5 = Strongly Agree), the frequency of a specific event or behavior (1 = Never, 5 = Always), or the presence of a quantity (e.g. quantity of support; 1 = Not at All, 5 = Very Much). Differing scale anchors were used to increase the variety of item content, enhance rater engagement, and decrease rater response sets that can artificially inflate internal consistency reliability and intercorrelations between separate constructs (Rosenthal and Rosnow, 1991). Items were written by a clinician experienced in the evaluation of individuals with ASD/DD, independently reviewed by a second experienced clinician, and pilot tested with two caregivers to ensure that item face validity was consistent with expectation. In this study, the CFQL was typically completed in 5–10 min (estimate reported by parent completing) by parents or legally authorized representatives, hereafter referred to as caregivers. CFQL questionnaires were distributed prior to the evaluation, and the majority of questionnaires were completed 1–4 weeks prior to the initial diagnostic evaluation appointment. Supplement 1 provides example item content for each CFQL scale.

Vineland Adaptive Behavior Scales, second edition, Parent/Caregiver Rating Form

The Vineland Adaptive Behavior Scales, second edition (VABS-II), Parent/Caregiver Rating Form (Sparrow et al., 2005) is a parent-report measure used to gain adult perception of the child’s communication, daily living, socialization, and motor skills. Responses are measured on a 3-point Likert scale representing the level of agreement with the statement (2 = Usually, 1 = Sometimes or Partially, and 0 = Never). Total scores are summed and yield norm-referenced standard scores within communication, daily living skills, socialization, and motor skills domains as well as a composite index.

Child Behavior Checklist

The Child Behavior Checklist (CBCL; Achenbach and Rescorla, 2001) is a parent-report questionnaire completed by the child’s parent/guardian to assess a child’s behavioral and emotional problems, including internalizing and externalizing behaviors. Responses are measured on a 3-point Likert scale representing the level of agreement with the statement (2 = Very or Often True, 1 = Somewhat or Sometimes True, and 0 = Not True). T-scores from the Total, Internalizing, and Externalizing problems scales were used to characterize the sample and evaluate construct validity.

Preschool Language Scales, fourth edition

The Preschool Language Scales, fourth edition (PLS-4; Zimmerman et al., 2011) is an individually administered test used to evaluate language. Standard scores for the auditory comprehension, expressive communication, and total language scales were included.

All questionnaires were collected prior to the first clinical contact. Language test results were collected at the time of the first visit.

Consensus diagnoses

ASD diagnoses were derived from clinical consensus and based on information from the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2002) completed with a reliable administrator, Autism Diagnostic Interview–Revised (ADI-R; Lord et al., 1994) parent interview with an experienced clinical psychologist, and evaluation by a physician (pediatric neurologist, developmental pediatrician, or pediatrician). Clinical consensus was obtained through team review at a case conference. Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR) diagnoses were collapsed into a single ASD category consistent with epidemiological catchment protocols (Centers for Disease Control and Prevention, 2014). Other DD diagnoses (including ADHD, speech or language disorders, disruptive behavior disorders, anxiety disorder, mood disorders, and no diagnosis) were lumped into a single non-ASD category. ADOS (total, social affect, and restricted/repetitive) and ADI-R (total score across social interaction, communication, and restricted/repetitive behavior domains) summary scores were used to evaluate convergent and discriminant validity.

Statistical analyses

Data entry quality control was completed by randomly selecting 10% of patients and re-entering their data, and cross-checking records. No discrepancies were identified, and data screening for univariate and multivariate outliers did not identify any high leverage or outlier data points (Tabachnick and Fidell, 2001).

Sample description

Demographic (age and sex) and clinical (DSM-IV-TR diagnoses, ADOS, VABS-II, CBCL, and PLS-4) characteristics were compared between children with and without ASD diagnoses using independent samples t-tests, chi-square, or Fisher’s exact tests where appropriate.

Factor structure

The first step in psychometric evaluation of the CFQL was to compare the factor structure of the instrument relative to a priori scale scoring. To determine the factor structure, a principal component analysis generated eigenvalues for evaluation of the scree plot and comparison to the results of Horn’s parallel analysis (HPA; Velicer et al., 2000). HPA has been empirically demonstrated to assist in identifying the true number of factors/components to retain (Glorfeld, 1995). In addition, we also conducted an item-level exploratory factor analysis using weighted least squares estimation for ordinal items (Muthén and Muthén, 2007). The comparative fit index (CFI), Tucker–Lewis index (TLI), root mean square error of approximation (RMSEA), and ΔCFI with increasing factor number were used as additional metrics for choosing the optimal factor number (Bentler, 1988; Hu and Bentler, 1999). Previous empirical literature has suggested that CFI values >0.92 indicate adequate and values >0.95 indicate excellent overall fit (Marsh et al., 2004). RMSEA values <0.10 were considered adequate fit and <0.08 were judged as good fit (Kline, 1998). Improvements of CFI > 0.01 and reductions of RMSEA > 0.01 suggest improved fit of nested models (Cheung and Rensvold, 2002). The factor loading matrix from weighted least squares analysis was examined to compare item loadings to a priori scale structure, with the general rule-of-thumb being that coefficients ~0.35 imply meaningful item loadings (Stevens, 1992; Tabachnick and Fidell, 2001).

Reliability

After identifying factor structure and comparing empirical structure to a priori scales, the reliability for each of the factor derived and a priori scales was estimated by computing internal consistency reliability coefficients (Cronbach’s α). Additionally, to provide a finer-grained analysis of reliability as a function of score level, item response theory analyses were conducted for each a priori scale. These analyses examine reliability across the full latent trait and assist with clinical interpretation of high, moderate, and low QoL scores. Item response theory-derived information values can be converted to reliability coefficients with information values of 3, 5, and 10 reflecting reliability coefficients of 0.67, 0.80, and 0.90, respectively. Next, we examined the convergent and discriminant validity of a priori CFQL scales with other clinical measures using Pearson correlation coefficients. The expectation was that Child QoL would show moderate relationships with autism symptoms, adaptive function, behavior problems, and language test results, but that other QoL scales would only show relationships with behavior problems. This expectation was based on the previously identified impact of child behavior problems and adaptive function on caregiver and family functioning (Emily and Grace, 2015; Kamio et al., 2012).

QoL in ASD versus non-ASD

To test the hypothesis that families with an ASD-affected child would show lower QoL relative to families with a non-ASD affected child, we computed a mixed effects regression model with CFQL scales nested within child. ASD diagnosis (ASD vs. non-ASD), CFQL scale (Child, Family, Caregiver, Financial, External Support, Partner Relationship, and Coping) and their two-way interaction were included as fixed effects factors. Age and sex of the child were included as covariates. Mixed effects regression models were estimated using maximum likelihood, and fit was considered by iteratively examining alternative covariance structures. Final models were presented based on diagonal covariance structure, which fit comparably and yielded a highly similar pattern of results to other covariance structures (e.g. unstructured, independent, and autoregressive). Statistical power was good (>0.80) for detecting a small-to-medium difference (Cohen’s d ⩾ 0.40, two-tailed p < 0.05) between ASD and non-ASD groups (Erdfelder et al., 1996).

QoL profiles

Finally, to increase clinical utility and provide clinicians with guidance for pattern interpretation of the CFQL, we also explored whether profiles of QoL could be identified on the CFQL using mixture modeling. For these analyses, one- to five-class solutions were examined with each of the seven CFQL scale raw scores as indicators. Optimal class number was determined by examining four fit statistics: Akaike Information Criterion (AIC; Akaike, 1987), Bayesian Information Criterion (BIC; Schwarz, 1978), sample-adjusted Bayesian Information Criterion (saBIC), and entropy. These statistics are useful for comparing the relative fit of models, with lower values of AIC, BIC, and saBIC indicating better fit and higher values of entropy indicating better fit. For model comparison, differences between models were considered significant if the difference in BIC and sample-adjusted BIC were >10 (Kass and Raftery, 1995). The BIC and saBIC penalize more heavily for model complexity, and preference was given to the saBIC given the modest sample size of this study and recommendations from the model fitting literature (Nylund et al., 2007).

Principal components analysis, Pearson correlations, and ASD versus non-ASD group comparison analyses were conducted in SPSS version 21 (IBM Corp, 2012). Weighted least squares exploratory factor, item response theory, and mixture modeling analyses were conducted in MPlus v7.2 (Muthén and Muthén, 2007). Statistical significance was determined using p < 0.05 with the exception of convergent and discriminant validity analysis which used a more stringent p < 0.01 to attempt to control the type 1 error rate due to multiple comparisons.

Results

Sample description

ASD and non-ASD participant groups were similar in age and total, internalizing, and externalizing behavior problems (Table 1). Not surprisingly, ASD-affected children were slightly, but nonsignificantly, more likely to be male and had lower motor function scores on the VABS-II. Children with ASD were significantly less likely to receive a comorbid anxiety disorder or other DSM-IV-TR diagnosis, potentially due to anxiety and other behavioral symptoms being considered secondary to autism symptoms in many cases. As expected, ASD-affected children had higher autism symptoms on the ADI-R and ADOS, lower adaptive function on the composite and nonmotor subscales, and lower language ability.

Table 1.

Sample demographic and clinical characteristics.

	Non-ASD	ASD	χ²/t (p)
	M (SD)	M (SD)
N	91	121
Age (range, months)	3.7 (1.1–6.6)	3.5 (1.2–7.2)	0.85 (0.399)
Male (n, %)	68 (74.7%)	102 (84.3%)	3.00 (0.083)
DSM-IV-TR diagnoses (n, %)
Any language disorder	74 (81.3%)	110 (90.9%)	4.17 (0.041)
Any anxiety disorder	20 (22.0%)	0 (0.0%)	29.36 (<0.001)
Any ADHD	10 (11.0%)	–
Other diagnosis	21 (23.1%)	2 (1.7%)	24.65 (<0.001)
No diagnosis	4 (4.4%)	–
Autism Diagnostic Interview–Revised (ADI-R)
Total score	10.5 (8.1)	22.2 (10.4)	7.60 (<0.001)
Autism Diagnostic Observation Schedule
Total score	6.3 (5.2)	16.7 (6.7)	−12.19 (<0.001)
Social affect	5.1 (4.3)	13.1 (5.5)	−11.36 (<0.001)
Restricted/repetitive	1.2 (1.5)	3.6 (2.2)	−8.73 (<0.001)
Vineland Adaptive Behavior Scales (SS)
Composite	82.6 (23.3)	71.2 (21.4)	3.69 (<0.001)
Communication	84.3 (16.0)	72.3 (17.9)	5.08 (<0.001)
Daily living skills	89.9 (18.9)	77.9 (14.9)	5.19 (<0.001)
Socialization	88.7 (16.4)	76.4 (15.5)	5.58 (<0.001)
Motor	86.3 (22.3)	80.5 (22.4)	1.87 (0.063)
Child Behavior Checklist (T-score)
Total problems	61.1 (13.3)	62.0 (10.4)	−0.52 (0.604)
Internalizing problems	59.6 (12.1)	61.0 (9.6)	−0.93 (0.394)
Externalizing problems	59.9 (14.6)	58.9 (11.4)	0.57 (0.568)
Preschool Language Scales, fourth edition (SS)
Total language	82.6 (16.5)	64.6 (14.5)	7.10 (<0.001)
Auditory comprehension	83.2 (17.2)	63.8 (15.3)	7.31 (<0.001)
Expressive communication	84.1 (15.2)	70.0 (14.0)	5.92 (<0.001)

ASD: autism spectrum disorder; SD: standard deviation; SS: standard score; DSM-IV-TR: Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.); ADHD: attention-deficit or hyperactivity disorder.

Note: For anxiety diagnoses, Fisher’s exact test also indicated significantly fewer anxiety diagnoses in the ASD group. For ADI-R, sample sizes were reduced as the interview was not conducted due to time constraints (non-ASD: n = 56, ASD: n = 92).

Factor structure

Comparison of results from principal component analysis and HPA suggested retention of a five- or six-factor solution (Table 2). Inspection of the scree plot supported a six-factor solution (Supplement 2), although increases in eigenvalues were somewhat gradual between five and eight factors. Fit indices from weighted least square exploratory factor analysis indicated excellent fit to five- to seven-factor solutions, with ΔCFI indicating improvements in fit from five to six factors, but no improvement from six to seven factors retained (Table 2). Inspection of the factor loading matrix indicated that retention of six factors yielded an interpretable solution with no additional clinical value obtained by retaining seven factors. For these reasons, a six-factor solution was retained and factor loadings compared to a priori scales.

Table 2.

Fit statistics from exploratory factor analysis.

	Observed eigenvalue	HPA eigenvalue—upper 95 percentile	χ²	DF	CFI	TLI	RMSEA (95% CI)	ΔCFI
1f	11.26	1.87	3373	464	0.823	0.811	0.172 (0.167–0.177)	–
2f	4.00	1.74	1968	433	0.907	0.893	0.129 (0.124–0.135)	0.084
3f	2.60	1.66	1383	403	0.940	0.927	0.107 (0.101–0.113)	0.033
4f	2.14	1.58	983	374	0.963	0.951	0.088 (0.081–0.094)	0.023
5f	1.62	1.51	727	346	0.977	0.967	0.072 (0.065–0.079)	0.014
6f	1.44	1.44	499	319	0.989	0.983	0.052 (0.043–0.060)	0.012
7f	1.06	1.39	402	293	0.993	0.989	0.042 (0.031–0.052)	0.004

HPA: Horn’s parallel analysis; DF: degrees of freedom; CFI: comparative fit index; TLI: Tucker–Lewis index; RMSEA: root mean square error of approximation; CI: confidence interval.

A priori structure of CFQL scales was largely consistent with the six-factor solution. The only exceptions were that the family and caregiver scales merged into a single factor, and there were minor differences in item loadings—a Child QoL item and an External Support QoL item loaded with Family QoL and Caregiver QoL items (Table 3). Additionally, a Child QoL item and a Coping QoL item cross-loaded with the Family QoL and Caregiver QoL items. However, even with these exceptions, the a priori scale structure corresponded closely to the empirically estimated factor structure.

Table 3.

Item descriptives, salient factor loadings, and internal consistency reliability for each CFQL scale.

No.	A priori scale	M	SD	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6
1	Child	3.07	1.11	0.53	0.35	–	–	–	–
2	Child	2.56	1.12	–	0.65	–	–	–	–
3	Child	2.57	1.22	–	0.88	–	–	–	–
4	Child	2.88	0.95	–	0.65	–	–	–	–
5	Child	4.02	0.72	0.53	–	–	–	–	–
6	Family	2.46	1.21	0.69	–	–	–	–	–
7	Family	2.86	1.30	1.04	–	–	–	–	–
8	Family	2.83	1.28	1.04	–	–	–	–	–
9	Family	3.44	1.18	0.88	–	–	–	–	–
10	Family	3.14	1.21	0.87	–	–	–	–	–
11	Caregiver	2.95	1.17	0.76	–	–	–	–	–
12	Caregiver	2.55	1.15	0.53	–	–	–	–	–
13	Caregiver	3.12	1.26	0.76	–	–	–	–	–
14	Caregiver	3.16	1.22	0.58	–	–	–	–	–
15	Caregiver	3.60	0.92	–	–	–	–	–	–
16	Financial	4.04	1.18	–	–	0.94	–	–	–
17	Financial	4.27	1.11	–	–	0.93	–	–	–
18	Financial	4.09	1.21	–	–	0.90	–	–	–
19	Financial	4.22	1.13	–	–	0.53	–	–	–
20	Support	3.30	1.23	–	–	–	–	0.74	–
21	Support	3.56	1.19	–	–	–	–	0.73	–
22	Support	4.30	1.01	0.56	–	–	–	–	–
23	Support	3.68	1.14	–	–	–	–	0.87	–
24	Support	3.85	1.13	–	–	–	–	0.57	–
25	Partner Rel.	3.78	1.40	–	–	–	0.93	–	–
26	Partner Rel.	3.53	1.41	–	–	–	0.91	–	–
27	Partner Rel.	3.84	1.38	–	–	–	0.94	–	–
28	Partner Rel.	3.67	1.41	–	–	–	0.98	–	–
29	Coping	3.32	0.93	0.39	–	–	–	–	0.42
30	Coping	3.77	1.16	–	–	–	–	–	0.79
31	Coping	4.15	0.99	–	–	–	–	–	0.69
32	Coping	4.73	0.67	–	–	–	–	–	–
Reliability (factor)			α	0.92	0.78	0.89	0.97	0.81	0.75
Reliability (a priori scale)			α	0.91/0.79	0.77	0.89	0.97	0.79	0.75

CFQL: Child and Family Quality of Life; SD: standard deviation; Partner Rel.: Partner Relationship Quality of Life.

Factor 1 combines the a priori scales of Family QoL and Caregiver QoL and includes salient loadings from Child QoL and External Support. Internal consistency reliability coefficients for the a priori scales dominating factor 1 reflect separate analyses of the Family QoL and Caregiver QoL scales.

Scale reliability

Internal consistency reliability was generally good to excellent, and values were largely unchanged when items were grouped by empirically derived factor or by a priori scale (Table 3). Notably, the aggregation of Family and Caregiver QoL items was likely due to the young age of the sample, where caregiver’s impressions of the family’s well-being are expected to be highly linked to the caregiver’s own sense of well-being. Also, clinician feedback suggested that the Family and Caregiver QoL scales provided distinct information in ongoing pilot use with older children. Thus, the remaining analyses focused on the a priori scales.

Reliability across the latent trait

Item response theory information curves indicated variable reliability across levels of the latent trait and across CFQL scales (Figure 1). The Child, Caregiver, External Support, and Coping QoL scales maintained adequate reliability (>0.67) in the low to moderate score ranges (θ = −1.5 to 0), but reliability was poor in very low (θ < −1.5), high (θ = 1–2), and very high (θ > 2) score ranges. This pattern implies good ability to discriminate adequate from low levels of QoL but that discrimination of low from very low QoL and discrimination of adequate to very high levels require larger score differences. The Family QoL scale had excellent reliability from low to high scores. Reliability only dropped off for very low and very high QoL, implying that this scale is useful for grading level of QoL in a wide range of presentations. The Partner Relationship QoL scale had a similar pattern with very high reliability from low to above average levels of QoL, implying excellent ability to discriminate well-functioning relationships from those in significant distress. The Financial QoL scale showed excellent reliability from very low to average levels of the latent trait. This implies strong ability to discriminate individuals who are subjectively experiencing distress related to their financial situation versus those who are not.

Figure 1.

Item information curves for each CFQL scale: Child, Family, Caregiver, Financial, External Support, Partner Relationship, and Coping.

Convergent and discriminant validity

Low scores on the Child QoL scale were significantly associated with greater autism symptom severity on the ADI-R, lower adaptive behavior level, lower expressive and receptive language level, and greater behavior problems (Table 4). Interestingly, there was no association between Child QoL and autism symptom severity on the ADOS, implying that the relationship with symptom severity may be driven by parent perception. Low scores on the Family, Caregiver, Financial, External Support, and Coping QoL scales were significantly associated with greater behavior problems. However, these scales had no significant relationships with other clinical measures, and the Partner Relationship QoL scale was not significantly related to any measure.

Table 4.

Convergent and discriminant validity of CFQL scales with autism symptom, behavioral problems, and cognitive measures.

	ADOS total	ADI-R total	VABS-II composite	CBCL total problems	PLS-4 auditory comprehension	PLS-4 expressive communication
Child	−0.01	−0.29*	0.22*	−0.62*	0.20*	0.21*
Family	0.06	−0.17	0.14	−0.56*	0.11	0.08
Caregiver	0.03	−0.06	0.07	−0.45*	−0.01	0.04
Financial	−0.08	−0.05	0.03	−0.26*	0.07	0.11
External Support	0.11	−0.07	−0.01	−0.24*	0.08	0.06
Partner Relationship	0.20	0.02	−0.02	−0.16	−0.11	−0.13
Coping	−0.09	−0.16	0.08	−0.31*	0.12	0.14

CFQL: Child and Family Quality of Life; ADOS: Autism Diagnostic Observation Schedule; ADI-R: Autism Diagnostic Interview–Revised; CBCL: Child Behavior Checklist; VABS-II: Vineland Adaptive Behavior Scales, second edition; PLS-4: Preschool Language Scales, fourth edition.

p < 0.01.

ASD versus non-ASD

Children with ASD showed generally lower scores across scales relative to children with non-ASD diagnoses (ASD diagnosis main effect F(1, 1336) = 9.13, p = 0.003), but the pattern was variable across scales (ASD diagnosis by CFQL scale interaction F(6, 374) = 2.24, p = 0.035). In particular, children with ASD had significantly lower Family QoL scores at the time of diagnosis relative to non-ASD children (Figure 2). There was a trend toward decreased Child QoL and increased Partner Relationship QoL in families containing children with ASD relative to families with non-ASD children (p < 0.10). Including VABS-II communication or adaptive composite scores as a covariate did not alter the pattern of findings (smallest ASD diagnosis by CFQL scale interaction F(6, 382) = 2.25, p = 0.038), indicating that this effect was specific to ASD diagnosis and not influenced by functional level of the child.

Figure 2.

CFQL scores (M ± 95% CI) across scales in ASD and non-ASD diagnosed children.

QoL levels in all children

Considering all children jointly, QoL levels were lowest on the Child, Family, and Caregiver QoL scales (M = 3.1–3.2, standard error (SE) = 0.05–0.07) and highest on the Financial, Partner Relationship, and Coping QoL scales (M = 4.0–4.2, SE = 0.05–0.07; F(6, 374) = 83.74, p < 0.001; Figure 2). Older children had generally lower QoL across scales (F(1, 1320) = 39.00, p < 0.001). There were no significant sex differences in QoL ratings (F(1, 1320) = 2.49, p = 0.115).

QoL profiles

Mixture modeling supported both two- and three-class solutions, with the lowest BIC observed for a two-class solution and only modest improvement in saBIC from the three- to four-class solutions (Supplement 3). These results suggest that the primary distinction in QoL was between low QoL and moderate-to-high QoL across all scales. However, to enhance clinical interpretation, we focused on a three-class solution, which showed modest improvement over the two-class solution on both saBIC and AIC. This model yielded a low base rate class (class 1: n = 34, 16%) with low QoL but high levels of Partner Relationship QoL, an additional low base rate class (class 2: n = 28, 13%) with low QoL but high Financial QoL, and a high base rate class (class 3: n = 150, 71%) with moderate-to-high QoL across scales (Figure 3). Interestingly, the low QoL classes (classes 1 and 2) were older (F(2, 209) = 5.67, p = 0.004), had higher levels of reported behavior problems on the CBCL (F(2, 206) = 12.28, p < 0.001), and were more likely to have received an ADHD diagnosis (χ²(2) = 19.25, p < 0.001). There were no differences across classes in the proportion of males/females; level of autism symptom severity on the ADOS or ADI-R; level of language on the PLS-4; adaptive function on the VABS-II; or in the proportion of language disorder, anxiety disorder, or other psychiatric diagnoses (all p > 0.100).

Figure 3.

CFQL scale scores (M + 95% CI) for empirically derived classes.

Discussion

The present findings support the CFQL as a psychometrically sound instrument for measuring psychosocial QoL in families of young children with ASD/DD. The CFQL was found to be reliable, particularly in the score ranges most useful for distinguishing adequate and low QoL and had expected convergent and discriminant validity with other commonly used clinical evaluation measures. The instrument appeared to have utility in helping clinicians screen for low QoL across psychosocial domains and identify areas in need of more detailed exploration. Clinical experience with the CFQL, obtained in the course of this study period, suggested that it may be particularly helpful in circumventing social desirability biases that emerge in direct clinician–parent interactions. Parents sometimes reported low Financial or Partner Relationship QoL when completing the CFQL questionnaire that they did not report face to face during clinical interview. Additional research is needed to explore this potential advantage. It is important to note although that the CFQL does not identify the source of low QoL in each domain, and thus, additional information collected during the clinical interview is essential before generating treatment recommendations.

The CFQL factor structure was clear and largely consistent with a priori scale scoring in this sample. The few minor differences may be a function of idiosyncrasies of the present sample. Specifically, the young age of the sample may have contributed to merging of the a priori Family and Caregiver QoL into a single empirically derived scale. Raters were the primary caregiver (typically mothers), and caregiver experience is a dominant component of the family at this young age when children have minimal independence. Regardless, clinicians should be aware of the high correlation between these scales in young children. Dissociations may signal important information distinguishing QoL between the caregiver and other family members. Additional studies are needed to replicate the factor structure in older samples and determine the most appropriate item scoring algorithm. At present, a priori scale scoring is recommended.

Beyond psychometric validation, the most novel and interesting finding in this study was that caregivers of ASD-affected children were already reporting significantly lower Family QoL immediately prior to their child receiving the diagnosis. This highlights the fact that many caregivers and other family members are feeling the impact of having a child with significant social and behavioral challenges before that child is brought to clinical attention. In at least some cases, these challenges may be a primary motivator for obtaining clinical evaluation. The reduction in Family QoL also suggests that family members of young children with ASD may feel overwhelmed very early in the child’s development, reinforcing the need for early screening and referral as well as treatment programs that broadly consider child and family functioning. It will be crucial to track this phenomenon after diagnosis to determine whether this very early trend can be attenuated or reversed with appropriate treatment. Regardless, this finding suggests a need for early, broad, family-based treatments beyond simply improving the symptoms and functioning of the identified child.

A second interesting finding of this investigation was that, regardless of ASD or other DD diagnosis, the primary distinction was between low and moderate-to-high QoL. This suggests that children and families with ASD or related DDs tend to show a qualitative drop in QoL that occurs at least by the time of diagnosis. Furthermore, caregivers reporting low QoL appear to split into those who subjectively perceive good partner relationship support but weaker financial support and those with stronger financial support but weaker partner relationship support. This finding has three important ramifications. First, clinicians should be on the lookout for distinctly low levels of QoL on the CFQL as these signal families in distress. Second, it suggests that the CFQL, and possibly other QoL measures, would be wise to focus on measuring reliably in the moderate to low score ranges as that is where the primary distinction between low and moderate-to-high classes was observed. Fortunately, all of the CFQL scales showed adequate-to-excellent reliability in these score ranges. Finally, clinicians should consider tailoring recommendations to families with weaker financial support to identify additional community resources (e.g. county or state support programs) and avoid actions that might further strain perception of their financial situation, while families with weaker partner relationship support may be considered for family or couples therapy referrals.

Limitations and future directions

Limitations of this study included a modest sample size, lack of within-sample replication of factor structure, limited age range, and lack of a healthy comparison sample. While the current sample size was modest by latent structure analysis standards, it was adequate for a preliminary psychometric evaluation, and the sample had good statistical power for detecting ASD versus non-ASD differences across CFQL scales. Unfortunately, within-study replication of the factor structure and other psychometric findings was not possible due to the modest sample size. The young sample, obtained at the time of diagnosis, limited the age range and restricted generalizability of these findings to young children seen in clinical diagnostic settings. Fortunately, the ages included in this study are typical of patients seen in most ASD or other DD diagnostic settings and permitted us to identify a very early pattern of reduced family QoL in ASD.

Future research on the use of the CFQL in clinical populations is needed to replicate these findings and extend them across ages and other DD diagnoses. At present clinical interpretation of CFQL raw scores is based on scale anchors and results in crude grading of QoL—very low, low, adequate, high, and very high. Collection of data in nonclinical healthy populations will permit calculation of norm-referenced scores. Norm-referenced scores may indicate that adequate levels of QoL already signal a reduction meriting clinical attention. If identified, this would reinforce the need for both content and norm-referenced interpretation for QoL scales. Collecting a larger sample of clinically ascertained non-ASD children will also be important for identifying specific patterns of psychosocial QoL across common disorders of childhood including speech/language, ADHD, disruptive behavior, mood, and anxiety disorders. Test–retest reliability studies are needed to determine the stability of CFQL scales and their potential sensitivity to treatment effects. Finally, adoption of the CFQL in longitudinal studies will help to clarify the effects of development and adaptation to the child’s diagnosis on psychosocial QoL.

With these extensions in mind, the present data support use of the CFQL as a brief, easy-to-acquire screener for psychosocial QoL in DD populations referred for clinical evaluation. Clinicians may routinely consider this information to capture a broader picture of child and family functioning with direct relevance to treatment planning and to determine the range of services needed to assist children and families affected by ASD or other DD.

Footnotes

Acknowledgements

The authors wish to acknowledge the important contribution of the participants with autism and their families. All authors participated in framing the research question, designing the analysis plan, and/or interpreting results. Dr Frazier conducted all analyses. Drs Markowitz, Reyes, and Frazier prepared the first draft of the manuscript. All authors participated in revising the manuscript after the first draft was prepared. Dr Frazier takes responsibility for the integrity of the data and accuracy of the data analysis.

Funding

This work was supported by funding from the Stephan and Allison Cole Family Research Fund and by the Developmental Synaptopathies Consortium (U54NS092090). The Developmental Synaptopathies Consortium is part of NCATS Rare Disease Clinical Research Network (RDCRN), an initiative of the Office of Rare Disease Research (ORDR). This consortium is funded through collaboration between NCATS and the National Institute of Neurological Disorders and Stroke (NINDS) of the National Institutes of Health. The content of this publication is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

References

Abbeduto

Seltzer

Shattuck

. (2004) Psychological well-being and coping in mothers of youths with autism, Down syndrome, or fragile X syndrome. American Journal of Mental Retardation 109(3): 237–254.

Achenbach

Rescorla

(2001) Manual for the ASEBA School-Age Forms and Profiles. Burlington, VT: Department of Psychiatry, University of Vermont.

Akaike

(1987) Factor analysis and AIC. Psychometrika 52: 317–332.

Bentler

(1988) Comparative fit indexes in structural equation models. Psychological Bulletin 107: 238–246.

Boulet

Boyle

Schieve

(2009) Health care use and health and functional impact of developmental disabilities among US children, 1997–2005. Archives of Pediatrics and Adolescent Medicine 163(1): 19–26.

Boyle

Boulet

Schieve

. (2011) Trends in the prevalence of developmental disabilities in US children, 1997–2008. Pediatrics 127(6): 1034–1042.

Boyle

Decoufle

Yeargin-Allsopp

(1994) Prevalence and health impact of developmental disabilities in US children. Pediatrics 93(3): 399–403.

Bronfenbrenner

(1989) Ecological systems theory. Annals of Child Development 6: 187–249.

Cappe

Wolff

Bobet

. (2011) Quality of life: a key variable to consider in the evaluation of adjustment in parents of children with autism spectrum disorders and in the development of relevant support and assistance programmes. Quality of Life Research 20(8): 1279–1294.

10.

Centers for Disease Control and Prevention (2014) Prevalence of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2010. MMWR 63(2): 1–21.

11.

Cheung

Rensvold

(2002) Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal 9(2): 233–255.

12.

De Vries

Geurts

(2015) Influence of Autism Traits and Executive Functioning on Quality of Life in Children with an Autism Spectrum Disorder. Journal of Autism and Developmental Disorders 45: 2734–2743.

13.

Eaves

(2008) Young adult outcome of autism spectrum disorders. Journal of Autism and Developmental Disorders 38(4): 739–747.

14.

Emily

Grace

(2015) Family quality of life and ASD: the role of child adaptive functioning and behavior problems. Autism Research 8(2): 199–213.

15.

Erdfelder

Faul

Buchner

(1996) GPOWER: a general power analysis program. Behavior Research Methods, Instruments, and Computers 28: 1–11.

16.

Gardiner

Iarocci

(2012) Unhappy (and happy) in their own way: a developmental psychopathology perspective on quality of life for families living with developmental disability with and without autism. Research in Developmental Disabilities 33(6): 2177–2192.

17.

Glorfeld

(1995) An improvement on Horn’s parallel analysis methodology for selecting the correct number of factors to retain. Educational and Psychological Measurement 55(3): 377–393.

18.

Gong

Young

Dempster

. (2007) The Quality of My Life questionnaire: the minimal clinically important difference for pediatric rheumatology patients. Journal of Rheumatology 34(3): 581–587.

19.

Hall

Graff

(2010) Parenting challenges in families of children with autism: a pilot study. Issues in Comprehensive Pediatric Nursing 33(4): 187–204.

20.

Hall

Graff

(2011) The relationships among adaptive behaviors of children with autism, family support, parenting stress, and coping. Issues in Comprehensive Pediatric Nursing 34(1): 4–25.

21.

Head

Abbeduto

(2007) Recognizing the role of parents in developmental outcomes: a systems approach to evaluating the child with developmental disabilities. Mental Retardation and Developmental Disabilities Research Reviews 13(4): 293–301.

22.

Hoefman

Payakachat

van Exel

. (2014) Caring for a child with autism spectrum disorder and parents’ quality of life: application of the CarerQol. Journal of Autism and Developmental Disorders 44(8): 1933–1945.

23.

Bentler

(1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling 6: 1–55.

24.

Summers

Turnbull

. (2011) The quantitative measurement of family quality of life: a review of available instruments. Journal of Intellectual Disability Research 55(12): 1098–1114.

25.

Wang

Fei

(2012) Family quality of life of Chinese families of children with intellectual disabilities. Journal of Intellectual Disability Research 56(1): 30–44.

26.

Hullmann

Ryan

Ramsey

. (2012) Measures of general pediatric quality of life: Child Health Questionnaire (CHQ), DISABKIDS Chronic Generic Measure (DCGM), KINDL-R, Pediatric Quality of Life Inventory (PedsQL) 4.0 Generic Core Scales, and Quality of My Life Questionnaire (QoML). Arthritis Care Res (Hoboken) 63(Suppl. 11): S420–S430.

27.

IBM Corp (2012) IBM Statistics for Windows. 21.0 ed. Armonk, NY: IBM Corp.

28.

Isaacs

Wang

Samuel

. (2012) Testing the factor structure of the Family Quality of Life Survey—2006. Journal of Intellectual Disability Research 56(1): 17–29.

29.

Kamio

Inada

Koyama

(2012) A nationwide survey on quality of life and associated factors of adults with high-functioning autism spectrum disorders. Autism 17(1): 15–26.

30.

Kass

Raftery

(1995) Bayes factors. Journal of the American Statistical Association 90(430): 773–795.

31.

Kline

(1998) Principles and Practice of Structural Equation Modeling. New York: Guilford.

32.

Landgraf

Rich

Rappaport

(2002) Measuring quality of life in children with Attention-Deficit/Hyperactivity Disorder and their families. Archives of Pediatrics and Adolescent Medicine 156: 384–391.

33.

Lee

Harrington

Louie

. (2008) Children with autism: quality of life and parental concerns. Journal of Autism and Developmental Disorders 38(6): 1147–1160.

34.

Lin

(2014) Quality of life of Taiwanese adults with autism spectrum disorder. PLoS ONE 9(10): e109567.

35.

Lord

Rutter

LeCouteur

(1994) Autism Diagnostic Interview–Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders 24: 569–685.

36.

Lord

Rutter

DiLavore

. (2002) Autism Diagnostic Observation Schedule: ADOS Manual. Los Angeles, CA: Western Psychological Services.

37.

Lucas-Carrasco

Salvador-Carulla

(2012) Life satisfaction in persons with intellectual disabilities. Research in Developmental Disabilities 33: 1103–1109.

38.

McStay

Trembath

Dissanayake

(2014) Maternal stress and family quality of life in response to raising a child with autism: from preschool to adolescence. Research in Developmental Disabilities 35(11): 3119–3130.

39.

Marsh

Hau

Wen

(2004) In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indices and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling 11: 320–341.

40.

Muthén

(2007) Mplus User’s Guide. Los Angeles, CA: Muthén & Muthén.

41.

Landgraf

Chiu

. (2005) Preliminary evidence on the measurement properties of the Chinese version of the Child Health Questionnaire, parent form (CHQ-pF50) and child form (CHQ-CF87). Quality of Life Research 14(7): 1775–1781.

42.

Nylund

Asparouhov

Muthén

(2007) Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Structural Equation Modeling 14(4): 535–569.

43.

Orsmond

Shattuck

Cooper

. (2013) Social participation among young adults with an autism spectrum disorder. Journal of Autism and Developmental Disorders 43(11): 2710–2719.

44.

Park

Hoffman

Marquis

. (2003) Toward assessing family outcomes of service delivery: validation of a family quality of life survey. Journal of Intellectual Disability Research 47(Pt 4–5): 367–384.

45.

Perry

Isaacs

(2015) Validity of the family quality of life survey-2006. Journal of Applied Research in Intellectual Disabilities 28: 584–588.

46.

Rana

Mishra

(2015) Quality of life of unaffected siblings of children with chronic neurological disorders. Indian Journal of Pediatrics 82(6): 545–548.

47.

Rosenthal

Rosnow

(1991) Essentials of Behavioral Research: Methods and Data Analysis. New York: McGraw-Hill, Inc.

48.

Schwarz

(1978) Estimating the dimension of a model. The Annals of Statistics 6: 461–464.

49.

Shattuck

Narendorf

Cooper

. (2012) Postsecondary education and employment among youth with an autism spectrum disorder. Pediatrics 129(6): 1042–1049.

50.

Sparrow

Cicchetti

Balla

(2005) Vineland Adaptive Behavior Scales. 2nd ed. San Antonio, TX: Pearson.

51.

Stevens

(1992) Applied Multivariate Statistics for the Social Sciences. Hillsdale, NJ: Erlbaum.

52.

Suzumura

(2015) Quality of life in mothers of preschoolers with high-functioning pervasive developmental disorders. Pediatrics International 57(1): 149–154.

53.

Tabachnick

Fidell

(2001) Using Multivariate Statistics. New York: HarperCollins.

54.

The WHOQOL Group (WHO) (1995) The World Health Organization Quality of Life assessment (WHOQOL): position paper from the World Health Organization. Social Science and Medicine 41(10): 1403–1409.

55.

Varni

Seid

Rode

(1999) The PedsQL: measurement model for the pediatric quality of life inventory. Medical Care 37(2): 126–139.

56.

Velicer

Eaton

Fava

(2000) Construct explication through factor or component analysis: a review and evaluation of alternative procedures for determining the number of factors or components. In: Goffin

Helmes

(eds) Problems and Solutions in Human Assessment: Honoring Douglas N. Jackson at Seventy. Boston, MA: Kluwer Academic Press, pp. 41–71.

57.

Vohra

Madhavan

Sambamoorthi

. (2014) Access to services, quality of care, and family impact for children with autism, other developmental disabilities, and other mental health conditions. Autism 18(7): 815–826.

58.

Zimmerman

Steiner

Pond

. (2011) Preschool Language Scales-Fifth Edition (PLS-5). San Antonio, TX: Pearson.