Psychometric examination of a Spanish translation of a developmental screening instrument

Abstract

Immigrant populations are growing and permanently changing the demographic profile of the United States. Diverse cultural and linguistic backgrounds are imposing demands and challenges upon agencies serving young children and families. Culturally sensitive assessments are not always available for these populations, and psychometric properties of available instruments have not always been investigated. This study examined one developmental screening instrument translated and adapted for Spanish-speaking families, the Ages and Stages Questionnaires, Third Edition, and evaluated how scores of children on the 9-, 18-, and 30-month Spanish translation differed from scores of children on the English version of the Ages and Stages Questionnaires, Third Edition. Item characteristics across the English and Spanish versions of the Ages and Stages Questionnaires, Third Edition, were investigated. Findings indicated that most of the Ages and Stages Questionnaires, Third Edition, items functioned invariantly across language versions, indicating that these items were productive for gathering information, presented an adequate hierarchy difficulty for order of items, and were eliciting a range of response options included on the tool. Implications for practice and research are discussed, supporting cross-cultural studies on parent-completed questionnaires as necessary for effective screening and monitoring of young children’s development in a context of cultural and linguistic diversity.

Keywords

early childhood education and care early identification early infant development early intervention Latino families test adaptation

Introduction

Immigration of families to the United States has dramatically increased in the last two decades, transforming the demographic landscape and resulting in a population increase of 9 percent between 2000 and 2009, with numbers of immigrants rising from 281 to 307 million. At the same time, the Latino population has grown four times more rapidly than the overall population—by 37 percent—accounting for more than half of the additional 26 million people added to the US population over the past decade. Latinos account for almost one-sixth of the total US population and are estimated to represent 3 of every 10 people by 2050 (Saenz, 2010).

For Latino families with young children who immigrate to the United States, the acculturation process is often stressful, and children may be exposed to multiple risk factors that influence development such as poverty, poor housing, and lack of quality medical care. In addition, many immigrant families are involved in seasonal work in which children move frequently and often receive inconsistent educational and medical services. For children and their families exposed to these risk conditions, supports such as high quality early childhood services can be pivotal for optimizing developmental outcomes (Burger, 2010; Buysse et al., 2005; Karoly and Gonzalez, 2011; Reynolds et al., 2010). Access to early intervention/early childhood special education (EI/ECSE) services is also critical in terms of improving developmental outcomes by providing individualized targeted services when problems occur. Promoting optimal developmental and family outcomes (Bagnato, 2007) in the context of family proximal experiences, routines, and customs is of prime importance.

The need for accurate early identification processes for EI/ECSE and other specialized services that respect families’ culture and native language is evident. Several challenges are present, however, including a lack of financial resources to support the development of culturally appropriate assessments, lack of articulated program guidelines about how to assess young children whose language is other than English, and lack of quality developmental screening tools translated into languages other than English. Finally, related to culturally appropriate assessment, there exists a lack of psychometric studies conducted on linguistic and cultural adaptations for ethnic minorities (Peña, 2007; Squires et al., 2009).

Low-cost screening tools that are culturally sensitive and provide an efficient “first look” at a young child’s developmental repertoire are critical for early identification and targeted interventions for children in need of developmental supports (Guevara et al., 2013). The Ages and Stages Questionnaires, Third Edition (ASQ-3) (Squires and Bricker, 2009) is one cost-effective parent-completed screening tool widely used in the United States and translated into several languages (e.g. Spanish, Korean, Chinese, Turkish) for use with diverse young children and families. Although a large population of Latino infants and toddlers across the United States are being screened using the Spanish translation of the ASQ, only preliminary psychometric studies examining the translated form have been conducted. Preliminary field testing with Spanish-speaking families has been completed, but cutoff points have not been empirically tested. More evidence is needed on the psychometric properties of the ASQ-3 Spanish translation for early identification of young Latino children at risk for developmental delays.

Psychometric properties of the Spanish translation of the Ages and Stages Questionnaires, Third Edition (Spanish ASQ-3) including score distribution, equivalence, and utility were the focus of this research. The 9-, 18-, and 30-month intervals were selected as these are recommended by the American Academy of Pediatrics (AAP) for screening infants and toddlers (AAP, 2006). Research questions included the following: (1) How do scores of children on the 9-, 18-, and 30-month Spanish ASQ-3 differ from scores of children on the English version?; (2) Do the items on the 9-, 18-, and 30-month Spanish ASQ-3 function the same as items on the English version?; and (3) What is the ease of understanding and completion of the Spanish ASQ-3 for Spanish-speaking parents?

Material and methods

A non-experimental psychometric study was conducted to examine the equivalence of the 9, 18 and 30 month intervals of the ASQ-3 Spanish translation. Item response theory (IRT) modeling and traditional analyses (e.g. multivariate analysis of variance (MANOVA)) were used to examine whether items on the Spanish translation of the ASQ-3 functioned similarly to original ASQ-3 items at the 9-, 18-, and 30-month intervals. Utility data for Spanish-speaking parents were gathered via telephone interviews on a subset of parents.

Participants

A total of 798 English- and Spanish-speaking parents and their children ages 9, 18, and 30 months living in the United States participated; 424 parents were Spanish-speaking and 374 were English-speaking, as shown in Table 1. Children with disabilities were excluded; children who were typically developing as well as those experiencing risk factors (e.g. living below the federal poverty level, mother’s age of 19 years or younger at child’s birth) were included. Data were collected in public preschool programs; childcare centers; nonprofit agencies; health centers; publicly funded programs such as Woman, Infant and Children’s program (WIC); and online through the Internet (i.e. ASQ Oregon website and ASQ-3 publisher online data base). Ethnicity and language used by participants are summarized in Table 2. Approved human subjects and informed consent procedures approved by a university Institutional Review Board were used throughout the study.

Table 1.

Participants by ASQ-3 interval completed and language version.

ASQ-3 interval	Version
ASQ-3 interval	English	Spanish	Total
9 months	112	108	220
18 months	135	139	274
30 months	127	177	304
Total	374	424	798

ASQ-3: Ages and Stages Questionnaires, Third Edition.

Table 2.

Scores for each ASQ-3 age and domain by language version.

ASQ-3 interval	Domain	Language		ASQ-3, M (cutoffs)
ASQ-3 interval	Domain	English (n = 374), M (SD)	Spanish (n = 424), M (SD)	ASQ-3, M (cutoffs)
9 months	Communication	44.37 (12.95)	48.74 (10.90)	38.55 (13.97)
	Gross motor	44.69 (15.88)	44.03 (15.60)	46.72 (17.82)
	Fine motor	51.99 (12.10)	53.13 (9.92)	52.31 (31.32)
	Problem solving	50.41 (11.75)	50.92 (11.63)	49.51 (28.72)
	Personal social	42.12 (12.12)	47.01 (11.79)	42.47 (18.91)
18 months	Communication	34.09 (17.22)	36.75 (16.95)	42.30 (13.06)
	Gross motor	52.51 (12.71)	52.54 (13.84)	55.46 (37.38)
	Fine motor	48.89 (11.51)	48.54 (12.58)	52.44 (34.32)
	Problem solving	41.58 (13.12)	43.92 (12.24)	45.99 (25.74)
	Personal social	44.26 (12.99)	50.96 (10.30)	47.90 (27.19)
30 months	Communication	46.82 (15.37)	44.05 (16.53)	43.81 (33.30)
	Gross motor	51.19 (11.54)	51.30 (10.46)	53.54 (36.14)
	Fine motor	38.70 (16.80)	42.64 (14.53)	46.78 (19.25)
	Problem solving	43.58 (14.69)	41.17 (14.53)	50.18 (27.08)
	Personal social	46.25 (12.31)	47.08 (12.37)	51.87 (32.01)

ASQ-3: Ages and Stages Questionnaires, Third Edition; M: mean; SD: standard deviation.

Note: ASQ-3 normative means and cutoff scores were obtained from “ASQ-3 User’s Guide,” by Squires et al. (2009: 171). M and SD were derived from this sample. M in bold indicates statistically significant differences between language version used.

Demographic data were obtained according to procedures used in participating programs and agencies. Thus, demographic data were not collected on all participants or variables and parents were not required to provide demographic data to participate in the study.

Measures

Four measures were used: (1) ASQ-3, English version, (2) ASQ-3, Spanish version, (3) demographic form, and (4) utility interview.

ASQ

The Ages and Stages Questionnaires: A Parent-Completed, Child-Monitoring System, Third Edition (ASQ-3; Squires and Bricker, 2009) is a screening tool designed to accurately identify infants and young children with suspected developmental delay who may benefit from further intervention (Squires et al., 2009). Three of 21 questionnaire intervals were used in this research—9, 18, and 30 months. Each interval contains 30 items addressing skills organized in five developmental areas: communication, gross motor, fine motor, problem solving, and personal social with a section with general questions to address parental concerns related to their child’s development. For items addressing skills, parents answer Yes, when indicating that their child was able to consistently perform the specified skill; Sometimes, to indicate that the described skills were emerging; and Not yet, indicating that the skill described was not performed by the child at that time. Each developmental area has a possible total score of 60 points and each age interval has a specific cutoff score (i.e. 2 standard deviations below the mean domain score) used to determine whether a child appears to be developing as expected or if he or she may be at risk of developmental delays.

Test–retest reliability of the ASQ-3 was investigated by comparing two questionnaires completed by the same parent (N = 145) at a 2-week time interval, with 92 percent agreement. Interclass correlations ranged from 0.75 to 0.82, and interobserver reliability was established by comparing questionnaires completed by 107 trained examiners and parents with 93 percent agreement on children’s classifications (OK, risk), with interclass correlations ranging from 0.43 to 0.69. Coefficient alpha indicated moderate to strong internal consistency and correlations ranged from 0.60 to 0.85 between total score and domain scores. Concurrent validity was measured by comparing children’s performance on the Battelle Developmental Inventory (Battelle Developmental Inventory, Newborg, 2005)., and the results were obtained on the ASQ-3. Agreement for a group of 579 children suggested moderate to high agreement between classifications (i.e. both tests classified the child as typical, or as identified with delays). Sensitivity ranged from 85 percent to 92 percent, specificity ranged from 78 percent to 92 percent, and positive predictive value ranged from 32 percent to 64 percent. Overidentification across intervals ranged from 6 percent to 13 percent, and underidentification ranged from 1 percent to 13 percent. IRT analyses of web-based and paper–pencil completion suggested few differences between modes of completion (Yovanoff et al., 2013), and both formats are available for screening and monitoring children from 2 months to 5.5 years.

Spanish ASQ-3 (Squires and Bricker, 2009) is identical in format to the English translation, with revisions made to convey item intent to Spanish-speaking parents. The authors suggested similar cutoff scores for the Spanish translation as for the English after pilot studies with Spanish-speaking families living in the United States were conducted. Differences in scores were present in both negative and positive directions when English and Spanish risk samples were compared (Squires et al., 2009).

Only preliminary unpublished studies (Chen et al., 2010) have been conducted to study the item equivalence between the English and the Spanish translation of the ASQ, Second Edition (Squires and Bricker, 1999). The results showed that 76 out of 253 unique ASQ-2 items (30%) indicated differential item functioning (DIF) across all age intervals and domains. On the Spanish ASQ-3, 17 of the 76 unique items were discovered to either provide fewer examples than in the English version or contain minor grammatical errors.

Demographic questionnaire

A demographic questionnaire including general questions related to the family characteristics such as the child’s developmental status, gender, date of birth, mother’s age at child’s birth, mother’s level of education, and family income level was completed by parents using the pencil/paper ASQ-3 completion format. An abbreviated form (i.e. child’s gender, date of birth, risk status) was completed by some participants according to agency preference.

Parent interview

The parent interview was a structured interview with closed- and open-ended questions conducted over the phone and included questions related to language spoken at home, years in the country, attendance at childcare, language spoken at childcare, and questions about items on the ASQ-3 that were not applicable or were difficult to answer. The questions were organized and worded to facilitate accurate and complete parent responses (Sheatsley, 1983). A subset of parents completing the pencil/paper ASQ-3 Spanish version participated in the parent interview.

Results

Of the 798 participants, 374 parents completed the English ASQ-3 and 424 parents completed the Spanish ASQ-3 (see Table 1). A total of 92 percent of the ASQ-3 was completed independently by mothers or other caregivers (e.g. father, grandparents, foster parents) and the remaining 8 percent was completed during home visits with the help of a teacher or specialist. In all, 94 percent of the participants reported their ethnicity: Hispanic/Latino (n = 52.1%), White/Caucasian (n = 29.2%), mixed ethnicity (n = 7.3%), African-American (n = 2.5%), and Asian (n = 1.4%). For the English ASQ, over 80 percent of those parents had at least a high school degree; almost 50 percent had a college degree. A total of 19 percent had an annual income less than US$12,000.00; 34 percent had an income between US$12,000.00 and US$24,000.00; the remaining 47 percent had annual earnings above US$24,000.00. Insufficient data were gathered on Spanish-speaking families across education and income variables to report.

Approximately 47 percent of the children for whom the ASQ was completed were female, irrespective of language version. Less than 1 percent of the target children were born prematurely or below 3.5 pounds at birth or presented any medical or environmental risk factors.

ASQ-3 score distribution

Analysis of ASQ-3 scores was conducted to describe the sample and identify how results were distributed across groups (see Table 2). A MANOVA was performed for each ASQ-3 interval (i.e. 9, 18, and 30 months) to test the differences between groups, using SPSS 17.0 for Windows (Table 3). Using Wilk’s test of multivariate significance for the 9-month interval, with language version considered as independent variable (i.e. English or Spanish), outcomes were related statistically to the weighted multivariate combination of dependent variable measures, Λ = 0.89, F (5, 208) = 5.17, p < .05, η² = 0.11. For the 18-month ASQ-3 interval, language version was also related statistically to the multivariate combination of dependent variables, Λ = 0.88, F (5, 256) = 6.90, p < .05, η² = 0.12. Similarly, for the 30-month interval, the independent variable was related statistically to the multivariate combination of dependent variables, Λ = 0.94, F (5, 281) = 3.89, p < .05, η² = 0.06.

Table 3.

ASQ-3 domain score MANOVA results by interval.

ASQ-3 age	Source	Wilk’s Lambda Λ	F	df	Error	η²
9 months	Language	0.89	5.17**	5	208	0.11
18 months	Language	0.88	6.90**	5	256	0.12
30 months	Language	0.94	3.89**	5	281	0.06

ASQ-3: Ages and Stages Questionnaires, Third Edition; MANOVA: multivariate analysis of variance; df: degree of freedom.

Note: MANOVA-dependent variables consist of ASQ-3 domains (i.e. communication, gross motor, fine motor, problem solving, and personal social). All F statistics are exact.

p < .05; **p < .01.

In addition, univariate ANOVAs on each of the five domains comprising the multivariate composite on each age interval were conducted. The analysis for the 9-month interval revealed statistically significant mean differences between language versions on the communication, F (1, 212) = 8.33, mean squared error (MSE) = 141.24, p < .05, and personal social domains, F (1, 212) = 14.41, MSE = 129.96, p < .05. For the 18-month interval, statistically significant mean differences were obtained on the personal social domain, F (1, 260) = 23.87, MSE = 137.38, p < .05. Finally, for the 30-month interval, statistically significant mean differences were found on the fine motor domain, F (1, 285) = 5.85, MSE = 240.33, p < .05.

DIF

Interpretation of the group mean differences is based on the assumption that the ASQ functions invariantly, that is, equivalently, for the English and Spanish versions. DIF was selected as a model for examining the test’s internal features to detect whether the item characteristics (difficulty, fit to the psychometric model) varied across the language forms (i.e. Spanish and English). Using IRT, a Rasch partial credit (Masters and Wright, 1984) DIF model for polytomous items was fitted to the data to test the hypothesized invariance model. Following the partial credit model requirements, only the difficulty parameter was considered. Consistent with the Rasch model, the item discrimination and guessing parameters were constrained to 1 and 0, respectively. Winsteps 3.66 computer software (Linacre, 2008) was used to test the invariance model specifying the English participants as the reference group and the Spanish sample as the focal group. The measured order of items, the item fit statistics, and how the response structure was predicted were investigated before conducting the DIF analysis in order to confirm whether the model fits the ASQ-3 data for the English and Spanish translations.

Item fit

The Rasch model provides a probabilistic relation between observed responses and item difficulty, given the examinee’s trait level. The mean square (MNSQ) fit statistic indicates whether the Rasch partial credit model fits the observed data. An MNSQ range from 0.5 to 1.5 is considered acceptable fit. Values above 1.5 are considered an underfit to the data, indicating that items do not provide sufficient information about what is being measured due to excessive randomness. In this case, the item may not be sensitive enough. Values below 0.5 overfit the data, indicating overpredictability. A total of 90 ASQ-3 items in each language were examined. Only 16 items presented misfit. Table 4 presents the findings regarding misfit items by ASQ domain and age interval.

Table 4.

Mean square statistic by ASQ-3 language, interval, and domain.

Version/age interval	Domain	Item (MNSQ)
English 9 months	Gross motor	Item 1 (9.90) Balancing holding hands
		Item 2 (4.40) Sitting up straight
		Item 3 (0.38) Standing up holding furniture
Spanish 9 months	Gross motor	Item 1 (2.24) Supporting own weight while standing
Spanish 9 months	Gross motor	Item 2 (2.98) Sitting up straight on floor
English 18 months	Communication	Item 1 (2.81) Telling by pointing
English 18 months	Problem solving	Item 5 (4.20) Copying a single line
Spanish 18 months	Communication	Item 1 (3.52) Asks for item by pointing
	Gross motor	Item 1 (2.33) Bending over to pick up a toy
	Gross motor	Item 2 (0.41) Moving around by walking
	Problem solving	Item 3 (0.39) Getting a crumb from a bottle
English 30 months	Gross motor	Item 2 (0.44) Walking up or down 2 steps
	Problem solving	Item 2 (2.51) Finding a chair to reach object
	Personal social	Item 3 (2.22) Pushing a toy on wheels
Spanish 30 months	Fine motor	Item 1 (2.06) Turning motion with a hand
Spanish 30 months	Problem solving	Item 2 (2.78)

ASQ-3: Ages and Stages Questionnaires, Third Edition; MNSQ: mean square.

Note: mean square fit statistic value > 1.5 item is off-variable, noise is greater than useful information. Value < 0.5 item is overly predictable, item are measuring unexpected variables.

Response categories

A partial credit model was used to examine how probable the observation of each category relative to the item measure was and how well categories (i.e. Yes, Sometimes, Not yet) were being used by respondents. Each item was studied accordingly and its response structure was modeled. The probability curves indicated how the response structure is predicted to work for any future sample. A disordered threshold indicated that the category is relatively rarely observed, (i.e. uses a narrow interval on the latent variable, and so may indicate substantive problems with the rating or partial credit). Thresholds are the points at which adjacent category probability curves intersect. Figure 1 shows an ideally structured item fit (18-month communication item 2) that used all the category responses.

Figure 1.

Structure of an item with all category responses well used.

Out of 180 items examined, only 10 presented an unexpected structure. The most recurrent situation was having only two category responses used by respondents (i.e. 5 = sometimes and 10 = yes), observed on the English 9-month interval for communication (items 1 and 3), and gross motor (item 2), and on the Spanish translation for the fine motor (item 1) domain. Also, it was observed on the 18-month English version for fine motor (item 5), and on the 30-month interval, gross motor domain (item 1).

DIF analyses

Item function analyses for the reference group (English-speaking families) compared with the focal group (Spanish-speaking families), by domain and by each age interval, were completed. DIF analyses identified 27 of 90 items (30%) functioned differently for each group (alpha value 0.05), showing a significant difference in difficulty. Table 5 shows the ASQ-3 items that were found to function differently between the two language versions by interval and developmental domain. The calibration of items seemed to be represented evenly across languages and domains: 14 items appeared to be more difficult for the Spanish-speaking population and 13 items were more difficult for English-speaking participants.

Table 5.

ASQ-3 items that function differently by age and domain.

ASQ-3 interval	ASQ-3 domain	DIF items
9 month (n = 5)	Communication	Item 1(S) Making sounds “da”-“ga”
		Item 3(S) Making two similar sounds “ba-ba”
		Item 5(E) Following one simple command
	Personal social	Item 5(S) Helping with dressing
	Personal social	Item 6(E) Giving a toy
18 month (n = 11)	Communication	Item 3(S) Saying eight or more words
	Communication	Item 5(E) Pointing the correct picture
	Gross motor	Item 1(S) Bending over to pick an object
		Item 3(S) Walking well
		Item 4(E) Reaching objects climbing on
		Item 5(E) Walking down stairs with holding one hand
	Fine motor	Item 5(S) Turning the pages
	Problem solving	Item 2(E) Copying a single line,
	Problem solving	Item 6(S) Getting a crumb from a bottle
	Personal social	Item 1(E) Looking herself in a mirror
	Personal social	Item 2(S) Hugging a stuffed animal
30 month (n = 11)	Communication	Item 3(S) Pointing body parts
	Communication	Item 5(E) Following verbal directions
	Gross motor	Item 2(E) Walking up or down steps
		Item 3(S) Kicking a ball
		Item 5(E) Walking up steps with one foot
		Item 6(S) Standing on one foot
	Fine motor	Item 3(E) Stringing small items
	Fine motor	Item 5(S) Making a circle
	Problem solving	Item 2(E) Finding a chair to reach objects
		Item 3(E) Lining up four objects
		Item 5(S) Repeating numbers

ASQ-3: Ages and Stages Questionnaires, Third Edition; DIF: differential item functioning.

Note: item calibration for each language is represented as more difficult. S = item more difficult for Spanish-speaking participants. E = item more difficult for English-speaking participants.

Utility

Follow-up phone interviews were conducted with 31 Latino parents who agreed to participate after completing pencil/paper Spanish ASQ-3 on their children. The time elapsed between the completion of the ASQ-3 and the interview was less than 7 days. A total of 29 parents completed only one ASQ-3 and two parents completed two questionnaires on children at different ages. All interviews were completed in Spanish by the researcher over the phone and took 10–15 minutes each. The results are shown in Table 6 regarding ease of understanding, appropriateness of questions.

Table 6.

Readability and utility of the Spanish ASQ-3.

Interview question	Yes (%)	No (%)	Sometimes (%)
Was the ASQ-3 easy to understand?	30 (96.7)	–	1 (3.2)
The questions were appropriate for my child’s age	31 (100)	–	–
Was it easy for you to understand how items were presented?	31 (100)	–	–
Were the pictures and examples helpful for answering the items?	30 (96.7)	–	1 (3.2)

ASQ-3: Ages and Stages Questionnaires, Third Edition.

When parents were asked how they would change the ASQ-3 to make it better, 27 (87%) indicated that they would not introduce any changes to the questionnaire; they included comments such as “it is excellent,” “it is very complete,” “it is well written and clear,” and “it was easy to complete.” Four parents (12.9%) made minor suggestions such as using consistent gender pronouns instead of alternating between he and she and minor wording changes.

Discussion

The results obtained from the examination of the English and Spanish versions of the ASQ-3 items provided an initial examination of psychometric properties and cultural equivalence of the Spanish ASQ-3. When comparing overall ASQ-3 scores by version using one-factor MANOVA, the independent variable (i.e. ASQ-3 language version used by parents) was statistically related to the weighted multivariate combination of dependent variable measures (i.e. scores in communication, gross motor, fine motor, problem solving, and personal social domains). Univariate ANOVAs showed statistically significant mean differences in scores on communication and personal social domains on the 9-month interval, personal social domain on the 18-month interval, and on the fine motor domain on the 30-month interval. In all cases, mean scores were higher for the Spanish-speaking population. These differences may have been due to differences in the population. Out of 527 participants that provided recruitment information, 172 Spanish-speaking mothers were recruited from agencies that frequently monitor children’s development as part of their routines, while most of the English-speaking parents used the online system. Parents who voluntarily decide to complete a developmental screening instrument may be trying to find additional information and answers to their concerns, so this sample of English-speaking families might include more children at risk for developmental delays than the Spanish-speaking families. However, these findings are consistent with those reported in several international studies in which ASQ domain scores (e.g. communication, personal social, fine motor) showed differences across groups due to differences in cultural practices such as eating with chopsticks rather than forks or spoons and grammatical differences among languages such as use of pronouns and order of words in sentences (e.g. Borman et al., 2010; Dionne et al., 2006; Heo et al., 2008; Kapci et al., 2010; Kerstjens et al., 2009; Sarmientoe et al., 2010; Tsai et al., 2006).

Additionally, interpreting English- and Spanish-speaking children’s results in the ASQ-3 must be considered with caution because differences across versions may be due to nonequivalence in the way that items might be working. Items that exhibit DIF may be introducing unexpected variability across the samples.

Regarding how items functioned across the two ASQ-3 versions, only 16 of 180 items presented misfit. Equal numbers of items were found to be less productive across ASQ-3 versions (n = 8 English, n = 8 Spanish). DIF analyses identified 27 of 90 items that were functioning differently across groups, mostly at older age intervals. The remainder of the ASQ-3 items on the 9-, 18-, and 30-month intervals were functioning invariantly across versions, meaning that they are providing similar information about children’s performance. In addition, items appeared to be productive in gathering the expected information and response categories were properly selected by parents. Gross motor was the domain that had the most nonequivalent items between the focal and reference groups at 18 and 30 months, followed by the personal social domain at 30 months and the communication domain at 9 months. Direction of the difficulty level was distributed evenly across intervals and domains. Fourteen items appeared to be more difficult for the Spanish-speaking group and 13 presented higher difficulty for the English-speaking group. As mentioned above, more DIF items were found at older age intervals. These disparities may be associated with cultural differences that become more evident at older ages, and also they might be related to language discrepancy and/or differences in meaning due to item translation.

As a follow-up to the DIF analysis, three native Spanish-speaking specialists each with over 10 years of experience in the field of EI and/or linguistics reviewed DIF items to determine whether modifications that needed to be made (Peña, 2007). Based on the linguistic equivalence examination, semantic and syntactic concerns about the Spanish translation were indicated. The experts suggested modifications to items that were related to the meaning of the expressions translated from English to Spanish and identified some words used in the Spanish version that did not have the same interpretation across languages. Experts also mentioned that some Spanish questions seemed ambiguous in the way that they were worded or structured. Therefore, it will be important to introduce minor changes in wording using more precise Spanish terms and reorganizing the structure of ambiguous questions in order to accurately match the meaning and intention of the original English items.

Limitations

Although results of the examination of ASQ-3 items and parental feedback were positive, several limitations should be considered. These limitations mainly relate to the recruitment and characteristics of the sample.

Even though the sample size was adequate for applying the selected procedures of analysis, limited demographic information was included. English-speaking families were largely recruited from a screening website using the ASQ where parents voluntarily completed age-appropriate questionnaires along with demographic data (e.g. income level, mother’s educational level). These data were not gathered for the entire Spanish-speaking group as demographic information was optional for families to provide. Therefore, comparison between groups was difficult. Although ethnicity was a known variable for the entire sample, it was not possible to determine the country of origin for Latino participants or the immigration generation status, which are important variables to consider especially for conducting cross-cultural studies. The Latino population is not a homogeneous population and particularities from each subgroup should be considered and analyzed in future studies.

Related to recruitment, it was difficult to find Latino families who were willing to talk with university-related researchers. This was most likely due to a general suspicion of governmental authorities that may appear threatening to a subset of Latino families that have immigrated to the United States illegally and/or are suspicious of governmental authorities. Recruitment was more successful when a face-to-face contact was used or when the contact was made via a family member or a friend, but the process was extremely time consuming and resulted in a small sample. The results, especially those related to the utility and readability of items, may be associated with a biased sample of people who were more acculturated and wanted to know more about their children’s development. Utility and readability evaluation may change when a random sample is interviewed.

Whereas the objective of this study was to examine only the 9-, 18-, and 30-month intervals in their English and Spanish versions, this narrow focus might limit the possibility to generalize the results to the entire set of ASQ-3 intervals. Future studies should include the rest of the intervals to draw conclusions about the Spanish ASQ-3 cultural and linguistic appropriateness.

Another limitation related to the recruitment process was that different methodologies were used by parents to complete the questionnaires. English-speaking participants primarily used the online format through ASQ parent websites, and Spanish-speaking parents used both formats, although the numbers of each could not be determined because many agency personnel assisted parents with completion then entered the ASQ data to an online site. A recent study compared paper and the online versions of the ASQ-2 (Bricker and Squires, 1999) completed by parents and found few differences between completion methods (Yovanoff et al., 2013). If possible, the ASQ-3 completion format should be controlled in future research, especially when cross-cultural studies are being conducted to avoid additional sources of potential variability.

Conclusion

Implications for practice

The results of this study affirm the importance of providing appropriate and effective screening and monitoring systems for young children’s development in a context of cultural and linguistic diversity. All children and their families should have access to appropriate services, especially culturally sensitive assessment options. Having an accurate and unbiased screening tool that is appropriate for families from diverse linguistic and cultural backgrounds in the United States will increase the likelihood that children who might be at risk for developmental delays will be effectively identified. Study findings may guide the selection of intervention materials and screening practices that will enhance the inclusion of families from diverse cultural and linguistic backgrounds. Ongoing research oriented to study the equivalence of items in different cultural environments is highly recommended. The results of this study support the recommendations provided by early childhood professional organizations for developmentally and culturally appropriate practices (Copple and Bredekamp, 2009; National Association of for the Education of Young Children (NAEYC) and National Association of Early Childhood Specialists in State Departments of Education (NAECS/SDE), 2003; Sandall et al., 2006). Translation flaws detected on the Spanish ASQ-3 may guide improvements on item accuracy, adequacy, and productivity of screening materials and enhance opportunities for timely identification of developmental delays, especially for Latino children and families.

Cross-cultural research needs to be expanded and improved to the extent in which measurement instruments can be calibrated to the specific properties of a growing diverse cultural population in the United States. Research studies designed to examine test and item bias on developmental screening measures will assist the process of guaranteeing fairness in testing. Thus, examining item functioning and cultural appropriateness of the Spanish ASQ-3 from a comprehensive perspective may contribute to the growing body of knowledge on cross-cultural research.

Differences between mean groups on the ASQ-3 results might suggest the need for reviewing referral cutoff scores for the Spanish translation to avoid the possibility of test bias and misidentification of Latino children who might be at risk for developmental delays or disabilities. Additionally, future research is needed on the remaining ASQ-3 intervals to investigate item functioning and cutoff scores for the Spanish version. Culturally sensitive referral cutoff scores are a crucial aspect of effective developmental screening and monitoring instruments.

Naturalistic and culturally sensitive assessment procedures will offer reliable ways to recognize children’s abilities, needs, strengths, developmental patterns, and emergent skills without test bias (Hambleton, 2005; Hambleton and Zenisky, 2011). These opportunities should be available to a vast population of children during their first years of life to facilitate their access to specialized supportive services and improved developmental outcomes.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

American Academy of Pediatrics (AAP) (2006) Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening. Pediatrics 118: 405–420.

Bagnato

(2007) Authentic Assessment for Early Childhood Intervention. New York: The Guilford Press.

Borman

Sevcik

Romski

. (2010) Successfully translating language and culture when adapting assessment measures. Journal of Policy and Practice in Intellectual Disabilities 7: 111–118.

Bricker

Squires

(1999) Ages & Stages Questionnaires: A Parent-Completed, Child-Monitoring System, 2nd ed. Baltimore, MD: Paul H. Brookes.

Burger

(2010) How does early childhood care and education affect cognitive development? An international review of the effects of early interventions for children from different social backgrounds. Early Childhood Research Quarterly 25: 140–165.

Buysse

Castro

West

. (2005) Addressing the needs of Latino children: a national survey of state administrators of early childhood programs. Early Childhood Research Quarterly 20: 146–163.

Chen

Squires

Pomés

(2010) Testing cultural fairness of translations of a developmental screening test. Unpublished manuscript, October.

Copple

Bredekamp

(eds) (2009) Developmentally Appropriate Practice in Early Childhood Programs Serving Children from Birth through Age 8. Washington, DC: National Association for the Education of Young Children (NAEYC).

Dionne

Squires

Leclerc

. (2006) Cross-cultural comparison of a French Canadian and U.S. developmental screening test. Developmental Disabilities Bulletin 34(1–2): 1–56.

10.

Guevara

Gerdes

Localio

. (2013) Effectiveness of developmental screening in an urban setting. Pediatrics 131(1): 30–37.

11.

Hambleton

(2005) Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In: Hambleton

Merenda

Spielberger

(eds) Adapting Educational and Psychological Test for Cross-Cultural Assessment. Mahwah, NJ: Lawrence Erlbaum Associates, pp. 3–38.

12.

Hambleton

Zenisky

(2011) Translating and adapting test for cross-cultural assessments. In: Matsumoto

Van de Vijver

FJR

(eds) Cross-Cultural Research Methods in Psychology. New York: Cambridge University Press, pp. 46–74.

13.

Heo

Squires

Yovanoff

(2008) Cross-cultural adaptation of a pre-school screening instrument: comparison of Korean and U.S. populations. Journal of Intellectual Disability Research 52: 195–206.

14.

Kapci

Kucuker

Uslu

(2010) How applicable are Ages and Stages Questionnaires for use with Turkish children? Topics in Early Childhood Special Education 30: 176–188.

15.

Karoly

Gonzalez

(2011) Early care and education for children in immigrant families. Future of Children 21: 71–101.

16.

Kerstjens

Bos

Vergert

. (2009) Support for the global feasibility of the Ages and Stages Questionnaire as developmental screener. Early Human Development 85: 443–447.

17.

Linacre

(2008) Winsteps Rasch Measurement Computer Program. Chicago, IL: Winsteps.com.

18.

Masters

Wright

(1984) The essential process in a family of measurement models. Psychometrika 49: 529–544.

19.

National Association of for the Education of Young Children (NAEYC) and National Association of Early Childhood Specialists in State Departments of Education (NAECS/SDE) (2003) Early childhood curriculum, assessment, and program evaluation: building and effective, accountable system in programs for children birth through age 8. Available at: http://www.naeyc.org/files/naeyc/file/positions/CAPEexpand.pdf

20.

Newborg

(2005) Battelle Developmental Inventory, 2 ed. Chicago, IL: Riverside Publishing.

21.

Peña

(2007) Lost in translation: methodological considerations in cross-cultural research. Child Development 78: 1255–1264.

22.

Reynolds

Magnuson

(2010) Preschool-to-third grade programs and practices: a review of research. Children and Youth Services Review 32: 1121–1131.

23.

Saenz

(2010) Latinos in America 2010: population bulletin. Available at: http://www.prb.org/Publications/PopulationBulletins/2010/latinosupdate1.aspx (accessed December 2010).

24.

Sandall

Hemmeter

Smith

. (2006) DEC Recommended Practices: A Comprehensive Guide for Practical Applications in Early Intervention/Early Childhood Special Education. Denver, CO: Sopris West.

25.

Sarmiento

Squires

Ponte

(2010) Universal developmental screening: preliminary studies in Galicia, Spain. Early Child Development and Care 1: 1–11.

26.

Sheatsley

(1983) Questionnaire construction and item writing. In: Rossi

Wright

Anderson

(eds) Handbook of Survey Research. New York: Academic Press, pp. 195–230.

27.

Squires

Bricker

(1999) Ages and Stages Questionnaires: A Parent-Completed, Child-Monitoring System. 2nd ed. Baltimore, MD: Paul H. Brookes Publishing.

28.

Squires

Bricker

(2009) Ages and Stages Questionnaires: A Parent-Completed, Child-Monitoring System. 3rd ed. Baltimore, MD: Paul H. Brookes Publishing.

29.

Squires

Twombly

Bricker

. (2009) ASQ-3 User’s Guide. Baltimore, MD: Paul H. Brookes Publishing.

30.

Tsai

McClelland

Pratt

. (2006) Adaptation of the 36-month Ages and Stages Questionnaire in Taiwan: results from a preliminary study. Journal of Early Intervention 28: 213–225.

31.

Yovanoff

Squires

McManus

(2013) Adaptation from paper-pencil to web-based administration of a developmental questionnaire for young children. Infants and Young Children 26: 318–332.