Preschool mental health: The Brief Child and Family Intake and Outcomes System

Abstract

Despite the availability of effective early interventions, few preschoolers with mental health issues receive these services. This situation exists partly due to challenges in the identification of emotional and behavioral issues in young children. We developed the Brief Child and Family Intake and Outcomes System for Preschoolers, which is a 60-item standardized online parent questionnaire including three externalizing scales (Regulating Attention, Impulsivity, and Activity; Cooperating; Regulating Conduct), five internalizing scales (Separating from Parents; Managing Anxiety; Managing Social Anxiety; Regulating Compulsive Behaviour; Managing Mood), and two regulating states scales (Eating; Sleeping). We conducted a normative study of 1,200 Canadian children 3–5 years old, stratified by sex, age, geographic region, and parents’ marital status, income, and education. Confirmatory factor analyses demonstrated good model fit, and the relationship between items and scales did not vary significantly between boys and girls or among 3-, 4-, and 5-year-old children. Reliability estimates indicated high internal consistency and 2-month test–retest reliability for a subsample (n = 100) ranging from .44 to .73. Providing preliminary evidence of validity, scale scores had positive relations with measures of child functioning challenges, family distress, caregiver mood, and demographic risk variables. We extend earlier work by including clinically relevant emotional-behavioral scales while at the same time minimizing respondent burden and providing norms for Canadian preschoolers. The questionnaire could be used in children’s mental health settings, primary care, child welfare, and day-care and school facilities, for intake, triage, and describing 3- to 5-year-old children.

Keywords

Preschoolers emotional behavioral intake questionnaire

Accumulating evidence suggests that emotional and behavioral problems in preschoolers are fairly common (Egger & Angold, 2006; Gudmundsson et al., 2013; present in 10%–26% of 3- to 5-year-old children) and predict later mental health problems (Caspi et al., 1996; Woodward et al., 2017). However, despite the availability of effective early interventions, few young children with emotional-behavioral issues receive intervention (Oh & Bayer, 2015). This situation may be partly due to challenges in identifying mental health issues in young children (Oh & Bayer, 2015). In early development, there is rapid cognitive and behavioral change and problem behaviors may be common. For example, many young children have temper tantrums at least occasionally (Gardner & Shaw, 2008). In fact, research suggests that pediatric primary care providers and parents tend to under-identify emotional-behavioral problems in young children (Sheldrick et al., 2011).

Measurement tools with normative data can be helpful in identifying those who might benefit from early intervention (Duncan et al., 2018). Standardized questionnaires may reveal problems which parents fail to share or fully describe during intake interviews, and normative data can help interpret scores. Another purpose of the questionnaire could be to inform decisions regarding triage to appropriate services. Measures with multiple scales may support more comprehensive (or differentiated) and client-centerd treatment plans.

Questionnaires for parents of young children already exist (e.g., Achenbach & Rescorla, 2000; Eyberg & Pincus, 2012; Gleason et al., 2010; Goodman, 1997; Reynolds & Kamphaus, 2015; Saylor et al., 1999; Squires et al., 2015; see Table 1 for a summary of some commonly used measures). Only two questionnaires have more than a few scales: the Child Behavior Checklist 1.5–5 (Achenbach & Rescorla, 2000) and the Behavior Assessment System for Children, Third Edition Parent Rating Scale Preschool (Reynolds & Kamphaus, 2015), both of which are long (99 and 139 items, respectively), entailing respondent burden. Although epidemiological research suggests national variation in rates of children’s mental health problems (e.g., Merikangas et al., 2009), no existing questionnaires provide Canadian norms. We extend earlier work by including clinically relevant scales while at the same time minimizing respondent burden and providing norms for Canadian preschoolers.

Table 1.

Summary of Some Commonly Used Measures.

Measure	Ages	Number of items	Scales
Ages and Stages Questionnaire Socio-Emotional Scale-2 (Squires et al., 2015)	2, 6, 12, 18, 24, 30, 36, 48, and 60 months	40	Self-regulation; Compliance; Communication; Adaptive Functioning; Autonomy; Affect; Interaction with People
Behavior Assessment System for Children, Third Edition Parent Rating Scale Preschool (Reynolds & Kamphaus, 2015)	2–5 years	139	Hyperactivity; Aggression; Anxiety; Depression; Somatization; Attention Problems; Atypicality; Withdrawal; Adaptability; Social Skills; Functional Communications; Activities of Daily Living
Child Behavior Checklist 1.5–5 (Achenbach & Rescorla, 2000)	18–60 months	99	Emotionally Reactive; Anxious/Depressed; Somatic Complaints; Withdrawn; Sleep Problems; Attention Problems, Aggressive Behavior
Early Childhood Screening Assessment (Gleason et al., 2010)	18–60 months	40	Child Emotional-Behavioral Problems; Parental Depression; Parental Distress
Eyberg Child Behavior Inventory (Eyberg & Pincus, 2012)	2–16 years	36	Intensity scale; Problem scale
Pediatric Emotional Distress Scale (Saylor et al., 1999)	2–10 years	21	Anxious/Withdrawn; Fearful; Acting Out
Strengths and Difficulties Questionnaire (Goodman, 1997)	3–16 years	25	Emotional Symptoms; Conduct Problems; Hyperactivity/Inattention; Peer Relationship Problems; Prosocial Behaviour

The Present Study

The Brief Child and Family Intake and Outcomes System (BCFIOS) originally was developed for 6–12 year olds and 13–18 year olds (Boyle et al., 2009; Cunningham et al., 2009). As part of a program of research to develop versions of this measure for younger children, we created questionnaires to be used with parents of infants and toddlers (described in Niccols et al., 2018, 2019). In this article, we describe the development of the preschool version, the results of our normative study of 1,200 Canadian children 3–5 years old, and psychometric properties.

Method

The Development of the Questionnaire

We consulted experts in early child development and existing diagnostic classification systems in order to develop items and scales that were relevant to young children’s emotional-behavioral regulation. For each item, parents are instructed “Following are statements that describe some of the feelings and behaviours of children. Please read each item and select the response that best describes your child in the last 6 months” and then asked to respond to the individual items (e.g., seems sad or unhappy). Response choices are on a 3-point scale (0 = never; 1 = sometimes; 2 = often).

In a pilot study, a third-party survey firm, Research Now, conducted anonymous online surveys of parents of 301 children 3–5 years old who were representative of the Canadian population in terms of geographic region, marital status, income, and education (cf. Statistics Canada, 2011). There were no missing data as Research Now’s survey process did not allow participants to skip questions. Exploratory factor analyses were conducted (see Appendix). The final questionnaire contained 60 items, with 10 scales of 6 items each: 3 externalizing scales (Regulating Attention, Impulsivity, and Activity; Cooperating; Regulating Conduct), 5 internalizing scales (Separating from Parents; Managing Anxiety; Managing Social Anxiety; Regulating Compulsive Behaviour; Managing Mood), and 2 regulation of states scales (Eating; Sleeping) (see Table 2). The items are at a Flesch–Kincaid reading level of Grade 6.5.

Table 2.
Scales and Items on the Child and Family Intake and Outcomes System for Preschoolers.

Scales and items

Regulating Attention, Impulsivity and Activity

Distractible, has trouble sticking to an activity

Fails to finish things he/she starts

Has difficulty following directions or instructions

Impulsive or acts without stopping to think

Jumps from one activity to another

Fidgets

Cooperating

Defiant, talks back to adults

Blames others for his/her own mistakes

Easily annoyed by others

Angry and resentful

Loses temper

Refuses to follow rules

Regulating Conduct

Acts physically aggressive (hits, bites, or kicks)

Starts fights with other children

Does things that hurt others

Acts mean to other children

Breaks, damages, or destroys thing

Steals or takes things from others

Separating from Parents

Worries bad things will happen to loved ones

Worries about being separated from loved ones

Scared to sleep without parents nearby

Overly upset when leaving loved ones

Complains of feeling sick before separating

Overly upset while away from loved ones

Managing Anxiety

Worries about doing better at things

Worries about past behavior

Worries about doing the wrong thing

Worries about things in the future

Afraid of making mistakes

Overly anxious to please people

Managing Social Anxiety

Shy around other children

Avoids meeting new children

Avoids groups with unfamiliar children

Anxious or worried in social situations with other children

Needs a long time to feel comfortable with other children

Easily embarrassed in social situations with other children

Regulating Compulsive Behaviour

Does things like washing hands or checking doors or lights over and over again

Spends too much time ordering or arranging things in a particular way

Insists that things be unusually clean or germ free

Has difficulty completing tasks because they need to be “perfect”

Worries a lot about dirt and germs

Gets upset if prevented from doing things in a certain order

Managing Mood

Has no interest in his/her usual activities

Gets no pleasure from usual activities

Has trouble enjoying himself/herself

Not as happy as other children

Feels hopeless

Seems sad or unhappy

Eating

Eats too little

Refuse to eat foods that you think are healthy

Resists eating

Not hungry when it is time to eat

Eats only a few particular foods

Has an irregular eating pattern

Sleeping

Sleeps less than other children of the same age

Has difficulty staying asleep

Has an irregular sleep schedule

Has difficulty falling asleep

Has difficulty falling back asleep after waking at night

Has difficulty separating at bedtime

The Normative Study

As described in our papers on the infant and toddler questionnaires (Niccols et al., 2018, 2019), Research Now again conducted anonymous online surveys of parents for the present study. In addition to demographic information and the preschool questionnaire, we collected information on child functioning, family distress, and caregiver mood. The measures were translated into French by Traductions A la Page, informally checked by several bilingual individuals, and made available in French to those who selected the French version. The Hamilton Integrated Research Ethics Board deemed this study exempt from review, as per Article 2.4 of the Tri-Council Policy Statement (Canadian Institutes of Health Research et al., 2014).

Participants and procedure

Research Now e-mailed a survey link to adult members of their survey panel living in Canadian households with at least one child 3–5 years old, with only one person per household receiving the invitation. Participants completed an electronic consent and answered questions regarding child sex, child age, and demographic characteristics. For those with more than one child in the target age range, respondents were alternatively instructed to complete the questionnaire regarding their first born or their second born child. The sample was stratified by child sex (boys and girls) and child age (3, 4, and 5 years), with approximately 200 children in each of the six strata. The sample also was stratified on demographic characteristics (geographic region, parental marital status, income, and education; as per Statistics Canada, 2011).

Our focus was to obtain a sample representative of the Canadian population (not a random sample); therefore, as stratification cells were filled, potential participants were gated in only if their responses to the demographic questions indicated that they met criteria for any stratification cells remaining to be filled. Participants who were gated in then completed the questionnaires. Two weekly reminders were sent. Upon completion of the survey, Research Now provided each respondent with virtual currency (e.g., air miles). In order to provide a reasonable estimate of test–retest reliability, 100 participants completed the survey 6–8 weeks later (cf. Streiner & Norman, 2014). All participants received the survey link and as soon as the subsample of 100 was obtained (with 16–17 in each age and sex cell), the survey was terminated.

Child functioning

Child functioning was assessed using the Child Functioning Scale, which has 6 items on day-to-day functioning (e.g., “has difficulty with activities such as school, sports, lessons, or playdates”). These items are rated on a 3-point scale from 1 (never) to 3 (often). Higher scores indicate more difficulties with day-to-day-functioning. Internal consistency (Cronbach’s α) was .80 for our sample and 2-month test–retest reliability was .65.

Family distress

Family distress was assessed using the Impact on Family Scale, which has shown reliability and validity in previous studies (e.g., Niccols et al., 2018, 2019). The Impact on Family Scale has 6 items on family activities, conflict, and anxiety (e.g., “How frequently has your child’s behaviour prevented you from taking him/her out shopping or visiting?”; “How frequently has your child’s behaviour caused you to be anxious or worried about his/her chances for doing well in the future?”). These items are rated on a 4-point scale from 1 (never) to 4 (always). Higher scores indicate more distress. Internal consistency (Cronbach’s α) was .82 and 2-month test–retest reliability was .79.

Caregiver mood

Caregiver mood was assessed using the Center for Epidemiologic Studies Depression Scale (Eaton et al., 2004), which has good psychometric properties (e.g., Devins et al., 1988). We used a 6-item short form that has been used in previous studies (e.g., Niccols et al., 2018, 2019). Informants are asked “During the past week, how often have you…?” (e.g., felt depressed). Response choices are on a 4-point scale from 1 (less than 1 day) to 4 (5–7 days). Higher scores indicate more distress. Internal consistency (Cronbach’s α) was .85 and 2-month test–retest reliability was .40.

Data analyses

As described in our papers on the infant and toddler questionnaires (Niccols et al., 2018, 2019), we used the following statistical methods. Based on the results of the exploratory factor analyses, we used Mplus 7.4 (Muthén & Muthén, 1998–2015) to conduct a categorical confirmatory factor analysis (CCFA) on the total normative sample and by sex and age. To assess model fit, we used multiple indices: the Tucker–Lewis Index (TLI), the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the weighted root mean square residual (WRMR). Although a value of .90 or greater has been considered acceptable, Hu and Bentler (1999) recommended that the TLI or CFI needs to be greater than .95 and that the RMSEA should be .06 or lower. In addition, DiStefano et al. (2018) recommend a cutoff value of 1.0 for the WRMR to be considered to indicate good fit. We did not use the χ² statistic as it is sensitive to sample size and is a test of exact fit (Little, 2013, pp. 107–108).

We used multiple group categorical confirmatory factor analyses (MGCCFA) to test for measurement invariance across sex (boys and girls) and age (3, 4, and 5 years) subgroups within a structural equation modeling framework. Muthén and Asparouhov (2002) suggest performing MGCCFA using different approaches depending on the objectives of the multiple group comparisons. We adopted the Delta approach or parameterization because, based on analysis of models with continuous indicators, it has been shown that the main requirement for multiple group comparisons is ensuring factor loadings and thresholds are invariant across groups as residual variances are usually not invariant. Using the weighted least square mean- and variance-adjusted estimator, first, thresholds and factor loadings are unconstrained across subgroups, while the scale factors are fixed at one and factor means fixed at zero in the subgroups. Next, thresholds and factor loadings are constrained to be equal across the subgroups with the scale factors fixed at one for one subgroup and unconstrained in the other subgroup(s) and factor means fixed at zero in one subgroup and unconstrained in the other subgroup(s) (constrained model). A difference of χ² test was computed using scaling correction factors with the DIFFTEST option where a statistically significant χ² indicates that the hypothesis that the constrained parameters are invariant across subgroups (i.e., the equality constraints are violated) is rejected. To obtain evidence of measurement invariance, we compared the model with constrained parameters to the models where parameters were not constrained and used the χ² difference test as well as the criterion that change in CFI or TLI was less than .01 (Cheung & Rensvold, 2002). It should be noted that these guidelines and criteria were derived for models with continuous indicators using the maximum likelihood estimator.

To examine the distribution of scores and reliability of the scales, we calculated means, standard deviations, and Cronbach’s αs for the total sample, and 2-month test–retest reliability correlations for a subsample using IBM SPSS version 26.0 (IBM Corp, 2019). To examine validity, we computed correlations between the scale scores and measures of child functioning, family distress, and caregiver mood, and t-tests and analyses of variance to examine the association between scale scores and marital status, education, and income using IBM SPSS version 26.0 (IBM Corp, 2019). To assess whether the theoretically different scales were measuring distinct constructs, we conducted analyses of discriminant validity in the latent variables (Hair et al., 2014).

Results

Descriptive Information

As described in our papers on the infant and toddler versions of the tool (Niccols et al., 2018, 2019), survey invitations were e-mailed to adult members of the Research Now survey panel living in Canadian households with at least one child. In February 2017, approximately 11,000 adults received the e-mail invitation. Of the 1,703 adults who entered the survey, 503 did not complete the survey (304 had no children 3–5 years old and 199 did not meet stratification requirements). Therefore, 1,200 parents of 3- to 5-year-old children completed the questionnaires (70% of those who entered the survey) and, 6–8 weeks later, 100 of these parents completed the questionnaires a second time. There were no missing data as Research Now’s survey process did not allow participants to skip questions. Demographic information on survey respondents is provided in Table 3, which demonstrates the representativeness of the sample and subsample (in terms of geographic region, marital status, income, and education) compared to the Canadian population.

Table 3.
Demographic Characteristics for the Total Study Sample of Children 3–5 Years Old (N = 1,200) and Test–Retest Subsample (N = 100).

Variable Canadian population^a Study sample Test–retest subsample

% % N % N

Region

Atlantic 7.0 7.0 84 11.0 11

Quebec 23.0 23.0 276 19.0 19

Ontario 40.0 40.0 480 38.0 38

West 30.0 30.0 360 32.0 32

Marital status

Single 16.0 16.0 192 18.0 18

Married/common law 84.0 84.0 1,008 82.0 82

Family income/year

<$29,999 14.0 12.4 148 12.0 12

$30,000–$79,999 37.0 35.2 423 34.0 34

>$80,000 49.0 45.9 551 53.0 53

Preferred not to answer 6.5 78 1.0 1

Education

High school 25.0 25.0 300 25.0 25

College 45.0 43.9 527 44.0 44

University 30.0 30.0 360 30.0 30

Preferred not to answer 1.1 13 1.0 1

Child age

3 years 33.3 400 34.0 34

4 years 33.3 400 33.0 33

5 years 33.3 400 33.0 33

Child sex

Male 50.0 600 50.0 50

Female 50.0 600 50.0 50

Siblings

0 25.9 311 31.0 31

>1 74.1 889 69.0 69

Respondent sex

Female 65.8 790 62.0 62

Male 34.2 410 38.0 38

Respondent parent status

Biological parent 95.4 1,145 93.0 93

Adoptive, foster, other 4.6 55 7.0 7

Language survey completed in

English 77.0 924 81.0 81

French 23.0 276 19.0 19

Primary language spoken in the home

English 72.9 875 78.0 78

French 23.4 281 20.0 20

Other 3.7 44 2.0 2

Respondent country of birth

Canada 84.5 1,014 90.0 90

Other 15.5 186 10.0 10

^a Statistics Canada (2011).

Confirmatory Factor Analyses

In Table 4, results of the CCFA for the total sample and each of the sex and age subgroups are presented. Even though the χ² statistics for the total sample, boys, girls, and 3-, 4-, and 5-year-old children were significant (ps < .001), the fit statistics of the models were adequate and met our criteria. These findings suggest that the measurement model fit each of the subgroups and we could proceed to testing the equivalence of this model across the subgroups.

Table 4.
Fit Statistics from the CCFA (N = 1,200).

Model χ² df p Value CFI TLI RMSEA (95% CI) WRMR

Total sample 4,076.251 1,665 <.001 .935 .931 .035 (.035, .036) 1.497

Boys (n = 600) 2,822.227 1,665 <.001 .943 .939 .034 (.032, .036) 1.227

Girls (n = 600) 2,559.523 1,665 <.001 .949 .945 .030 (.028, .032) 1.185

3 years (n = 400) 2,317.193 1,665 <.001 .955 .952 .031 (.028, .034) 1.124

4 years (n = 400) 2,195.886 1,665 <.001 .955 .952 .028 (.025, .031) 1.060

5 years (n = 400) 2,345.257 1,665 <.001 .939 .935 .032 (.029, .035) 1.085

Note. CCFA = categorical confirmatory factor analysis; CFI = comparative fit index; TLI = Tucker–Lewis Index; RMSEA = root mean square error of approximation; WRMR = weighted root mean square residual; CI = confidence interval.

In Table 5, we present the results of the MGCCFA for the sex and age subgroups, using the Muthén and Asparouhov (2002) approach to testing measurement equivalence for categorical indicators across subgroups. Although the χ² tests of difference in the two models for age were significant, the changes in the χ² statistics suggested an improvement in the model fit. The test of difference in models showed that the hypothesis of equality of factor loadings and thresholds across sex could not be rejected, DIFFTEST: χ²(100) = 118.541, p = .0995; ΔCFI = −.001, ΔTLI = −.003, ΔRMSEA = .001. The differences in fit indices for the test of measurement equivalence of the model for age also were within the acceptable limits of the criteria suggesting lack of evidence of violation of equality of factor loadings and thresholds, DIFFTEST: χ²(200) = 295.923, p < .0001; ΔCFI = −.001, ΔTLI = −.003, ΔRMSEA = .001. Hence, we cannot reject the hypothesis of similarity of models and consequently equality of factor loadings and thresholds.

Table 5.
MGCCFA Results Testing Measurement Equivalence (N = 1,200).

Model χ² df p Value CFI TLI RMSEA (95% CI) WRMR

Sex 118.541^a 100 .0995

Model 1 .946 .942 .032 (.030, .034) 1.706

Model 2 .947 .945 .031 (.030, .033) 1.724

Age 295.923^a 200 <.0001

Model 1 .949 .946 .031 (.029, .032) 1.888

Model 2 .950 .949 .030 (.028, .032) 1.932

Note. MGCCFA = multiple group categorical confirmatory factor analyses; CFI = comparative fit index; TLI = Tucker–Lewis Index; RMSEA = root mean square error of approximation; WRMR = weighted root mean square residual; CI = confidence interval.

^aχ² difference test between Model 1 and Model 2. χ² difference testing was carried out using scaling correction factors with the DIFFTEST option in Mplus.

In Online Supplemental Tables S1, S2, S5, and S6, we present the results of the multigroup analysis with parameter estimates constrained to equality between subgroups. The parameter estimates reported include unstandardized and standardized factor loadings and thresholds with standard errors for the unstandardized parameters. In Online Supplemental Tables S3 and S7, correlations among the latent variables are reported by age and sex subgroups. In Online Supplemental Tables S4 and S8, we report subgroup (sex and age) differences in latent variable means and the standard errors associated with the differences. Boys had significantly lower means than girls for four latent variables: Cooperating, Regulating Conduct, Regulating Compulsive Behaviour, and Regulating Attention, Impulsivity, and Activity. Means were significantly lower for 5-year-olds compared to 3-year-olds for two latent variables, Regulating Conduct and Sleeping, and higher for Managing Anxiety. Means were significantly lower for 4-year-olds compared to 3-year-olds for one latent variable, Regulating Conduct, and higher for Managing Anxiety.

Distribution of Scores and Reliability

Scale means ranged from 1.29 to 5.41 (possible range 0–12) and standard deviations ranged from 1.95 to 3.07. Scale scores were skewed toward low scores, except for Cooperation and Regulating Attention, Impulsivity, and Activity which were not skewed. Skewness ranged from 0.149 (Regulating Attention, Impulsivity, and Activity) to 2.058 (Managing Mood; SE of skewness = 0.141). Reliability estimates are presented in Table 6. Total sample internal consistency (Cronbach’s αs) ranged from .78 to .88, values which are considered high (cf. Streiner & Norman, 2014). Two-month test–retest reliability for a subsample (n = 100) ranged from .44 to .73. Only one of the scales (Managing Mood) had unacceptable test–retest reliability (<.50).

Table 6.
Internal Consistency (Cronbach’s α) for Total Sample (N = 1,200) and 2-Month Test–Retest Reliability Using Test Data for the Total Sample (N = 1,200) and Retest Data for a Subsample (n = 100).

Scale Cronbach’s α Test–retest r ^a (95% CI)

Regulating Attention, Impulsivity, and Activity .79 .56 (.26, .77)

Cooperating .78 .56 (.25, .77)

Regulating Conduct .81 .62 (.33, .80)

Separating from Parents .78 .57 (.27, .77)

Managing Anxiety .81 .64 (.36, .82)

Managing Social Anxiety .88 .64 (.37, .82)

Regulating Compulsive Behaviour .84 .50 (.17, .73)

Managing Mood .84 .44 (.09, .69)

Eating .87 .73 (.50, .86)

Sleeping .83 .69 (.44, .84)

Note. CI = confidence interval.

^a Estimates were obtained performing the analysis using scale scores in Mplus with FIML option and controlling for age group, sex, region, parental education, marital status, and family income as auxiliary variables to account for missingness.

Preliminary Validity

Most scales showed significant moderate to strong positive correlations with Child Functioning and Impact on Family scores and significant small positive correlations with caregiver mood scores (see Table 7): Parents who rated their children as having higher scores on the scales reported more child functioning difficulties, more family distress, and more problems with depressed mood.

Table 7.
Correlations for the Preschooler Scales and Measures of Child Functioning, Family Distress, and Parental Mood (N = 1,200).

Scale Child Functioning Impact on Family Parental mood

Regulating Attention, Impulsivity, and Activity .43* .37* .24*

Cooperating .51* .48* .25*

Regulating Conduct .59* .52* .25*

Separating from Parents .42* .39* .27*

Managing Anxiety .43* .39* .23*

Managing Social Anxiety .41* .34* .17*

Regulating Compulsive Behaviour .44* .46* .22*

Managing Mood .60* .60* .24*

Eating .39* .37* .28*

Sleeping .52* .49* .30*

***p < .001

Some scale scores also were related to family demographic characteristics. Single parents reported that their children had more difficulties with Regulating Attention, Impulsivity, and Activity, Separating from Parents, Managing Anxiety, Regulating Compulsive Behaviour, Managing Mood, and Sleeping, ts(1199) = 4.49, 43.15, 8.68, 8.70, 3.89, and 7.54, respectively, ps < .05. Parental education was related to preschoolers’ scores on Regulating Attention, Impulsivity, and Activity, F(3, 1199) = 4.36, p = .005. Post hoc Tukey tests revealed that, compared to parents with university education, parents with college education reported that their preschoolers had more difficulties with Regulating Attention, Impulsivity, and Activity, p < .05. Parental income was related to preschoolers’ scores on Separating from Parents, Managing Compulsive Behaviour, Sleeping, and Eating, Fs(3, 1199) = 10.80, 10.49, 4.37, and 3.98, respectively, ps < .05, with post hoc Tukey and Games–Howell tests revealing that low-income parents reported more difficulties in these areas than high-income parents, ps < .05.

In Online Supplemental Table S9, we present the results of analyses of discriminant validity in the latent variables. Providing good evidence of discriminant validity of the scales, the square root of the average variance extracted for each latent variable was larger than the intercorrelations between each particular latent variable and the other latent variables (Hair et al., 2014), with the exception of Cooperating and Regulating Conduct.

Discussion

After developing standardized online questionnaire versions of the BCFIOS for school-age children and adolescents (Boyle et al., 2009; Cunningham et al., 2009) and infants and toddlers (Niccols et al., 2018, 2019), we created a version for preschool-age children. Confirmatory factor analysis of this 60-item questionnaire supported the item structure of the scales. Data from a representative Canadian sample of parents of 1,200 children 3–5 years old revealed 10 factors: 3 externalizing scales (Regulating Attention, Impulsivity, and Activity; Cooperating; Regulating Conduct), 5 internalizing scales (Separating from Parents; Managing Anxiety; Managing Social Anxiety; Regulating Compulsive Behaviour; Managing Mood), and 2 regulation of states scales (Eating; Sleeping). Young children’s mental health problems are presumed to reflect difficulties in these areas (thought to be precursors of later disorders such as attention deficit disorder, oppositional defiant disorder, conduct disorder, anxiety, depression, and challenges regulating physiological senses and states) (e.g., Woodward et al., 2017). Therefore, the factor structure of the scales is consistent with our goals for item selection, in that they are functionally meaningful and clinically relevant. The scales represent common reasons for referral for preschoolers and can be part of a clinical presentation reflecting dysregulation and distress.

Psychometric properties of the instrument are good. Results of the confirmatory factor analysis demonstrated good model fit. The relationship between items and scales did not vary significantly by sex or age, demonstrating reliable model fit. Reliability estimates indicated high internal consistency of the scales and that only one of the scales (Managing Mood) had unacceptable stability over 2 months. There also was evidence of discriminant validity in almost all of the latent variables, suggesting that the theoretically different scales were measuring distinct constructs. The exceptions were two scales (Cooperating and Regulating Conduct) representing disruptive behavioral challenges that are commonly comorbid (Dougherty et al., 2014).

Providing preliminary evidence of validity, preschooler scales were related to measures of child functioning, family distress, caregiver mood, and demographic characteristics. Preschoolers with lower scores on the scales had parents who reported less challenging family circumstances (married/common law, university education, high income) and preschoolers with higher scores on the scales had parents who reported more child functioning difficulties, higher levels of family distress, and more problems with depressed mood. In previous studies, preschooler emotional-behavioral problems also were related to demographic characteristics, family distress, and parental mood (e.g., Weitzman et al., 2014). Clinical trials have shown that parenting programs that reduce emotional-behavioral problems in preschoolers also reduce parental distress (e.g., Sourander et al., 2016). Our results add to these findings and provide preliminary evidence of validity of the preschool scales.

Limitations and Recommendations for Future Research

This study has some limitations that warrant caution in interpretation of the results. First, there is no information on sensitivity and specificity, which would assist in assessing this questionnaire’s suitability as a screening measure. In future studies, the questionnaire could be compared to an established measure of preschooler mental health such as the Diagnostic Interview for the Preschool Age (Scheeringa & Haslett, 2010) or observational, psychophysiological, or other measures of preschooler emotional-behavioral functioning. Nevertheless, the study provides information on a representative sample of Canadian preschoolers and concurrent external validity.

Second, we did not evaluate predictive validity. Others have demonstrated the link between preschool emotional-behavioral problems and later mental health (e.g., Woodward et al., 2017). Prospective follow-up studies of preschoolers previously assessed with the BCFIOS for Preschoolers would be helpful.

Third, we investigated the online version of the questionnaire, although it also could be administered by clinicians as an interview or self-completed by parents on paper. Also, the internet response sample is likely a self-selecting group. Future studies comparing the different methods of administration and using other recruitment methods may be informative.

Fourth, we used simple direct translation to make the questionnaire available to French participants, however, this method does not ensure that it is culturally appropriate for French Canadians. Also, we did not have enough French questionnaires completed to assess the equivalence of the factor structure for the English and French versions. Future research should include additional cross-cultural translation procedures and evaluation.

Last, in order to use this questionnaire as an outcome measure to assess treatment effects, future research would need to demonstrate its utility in an intervention evaluation. Investigating sensitivity to change and differential sensitivity to change compared to existing tools would be useful in this regard. Also, updated school-age and adolescent versions would be helpful to track change over time.

Conclusions

In this study, we demonstrated the factor structure, internal consistency, and test–retest reliability of the BCFIOS for Preschoolers. Confirmatory factor analysis of data from a representative sample of parents of 1,200 Canadian children 3–5 years old revealed 10 factors relevant to preschooler mental health: 3 externalizing scales (Regulating Attention, Impulsivity, and Activity; Cooperating; Regulating Conduct), 5 internalizing scales (Separating from Parents; Managing Anxiety; Managing Social Anxiety; Regulating Compulsive Behaviour; Managing Mood), and 2 regulatory scales (Eating; Sleeping). There also was preliminary evidence of validity, as scale scores were associated with measures of child functioning, family distress, caregiver mood, and demographic characteristics.

The BCFIOS for Preschoolers may be of interest to service providers, researchers, and individuals in leadership and governance roles. While acknowledging the limitations noted above, it could be used as an online questionnaire in children’s mental health settings, primary care, child welfare, schools, and day-care facilities, for intake and triage, tracking, and describing young children (cf. Duncan et al., 2018). Hopefully, its use will lead to increased access to effective early interventions, thereby reducing unmet mental health needs among young children (cf. Duncan et al., 2018).

Supplemental material

Supplemental Material, JBD951248_supplemental_tables - Preschool mental health: The Brief Child and Family Intake and Outcomes System

Supplemental Material, JBD951248_supplemental_tables for Preschool mental health: The Brief Child and Family Intake and Outcomes System by Alison Niccols, Charles Cunningham, Peter Pettingill, Donna Bohaychuk and Eric Duku in International Journal of Behavioral Development

Scales and items
Regulating Attention, Impulsivity and Activity
Distractible, has trouble sticking to an activity
Fails to finish things he/she starts
Has difficulty following directions or instructions
Impulsive or acts without stopping to think
Jumps from one activity to another
Fidgets
Cooperating
Defiant, talks back to adults
Blames others for his/her own mistakes
Easily annoyed by others
Angry and resentful
Loses temper
Refuses to follow rules
Regulating Conduct
Acts physically aggressive (hits, bites, or kicks)
Starts fights with other children
Does things that hurt others
Acts mean to other children
Breaks, damages, or destroys thing
Steals or takes things from others
Separating from Parents
Worries bad things will happen to loved ones
Worries about being separated from loved ones
Scared to sleep without parents nearby
Overly upset when leaving loved ones
Complains of feeling sick before separating
Overly upset while away from loved ones
Managing Anxiety
Worries about doing better at things
Worries about past behavior
Worries about doing the wrong thing
Worries about things in the future
Afraid of making mistakes
Overly anxious to please people
Managing Social Anxiety
Shy around other children
Avoids meeting new children
Avoids groups with unfamiliar children
Anxious or worried in social situations with other children
Needs a long time to feel comfortable with other children
Easily embarrassed in social situations with other children
Regulating Compulsive Behaviour
Does things like washing hands or checking doors or lights over and over again
Spends too much time ordering or arranging things in a particular way
Insists that things be unusually clean or germ free
Has difficulty completing tasks because they need to be “perfect”
Worries a lot about dirt and germs
Gets upset if prevented from doing things in a certain order
Managing Mood
Has no interest in his/her usual activities
Gets no pleasure from usual activities
Has trouble enjoying himself/herself
Not as happy as other children
Feels hopeless
Seems sad or unhappy
Eating
Eats too little
Refuse to eat foods that you think are healthy
Resists eating
Not hungry when it is time to eat
Eats only a few particular foods
Has an irregular eating pattern
Sleeping
Sleeps less than other children of the same age
Has difficulty staying asleep
Has an irregular sleep schedule
Has difficulty falling asleep
Has difficulty falling back asleep after waking at night
Has difficulty separating at bedtime

Variable	Canadian population^a	Study sample	Test–retest subsample
Region
Atlantic	7.0	7.0	84	11.0	11
Quebec	23.0	23.0	276	19.0	19
Ontario	40.0	40.0	480	38.0	38
West	30.0	30.0	360	32.0	32
Marital status
Single	16.0	16.0	192	18.0	18
Married/common law	84.0	84.0	1,008	82.0	82
Family income/year
<$29,999	14.0	12.4	148	12.0	12
$30,000–$79,999	37.0	35.2	423	34.0	34
>$80,000	49.0	45.9	551	53.0	53
Preferred not to answer		6.5	78	1.0	1
Education
High school	25.0	25.0	300	25.0	25
College	45.0	43.9	527	44.0	44
University	30.0	30.0	360	30.0	30
Preferred not to answer		1.1	13	1.0	1
Child age
3 years		33.3	400	34.0	34
4 years		33.3	400	33.0	33
5 years		33.3	400	33.0	33
Child sex
Male		50.0	600	50.0	50
Female		50.0	600	50.0	50
Siblings
0		25.9	311	31.0	31
>1		74.1	889	69.0	69
Respondent sex
Female		65.8	790	62.0	62
Male		34.2	410	38.0	38
Respondent parent status
Biological parent		95.4	1,145	93.0	93
Adoptive, foster, other		4.6	55	7.0	7
Language survey completed in
English		77.0	924	81.0	81
French		23.0	276	19.0	19
Primary language spoken in the home
English		72.9	875	78.0	78
French		23.4	281	20.0	20
Other		3.7	44	2.0	2
Respondent country of birth
Canada		84.5	1,014	90.0	90
Other		15.5	186	10.0	10

Model	χ²	df	p Value	CFI	TLI	RMSEA (95% CI)	WRMR
Total sample	4,076.251	1,665	<.001	.935	.931	.035 (.035, .036)	1.497
Boys (n = 600)	2,822.227	1,665	<.001	.943	.939	.034 (.032, .036)	1.227
Girls (n = 600)	2,559.523	1,665	<.001	.949	.945	.030 (.028, .032)	1.185
3 years (n = 400)	2,317.193	1,665	<.001	.955	.952	.031 (.028, .034)	1.124
4 years (n = 400)	2,195.886	1,665	<.001	.955	.952	.028 (.025, .031)	1.060
5 years (n = 400)	2,345.257	1,665	<.001	.939	.935	.032 (.029, .035)	1.085

Model	χ²	df	p Value	CFI	TLI	RMSEA (95% CI)	WRMR
Sex	118.541^a	100	.0995
Model 1				.946	.942	.032 (.030, .034)	1.706
Model 2				.947	.945	.031 (.030, .033)	1.724
Age	295.923^a	200	<.0001
Model 1				.949	.946	.031 (.029, .032)	1.888
Model 2				.950	.949	.030 (.028, .032)	1.932

Scale	Cronbach’s α	Test–retest r ^a (95% CI)
Regulating Attention, Impulsivity, and Activity	.79	.56 (.26, .77)
Cooperating	.78	.56 (.25, .77)
Regulating Conduct	.81	.62 (.33, .80)
Separating from Parents	.78	.57 (.27, .77)
Managing Anxiety	.81	.64 (.36, .82)
Managing Social Anxiety	.88	.64 (.37, .82)
Regulating Compulsive Behaviour	.84	.50 (.17, .73)
Managing Mood	.84	.44 (.09, .69)
Eating	.87	.73 (.50, .86)
Sleeping	.83	.69 (.44, .84)

Scale	Child Functioning	Impact on Family	Parental mood
Regulating Attention, Impulsivity, and Activity	.43***	.37***	.24***
Cooperating	.51***	.48***	.25***
Regulating Conduct	.59***	.52***	.25***
Separating from Parents	.42***	.39***	.27***
Managing Anxiety	.43***	.39***	.23***
Managing Social Anxiety	.41***	.34***	.17***
Regulating Compulsive Behaviour	.44***	.46***	.22***
Managing Mood	.60***	.60***	.24***
Eating	.39***	.37***	.28***
Sleeping	.52***	.49***	.30***

Footnotes

Authors’ note

Peter Pettingill is the owner of BCFPI Inc., which distributes child and youth mental health software in Canada and the European Union, and Alison Niccols, Donna Bohaychuk, and Eric Duku are consultants to BCFPI Inc. Charles Cunningham received salary support from BCFPI Inc., royalties from materials and training related to large group parent training programs and held shares in BCFPI Inc.

Acknowledgment

The authors are grateful for assistance from Stephanie Mielko, Heather Rimas, Research Now, Ainsley Smith, and all the participating parents.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by BCFPI Inc.

ORCID iD

Alison Niccols

Supplemental material

Supplemental material for this article is available online.

Appendix

References

Achenbach

T. M.

Rescorla

L. A.

(2000). Manual for the ASEBA preschool forms and profiles. University of Vermont.

Boyle

M. H.

Cunningham

C. E.

Georgiades

Cullen

Racine

Pettingill

(2009). The Brief Child and Family Phone Interview (BCFPI): 2. Usefulness in screening for child and adolescent psychopathology. Journal of Child Psychology and Psychiatry, 50(4), 424–431.

Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, & Social Sciences and Humanities Research Council of Canada. (2014). Tri-council policy statement: Ethical conduct for research involving humans. Government of Canada.

Caspi

Moffitt

T. E.

Newman

D. L.

Silva

P. A.

(1996). Behavioral observations at age 3 years predict adult psychiatric disorders: Longitudinal evidence from a birth cohort. Archives of General Psychiatry, 53(11), 1033–1039.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255.

Cunningham

C. E.

Boyle

M. H.

Hong

Pettingill

Bohaychuk

(2009). The Brief Child and Family Phone Interview (BCFPI): 1. Rational, development, and description of a computerized children’s mental health intake and outcome assessment tool. Journal of Child Psychology and Psychiatry, 50(4), 416–423.

Devins

G. M.

Orme

C. M.

Costello

C. G.

Blinik

Y. M.

Frizell

Stam

H. J.

Pullin

W. M.

(1988). Measuring depressive symptoms in illness populations: Psychometric properties of the Center for Epidemiologic Studies Depression (CES-D) scale. Psychology & Health, 2(2), 139–156.

DiStefano

Liu

Jiang

Shi

(2018). Examination of the weighted root mean square residual: Evidence for trustworthiness? Structural Equation Modeling, 25(3), 453–466.

Dougherty

L. R.

Smith

V. C.

Bufferd

S. J.

Carlson

G. A.

Stringaris

Leibenluft

Klein

D. N.

(2014). DSM-5 disruptive mood dysregulation disorder: Correlates and predictors in young children. Psychological Medicine, 44(11), 2339–2350.

10.

Duncan

Boyle

M. H.

Abelson

Waddell

(2018). Measuring children’s mental health in Ontario: Policy issues and prospects for change. Journal of the Canadian Academy for Child and Adolescent Psychiatry, 27(2), 88–98.

11.

Eaton

W. W.

Muntaner

Smith

Tien

Ybarra

(2004). Center for Epidemiologic Studies Depression Scale: Review and revision (CESD and CESD-R). In Maruish

M. E.

(Ed.), The use of psychological testing for treatment planning and outcomes assessment (3rd ed., pp. 363–377). Erlbaum.

12.

Egger

H. L.

Angold

. (2006). Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiology. Journal of Child Psychology and Psychiatry, 47(3–4), 313–337.

13.

Eyberg

Pincus

(2012). Eyberg child behavior inventory & Sutter-Eyberg student behavior inventory-revised. PAR.

14.

Gardner

Shaw

(2008). Behavioral problems of infancy and preschool children (0–5). In Rutter

Bishop

D. V. M.

Pine

D. S.

Scott

Stevenson

Taylor

Thapar

(Eds.), Rutter’s child and adolescent psychiatry (5th ed., pp. 882–893). Blackwell.

15.

Gleason

M. M.

Zeanah

C. H.

Dickstein

(2010). Recognizing young children in need of mental health assessment: Development and preliminary validity of the early childhood screening assessment. Infant Mental Health Journal, 31(3), 335–357.

16.

Goodman

(1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581–586.

17.

Gudmundsson

O. O.

Magnusson

Saemundsen

Lauth

Baldursson

Skarphedinsson

Fombonne

(2013). Psychiatric disorders in an urban sample of preschool children. Child and Adolescent Mental Health, 18(4), 210–217.

18.

Hair

J. F.

Jr Black

W. C.

Babin

B. J.

Anderson

R. E.

(2014). Multivariate data analysis (7th ed.). Pearson.

19.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

20.

IBM Corp. (2019). IBM SPSS statistics for Windows (Version 26.0) [Computer software] . IBM Corp.

21.

Little

T. D.

(2013). Longitudinal structural equation modeling. Guilford.

22.

Merikangas

K. R.

Nakamura

E. F.

Kessler

R. C.

(2009). Epidemiology of mental disorders in children and adolescents. Dialogues of Clinical Neuroscience, 11(1), 7–20.

23.

Muthén

Asparouhov

(2002). Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus. Mplus Web Notes: No. 4.

24.

Muthén

L. K.

Muthén

B. O.

(1998–2015). Mplus user’s guide: Statistical analysis with latent variables (7th ed.).

25.

Niccols

Cunningham

Pettingill

Bohaychuk

Duku

(2018). Infant mental health: The Brief Child and Family Intake and Outcomes System. International Journal of Behavioral Development, 42(6), 588–596. https://doi.org/10.1177/0165025417752497

26.

Niccols

Cunningham

Pettingill

Bohaychuk

Duku

(2019). Toddler mental health: The Brief Child and Family Intake and Outcomes System. International Journal of Behavioral Development. Epub ahead of print October 16 2019. DOI: 10.1177/0165025419880618.

27.

Bayer

(2015). Parents’ help-seeking processes for early childhood mental health problems. Child and Adolescent Mental Health, 20(3), 149–154.

28.

Reynolds

C. R.

Kamphaus

R. W.

(2015). Behavior assessment system for children (3rd ed.). Pearson.

29.

Saylor

C. F.

Swenson

C. C.

Reynolds

S. S.

Taylor

(1999). The Pediatric Emotional Distress Scale: A brief screening measure for young children exposed to traumatic events. Journal of Clinical Child Psychology, 28(1), 70–81.

30.

Scheeringa

M. S.

Haslett

(2010). The reliability and criterion validity of the Diagnostic Infant and Preschool Assessment: A new diagnostic instrument for young children. Child Psychiatry and Human Development, 41(3), 299–312.

31.

Sheldrick

R. C.

Merchant

Perrin

E. C.

(2011). Identification of developmental-behavioral problems in primary care: A systematic review. Pediatrics, 128, 356–363.

32.

Sourander

McGrath

P. J.

Ristkari

Cunningham

Huttunen

Lingley-Pottie

Hinkka-Yli-Salomäki

Kinnunen

Vuorio

Sinokki

Fossum

Unruh

(2016). Internet-assisted parent training intervention for disruptive behavior in 4-year-old children: A randomized clinical trial. Journal of the American Medical Association Psychiatry, 73(4), 378–387.

33.

Squires

Bricker

Twombly

(2015). Ages & stages questionnaires: Social-emotional-2. Brookes.

34.

Statistics Canada. (2011). Census of Population, National Household Survey, and Labour Force Survey (Statistics Canada Catalogue nos. 98-312-XCB2011017, 98-312-XCB2011024, 99-014-X2011048, and Table 282-0004). Government of Canada.

35.

Streiner

D. L.

Norman

G. R.

(2014). Health measurement scales: A practical guide to their development and use (5th ed.). Oxford University.

36.

Weitzman

Edmonds

Davagnino

Briggs-Gowan

M. J.

(2014). Young child socioemotional/behavioral problems and cumulative psychosocial risk. Infant Mental Health Journal, 35(1), 1–9.

37.

Woodward

L. J.

Morris

A. R.

Healey

D. M.

(2017). Preschool self regulation predicts later mental health and educational achievement in very preterm and typically developing children. The Clinical Neuropsychologist, 31(2), 404–422.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.40 MB