When Parent and Teacher Ratings Don't Agree: The Tracking Adolescents' Individual Lives Survey (TRAILS)

Abstract

Objectives:

A commonly encountered situation for evaluating clinicians is a history of significant problems in one setting with little or no difficulties in another. This study aims to describe this phenomenon and to examine its relations with other child and family characteristics.

Method:

A total of 1,730 children (mean age 11.05 years) was studied from the first wave of the Tracking Adolescents' Individual Lives Survey (TRAILS), a large population-based study of Dutch youth. Parent and teacher ratings of aggression, rule breaking, inattention, and hyperactivity were obtained. Children were assigned to groups according to the presence of clinically relevant problems at home only, at school only, or in both settings. The rate of setting specific problems was calculated and comparisons between groups were made.

Results:

Setting specific, especially home-specific, problems were quite common. Among children whom parents rated as having at least borderline-clinical problems, teachers reported clear or very clear behaviors at school at the following rates: aggression (22%), rule breaking (12.5%), inattention (55%), and hyperactivity/impulsivity (33%). Compared with the school-specific group, the home-specific group contained a significantly higher percentage of girls with regard to inattention or hyperactivity and a significantly lower percentage of girls with regards to rule breaking. Logistic regression analyses revealed that home- versus school-specific problems were related to sex, child effortful control, and parental stress.

Conclusion:

Externalizing problems are frequently encountered only in one setting between home and school and are related to sex, child effortful control, and parental stress.

Introduction

F or good reason, most clinicians place a great deal of emphasis on parental report when making diagnostic decisions about children. At the same time, however, clinicians are urged if not required to obtain information from others such as teachers (King 1995). During this process, it is common to hear significant discrepancies in the reported levels of problem behavior across different settings (Schachar et al. 1981; Lahey et al. 2004; Soma et al. 2009). Researchers have similarly been aware of these differences and have approached this question for decades from various perspectives, as has been recently described (Achenbach 2011). Examinations of the level of agreement across informants of child behavior has revealed very modest correlations between informants like parents and teachers (Achenbach et al. 2005; Kanne et al. 2009). This low agreement has been hypothesized to result from a number of factors, including (1) actual behavior differences in the child across settings, (2) memory and judgment differences by the informant, (3) site differences in ability to elicit behavior being assessed, (4) candor of informants, and (5) measurement error (Kraemer et al. 2003; Achenbach 2009). Informant disagreement has also been found to depend somewhat on the clinical setting (MacLeod et al. 1999). Twin data have revealed that different informants offer valuable information on the genetic and environmental contributions to child behavior (Hudziak et al. 2003). Moreover these discrepancies have been reliably found across a wide range of behavioral problems in addition to patterns of comorbidity (Yeh and Weisz 2001; Youngstrom et al. 2003).

One of the most familiar situations in clinical practice is the reporting of significant problems reported at home with minimal or no difficulties reported by teachers and other school staff (Steiner and Remsing 2007). We refer to this phenomenon as home-specific problems (HSP) with the corresponding school-specific problems (SSP) referring to the opposite pattern. These discrepancies often present a major diagnostic dilemma for the evaluating clinician, and there is surprisingly little clinical guidance about how to understand and reconcile these differences in reports (MacLeod et al. 1999; Kraemer et al. 2003).

Here, we attempt to address this gap by analyzing parental and teacher report data from the Tracking Adolescents' Individual Lives Survey (TRAILS), a large prospective study of 2,230 Dutch adolescents followed since age 10. Data from these children were obtained from multiple informants using a variety of instruments (Huisman et al. 2008). Our aim was to provide a quantitative description of the prevalence of children who display HSP and SSP across various types of externalizing psychopathology and to identify specific child and family factors which may underlie differences in reported behaviors across settings.

TRAILS data from home and school have been previously published. One report by Noordhof et al. (2008) using principal component analyses demonstrated the presence of a significant context dependent factor that captured behavior problems observed only in one setting. This factor was distinct to those related to symptom severity, direction, and perspective. Another investigation by Veenstra et al. (2008) examined prosocial and antisocial behavior across school and home domains using cluster analyses to identify groups of children with consensual or discrepant behavior across samples. Differences were found in both child and family characteristics, including the child temperament dimension of effortful control, academic performance, as well as parental stress.

The purpose of this study is to extend the findings of these prior investigations into the area of inattention and hyperactivity in one of the largest samples to date and to examine the question from a perspective that would be most relevant for clinicians. To date, the majority of the literature has used percentage cutoffs or statistical modeling procedures to divide the sample into groups that reflect problems reported by single or multiple informants. While such procedures are highly valuable for empirical validation of these subgroups, the present study groups children into clinical groups based on established cutoffs of ratings scales that are directly used in clinical practice, thus maximizing clinical utility and generalizability of key questions. How often do parents report clinical levels of problems when teachers do not and vice versa? What factors deserve investigation when a clinician encounters the situation of parent and school disagreement? Overall, we hypothesized that home and school specific problems would be a relatively common phenomenon as has been previously found in the literature (De Los Reyes and Kazdin 2005; Jensen et al. 1988). Further, we expected that HSP will be related to both family and child characteristics such as child effortful control and sex, socioeconomic status, and parental stress as initially explored by others (Collishaw et al. 2009; Offord et al., 1996; Treutler and Epkins 2003; Veenstra et al. 2008).

To extend previous studies, we also included variables potentially related to setting specific problems that have not been well studied in the past such as specific parent personality dimensions that have been shown to be related to child diagnoses such as ADHD (Nigg and Hinshaw 1998) but not to informant discrepancies. An additional variable that has not been extensively studied in these investigations is family composition and in particular the number of other children in the household. Given the heritability of externalizing disorders and rate of psychopathology in siblings (Listug-Lunde et al. 2008; Rettew and Hudziak 2009), we wanted to explore the possibility that having multiple siblings around the same age might lead to increased disruptive behavior at home that might not be reflected in the school setting.

Method

Subjects

Details of the TRAILS study design and participants have been reported elsewhere (De Winter et al. 2005; Huisman et al. 2008). The current investigation utilizes data from the first wave of data collection. The TRAILS study targeted all children between the ages of 10 and 12 living in five municipalities in the northern part of the Netherlands, including both urban and rural areas. Approximately 76% of eligible children (2,230/2,934) participated in the first wave of the study. Previous work did not find significant differences in demographic variables, teacher-reported behavior problems, and other mental health indices between responders and nonresponders (De Winter et al. 2005). Written informed consent was obtained from both the preadolescent and a parent and the study was approved from the Central Committee on Research Involving Human Subjects.

Measures

Children and families were assessed both at school and during a home visit by well-trained interviewers. The vast majority (96%) of parental respondents were mothers. Children completed components of intelligence quotient (IQ) testing and questionnaires at school under supervision of the TRAILS staff. To minimize the number of statistical tests and comparisons performed, selected variables were chosen based on the priori hypotheses of the investigators.

Demographic information

One focus of this study was to examine possible effects related to having multiple siblings around the same age in the household. As subjects were between 10 and 12 years old, we recorded the number of siblings living in the household between the ages of 5 and 17. Socioeconomic status was measured using a standardized value that reflects the average of family income as well as parental educational and occupational level (Veenstra et al. 2008). We also examined ethnicity and being a single parent.

Parent-rated child psychopathology

Parent-rated psychopathology was assessed using the Child Behavior Checklist (CBCL; Achenbach 2009). The CBCL is a sex- and age-sensitized questionnaire used for parents to respond to 118 problem behaviors exhibited by their child over the previous 6 months. The characteristics and psychometric stability of the CBCL have been well established. Internal consistency of the four scales used in this study has been found to be high, with Chronbach's alpha ranging from 0.90 to 0.93 (Achenbach and Rescorla 2001). This study focuses on the subscales of aggressive problems, rule-breaking problems, and attention problems from the empirically derived subscales and the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) (American Psychiatric Association 1994)–oriented attention-deficit/hyperactivity problems. Clinically significant scores were defined as a T score of 64 or higher, which takes account of both sex and age. This cutoff parallels the “borderline-clinical” convention of the scales themselves.

Teacher-rated child psychopathology

Teacher's ratings of problem behaviors were obtained using the Teacher's Checklist of Psychopathology (TCP) which contains nine items with descriptions of the following domains: withdrawn, somatic complaints, anxious/depressed, social problems, thought problems, inattention, activity/impulsivity, aggressive behavior, and rule breaking (De Winter et al. 2005). The format of the TCP parallels that of the Teacher Report Form (TRF) (Achenbach 1991) and contains a list of behaviors (below) that corresponds to items contained in the parallel scale of the TRF. Instead of the TRF three point scale for each individual item that loads onto a particular dimension, however, teachers look at all the items for a particular problem area and give a single number along a five-point scale. This study uses the problem areas of aggression, rule-breaking behavior, inattention, and hyperactivity/impulsivity. Items included for four subscales include the following.

Aggressive behavior:

Argues a lot, defiant, talks back to staff, bragging, boasting, cruelty, bullying or meanness to others, demands a lot of attention, destroys his/her own things, destroys property belonging to others, disobedient at school, disturbs other pupils, disrupts class discipline, easily jealous, gets in many fights, physically attacks people, screams a lot, explosive or unpredictable behavior, demands must be met immediately, easily frustrated, stubborn, sullen or irritable, sudden changes in mood or feelings, teases a lot, temper tantrums or hot temper, threatens people.

Rule-breaking behavior:

Does not seem to feel guilty after misbehaving, hangs around with others who get in trouble, lying or cheating, prefers being with older children, steals, swearing or obscene language, tardy to school. Truancy or unexplained absence, uses alcohol or drugs.

Attention problems:

Fails to finish things he/she starts, can not concentrate, can not pay attention for long, confused or seems to be in a fog, daydreams or gets lost in his/her thoughts, has difficulty learning, poorly coordinated or clumsy, inattentive or easily distracted, underachieving, not working up to potential, fails to carry out assigned tasks.

Hyperactivity/impulsivity:

Hums or makes odd noises in class, can not sit still, restless, hyperactive, fidgets, difficulty following directions, impulsive or acts without thinking, messy work.

Child temperament

The temperament trait of effortful control was assessed using the parent-rated Early Adolescence Temperament Questionnaire (Kim et al. 2003). Effortful control pertains to a child's ability to regulate behavior and delay gratification toward the fulfillment of long-term goals. This particular dimension was chosen due to accumulating reports that this dimension and others like it related to self-regulation is a critical trait that may underlie a broad array of psychopathology (Oldehinkel et al. 2004; Rettew et al. 2004).

Parent personality

The NEO Five Factor Inventory (Costa and McCrae 1992) is a widely used self-report personality instrument. For this study, we used the facets of angry hostility (from neuroticism), vulnerability (from neuroticism), and self-discipline (from consciousness) based upon prior research and study hypotheses (Nigg and Hinshaw 1998). Angry hostility refers to a tendency toward experiencing anger and frustration. The vulnerability scale assesses a predisposition to feeling overwhelmed and dependent under stress. The self-discipline dimension measures the ability to execute tasks and goals in the face of obstacles and distractions.

Intelligence

Child intelligence was assessed using the block design and vocabulary subtests of the Wechsler Intelligence Scale for Children–Revised (Silverstein 1972).

Parental stress

The level of parental stress was assessed using the 24-item Parental Stress Index (Janssens et al. 2009). The total score is the average of two subscales referring to child characteristics and parent characteristics within the caregiving context.

Data analysis

Children with HSP and SSP were identified by contrasting levels of behavior problems between the parent-report CBCL and teacher-report TCP. On an a priori basis, subjects were selected into one of four groups. The HSP group was defined as having at least borderline-clinical syndrome scores (T score greater than 64) according to the parent-rated CBCL while having a teacher rating of 0 (not applicable) or 1 (rarely applicable) for that problem. The SSP group was defined as having a teacher-rated score of 3 (problem clearly or often applicable) or 4 (very clearly or very often applicable) while having a parent-rated CBCL T score for that problem at less than 60 (1 standard deviation [SD] of average). The both home and school problems (Both) group was defined as having at least a CBCL T score of greater than 64 and a teacher rating of 3 or 4, whereas the neither home nor school (Neither) group had a CBCL rated T score less than 60 and a teacher rated score of 0 or 1. These cut-offs are based on the statistical distribution of scores and the given cut-offs of the instruments that have been derived from empirical study. The percentage of children grouped into the “clinical” range by parent report (CBCL T score greater than 64) versus teacher report (score of 3 or 4 with problem being clearly or very clearly present) were 15.2% vs. 6.9% for aggression, 7.3% vs. 2.4% for rule breaking, 12.1% vs. 13.6% for attention problems, and 12.3% vs. 8.3% for attention-deficit hyperactivity. Because of the frequency differences found especially for aggression and rule breaking behavior, with parents reporting more baseline problematic behavior than teachers, we also present prevalence of HSP and SSP using a more lenient cut-off in which a teacher rating of 2 (sometimes applicable) counts as problematic. Adjusting this cutoff for aggression and rule breaking results in teacher reported problems at a rate of 17.3% for aggression and 6.7% for rule breaking. These adjustments lead to a more parallel structure for parents and teachers.

Comparisons between groups were made using univariate analysis of variance for continuous variables and chi-square analysis for binary data. Logistic regression analyses were used to examine associations between group status and the predictor variables among variables found to be significant through the preliminary analyses.

To examine the possibility that the rates of child behavior problems vary markedly between schools, thus affecting the rates of HSP and SSP, the frequency of reporting significant problems (i.e., teacher reporting often or very often symptoms) was calculated for each of the 35 schools that had at least 20 subjects in that school and whose subjects' mean socioeconomic status was within one SD of the mean.

Results

Of the 2,230 children who were in the first wave of the TRAILS study, the total number of children with valid parent and teacher rated data was 1,780 (51.4% female; 48.6% male). The average age of subjects was 11.05 years (SD=0.53). Of these subjects, 87.9% were ethnic Dutch with Surinam (1.9%), Indonesian or Mollucan (1.7%), Antillean (1.3%), Moroccan (0.6%), and Turkish (0.4%) comprising the ethnic background of non-Dutch subjects. Analyses between subjects who were included versus not included in the study revealed no significant sex differences, although nonparticipants were more likely to be older, of non-Dutch ethnicity, and have lower SES and lower full scale IQ. Problem score comparisons between children who had ratings from only one setting versus both settings revealed significant higher problem scores in children with parent and teacher ratings compared with parent only ratings for rule-breaking only (p<0.05). In all four problem domains, however, children with teacher ratings only had higher problem scores than those with teacher and parent ratings (all p<0.01). Pearson correlations between a child's score for a particular behavioral domain between parent and teacher ratings were as follows: aggression, r=0.32; rule-breaking, r=0.27, attention problems, r=0.47, attention-deficit/ hyperactivity/impulsivity, r=0.40 (all p<0.001).

Prevalence of setting specific problems

Grouping of subjects into those with prominent difficulties across the home and school setting revealed a high degree of discrepancy between locations. As shown in Table 1, it was generally more common for problems to be reported only in one setting compared with both. Among children rated by the parent as having problematic behavior, agreement by the teacher was generally low. Of children whom a parent rated as being in the borderline-clinical or clinical range for aggression, the teacher reported clear or very clear aggressive behaviors at school only 22% of the time (47 children in “Both Settings” for aggressive behavior divided by sum of this number and 166 children in “Home Specific” cell). Teachers agreed with parents for concerning levels of rule-breaking behavior only 12.5% of the time while for attention problems and DSM oriented attention-deficit/hyperactivity problems the rate rose to 55% and 33%, respectively. SSP were also present; for students who teachers rated as having clear or very clear problems, parents also reported at least borderline-clinical problems at the following rates: aggression (52%), rule-breaking (45%), attention problems (43%), and hyperactivity/impulsivity (49%). The SSP rates were lower than HSP for all areas studied except for attention problems, where it was more common than home-specific attention problems (X ²=226.69, p<0.001). There were also sizable numbers of children who were not grouped who presented with intermediate problems levels.

Table 1.

Frequency of Home- and School-specific Symptoms (Total n =1,780)

	Neither setting	Both settings	Home specific	School specific	Not grouped
Aggressive behavior	1,144 64.3%	47 2.6%	166 9.3%	43 2.4%	380 21.3%
Rule-breaking behavior	1,371 77.0%	15 0.8%	105 5.9%	18 1.0%	271 15.2%
Attention problems	1,094 61.5%	89 5.0%	73 4.1%	119 6.7%	405 22.8%
Attention-deficit/hyperactivity problems	1,177 66.1%	58 3.3%	118 6.6%	60 3.4%	367 20.6%

Using the less severe cutoff for teacher ratings of aggression and rule breaking in order to make overall levels of “clinical” behavior more parallel between teachers and parents did not change the overall patterns of results. Agreement by teachers on parent-rated aggression rose from 22% to 39% while agreement on rule-breaking rose from 12% to 17%. In the other direction, the rate of parental agreement with teacher ratings of clinical aggression dropped from 52% to 40% while for rule-breaking the rate dropped from 45% to 28%, as would be expected.

Variability of reported problems by school

For frequent rule-breaking behaviors, nearly all schools reported low levels in their students (less than 5%). The rate of reported attention problems, however, ranged widely. Five of 35 schools reported problematic levels of inattention in over 20% of the subjects in this study, whereas 15 schools reported rates less than 10%. Rates of frequent hyperactivity/impulsivity also varied with 17 schools reporting levels less than 5% and 10 schools reported rates greater than 10%. Children were not recruited selectively by school.

Group differences

One-way analysis of variances with Tukey B post-hoc comparisons were conducted to evaluate potential associations between clinical groups (HSP, SSP, and Both) and a number of variables. We also examined levels of parent- and teacher-rated problems across groups. As the groups were defined by these levels, significant differences were expected. Whereas absolute differences tended to be small regarding levels of parent-rated problems between the HSP and Both groups (on average about 11% higher in the Both group), differences for all symptom areas were statistically significant at the p<0.05 level with the exception of attention problems. Teacher-rated problems between the SSP and Both groups, by contrast, did not significantly differ with the exception of hyperactive/impulsive problems (Table 2).

Table 2.

Comparisons of Predictor Variables Between Children with Home-Specific, School-Specific, and Symptoms at Both Home and School (Mean±Standard Deviation)

	Aggressive behavior
	Home specific (n=166)	School specific (n=43)	Both home and school (n=47)	p value	Pairwise
Parent-rated severity	15.6±3.5	4.9±2.1	17.4±4.3	<0.001	B>H>S
Teacher-rated severity	0.3±0.5	3.1±0.3	3.2±0.4	<0.001	S,B>H
Have other children in household ages 5 to 17 (%)	74.7	72.1	74.5	ns	—
Single parent	18.1	32.6	19.1	ns	—
Sex (% female)	40.1	23.2	25.6	<0.05	H>S
Dutch ethnicity (%)	9.0	7.0	12.8	ns
SES	−0.2±0.8	−0.4±0.7	−0.6±0.7	<0.05	B<H
Child IQ	94.1±14.6	96.6±16.4	88.0±14.4	<0.05	B<H,S
Parent stress	2.9±0.9	1.7±0.7	3.1±0.9	<0.001	B,H>S
Child temperament–effortful control	2.7±0.71	3.0±0.6	2.5±0.6	<0.01	B, H<S
Parent personality
Vulnerability	20.6±4.2	19.3±4.7	21.1±4.1	ns	—
Annoyance	21.6±3.3	21.9±3.1	23.1±4.4	ns	—
Self-discipline	27.8±3.9	28.6±4.8	27.6±4.0	ns	—

	Rule-breaking behavior ^a
	Home specific (n=105)	School specific (n=18)	Both home and school (n=15)	p value	Pairwise
Parent-rated severity	7.6±2.5	1.5±1.2	9.1±3.5	<0.001	B>H>S
Teacher-rated severity	0.3±0.5	3.3±0.5	3.3±0.5	<0.001	B, S>H
Children in household ages 5 to 17	70.4	100.0	60.0	<0.05	S>H, B
Single parent	27.6	22.2	33.3	ns	—
Sex (% female)	6.7	38.9	13.3	<0.001	H<S
Dutch ethnicity (%)	12.4	22.2	13.3	ns	—
SES	−0.3±0.8	−0.2±0.7	−0.9±0.6	<0.05	B<H
Child IQ	93.4±14.3	95.2±15.6	79.2±14.1	<0.01	B<H, S
Parent stress	2.9±1.0	1.8±0.7	2.7±1.2	<0.001	B, H>S
Child temperament–effortful control	2.7±0.7	3.2±0.5	2.4±0.6	<0.01	B, H S

	Attention problems
	Home specific (n=73)	School specific (n=119)	Both home and school (n=89)	p value	Pairwise
Parent-rated severity	10.8±1.8	4.4±1.9	11.5±2.1	<0.001	B, H>S
Teacher-rated severity	0.6±0.5	3.2±0.4	3.3±0.4	<0.001	B, S>H
Children in household ages 5–17	76.7	75.6	67.4	ns
Single parent	19.2	21.0	23.4	ns	—
Sex (% female)	53.4	27.8	33.7	<0.001	H>S, B
Dutch ethnicity (%)	8.2	10.9	9.0	ns	—
SES	−0.2±0.8	−0.4±0.8	−0.3±0.8	ns	—
Child IQ	92.9±14.0	91.6±12.8	89.9±14.9	ns	—
Parent stress	2.6±1.0	1.6±0.6	2.9±1.0	<0.001	B, H>S
Child temperament–effortful control	2.4±0.6	2.9±0.5	2.3±0.5	<0.001	B, H<S
Parent personality
Vulnerability	19.2±3.6	18.5±3.4	20.8±4.4	ns	—
Annoyance	20.9±3.3	20.5±3.4	21.8±3.7	ns	—
Self-discipline	28.2±4.0	28.2±2.8	28.0±4.1	ns

	Hyperactivity/impulsivity
	Home specific (n=118)	School specific (n=60)	Both home and school (n=58)	p value	Pairwise
Parent-rated severity	9.8±1.6	3.8±1.6	10.9±1.6	<0.001	B>H>S
Teacher-rated severity	0.4±0.5	3.2±0.4	3.4±0.5	<0.001	B>S>H
Children in household ages 5 to 17	68.7	76.7	75.9	ns	—
Single parent (%)	19.5	23.3	24.1	ns	—
Sex (% female)	53.4	26.7	22.4	<.001	H>S, B
Dutch ethnicity (%)	10.1	15.0	8.6	0.001
SES	−0.4±0.8	−0.4±0.7	−0.4±0.8	ns	—
Child IQ	89.9±14.5	94.1+13.4	90.4±13.3	ns	—
Parent stress	2.7±1.0	1.7±0.7	2.9±0.9	<0.001	B, H>S
Child temperament–effortful control	2.5±0.6	3.0±0.5	2.4±0.5	<0.001	B, H<S
Parent personality
Vulnerability	19.8±4.4	18.5±3.7	20.2±4.5	ns	—
Annoyance	21.4±3.9	21.1±3.9	21.5±4.1	ns	—
Self-discipline	28.2±3.5	28.3±3.3	28.6±4.3	ns

Insufficient sample size to perform analyses with parental personality.

IQ=intelligence quotient; B=both; H=home; S=school; SES=socioeconomic status.

Many sex differences were found between groups. With regard to attention and hyperactivity, those in the SSP or Both group tended to be predominantly boys, whereas the HSP group contained a much higher percentage of girls. A similar trend was seen for aggressive behavior. This pattern changed for rule-breaking behavior, where there were almost no girls in the HSP group in contrast to the SSP group.

No significant differences were found in family composition in rates of single parenthood or the presence of other children in the household within 5 years of age of the subject. Other analyses not shown examining the number of children or the presence of children closer to the age to the index subject revealed similar results. In addition, no significant differences were found between groups on parental personality.

Lower IQ and SES, however, were found in the Both group compared with the other two groups for aggressive and rule-breaking behavior. Across problem areas, higher parental stress was found in the HSP and Both groups compared with the SSP group. Lower levels of the child temperament dimension of Effortful Control were found in the HSP and Both group compared with the SSP group for all four behavioral syndromes.

Regression analyses between home and school groups

Multivariate logistic regression analyses were performed to predict HSP versus SSP group status among the variables found to be statistically significant (excluding parent- and teacher-rated problem severity which, as mentioned, was used to define the groups). The results of these analyses are shown in Table 3 with the HSP group serving as the reference. Total parent stress was found to be a significant predictor for all 4 problem domains with less stress associated with the SSP group. Sex remained a significant predictor with female sex related to being in SSP group for rule-breaking behavior and male sex significantly related to the SSP group for inattention and hyperactivity/impulsivity. Higher child levels of Effortful Control remained a significant predictor of being in the SSP group for both inattention and hyperactivity/impulsivity.

Table 3.

Logistic Regressions Predicting School-specific Problems Groups Versus Home-specific Problems Group

	Aggression			Rule-breaking			Inattention			Hyperactivity/impulsivity
Predictor	OR	95% CI	p value	OR	95% CI	p value	OR	95% CI	p value	OR	95% CI	p value
Female sex	2.02	0.81–5.00	ns	0.03	0.01–0.29	<0.01	4.81	2.09–11.05	<0.001	4.79	2.01–11.04	<0.001
Parent Stress	0.14	0.07–0.28	<0.001	0.17	0.05–0.56	<0.01	0.21	0.12–0.37	<0.001	0.29	0.17–0.50	<0.001
Child Effortful Control	1.33	0.69–2.55	ns	1.62	0.55–5.15	ns	6.81	2.75–16.86	<0.001	4.28	1.96–9.31	<0.001

The home-specific problem group is the reference group.

Number of household children aged 5–17 not included in rule-breaking regression due to OR being undefined (100% of school-specific group had other children in household).

OR=odds ratio; CI=confidence interval.

Discussion

This study examined the characteristics of a general population sample of children who have externalizing problems in one setting (home or school) compared with those with problems in both settings. Subjects were grouped according to thresholds that would likely be encountered in clinical situations to maximize the applicability of the findings to general practice. Overall, we found that it was common for problems of aggression, rule-breaking, inattention, and hyperactivity to be present in one setting while being rated as rare or nonexistent in the other. Indeed, in the majority of cases, children rated as having high levels of aggression or rule-breaking at home were rated as having little or no symptoms at school. For inattention and hyperactivity, the teacher reported frequent or very frequent symptoms among children that parents rated in the borderline-clinical or clinical range roughly half of the time. This pattern was found even when cutoffs for “clinical” levels of problem behavior was set to include roughly the same percentage of children in parent and teacher ratings.

SSP were also present, although at rates lower than HSP for all areas studied except attention problems. This pattern is consistent with a study by Drabick et al. (2007) in which teacher-identified oppositional defiant disorder (ODD) was generally consistent with the mother's perception (52%), whereas the reverse pattern was less common (27%). Our rate of 5.7% of the sample manifesting significant rule-breaking problems by mother-report only is also similar to other community samples, such as the 5.8% rate of mother reported conduct disturbance in a slightly younger New Zealand sample (Fergusson et al., 2009). With the exception of attention problems, teachers tended to identify lower percentages of children with problematic levels of behavior than parents, a finding that has been found previously (Youngstrom et al. 2000).

It should also be noted that our method for identifying children with problematic levels of behavior was not parallel between parent and teacher ratings and thus may have contributed to proportionally higher rates of HSP versus SSP and perhaps to findings of related factors. It was somewhat reassuring, however, to find that the pattern of variables related to inattention, in which parents and teachers reported very comparable percentages of children with problematic behavior, showed very similar relations to child and family characteristics. Somewhat different procedures for determining clinical thresholds between parents and teacher reported problems was also present in a previous study (Offord et al., 1996) that also found substantial parent-teacher disagreement. Finally, a further indication that our measurement strategy did not overly influence patterns of disagreement is that the correlations between parent and teacher ratings ranged from .27 to .47 and is in line with other studies (Achenbach et al. 2005).

These data offer initial prevalence estimates and confirmation of the challenging and commonly encountered clinical situation of needing to synthesize discrepant information into an overall diagnostic formulation and treatment plan. One possible explanation for these differences is that the interpretation of behavioral symptoms reflects the goals of a particular setting. As attention is at such a premium at school, for example, some aggressive or rule-breaking behavior may tend to be viewed more as inattention/hyperactivity, whereas inattentive symptoms, such as not listening, may be viewed more as oppositionality at home (Abikoff et al. 1993). In an attempt to identify variables that might be related to the occurrence of setting specific problems, a number of factors were examined. Sex differences clearly emerged as the group of children with HSP compared with SSP tended to consist of proportionately more girls compared with boys for all areas except for rule-breaking, where only about 7% of the HSP group consisted of girls. Derks et al. (2007) similarly found teachers reporting less inattention in girls among children identified by mothers as having ADHD. These results may suggest that teachers may not detect inattention and hyperactivity to the same degree in girls as compared with boys. It is also possible that what is labeled as externalizing problems in girls at home is perceived as a different type of problem not measured in this study. Alternatively, girls may be able to modify their behavior more than boys in school settings. Why this pattern did not hold for rule-breaking is puzzling. In this area, the sex distribution for the HSP group was similar to that for the Both group, whereas for inattention and hyperactivity/impulsivity the SSP and Both groups were quite comparable.

Lower IQ and SES were found in the group with problems in both settings compared with children with setting specific problems (home or school) for aggression and rule-breaking but not for inattention and hyperactivity. The home-specific and both settings group also had higher levels of parental stress compared with the school-specific group. These results are comparable to earlier work that has also shown relations between externalizing problems identified by parents only and parental dysfunction (Offord et al. 1996; Collishaw et al., 2009), although lower SES in these studies was related to teacher reported externalizing problems. In a study of internalizing problems, a previous study demonstrated that maternal anxiety was related to increased perception of child problems compared with self-report, although the effects were small (van der Toorn et al. 2010).

Our findings of both child and family variables being related to informant disagreement should not be interpreted to mean that, in a particular case, rating discrepancy is related to one factor or another. Rather, the findings could result from the influence of one variable on another, such as the increased stress that is induced by a child with significant behavioral problems at home (Mebust et al. 2010) but may also reflect parental stress from other sources such as marital difficulties or parental psychopathology that impacts a parent's perception.

Logistic regression analyses comparing the home-specific and school-specific groups for inattention and hyperactivity/impulsivity revealed an independent association between being in the home-specific group and female sex, higher parental stress, and lower levels of Effortful Control. This trait of Effortful Control relates to ability to modulate attention and impulses and would be expected to be lower among children who display difficulties in multiple settings compared with one. The finding that Effortful Control is low in children with HSP is more unexpected and may be related to the fact that the dimension was also rated by the parent while teacher ratings were not obtained. While this potential rater bias may account for some of these findings with regard to parental stress and child Effortful Control, these results parallel a previous study that showed strong relations between Effortful Control and particularly externalizing problems as measured by the parent-rated CBCL (Oldehinkel et al. 2004).

Contrary to expectations, we did not find evidence that HSP were related to the number of parents or number of children around the same age in the household. These data suggest that parents who report clinically significant externalizing problems are not merely describing the phenomenon of siblings overstimulating each other within the confines of the household. For rule-breaking problems, having a household sibling was significantly related to the school-specific group. This unexpected finding may have been spurious given the low sample sizes in the rule-breaking groups. It is also possible that some parents misattribute rule-breaking behavior to other siblings.

For all domains except attention problems, parent-rated severity was modestly but significantly higher in the children with difficulties in both settings compared with those with problems at home only, indicating a possible threshold effect in that higher levels of symptoms are more likely to be detected by more informants. Interestingly, such an effect was not found regarding teacher rated severity between the SSP and Both group.

Teacher ratings across school revealed fairly wide ranging levels of reported problems particularly for inattention and hyperactivity/impulsivity. These calculations were based on schools drawing mainstream students from similar socioeconomic backgrounds; however, more systematic research is needed to control for potential confounds. It is possible that parents select particular schools for their children with ADHD symptoms based upon available services. The findings also support the possibility that schools have their own culture when it comes to thresholds for reporting and intervening with child behavior problems.

This study has important clinical implications. First, clinicians should commonly expect substantial differences in reports coming from different settings. When high levels of problems are found at home only, the pattern may indicate the presence of significant parental stress emerging in the context of interplay with family SES and other child risk factors. Efforts to address these difficulties directly may help lessen both the real and perceived behavioral disturbances in the children as part of an overall family-based treatment approach (Hudziak 2008). Indeed, there may be occasions when psychopharmacological attention may be better directed at the parents rather than the children. Furthermore, evaluators may further want to monitor and account for schools or teachers that consistently report either very low or very high levels of problem behavior in their students when making synthesized assessments in their patients. Further research is needed to compare clinical assessments from different informants on measurable outcomes. A recent study, for example, found that teacher- but not parent-rated ODD symptoms were predictive of depressive and conduct symptoms assessed 9 months later, after controlling for baseline symptoms (Drabick et al. 2011). Other studies, however, have shown that behavior problems reported by only one type of informant have significant and independent predictive value for future outcomes such as later symptomatology, socioeconomic status, criminal behavior, and substance abuse (Fergusson et al. 2009; Dirks et al. 2011).

While this study contains a number of strengths, including a relatively large sample that was assessed with multiple informants, there remain some limitations. These analyses were based on only a single time point, although we plan to assess the stability of setting specific problems in the future using this longitudinal dataset. To minimize fatigue and measurement error in having teachers rate many pupils, we also chose not to use the full TRF, but a shorter, vignette-based, version. As mentioned, the different definitions for problematic behavior between parent and teacher ratings may have influenced the pattern of results. In addition, the prevalence estimates for home or SSP obtained in this study may be somewhat inflated compared with clinically referred sample as subjects in this study came from the community. As there was some indication that informant disagreement was related to symptom severity level, it is possible that disagreement would be less in a clinical sample of children referred for clinical assessment and treatment. Finally, we did not obtain ratings from other trained observers as has been done in other investigations (Youngstrom et al. 2000; Chi and Hinshaw 2002). A previous study that compared parent and teacher ratings of child behavior to observed behavior in the laboratory found that informant rating discrepancies reflected actual differences in child behavior when observed interacting with parental and nonparental figures (De Los Reyes et al. 2009).

In summary, the presence of externalizing problems confined only to the home or school setting is quite common. These disagreements may be related to rater bias as well as a number of other child, parent, and school characteristics, including child sex, effortful control, and parental stress. As others have advocated, the common instances of disagreement between different informants of child behavior should optimally be viewed not simply as measurement noise to be overcome but important opportunities for the clinician to try and understand how and why children react with different environments (De Los Reyes et al., 2009; Hartley et al. 2011). Clinicians are urged to consider and to investigate these factors when weighing evidence from multiple factors in the course of a comprehensive assessment and treatment plan.

Disclosures

Dr. Verhulst is director at the Department of Child and Adolescent Psychiatry, Erasmus University Medical Center-Sophia Children's Hospital, which publishes the Dutch translations of the Achenbach System of Empirically Based Assessment and from which he receives remuneration.

Dr. Buitelaar has been in a consultant to/member of advisory board of/and/or speaker for Janssen Cilag BV, Eli Lilly, Organon/Shering Plough, UCB, Shire, Medice, and Servier. Drs. van Oort, Verhulst, Ormel, Hartman, Veenstra, Althoff, and Hudziak have no conflicts of interest or financial ties to disclose.

Footnotes

Acknowledgments

This research is part of the TRAILS. Participating centers include various departments of the University Medical Center and University of Groningen, the Erasmus University Medical Center Rotterdam, the University of Utrecht, the Radboud Medical Center Nijmegen, and the Trimbos Institute, all in the Netherlands. The authors are grateful to the adolescents, their parents and teachers, and to everyone who participated in this research and made it possible.

References

Abikoff

, Courtney

, Pelham

Jr , Koplewicz

. Teachers' ratings of disruptive behaviors: The influence of halo effects. J Abnorm Child Psychol, 21:519–533. 1993.

Achenbach

. Manual of the Teacher Report Form and 1991 profile. Burlington, VT: University of Vermont, 1991.

Achenbach

. Commentary: Definitely more than measurement error: But how should we understand and deal with informant discrepancies? J Clin Child Adols Psychol, 40:80–86. 2011.

Achenbach

. The Achenbach System of Empirically Based Assessment (ASEBA): Development, Findings, Theory, Applications. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families, 2009.

Achenbach

, Krukowski

, Dumenci

, Ivanova

. Assessment of adult psychopathology: Meta-analyses and implications of cross-informant correlations. Psychol Bull, 131:361–382. 2005.

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th (DSM-IV) Washington, DC: American Psychiatric Association, 1994.

Burns

, Walsh

, Gomez

. Convergent and discriminant validity of trait and score effects in ADHD-inattention and hyperactivity/impulsivity measures across a 3-month interval. J Abnorm Child Psychol, 31:529–541. 2003.

Chi

, Hinshaw

. Mother-child relationships of children with ADHD: The role of maternal depressive symptoms, depression-related distortions. J Abnorm Child Psychol, 30:387–400. 2002.

Collishaw

, Goodman

, Ford

, Rabe-Hesketh

, Pickles

. How far are associations between child, family and community factors and child psychopathology informant-specific and informant-general. J Child Psychol Psychiatry, 50:571–580. 2009.

10.

Costa

, McCrae

. Revised NEO Personality Inventory and NEO Five-Factor Inventory: Professional Manual. Odessa, FL: Psychological Assessment Resources, Inc., 1992.

11.

De Los Reyes

, Henry

, Tolan

, Wakschlag

. Linking informant discrepancies to observed variations in young children's disruptive behavior. J Abnorm Child Psychol, 37:637–652. 2009.

12.

De Los Reyes

, Kazdin

. Linking information discrepancies to observed variations in young children's disruptive behavior. Psychol Bull, 131:483–509. 2005.

13.

Derks

, Hudziak

, Boomsma

. Why more boys than girls with ADHD receive treatment: A study of Dutch twins. Twin Res Hum Genet, 10:765–770. 2007.

14.

De Winter

, Oldehinkel

, Veenstra

, Brunnekreef

, Verhulst

, Ormel

. Evaluation of non-response bias in mental health determinants and outcomes in a large sample of pre-adolescents. Eur J Epidem, 20:173–181. 2005.

15.

Dirks

, Boyle

, Georgiades

. Psychological symptoms in youth and later socioeconomic functioning: Do associations vary by informant? J Clin Child Adolesc Psychol, 40:10–22. 2011.

16.

Drabick

DAG

, Gadow

, Loney

. Source-specific oppositional defiant disorder: Comorbidity and risk factors in referred elementary schoolboys. J Am Acad Child Adolesc Psychiatry, 46:92–101. 2007.

17.

Fergusson

, Boden

, Horwood

. Situational and generalized conduct problems and later life outcomes: Evidence from a New Zealand birth cohort. J Child Psychol Psychiatry, 50:1084–1092. 2009.

18.

Hartley

, Zakriski

, Wright

. Probing the depths of informant discrepancies: Contextual influences on divergence and convergence. J Clin Child Adolesc Psychol, 40:54–66. 2011.

19.

Hudziak

, van Beijsterveldt

, Bartels

, Rietveld

, Rettew

, Derks

, Boomsma

. Individual differences in aggression: Genetic analyses by age, gender, and informant in 3-, 7-, and 10-year-old Dutch twins. Behav Genet, 33:575–589. 2003.

20.

Hudziak

. Genetic, environmental influences on wellness, resilience, psychopathology: A family-based approach for promotion, prevention, intervention. Developmental Psychopathology and Wellness: Genetic and Environmental Influences. Hudziak

J.J.

Washington, DC: American Psychiatric Publishing, Inc., 267–286. 2008.

21.

Huisman

, Oldehinkel

, de Winter

, Minderaa

, de Bildt

, Huizink

, Verhulst

, Ormel

. Cohort profile: The Dutch 'TRacking Adolescents' Individual Lives' Survey'; TRAILS. Int J Epidem, 37:1227–1235. 2008.

22.

Janssens

, Oldehinkel

, Rosmalen

. Parental overprotection predicts the development of functional somatic symptoms in young adolescents. J Pediat, 154:918–923e911. 2009.

23.

Jensen

, Xenakis

, Davis

, Degroot

. Child psychopathology rating scales and interrater agreement: II. Child and family characteristics. J Am Acad Child Adolesc Psychiatry, 27:1451–461. 1988.

24.

Kanne

, Abbacchi

, Constantino

. Multi-informant ratings of psychiatric symptom severity in children with autism spectrum disorders: The importance of environmental context. J Autism Dev Disord, 39:856–864. 2009.

25.

Kim

, Brody

, Murry

. Factor structure of the early adolescent temperament questionnaire and measurement invariance across gender. J Early Adolesc, 23:268–294. 2003.

26.

King

. Practice parameters for the psychiatric assessment of children and adolescents. J Am Acad Child Adolesc Psychiatry, 34:1386–1402. 1995.

27.

Kraemer

, Measelle

, Ablow

, Essex

, Boyce

, Kupfer

. A new approach to integrating data from multiple informants in psychiatric assessment and research: Mixing and matching contexts and perspectives. Am J Psychiatry, 160:1566–1577. 2003.

28.

Lahey

, Pelham

, Loney

, Kipp

, Ehrhardt

, Lee

, Willcutt

, Hartung

, Chronis

, Massetti

. Three-year predictive validity of DSM-IV attention deficit hyperactivity disorder in children diagnosed at 4–6 years of age. Am J Psychiatry, 161:2014–2020. 2004.

29.

Listug-Lunde

, Zevenbergen

, Petros

. Psychological symptomatology in siblings of children with ADHD. J Atten Disord, 12:239–247. 2008.

30.

MacLeod

, McNamee

, Boyle

, Offord

, Friedrich

. Identification of child psychiatric disorder by informant: Comparison of clinic and community samples. Can J Psychiatry, 44:144–150. 1999.

31.

Mebust

, Rettew

, Hudziak

. Effect of ADHD on parenting: Evidence for an evocative gene-environment correlation. Poster Presented at the 57th Annual Meeting of the American Academy of Child and Adolescent Psychiatry, New York, 2010.

32.

Nigg

, Hinshaw

. Parent personality traits and psychopathology associated with antisocial behaviors in childhood attention-deficit hyperactivity disorder. J Child Psychol Psychiatry, 39:145–159. 1998.

33.

Noordhof

, Oldehinkel

, Verhulst

, Ormel

. Optimal use of multi-informant data on co-occurrence of internalizing and externalizing problems: The TRAILS study. Int J Methods Psychiatr Res, 17:174–183. 2008.

34.

Offord

, Boyle

, Racine

, Szatmari

, Fleming

, Sanford

, Lipman

. Integrating assessment data from multiple informants. J Am Acad Child Adolesc Psychiatry, 35:1078–1085. 1996.

35.

Oldehinkel

, Hartman

, De Winter

, Veenstra

, Ormel

. Temperament profiles associated with internalizing and externalizing problems in preadolescence. Dev Psychopath, 16:421–440. 2004.

36.

Rettew

, Copeland

, Stanger

, Hudziak

. Associations between temperament and DSM-IV externalizing disorders in children and adolescents. J Dev Behav Pediatr, 25:383–391. 2004.

37.

Rettew

, Hudziak

. Genetics of ADHD. ADHD Comorbidities: Handbook for ADHD Complications in Children and Adults. Brown

T.E

. Arlington, VA: American Psychiatric Publishing, Inc., 23–36. 2009.

38.

Richters

. Depressed mothers as informants about their children: A critical review of the evidence for distortion. Psychol Bull, 112:485–499. 1992.

39.

Schachar

, Rutter

, Smith

. The characteristics of situationally and pervasively hyperactive children: Implications for syndrome definition. J Child Psychol Psychiatry, 22:375–392. 1981.

40.

Silverstein

. Validity of the WISC-R short forms. J Clin Psychol, 31:696–697. 1972.

41.

Soma

, Nakamura

, Oyama

, Tsuchiya

, Yamamoto

. Prevalence of attention-deficit/hyperactivity disorder (ADHD) symptoms in preschool children: Discrepancy between parent and teacher evaluations. Env Health Prev Med, 14:150–154. 2009.

42.

Steiner

, Remsing

. Practice parameter for the assessment and treatment of children and adolescents with oppositional defiant disorder. J Am Acad Child Adolesc Psychiatry, 46:126–141. 2007.

43.

Treutler

, Epkins

. Are discrepancies among child, mother, and father reports on children's behavior related to parents' psychological symptoms and aspects of parent-child relationships. J Abnorm Child Psychol, 31:13–27. 2003.

44.

van der Toorn

, Huizink

, Utens

, Verhulst

, Ormel

, Ferdinand

. Maternal depressive symptoms, and not anxiety symptoms, are associated with positive mother-child reporting discrepancies of internalizing problems in children: A report on the TRAILS study. Eur Child Adolescent Psychiatry, 19:379–388. 2010.

45.

Veenstra

, Lindenberg

, Oldehinkel

, De Winter

, Verhulst

, Ormel

. Prosocial and antisocial behavior in preadolescence: Teachers' and parents' perception of the behavior of boys and girls. Int J Behav Dev, 32:243–251. 2008.

46.

Yeh

, Weisz

. Why are we here at the clinic? Parent-child (dis)agreement on referral problems at outpatient treatment entry. J Consult Clin Psychol, 69:1018–1025. 2001.

47.

Youngstrom

, Findling

, Calabrese

. Who are the comorbid adolescents? Agreement between psychiatric diagnosis, youth, parent, and teacher report. J Abnorm Child Psychol, 31:231–245. 2003.

48.

Youngstrom

, Loeber

, Stouthamer-Loeber

. Patterns and correlates of agreement between parent, teacher, and male adolescent ratings of externalizing and internalizing problems. J Consult Clin Psychol, 68:1038–1050. 2000.