Abstract
This study examines similarities and differences in teacher ratings of behavioral problems and adaptive skills between a sample of 320 students from Anguilla, BWI and 315 children from the United States of America using the Behavior Assessment System for Children (BASC; Reynolds & Kamphaus, 1992). The study also compared teacher ratings of boys and girls in the Caribbean sample. Analyses of variance (ANOVA) revealed that Anguillian teachers rated their students as having higher mean school problems, while the mean score for adaptive skills was higher for US students. Effect sizes were small, though, indicating that little of the variance in ratings was attributable to cultural group. Boys in the Caribbean sample had significantly higher mean scores on externalizing and school problems. Girls were rated at having more adaptive skills. Again, effect sizes for the differences were small. Scientific and practice implications are offered.
Keywords
The field of school psychology is growing and evolving worldwide (Farrell, 2010; Farrell, Jimerson, & Oakland, 2007). While the exact number of school psychologist is unknown, a 2009 study estimated that over 76,100 school psychologists were working in the 48 countries from which data were available (Jimerson, Stewart, Skokut, Cardenas, & Malone, 2009). Moreover, among the 192 Member States of the United Nations, 83 countries had evidence of school psychologists (Jimerson, Skokut, Cardenas, Malone, & Stewart, 2008). Although the roles of school psychologists vary from country to country as well as within countries, in general, school psychologists are professionals who provide services in schools to enhance the academic, emotional, social, and behavioral adjustment and success of children (Cook, Jimerson, & Begeny, 2010). Given this responsibility, international research on competencies and behavioral problems among children seems critical for our profession.
As noted by Achenbach and Rescorla (2007), international or cross-cultural research on behavioral problems among children is valuable for a number of reasons. To begin, cultural values, norms, and perceptions may influence the behaviors that children express, while at the same time, certain child problems may be less susceptible to cultural differences. Disentangling these differences will most likely be achieved only through cross-cultural/international research. Cross-cultural research also can assist school psychologists in different countries on becoming aware of the problems that may be more or less prevalent among the students they serve (Little, Akin-Little, & Lloyd, 2011). Finally, cross-cultural research can contribute to information on the validity of using certain tests or assessment instruments with children in various cultures (Achenbach & Rescorla, 2007). All of these benefits can, in turn, aid school psychologists in developing and implementing approaches for the prevention and amelioration of emotional and behavioral problems among children in various countries and cultures.
Psychologists have conducted numerous epidemiological studies examining behavior problems among children in different parts of the world using behavior rating scales (e.g., Crijnen, Achenbach, & Verhulst, 1997, 1999; Kim, Kamphaus, Orpinas, & Kelder, 2010; Lambert, Knight, Taylor, & Achenbach, 1996; Rescorla, Achenbach, Ginzburg, et al., 2007; Rescorla, Achenbach, Ivanova, et al., 2007). The time demands upon school psychologists are usually many, and rating scales function as a time- and cost-efficient method for assessing functioning across domains (Hart & Lahey, 1999). Rating scales also allow school psychologists to evaluate behavior and compare it to normative or clinical samples, which helps to determine the severity of the behavior(s) in question. Additionally, the dimensional nature of rating scales allows psychologists to consider problematic behavior, even when the problems are not severe enough to warrant a diagnosis. One disadvantage of standardized rating scales is that they can be costly to develop.
Cross-cultural comparisons with parent ratings
A number of researchers have relied on parents’ ratings of their children using the Child Behavior Checklist (CBCL; Achenbach, 1991a; Achenbach & Rescorla, 2001) to study behavior problems in different countries. For example, Crijnen et al. (1997, 1999) used the CBCL to compare behavioral and emotional problems among 13,697 children from 12 countries. Rescorla, Achenbach, Ivanova, et al. (2007) built upon the work of Crijnen et al. (1997, 1999) by examining ratings on the CBCL across 31 societies. The 31 societies consisted of 12 from Western Europe, five from Eastern Europe, six from Asia, one from Africa, three from the Middle East, two from the Caribbean and one from the United States of America (USA) and Australia each. On the whole, results from this study suggested considerable consistency in parents’ ratings across different countries and cultures (Rescorla, Achenbach, Ivanova, et al., 2007). For instance, the mean ratings from 19 cultures were within 1 SD of the overall mean. The authors concluded that variability among parent ratings of behavior problems tends to be higher within cultures than between cultures (Rescorla, Achenbach, Ivanova, et al., 2007).
Rescorla, Achenbach, Ivanova, et al. (2007) did find differences between ratings of boys and girls. On average, boys tended to have higher scores for total problems. Moreover, boys obtained higher mean ratings of attention problems and externalizing problems such as delinquent or aggressive behavior, particularly at ages 6 to 11 years. Girls, on the other hand, received higher ratings for internalizing problems such as somatic complaints and anxious/depressed behaviors or symptoms, especially at ages 12 to 16. The authors concluded that gender differences in ratings of behavior problems appear to be common (Rescorla, Achenbach, Ivanova, et al., 2007).
Cross-cultural comparisons with teacher ratings
In addition to parent ratings, teacher ratings have also been used in international studies examining child behavior problems (e.g. Lambert, Lyubansky, & Achenbach, 1998; Rescorla, Achenbach, Ginzburg, et al., 2007). At noted by Weisz, Chaiyasit, Weiss, Eastman, and Jackson (1995), teachers’ day-to-day interactions allow for a comprehensive insight into a child’s behavior. Teachers are in a position to observe the child while the child is engaged in age-appropriate academic activities, as well as during interactions with their peers. Thus, teachers have an opportunity to observe behavior that may not be evident to parents (Lambert et al., 1998). Teachers also have the advantage of a built-in normative group in the form of other children in their class. A final advantage of teacher ratings is that teachers are typically easily accessible in schools.
As was the case with parent ratings, differences in teacher ratings of behavior problems exist, but there are also many consistencies. For example, Lambert et al. (1996) examined ratings of behavior problems between teachers from the USA and Jamaica using Achenbach’s (1991b) Teachers’ Report Form (TRF). The Jamaican sample consisted of teacher ratings of 359 students of whom 90% were of African descent. The remaining 10% were Chinese or East Indian. The US sample consisted of teacher ratings of 665 students. Seventy-seven percent of the US students were Caucasian, 15% were of African descent, and the remaining 8% were ‘other’. Although the mean Total score was higher for the ratings from Jamaican teachers, the difference based on nationality was small and accounted for only 2% of the variance. Mean ratings from Jamaican teachers were also higher on six of the eight clinical syndrome scales as well as externalizing and internalizing problems, but theses effects were small as well (Lambert et al., 1996).
In addition to differences by country, Lambert et al. (1996) found that boys were rated as having higher mean scores for attention problems and externalizing problems of delinquent and aggressive behavior. However, the gender differences on the externalizing composite accounted for only 4% of the variance. There were no gender differences on scores for internalizing problems.
Rescorla, Achenbach, Ginzburg, et al. (2007) expanded upon this two country comparison with a study examining teacher ratings from 21 countries. The countries included ten European, three Asian, three Middle Eastern, and two Caribbean countries in addition to the USA and Australia. The results were similar to those from previous studies (Rescorla, Achenbach, Ivanova, et al., 2007). On the total problems scale, 15 countries had mean scores within 1 SD of the overall mean, and culture rarely explained more than 10% of the variance in ratings. In fact, the overall effect size for culture was 0.08 for total problems. As was the case with parent ratings, the findings led the researchers to conclude that scores were more likely to vary within countries than between countries (Rescorla, Achenbach, Ginzburg, et al., 2007).
The small but significant differences by sex were also similar to results from previous comparisons. Boys were rated as having higher externalizing problems and attention problems in most countries (Rescorla, Achenbach, Ginzburg, et al., 2007). With teacher ratings, there were no significant sex differences for internalizing scores for children ages 6–11. The authors hypothesized that sex differences on internalizing scores from parents but not teachers may be because teachers are less sensitive to internalizing problems than parents, and students may be more likely to express internalizing symptoms at home (Rescorla, Achenbach, Ginzburg, et al., 2007).
Taken together, these international comparisons suggest that differences in levels of child problems across countries tend to be small, when they exist at all, and account for little of the variance in ratings. In comparing boys and girls, teachers and parents tend to rate boys as having higher externalizing behaviors and attention problems, while parents have a tendency to rate girls as having higher internalizing problems. The sex differences, like the cultural differences, do not account for large portions of the variance in ratings.
Although this body of work has contributed greatly to our knowledge and understanding of child behavior problems internationally, an issue that possibly limits the generalizability of the findings is that all studies used instruments from the Achenbach System of Empirically Based Assessment (ASEBA; Achenbach & Rescorla, 2001). Thus, further cross-cultural comparisons with other rating scales seem warranted. Results with other measures would confirm that findings by Achenbach and his colleagues are not scale specific but generalizable.
Cross-cultural research with the BASC
The Behavioral Assessment System for Children (BASC; Reynolds & Kamphaus, 1992) is another popular measure for assessing adjustment and behavior problems among children. One advantage of the BASC rating scales is the inclusion of adaptive competencies that are rated and scored in the same format as the clinical scales.
To date, few cross-cultural studies using the BASC have been published. Among those that have been published, none were conducted in the same format as those with the ASEBA measures. For example, a 2003 study (Zhou, Peverly, Xin, Huang, &Wang, 2003) examined mean differences among Chinese, Chinese-American, and European-American middle school students on the BASC Self-Report of Personality (SRP; Reynolds & Kamphaus, 1992). The researchers found that European-American and Chinese-American students had more negative attitudes toward school and Chinese-American students had more negative attitudes toward teachers than did European-American students or students from Mainland China (Zhou et al., 2003). Moreover, Chinese-American students reported higher levels of social stress, anxiety, and depression. However, effect sizes were not calculated, so it is difficult to determine the meaningfulness of the differences.
Another study compared behavior problems and adjustment among Korean, Korean American, and European-American students using the BASC Parent Rating Scale (PRS; Reynolds & Kamphaus, 1992) and the BASC SRP (Jung & Stinnett, 2005). A multivariate analysis of variance (MANOVA) of parent ratings revealed significant variance by group. In this study, 26% of the variance in test scores was attributable to the cultural groups. The authors conclude that the differences may be the result of differing social expectations by the parents (Jung & Stinnett, 2005). However, one potential methodological issue with the Jung and Stinnett study is that each group had only 40 students.
Goals of the current study
The current study sought to expand upon the international research with children using standardized, behavior rating scales. Specifically, this study investigated the consistencies and differences in teacher reports of child behavior between a sample of children from the USA and a sample of students from Anguilla, BWI (Caribbean Island) using the BASC. Due to the relatively consistent findings of mean differences by sex on rating scales, sex differences were also analyzed for the Caribbean sample.
Method
Participants
The data consisted of two sets of teacher ratings. The first group of participants was recruited through the schools of Anguilla, BWI. The second set of ratings was from US students who participated in the standardization of the BASC Teacher Rating Scales for Children (TRS-C; Reynolds & Kamphaus, 1992).
Anguillian data
The sample from Anguilla consists of ratings of 320 students, ages 6 through 11 years. All elementary schools on the island participated in the project, and the number of TRS-C collected represented 47 % of students ages 6–11 in the public schools in Anguilla. Children in Anguilla have the same teacher for all subjects, so ratings were made by that teacher. Of the 320 Caribbean students, 43% were male and 57% were female. The mean age was 8.79 years. Although individual information on ethnicity was not available, the majority of students were of African descent, as is the majority (90%) of the population in Anguilla.
USA data
The US data consisted of teacher ratings of 315 children ages 5 through 12. The participants were randomly selected from the original normative sample of 1,259 (Reynolds & Kamphaus, 1992). Of the 315 students, 52% were male and 48% were female. The mean age was 8.70 years. Sixty-nine percent of the sample was Caucasian, 24% African-American, 5% Hispanic, 1% Asian, and 1% other.
Instrument
The BASC-TRS-C (Reynolds & Kamphaus, 1992) has 148 items that are rated by teachers on a four-point Likert scale ranging from ‘Never’ to ‘Almost Always’. The BASC-TRS-C includes four scales measuring adaptive behaviors and ten clinical scales that measure problematic behaviors.
The BASC clinical scales yield four composite scores, while the adaptive scales combine to form an adaptive skills composite. The hyperactivity, aggression, and conduct problems scales combine to produce externalizing problems. The anxiety, depression, and somatization scores are summed for internalizing problems, and scores for the attention and learning problems scales make up the school problems composite. A final composite is the behavioral symptoms index (BSI), which represents an overall level of problems.
The BASC Manual (Reynolds & Kamphaus, 1992) provides evidence of reliability for the TRS. Internal consistency estimates for the normative sample were high, with a range of 0.90 for internalizing problems to 0.97 for adaptive skills and the BSI. The median test–retest reliability was 0.71. The teacher inter-rater reliability coefficients ranged from 0.60 to 0.91.
The BASC Manual presents factor-analytic results as evidence of construct validity using both exploratory (EFA) and confirmatory (CFA) factor analyses (Reynolds & Kamphaus, 1992). Additionally, the TRS scales have high correlations with other comparable teacher rating scales (Reynolds & Kamphaus, 1992). Rivers and Rowe (2006) used multi-sample CFA to compare the four-factor model from the BASC Manual (Reynolds & Kamphaus, 1992) using data from the normative sample and data from Caribbean teachers. The researchers used the comparative fit index (CFI) and root mean square error of approximation (RMSEA) to evaluate model fit at each step of the analyses. The values for the two fit indexes each changed only 0.01 from the constrained model to the model with all parameters estimated freely. The lack of a substantial change in the fit indices suggests that the model fit is essentially equivalent across the two data sets.
Data analyses
We began the current analyses by obtaining the internal consistency values for the composites with the data from Anguilla. Subsequently, we conducted a series of analyses of variance (ANOVA) on the five BASC composites to assess differences by cultural group. We also examined differences between ratings of boys and girls from Anguilla. Given the multiple comparisons, the final alpha level was .01. Missing values were treated by listwise deletion, and effect sizes were measured with Eta squared. We analyzed and present our results with T-scores. Reynolds and Kamphaus (1992) converted BASC raw scores to T-scores using a linear transformation. Given a linear transformation, analyses with T-scores are equivalent to using raw scores, except that T-scores provide an interpretive advantage.
Results
The scores from teacher ratings of students from Anguilla yielded a Cronbach’s alpha of 0.91 for the adaptive composite, 0.72 for the internalizing, 0.90 for the externalizing, 0.91 for the school problems, and 0.84 for the BSI.
Differences by culture
T-score Means, Standard Deviations, and differences of teacher BASC ratings
p ≤ 0.01; **p = 0.01.
Gender differences
T-score Means, Standard Deviations and differences of teacher BASC ratings from Anguilla by sex
p < 0.01.
Discussion
The values for Cronbach’s alpha indicate that the internal consistency of the composites ranged from acceptable to excellent with the data from Anguilla (Cicchetti, 1994). Though lower, the internal consistency value for Internalizing Problems was acceptable. Additionally, there were no significant differences in ratings between the two groups for total problems on the BSI. While there were significant differences on school problems and adaptive skills, the differences were small and do not appear to be clinically meaningful. The present study, then, provides support for the similarity in levels of behavior problems and adaptive skills as rated by teachers for children in the USA and Anguilla, BWI.
These results are consistent with the findings of Rescorla, Achenbach, Ginzburg, et al. (2007) in their comparison of teacher ratings from 21 countries. In that study, differences by culture rarely explained more than 10% of the variance in scores. The conclusion that ratings were more likely to vary within country than between country (Rescorla, Achenbach, Ginzburg, et al., 2007) applies to our findings as well. The similarity of these results and previous research using the ASEBA (Achenbach & Rescorla, 2001) serves to strengthen confidence in the degree to which one can generalize current and past findings regarding ratings of behavior problems among children internationally to other measures.
More specifically, the current findings are consistent with the comparison of teacher ratings with Achenbach’s scales for Jamaican and US students (Lambert et al., 1996). There is no composite on the Achenbach scales representing school problems, but the Lambert et al. study did reveal differences by country on the attention problem scale. The BASC attention problems scale is one of the two scales on to the school problems composite. Thus, the finding of differences in mean ratings of school problems in our study seems consistent with previous research. The overall similarity in these results is particularly salient, given that Jamaica and Anguilla are both Caribbean Islands. While differences in culture and norms exist among the islands and nations of the Caribbean, most would agree that there are substantial differences between US and Caribbean cultures. At the same time, teacher ratings of behavior problems, adjustment, and adaptive skills are remarkably similar. As noted by Lambert et al. (1998), such similarities in ratings of child behavior and adaptation may suggest common processes that influence child adaptation and pathology as well as informant ratings, despite the effects of culture.
Moreover, researchers have found considerable cross-cultural consistencies in the life events that children find most stressful (Leontopoulou, Jimerson, & Anderson, 2011; Yamamoto, et al., 1996). Such findings have led researchers to consider the concept of a common culture of childhood (Yamamoto et al., 1996). Future cross-cultural research should explore possible intersections or interactions between common processes that influence child adaptation and behavior problems and a common culture of childhood.
The current study also addressed the issue of differences in ratings of child behavior by sex. The significant differences between Anguillian boys and girls were higher than the differences by culture; however, the variance accounted for was still small. Although the pattern of boys having higher ratings on externalizing problems and no differences on internalizing problems was consistent with previous research on teacher ratings (Lambert et al., 1996; Rescorla, Achenbach, Ginzburg, et al., 2007), there is little comparable information on differences in school problems or adaptive skills. The issue of sex differences in school problems and adaptive skills is another topic for future cross-cultural research.
In terms of practical implications, many countries lack appropriate diagnostic tools that meet high psychometric standards and allow school psychologists to assess the emotional and behavioral needs of children. The results of this study and others suggest that school psychologists can feel confident in using measures developed in other countries for assessing adjustment and behavioral problems. This reduces the potential expense, cost, and time of having to develop instruments locally.
For school psychologists working to improve the lives of students, though, assessment is only the first step. It is prevention or intervention efforts that typically effect change. If, in fact, common processes do result in adaptation or behavior problems among children in different countries, then it may be that approaches to prevention and intervention which have been empirically supported in one country will yield similar results in other countries. For instance, Yeo and Choi (2011) implemented a cognitive-behavioral, group-based intervention in Singapore schools with positive results following reports of similar successful interventions in schools in the US and UK.
Another practical implication from the current study relates to findings of sex differences. Rescorla, Achenbach, Ivanova, et al. (2007) concluded that the consistency of sex differences in ratings provides support for the practice of using separate normative data for boys and girls. We would argue the opposite. Of course, a primary issue in the debate about using separate- or combined-sex norms is the purpose for which the rating information was obtained. If the purpose is to screen or identify children for targeted intervention, it seems likely that school psychologists would want the children who are highest on the behavior in question, regardless of sex. Using separate sex norms could result in the selection of a set of boys with the highest scores and a set of girls with the highest scores, even though some boys not selected could have higher raw scores than girls who were selected. Consequently, when differences in ratings between boys and girls appear to be real and the goal is intervention, we suggest the use of combined-sex norms.
In spite of the relevance of the findings, this research is not without limitations. A primary shortcoming is that the data consist entirely of teacher ratings, and some studies have found differences between teacher ratings and actual observations of behavior (Puig et al., 1999; Weisz et al., 1995). Moreover, best practices in assessment call for information from multiple informants who observe the child in different contexts. At the same time, teachers clearly have an important perspective on child behavior (Lambert et al., 1996).
Another limitation is that the current study investigated mean differences in teacher ratings of children’s behavior only in Anguilla, a small Caribbean Island, and the USA. However, as Oakland and Jimerson (2007) point out, school psychology, as an international profession, will benefit from extending beyond its traditional, Western influences, and international research is one way for this expansion to take place. At the same time, few school psychologists have the time or resources to engage in research (Merrell, Ervin, & Gimpel, 2006). One solution is the establishment of collaborations between researchers in school psychology and school psychologists living and working in areas where the profession is emerging (Oakland & Jimerson). Such partnerships will generate empirically-based findings that, when taken together, enhance our knowledge of children worldwide.
Thus, we conceptualize our findings as adding evidence to the hypothesis that international or cross-cultural assessment of behavioral and emotional problems among children is likely reflective of real similarities, rather than similarities due to instrument artifacts or cultural variation. Although, future research should continue to explore the epidemiology and measurement of constructs such as school problems, adaptive skills, externalizing problems, and internalizing problems with children internationally, the evidence to date suggests that school psychologists focus on the similarities of behavior problems and competencies among children versus potential differences due to culture.