Standardized rating scales remain the primary mechanism through which child behaviors are recorded. Despite the many advantages of such systems, the documented lack of agreement among different informants' ratings of the same child remains a pervasive problem for clinicians. This study examines the degree to which observed behavior ratings were influenced by different informants, relative to the traits these measures were designed to assess. A correlated trait-correlated method confirmatory factor analysis model is estimated on a multitrait-multimethod design in which five behavioral traits were completely crossed with three informants (mothers, fathers, and teachers). Behavioral ratings were completed on 562 first-grade children. Results indicate that ratings of child behavior are heavily influenced by the informant, that informant influences are relatively independent of one another, and that the influence of method variance is often greater than the influence of the trait being assessed.
Achenbach, T.M. (1991a). Manual for the Child Behavior Checklist/4-18 and 1991 Profile. Burlington: University of Vermont, Department of Psychiatry.
2.
Achenbach, T.M. (1991b). Manual for the Teacher's Report Form and 1991 Profile. Burlington: University of Vermont, Department of Psychiatry.
3.
Achenbach, T.M., & McConaughy, S.H. (1987). Empirically based assessment of child and adolescent psychopathology: Practical applications. Newbury Park, CA: Sage.
4.
Achenbach, T.M., McConaughy, S.H. & Howell, C.T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross informant correlations for situational specificity. Psychological Bulletin,101(2), 213-232.
5.
Achenbach, T.M., & Rescorla, L.A. (2000). Manual for the ASEBA preschool forms and profiles. Burlington: University of Vermont, Department of Psychiatry.
6.
Angold, A., & Costello, E.J. (1996). Toward establishing an empirical basis for the diagnosis of oppositional defiant disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 35(9), 1205-1212.
7.
Arbuckle, J.L., & Wothke, W. (1999). Amos 4.0 user's guideChicago : Small Waters.
8.
Arend, R., Lavigne, J.V., Rosenbaum, D., Binns, H.J., & Christoffel, K.K. (1996). Relations between taxonomic and quantitative diagnostic systems in preschool children: Emphasis on disruptive disorders. Journal of Clinical Child Psychology, 25, 388-397.
9.
Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin,88, 588-606.
10.
Bollen, K.A. (1989). Structural equations with latent variables. New York: John Wiley.
11.
Broidy, L., Nagin, D., Tremblay, R., Bates, J., Brame, B., Dodge, K. (2003). Developmental trajectories of childhood disruptive behaviors and adolescent delinquency: A six-site, cross-national study. Developmental Psychology, 39(2), 222-245.
12.
Browne, M.W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
13.
Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin,56, 81-105.
14.
Canivez, G.L., Watkins, M.W., & Schaefer, B.A. (2002). Interrater agreement for discriminant classifications for the adjustment scales for children and adolescents . Psychology in the Schools, 39, 375-384.
15.
Eisert, D.C., Sturner, R.A., & Mabe, A.P. (1991). Questionnaires in behavioral pediatrics: Guidelines for selection and use. Journal of Developmental and Behavioral Pediatrics, 1, 42-50.
16.
Fagot, B.I. (1995). Classification of problem behaviors in young children: A comparison of four systems. Journal of Applied Developmental Psychology, 16, 95-106.
17.
Fergusson, D.M., Horwood, L.J., & Lloyd, M. (1991). Confirmatory factor models of attention deficit and conduct disorder. Journal of Child Psychology and Psychiatry , 32, 257-274.
18.
Hu, L., & Bentler, P.M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76-99). Thousand Oaks, CA: Sage.
19.
Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
20.
Kamphaus, R.W., & Frick, P.J. (1996). Clinical assessment of child and adolescent personality and behavior. Boston, MA: Allyn & Bacon.
21.
Kaplan, D. (1990). Evaluating and modifying covariance structure models: A review and recommendation. Multivariate Behavioral Research, 25, 137-155.
22.
Keith, T.Z., & Witta, E.L. (1997). Hierarchical and cross-age confirmatory factor analysis of the WISC-III: What does it measure?School Psychology Quarterly, 12, 89-107.
23.
Kenny, D.A., & Kashy, D.A. (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin,112, 165-172.
24.
Kline, R.B. (1998). Principles and practice of structural equation modeling. New York: Guilford.
25.
Kline, R.B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York : Guilford.
26.
Knoff, H.M. (1995). Best practices in personality assessment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology−III (pp. 849-864). Washington, DC: National Association of School Psychologists .
27.
Konold, T.R., Brewster, J.C., & Pianta, R.C. (2004). The behavior of child behavior ratings: Measurement structure of the Child Behavior Checklist across time, informants, and child gender. Journal ofBehavioral Disorders, 29, 372-383.
28.
Kraemer, H.C., Measelle, J.R., Ablow, J.C., Essex, M.J., Boyce, W.T., & Kupfer, D.J. (2003). A new approach to integrating data from multiple informants in psychiatric assessment and research: Mixing and matching contexts and perspectives. American Journal of Psychiatry,160(9), 1566-1577.
29.
Lance, C.E., Noble, C.L., & Scullen, S.E. (2002). A critique of the correlated trait-correlated method and correlated uniqueness models for multitrait-multimethod data. Psychological Methods, 7, 228-244.
30.
Lewis, M. (2000). Toward a development of psychopathology: Models, definitions, and prediction. In A. J. Sameroff , M. Lewis, & S. M. Miller (Eds.), Handbook of developmental psychopathology (2nd ed., pp. 3-19). New York: Plenum.
31.
MacLeod, R.J., McNamee, J.E., Boyle, M.H., Offord, D.R., & Friedrich, M. (1999). Identification of childhood psychiatric disorder by informant: Comparisons of clinic and community samples. Canadian Journal of Psychiatry, 44, 144-150.
McConaughy, S.H. (1993). Advances in empirically based assessment of children's behavioral and emotional problems. School Psychology Review , 22(2), 285-307.
34.
McConaughy, S.H., & Ritter, D.R. (1995). Best practices in multidimensional assessment of emotional or behavioral disorders. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology−III (pp. 865-877). Washington, DC: National Association of School Psychologists .
35.
Muthen, B.O. (1989). Dichotomous factor analysis of symptom data. Sociological Methods and Research, 18, 19-65.
36.
National Center for Educational Statistics. ( 1999). Academic Skills Rating Scale: ECLS-K. Retrieved from http://nces.ed.gov/pubs2001/2001029_1_4.pdf
37.
National Institute of Child Health and Human Development (NICHD) Early Child Care Research Network. (1998). Early child care and self-control, compliance, and problem behavior at twenty-four and thirty-six months. Child Development, 69, 1145-1170.
38.
National Institute of Child Health and Human Development (NICHD). (2001). Children in working families: Parent employment and poverty as predictors of developmental outcomes. Manuscript under review.
39.
National Institute of Child Health and Human Development (NICHD) Early Child Care Research Network. (2004). Social functioning in first grade: Associations with earlier home and child care predictors and with current classroom experiences. Child Development, 74(6), 1639-1662.
40.
Offord, E.R., Boyle, M.H., Racine, Y., Szatmari, P., Fleming, J.E., Sanford, M., et al. (1996). Integrating assessment data from multiple informants. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 1078-1085.
41.
Piacentini, J. (1993). Checklists and rating scales. In T. H. Ollendick & M. Hersen (Eds.), Handbook of child and adolescent assessment (pp. 82-97). Boston: Allyn & Bacon.
42.
Piacentini, J., Cohen, P., & Cohen, J. (1992). Combining discrepant diagnostic information from multiple sources: Are complex algorithms better than simple ones? Journal of Abnormal Child Psychology, 20, 51-63.
43.
Ruffalo, S.L., & Elliott, S.N. (1997). Teachers' and parents' ratings of children's social skills: A closer look at cross-informant agreements through an item analysis protocol. School Psychology Review, 26, 489-501.
44.
Shaw, D., Gilliom, J., Ingoldsby, E., & Nagin, D. (2003). Trajectories leading to school-age conduct problems . Developmental Psychology, 39(2), 189-200.
45.
Spiker, D., Kraemer, H.C., Constantine, N.A., & Bryant, D. (1992). Reliability and validity of behavior problems checklists as measures of stable traits in low birth weight, premature preschoolers . Child Development, 63, 1481-1496.
46.
Suen, H.K., & Ary, D. (1989). Analyzing quantitative behavioral observation data. Hillsdale, NJ: Lawrence Erlbaum .
47.
Tabachnick, B.G., & Fidell, L.S. (2001). Using multivariate statistics. Needham Heights, MA: Allyn & Bacon.
48.
Tanaka, J.S. (1993). Multifaceted conceptions of fit in structural equation models. In K. S. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 10-39). Newbury Park, CA: Sage.
49.
Thomas, J.M., Benham, A.L., Gean, M., Luby, J., Minde, K., Turner, J.M., et al. (1997). Practice parameters for the psychiatric assessment of infants and toddlers (0-36 months). Journal of the American Academy of Child and Adolescent Psychiatry, 36, 21-46.
50.
Thompson, B. (2003). Score reliability: Contemporary thinking on reliability issues. Thousand Oaks, CA: Sage .
51.
U.S. Department of Labor, Bureau of Labor Statistics. (1994). Current population survey, March 1991, as cited in U.S. Department of Labor, Women's Bureau, 1993 Handbook on women workers: Trends and issues. Washington, DC: Government Printing Office.
52.
Wasserman, R.C., Kelleher, K.J., Bocian A., Baker, A., Childs, G.E., Indacochea, F., et al. (1999). Identification of attentional and hyperactivity problems in primary care: A report from pediatric research in office settings and the ambulatory sentinel practice network. Pediatrics, 103, 38-44.
