Development of a Change-Sensitive Outcome Measure for Children Receiving Counseling

Abstract

Contemporary testing standards place test purpose as the central focus during test development and subsequent use. This study describes the development of a measure for children designed explicitly to measure change resulting from psychosocial interventions. Parents completed the outcome measure for 896 elementary school-age children receiving psychotherapy interventions from community mental health agencies. Scales formed with change-sensitive items evidenced adequate reliability estimates and larger effect sizes than scales composed of the original item pool. When asked by stakeholders such as parents, principals, and school boards to investigate the effectiveness of provided interventions, school psychologists should consider the use of change-sensitive measures that are sensitive to small and moderately sized treatment effects.

De nos jours, il est pratique courante d'accorder, au cours de l'élaboration du test psychologique et de son utilisation ultérieure, une place centrale à l'objectif qu'il vise. La présente étude aborde l'élaboration d'un test pour enfants ayant pour objectif de mesurer d'une manière explicite les changements occasionnés par les interventions psychosociales. Des parents ont participé à ce test qui concernait 896 enfants d'âge primaire bénéficiant de l'aide de psychothérapeutes d'organismes communautaires de santé mentale. Les tests contenant des items aptes à détecter les changements attribuables au counseling ont prouvé leur efficacité et permettent de mieux mesurer ces changements que les tests constitués des items initiaux. Lorsque les intervenants, par exemple les parents, les chefs d'école et les conseils scolaires, leur demandent de déterminer l'efficacité des interventions offertes, les psychologues scolaires devraient envisager de recourir aux tests contenant des items aptes à détecter les changements attribuables au counseling.

Keywords

testing measurement outcome assessment counseling psychotherapy children and adolescents

Get full access to this article

View all access options for this article.

References

Achenbach, T.M. (1994). Child Behavior Checklist and related instruments . In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (pp. 517-549). Hillsdale, NJ: Lawrence Erlbaum.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing . Washington, DC: American Educational Research Association.

Conners, C.K. (1994). Conners Rating Scales. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (pp. 550-578). Hillsdale, NJ: Lawrence Erlbaum.

Cronbach, L.J. , Ambron, S.R. , Dornbursch, S.M. , Hess, R.D. , Hornik, R.C. , Phillips, D.C. , et al. (1980). Toward reform of program evaluation. San Francisco, CA: Jossey-Bass.

Finn, J.D. , Pannozzo, G.M. , & Voelkl, K.E. (1995). Disruptive and inattentive-withdrawn behavior and achievement among fourth graders. Elementary School Journal , 95, 421-434.

Frank, A.C. , & Kirk, B.A. (1975). Differences in outcome for users and nonusers of university counseling and psychiatric services: A 5-year accountability study. Journal of Counseling Psychology, 22, 252-258.

Hill, C.E. , & Lambert, M.J. (2004). Methodological issues in studying psychotherapy processes and outcome. In M. J. Lambert (Ed.), Bergin and Garfield's handbook of psychotherapy and behavior change (5th ed., pp. 84-135). New York: John Wiley.

Kazdin, A.E. (2000). Psychotherapy for children and adolescents: Directions for research and practice. New York: Oxford Press.

Lambert, M.J. , Hatch, D.R. , Kingston, M.D. , & Edwards, B.C. (1986). Zung, Beck, and Hamilton Rating Scales as measures of treatment outcome: A meta-analytic comparison. Journal of Consulting and Clinical Psychology, 54, 54-59.

10.

Lewis, J.D. , & Magoon, T.M. (1987). Survey of college counseling centers' follow-up practices with former clients. Professional Psychology: Research and Practice, 18(2), 128-133.

11.

Lipsey, M. (1998). Design sensitivity: Statistical power for applied experimental research. In L. Bickman & D. J. Rog (Eds.), Handbook of applied social research methods (pp. 39-68). Thousand Oaks, CA: Sage.

12.

Meier, S. (1997). Nomothetic item selection rules for tests of psychological interventions. Psychotherapy Research, 7, 419-427.

13.

Meier, S. (1998). Evaluating change-based item selection rules. Measurement and Evaluation in Counseling and Development, 31, 15-27.

14.

Meier, S. (2000). Treatment sensitivity of the PE Form of the Social Skills Rating Scales: Implications for test construction procedures. Measurement and Evaluation in Counseling and Development, 33, 144-156.

15.

Meier, S. , & Letsch, E. (2000). Data collection issues in an urban community mental health center: What is necessary and sufficient information for outcome assessment ? Professional Psychology: Research and Practice, 31, 409-411.

16.

Meier, S.T. (2004). Improving design sensitivity through intervention-sensitive measures. American Journal of Evaluation, 25, 321-334.

17.

Meier, S.T. , McDougal, J.L. , & Bardos, A. (in press). The Behavioral Intervention Monitoring Assessment System . Toronto: Multi-Health Systems .

18.

Meier, S.T. , & Vermeersch, D. (2007). What changes in counseling and psychotherapy? Manuscript submitted for publication.

19.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). Washington, DC: American Council on Education and National Council on Measurement in Education .

20.

Speer, D.C. , & Newman, F.L. (1996). Mental health services outcome evaluation. Clinical Psychology: Science and Practice, 3, 105-129.

21.

Stiffman, A.R. , Orme, J.G. , Evans, D.A. , Feldman, R.A. , & Keeney, P.A. (1984). A brief measure of children's behavior problems: The Behavior Rating Index for Children. Measurement and Evaluation in Counseling and Development, 16, 83-90.

22.

Vermeersch, D.A. , Lambert, M.J. , & Burlingame, G.M. (2000). Outcome questionnaire: Item sensitivity to change. Journal of Personality Assessment , 74, 242-261.

23.

Vermeersch, D.A. , Whipple, J.L. , Lambert, M.J. , Hawkins, E.J. , Burchfield, C.M. , & Okiishi, J.C. (2004). Outcome Questionnaire: Is it sensitive to changes in counseling center clients? Journal of Counseling Psychology , 51, 38-49.

24.

Webster-Stratton, C. (1996). Early onset conduct problems: Does gender make a difference. Journal of Consulting & Clinical Psychology , 64, 540-551.

25.

Weinstock, M. , & Meier, S.T. (2003). A comparison of two item selection methodologies for measuring change in university counseling center clients. Measurement and Evaluation in Counseling & Development, 36, 66-75.

26.

Weisz, J.R. , Huey, S.J. , & Weersing, V.R. (1998). Psychotherapy outcome research with children and adolescents: The state of the art. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in clinical child psychology (Vol. 20, pp. 49-91). New York: Plenum.

27.

Weisz, J.R. , Weiss, B. , Han, S.S. , Granger, D.A. , & Morton, T. (1995). Effects of psychotherapy with children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin, 117, 450-468.