Considerations for test selection: How do validity and reliability impact diagnostic decisions?

Abstract

Nine preschool and school-age language assessment tools found to have acceptable levels of identification accuracy were evaluated to determine their overall levels of psychometric validity for use in diagnosing the presence/absence of language impairment. Eleven specific criteria based on those initially devised by McCauley and Swisher (1984) were applied to each of the selected tests in order to determine each test’s overall level of psychometric validity. Results indicated that each of the selected assessment tools met at least eight of the 11 criteria used to evaluate each assessment tool. Five tests met 10 out of 11 criteria. Findings are discussed to assist clinicians in applying psychometric criteria to these selected tests, as well as those not reviewed as part of this current review of standardized assessment tools. A decision tree is included within the discussion of this study’s findings to aid clinicians in the selection of standardized assessment tools that are most appropriate for clinical use, based on their psychometric characteristics.

Keywords

identification accuracy psychometric validity reliability sensitivity specificity validity

Get full access to this article

View all access options for this article.

References

American Speech-Language-Hearing Association ( 2000) Guidelines for the roles and responsibilities of the school-based speech-language pathologist. Available online at www.asha.org/policy (November 2009).

Anastasi A. and Urbina S. ( 1997) Psychological testing. 7th edition. Upper Saddle River, NJ: Prentice-Hall .

Dawson J. , Stout C. , and Eyer J. ( 2003) Structured photographic expressive language test. 3rd edition. DeKalb, IL: Janelle.

Dawson J. , Stout C. , Eyer J. , Tattersall P. , Fonkalsrud J. , and Croley K. ( 2005) Structured photographic expressive language test: Preschool . DeKalb, IL: Janelle.

Dollaghan CA ( 2004) Evidence-based practice in communication disorders: What do we know and when do we know it? Journal of Communication Disorders 37: 391-400.

Dollaghan CA ( 2007) The handbook for evidence-based practice in communication disorders. Baltimore, MD: Paul H Brookes.

Entwisle DR and Astone NM ( 1994) Some practical guidelines for measuring youth’s race/ethnicity and socioeconomic status. Child Development 65: 1521-40.

Fan X. ( 1998) Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement 53(3): 1-17.

Gillam RB and Pearson NA ( 2004) Test of narrative language. Austin, TX: Pro-Ed.

10.

Gray S. , Plante E. , Vance R. , and Henrichsen M. ( 1999) The diagnostic accuracy of four vocabulary tests administered to preschool-age children. Language, Speech, and Hearing Services in Schools 30: 196-206.

11.

Hutchinson TA ( 1996) What to look for in the technical manual: Twenty questions for users. Language, Speech, and Hearing Services in Schools 27: 109-21.

12.

McCauley RJ and Swisher L. ( 1984) Psychometric review of language and articulation tests for preschool children. Journal of Speech and Hearing Disorders 49: 34-42.

13.

Merrell AW and Plante E. ( 1997) Norm-referenced test interpretation in the diagnostic process. Language, Speech, and Hearing Services in Schools 28: 50-58.

14.

Mikucki BA and Larrivee L. ( 2006) Validity and reliability of twelve child language tests . Unpublished poster session presented at the American Speech-Language-Hearing Association’s national convention, Miami, FL.

15.

Paul R. ( 2007) Language disorders from infancy through adolescence: Assessment and intervention. St Louis, MO: Mosby.

16.

Peña ED , Spaulding TJ , and Plante E. ( 2006) The composition of normative groups and diagnostic decision-making: Shooting ourselves in the foot. American Journal of Speech-Language Pathology 15: 247-54.

17.

Plante E. and Vance R. ( 1994) Selection of preschool language tests: A data based approach . Language, Speech, and Hearing Services in Schools 25: 15-24.

18.

Plante E. and Vance R. ( 1995) Diagnostic accuracy of two tests of preschool language . American Journal of Speech-Language Pathology 4: 70-76.

19.

Rice ML and Wexler K. ( 2001) Test of early grammatical impairment. San Antonio, TX: Psychological Corporation.

20.

Roseberry-McKibbin C. ( 2007) Language disorders in children: A multicultural and case perspective. Boston, MA: Allyn and Bacon.

21.

Roulstone S. , Peters TJ , Glogowska M. , and Enderby P. ( 2008) Predictors and outcomes of speech and language therapists’ treatment decisions. International Journal of Speech-Language Pathology 10(3): 146-55.

22.

Semel E. , Wiig EH , and Secord WA ( 2003) Clinical evaluation of language fundamentals. 4th edition. San Antonio, TX: Psychological Corporation.

23.

Shipley KG , Stone TA , and Sue MB ( 1983) Test for examining expressive morphology. Austin, TX: Pro-Ed.

24.

Spaulding TJ , Plante E. , and Farinella KA ( 2006) Eligibility criteria for language impairment: Is the low end of normal always appropriate? Language, Speech, and Hearing Services in Schools 37: 61-72.

25.

Wiig EH and Secord W. ( 1989) Test of language competence. Expanded edition. San Antonio, TX: Psychological Corporation.

26.

Wiig EH , Secord WA , and Semel E. ( 2004) Clinical evaluation of language fundamentals: Preschool . 2nd edition. San Antonio, TX : Psychological Corporation.

27.

Young EC and Perachio JJ ( 1993) Patterned elicited syntax test. Tucson, AZ: Communication Skill Builders.

28.

Zimmerman IL , Steiner VG , and Pond RE ( 2002) Preschool language scale. 4th edition. San Antonio, TX: Psychological Corporation.