Structured Teacher Ratings to Identify Students in Need of Academic Assistance

Abstract

The Brief Academic Competence Evaluation Screening System (BACESS) is a multiphase universal screening measure designed to assist educators in the identification of students who are likely to experience learning difficulties in elementary school. This study evaluated the reliability and validity of the measure for this purpose. The BACESS was used in 25 elementary classrooms in Wisconsin, and the entire sample included 285 students. The phases of the BACESS were each found to be highly reliable for their respective numbers of items. Internal structure evidence indicated that the phases functioned well together. The BACESS was found to share good concurrent validity with achievement test proficiency, approaching .70 on Bayesian conditional probability analysis. Receiver Operating Characteristic (ROC) analysis supported the use of the BACESS, incorporating different cutoff rules in different academic environments. Feedback via an evaluation survey indicated teacher opinion that the information gained from using the BACESS was valuable.

Keywords

screening validity sensitivity specificity teacher ratings academic competence

Get full access to this article

View all access options for this article.

References

Albers, C.A. , & Kettler, R.J. (2005). Universal screening: Teacher ratings to identify academic difficulties. Paper presented at the annual meeting of the National Association of School Psychologists, Atlanta, GA.

Bennett, K.J. , Lipman, E.L. , Brown, S. , Racine, Y. , Boyle, M.H. , & Offord, D.R. (1999). Predicting conduct problems: Can high-risk children be identified in kindergarten and grade 1? Journal of Consulting and Clinical Psychology, 67, 470-480.

Carran, D.T. , & Scott, K.G. (1992). Risk assessment in preschool children: Research implications for the early detection of educational handicaps. Topics in Early Childhood Special Education, 12, 196-211.

CTB/McGraw-Hill. (1997). TerraNova multiple assessments battery. Monterey, CA: Author.

DiPerna, J.C. , & Elliott, S.N. (2000). Academic competence evaluation scales manual K-12 . San Antonio, TX: The Psychological Corporation.

Donovan, M.S. , & Cross, C.T. (2002). Minority students in special and gifted education . Washington, DC: National Academy Press.

Elliott, S.N. , & DiPerna, J.C. (2003). Brief Academic Competence Evaluation Screening System. Unpublished instrument, University of Wisconsin-Madison .

Flynn, J.M. , & Rahbar, M.H. (1998). Improving teacher prediction of children at risk for reading failure. Psychology in the Schools, 35, 163-172.

Glover, T. , & Albers, C.A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, 117-135.

10.

Gredler, G.R. (1997). Issues in early childhood screening and assessment . Psychology in the Schools, 34, 99-106.

11.

Gresham, F.M. , MacMillan, D.L. , & Bocian, K.M. (1997). Teachers as "tests": Differential validity of teacher judgments in identifying students at risk for learning disabilities . School Psychology Review, 26, 47-60.

12.

Gresham, F.M. , Reschly, D.J. , & Carey, M.P. (1987). Teachers as "tests": Classification accuracy and concurrent validation on the identification of learning disabled children . School Psychology Review, 16, 543-553.

13.

Haynes, S.N. , & Lench, H.C. (2003). Incremental validity of new clinical assessment measures. Psychological Assessment, 15, 456-466.

14.

Hernandez-Orallo, J. (2004). Classifier evaluation in data mining: ROC analysis . Retrieved November 27, 2006, from http://www.dsic.upv.es/~jorallo/Albacete/ROCAnalysis-1.6-English.pdf

15.

Huai, N. , & Elliott, S.N. (2005). Validation of the Brief Academic Competence Evaluation Screening System (BACESS): A multi-phase process for identifying children at risk of poor academic performance. Manuscript submitted for publication.

16.

Scholastic. (2005). Scholastic reading inventory. Retrieved June 10, 2005, from http://teacher.scholastic.com/products/sri/index.htm

17.

U.S. Department of Education. (2001). No Child Left Behind. Retrieved August 21, 2001, from http://www.ed.gov/inits/nclb/titlepage.html

18.

U.S. Department of Education. (2004). Individuals with Disabilities Education Improvement Act of 2004. Retrieved December 6, 2006, from http://www.ed.gov/policy/speced/guid/idea/idea2004.html#law

19.

VanDerHeyden, A.M. , Witt, J.C. , & Naquin, G. (2003). Development and validation of a process for screening referrals to special education. School Psychology Review, 32, 204-227.

20.

Walker, H.M. , Severson, H. , Stiller, B. , Williams, G. , Haring, N. , Shinn, M. , et al. (1988). Systematic screening of pupils in the elementary age range at risk for behavioral disorders: Development and trial testing of a multiple gating model. Remedial and Special Education , 9(3), 8-14.

21.

Wisconsin Department of Public Instruction. ( 2002). Wisconsin knowledge and concepts examination-Criterion referenced test. Retrieved April 23, 2006, from http://www.dpi.state.wi.us/dpi/oea

22.

Wisconsin Department of Public Instruction. ( 2004). Wisconsin makes the connection: Teaching & testing reading comprehension. Madison: Author.