Initial Evidence for the Reliability and Validity of the Student Risk Screening Scale for Internalizing and Externalizing Behaviors at the Elementary Level
Restricted accessResearch articleFirst published online February, 2012
Initial Evidence for the Reliability and Validity of the Student Risk Screening Scale for Internalizing and Externalizing Behaviors at the Elementary Level
We report findings of an exploratory validation study of a revised instrument: the Student Risk Screening Scale-Internalizing and Externalizing (SRSS-IE). The SRSS-IE was modified to include seven additional items reflecting characteristics of internalizing behaviors, with proposed items generated from the current literature base, review of current measures, and teaching experience with students with emotional and behavioral disorders. The original seven items developed by Drummond (1994) were retained in the exact form, yielding an instrument containing 14 items (SRSS-IE14), each rated on a 4-point Likert-type scale. First, we explore the reliability of the SRSS-IE14 by examining item-level data, internal consistency, and factor structure with 2,460 elementary students. Results of a data analytic plan grounded in classical test theory support retention of five additional items, yielding the SRSS-IE12. Second, we established convergent validity of the SRSS-IE12 with two well-established screening tools: the Strengths and Difficulties Questionnaire (Goodman, 1997) and the Systematic Screening for Behavior Disorders (Walker & Severson, 1992). Limitations and future directions are offered.
Get full access to this article
View all access options for this article.
References
1.
AchenbachT. M. (1991). Integrative guide for the 1991 CBCU4-18, YRS, & TRF profiles.Burlington, VT: University of Vermont, Department of Psychiatry.
2.
AlfonsoM., & DedrickR. F. (2010). Self-injury among early adolescents.American Journal of Health Education, 41, 74–84.
3.
American Educational Research Association, American Psychological Association, & National Council for Measurement in Education. (1999). Standards for educational and psychological testing.Washington, DC: American Educational Research Association,
BeckerA., WoernerW., HasselhornM., Ba-naschewskiT., & RothenbergerA. (2004). Validation of the parent and teacher SDQ in a clinical sample.European Child and Adolescent Psychiatry, 13, 1111–1116. doi:10.1007/s00787-004-2003-5
6.
BensonJ. (1998). Developing a strong program of construct validation: A test anxiety example.Educational Measurement: Issues and Practice, 1, 10–17.
7.
BensonJ., & ClarkF. (1982). A guide for instrument development and validation.American Journal of Occupational Therapy, 36, 789–800.
8.
BradshawC. P., BuckleyJ., & IalongoN. (2008). School-based service utilization among urban children with early-onset educational and mental health problems: The squeaky wheel phenomenon.School Psychology Quarterly, 23, 169–186.
9.
BuneviciusA., PeceliunieneJ., MickuvieneN., ValiusL., & BuneviciusR. (2007). Screening for depression and anxiety disorders in primary care patients.Depression and Anxiety, 24, 455–460.
10.
CapraraG. V., BarbaranelliC., PastorelliC., BanduraA., & ZimbardoP. C. (2000). Prosocial foundations of children's academic achievement.Psychological Science, 11, 302–306.
11.
CattellR. B. (1966). The scree plot for the number of factors.Multivariate Behavioral Research, 1, 245–276.
12.
CostelloE. J., ErkanliA., & AngoldA. (2006). Is there an epidemic of child or adolescent depression?Journal of Child Psychology and Psychiatry, 47, 1263–1271.
13.
CostelloE. J., MustilloS., ErkanliA., KeelerC., & AngoldA. (2003). Prevalence and development of psychiatric disorders in childhood and adolescence.Archives of General Psychiatry, 60, 837–844.
14.
CrickN., CrotpeterJ., & BigbeeM. (2002). Relationally and physically aggressive children's intent attributions and feelings of distress for relational and instrumental peer provocations.Child Development, 73, 1134–1142.
15.
CronbachL. J., & ShavelsonR. J. (2004). My current thoughts on coefficient alpha and successor procedures.Educational and Psychological Measurement, 64, 391–418.
16.
DiStefanoC. A., & KamphausR. W. (2007). Development and validation of a behavioral screener for preschool-age children.Journal of Emotional and Behavioral Disorders, 15, 93–102.
17.
DrummondT. (1994). The Student Risk Screening Scale (SRSS).Grants Pass, OR: Josephine County Mental Health Program.
18.
DrummondT., EddyJ. M., ReidJ. B., & BankL. (1994, November). The Student Risk Screening Scale: A brief teacher screening instrument for conduct disorder. Paper presented at the Fourth Annual Prevention Conference, Washington, DC.
19.
Educational Data Partnership. (2011). Fiscal, demographic, and performance data on California K-12 school. Retrieved from http://www.ed-data.k12.ca.us
20.
EisenbergN., ValienteC., SpinradT. L., Cumbers-landA., LiewJ., & ReiserM. (2009). Longitudinal relations of children's effortful control, impulsivity, and negative emotionality to their externalizing, internalizing and co-occurring behavior problems.Developmental Psychology, 45, 988–1008.
21.
ElliottS. N., & GreshamF. M. (2007). Social Skills Improvement System: Performance screening guides.Bloomington, MN: Pearson Assessments.
22.
FornessS. R., FreemanS. F. N., PaparellaT., KauffmanJ. M., & WalkerH. M. (in press). Special education implications of point and cumulative prevalence for children with emotional or behavioral disorders.Journal of Emotional and Behavioral Disorders. doi:10.1177/1063426611401624
23.
FuchsD., & FuchsL. (2006). Introduction to response to intervention: What, why, and how valid is it?Reading Research Quarterly, 41, 93–99.
24.
GoodR. H., & KaminskiR. A. (Eds.). (2002). Dynamic indicators of basic early literacy skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement.
25.
GoodmanR. (1997). The Strengths and Difficulties Questionnaire: A research note.Journal of Child Psychology and Psychiatry, 38, 581–586.
26.
GoodmanR. (2001). Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ).Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345.
27.
GreshamF. M. (2002). Responsiveness to intervention: An alternative approach to learning disabilities. In BradleyR., DanielsonL., & HallahanD. (Eds.), Identification of learning disabilities: Research to practice.Mahwah, NJ: Erlbaum.
28.
GreshamF. M., & ElliottS. N. (1990). Social Skills Rating System (SSRS).Circle Pines, MN: American Guidance Service.
29.
GreshamF. M., & ElliottS. N. (2007). Social Skills Improvement System: Performance screening guide.Bloomington, MN: Pearson Assessments.
30.
HanleyJ. A., & McNeilB. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology, 143, 29–36.
31.
HatcherL. (1994). A step-by-step approach to using SAS system for factor analysis and structural equation modeling.Cary, NC: The SAS Institute.
32.
HenggelerS. (1998). Multisystemic therapy. In ElliottD. (Ed.), Blueprints for violence prevention.Boulder, CO: Center for the Study and Prevention of Violence.
KalbergJ. R., LaneK. L., & MenziesH. M. (2010). Using systematic screening procedures to identify students who are nonresponsive to primary prevention efforts: Integrating academic and behavioral measures.Education and Treatment of Children, 33, 561–584.
35.
KamphausR. W., & ReynoldsC. R. (2007). BASC-2 behavior and emotional screening system (BASC-2 BESS).San Antonio, TX: Pearson.
36.
KamphausR. W., ThorpeJ. S., WinsorA. P., KronckeA. P., DowdyE. T., & VanDeventerM. C. (2007). Development and predictive validity of a teacher screener for child behavioral and emotional problems at school.Educational & Psychological Measurement, 67, 342–356.
37.
KauffmanJ. M. (2001). Characteristics of emotional and behavioral disorders of children and youth (7th ed.). Columbus, OH: Merrill.
38.
KauffmanJ. M. (2007). Conceptual models and the future of special education.Education and Treatment of Children, 30, 241–258.
39.
KauffmanJ. M., & BrighamF. J. (2009). Working with troubled children.Verona, WI: Full Court.
40.
KazdinA. E. (1993). Treatment of conduct disorders: Progress and directions in psychotherapy research.Development and Psychopathology, 5, 277–310.
41.
KernL., & ManzP. (2004). A look at current validity issues of school-wide behavior support.Behavioral Disorders, 30, 47–59.
42.
KesslerR. C., BerglundP. A., DernierO., JinR., & WaltersE. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication (NCS-R).Archives of General Psychiatry, 62, 593–602.
43.
KingS. M., laconoW. C., & McGueM. (2004). Childhood externalizing and internalizing psychopathology in the prediction of early substance use.Addiction, 99, 1548–1559.
44.
KovacsM., & DevlinB. (1998). Internalizing disorders in childhood.Journal of Child Psychology and Psychiatry, 39, 47–63.
45.
KraemerH. C. (1992). Evaluating medical tests: Objective and quantitative guidelines.Newbury Park, CA: Sage Publications.
46.
LaneK. L., KalbergJ. R., LambertW., CmoboriM., & BruhnA. (2010). A comparison of systematic screening tools for emotional and behavioral disorders: A replication.Journal of Emotional and Behavioral Disorders, 18, 100–112.
47.
LaneK. L., KalbergJ. R., & MenziesH. M. (2009). Developing schoolwide programs to prevent and manage problem behaviors: A step-by-step approach.New York: Guilford.
48.
LaneK. L., KalbergJ. R., BruhnA. L., MahoneyM. E., & DriscollS. A. (2008). Primary prevention programs at the elementary level: Issues of treatment integrity, systematic screening, and reinforcement.Education and Treatment of Children, 31, 465–494.
49.
LaneK. L., LittleM. A., CaseyA. M., LambertW., WehbyJ. H., WeisenbachJ. L., & PhillipsA. (2009). A comparison of systematic screening tools for emotional and behavioral disorders: How do they compare?Journal of Emotional and Behavioral Disorders, 17, 93–105.
50.
LaneK. L., MenziesH. M., OakesW. P., & KalbergJ. R. (2012). Systematic screenings of behavior to support instruction: From preschool to high school.New York: Guilford.
51.
LaneK. L., OakesW. P., EnnisR. P., CoxM. L., SchatschneiderC., & LambertW. (in press). Additional evidence for the reliability and validity of the Student Risk Screening Scale at the high school level: A replication and extension.Journal of Emotional Behavioral Disorders.
52.
LaneK. L., OakesW. P., & MenziesH. M. (2010). Systematic screenings to prevent the development of learning and behavior problems: Considerations for practitioners, researchers, and policy makers.Journal of Disabilities Policy Studies, 21, 160–172.
53.
LaneK. L., ParksR. J., KalbergJ. R., & CarterE. W. (2007). Systematic screening at the middle school level: Score reliability and validity of the students risk screening scale.Journal of Emotional and Behavioral Disorders, 15, 209–222.
54.
LaneK. L., & WehbyJ. H. (2002). Addressing antisocial behavior in schools: A call for action.Academic Exchange Quarterly, 6, 4–9.
55.
LewisT. J., & SugaiG. (1999). Effective behavior support: A systems approach to proactive schoolwide management.Focus on Exceptional Children, 31, 1–24.
56.
MaleckiC. K., & ElliottS. N. (2002). Children's social behaviors as predictors of academic achievement: A longitudinal analysis.School Psychology Quarterly, 17, 1–23.
57.
MastenA. S., RoismanG. I., LongJ. D., BurtK. B., ObradovicJ., RileyJ. R., … TellegenA. (2005). Developmental cascades: Linking academic achievement and externalizing and internalizing symptoms over 20 years.Developmental Psychology, 41, 733–746.
58.
MattisonR. E., HooperS. R., & GlassbergL. A. (2002). Three-year course of learning disorders in special education students classified as behavioral disorder.Journal of the American Academy of Child & Adolescent Psychiatry, 41, 1454–1461.
59.
McEvoyA., & WelkerR. (2000). Antisocial behavior, academic failure, and school climate: A critical review.Journal of Emotional and Behavioral Disorders, 8, 130–140.
60.
MenziesH. M., & LaneK. L. (in press). Validity of the student risk screening scale: Evidence of predictive validity in a diverse, suburban elementary setting.Journal of Emotional and Behavioral Disorders.
61.
MeyerE. P., KaiserH. F., CernyB. A., & GreenB. G. (1977). MSA for a special Spearman matrix.Psychometrika, 42, 153–156.
62.
MorrisR.J., ShahK., & MorrisY. P. (2002). Internalizing behavior disorders. In LaneK. L., GreshamF. M., & O'ShaughnessyT. E. (Eds.), Interventions for children with or at risk for emotional and behavioral disorders (pp. 223–241). Boston, MA: Allyn & Bacon.
63.
NunnallyJ. C., & BernsteinI. H. (1994). Psychometric theory (3rded.). St. Louis, MO: McGraw-Hill.
64.
OakesW. P., WilderK., LaneK. L., PowersL., YokoyamaL., O'HareM. E., & JenkinsA. B. (2010). Psychometric properties of the Student Risk Screening Scale: An effective tool for use in diverse urban elementary schools.Assessment for Effective Intervention, 35, 231–239.
65.
Pearson Education. (2008). AIMSweb® Benchmark and progress monitoring system for grades K-8.Bloomington, MN: Pearson Education.
66.
PetrasH., ChilcoatH., LeafP. J., lalongoN. S., & KellamS. G. (2004). Utility of TOCA-R scores during the elementary school years in identifying later violence among adolescent males.Journal of the American Academy of Child and Adolescent Psychiatry, 43, 88–96.
67.
RapportM. D., DennyC. B., ChungK., & HustaceK. (2001). Internalizing behavior problems and scholastic achievement in children: Cognitive mediators and pathways as mediators of outcomes.Journal of Clinical Child Psychology, 30, 536–551.
68.
ReidR., GonzalezJ., NordnessP., TroutA., & EpsteinM. (2004). A meta-analysis of the academic status with emotional/behavior disturbances.Journal of Special Education, 38, 130–143.
69.
SipersteinG. N., WileyA. L., & FornessS. R. (2011). School context and the academic and behavioral progress of students with emotional disturbance.Behavioral Disorders, 36, 172–184.
70.
SugaiG., & HornerR. H. (2002). The evolution of discipline practices: School-wide positive behavior supports.Child & Family Behavior Therapy, 24, 25–50.
71.
TodisB., SeversonH. H., & WalkerH. M. (1990). The critical events scale: Behavioral profiles of students with externalizing and internalizing behavior disorders.Behavioral Disorders, 15, 75–86.
72.
U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics.Common Core of Data: Public school data 2008–2009.Washington, DC: Author.
73.
WagnerM., NewmanL., CametoR., LevineP., & GarzaN. (2006). An overview of findings from wave 2 of the National Longitudinal Transition Study-2 (NLTS2).Menlo Park, CA: SRI International.
74.
WalkerH. M. (2003, February 20). Comments on accepting the Outstanding Leadership Award from the Midwest Symposium for Leadership in Behavior Disorders.Kansas City, KS: Author.
75.
WalkerH. M., RamseyE., & GreshamF. M. (2004). Antisocial behavior in school: Evidence-based practices.Belmont, CA: Wadsworth.
76.
WalkerH. M., & SeversonH. (1992). Systematic Screening for Behavior Disorders: Technical manual.Longmont, CO: Sopris West.
77.
WalkerH. M., SeversonH., NicholsonF., KehleT., JensonW. R., & ClarkE. (1994). Replication of the Systematic Screening for Behavior Disorders (SSBD) procedure for the identification of at-risk children.Journal of Emotional and Behavioral Disorders, 2, 66–77.
78.
WalkerL. S., BeckJ. E., GarberJ., & LambertW. (2009). Children's Somatization Inventory: Psychometric properties of the revised form (CSI-24).Journal of Pediatric Psychology, 34, 430–440.
