Abstract
Test Description
The Social Skills Improvement System Social-Emotional Learning Edition (SSIS SEL; Gresham & Elliott, 2017) is a multicomponent rating scale that includes a criterion and norm-referenced measure of social-emotional and academic functioning—based on a reformulation of the Social Skills Improvement System (SSIS; Gresham & Elliott, 2008). Both measures can be used to assess the five social-emotional core competencies outlined by the Collaborative for Academic, Social, and Emotional Learning (CASEL, 2018). The SSIS SEL also includes a corresponding Classwide Intervention Program (CIP). The criterion-referenced measure is meant to be used as a screening and progress monitoring tool. Practitioners could utilize this scale to universally screen for social-emotional difficulties and progress monitor throughout the year. It includes eight competency domains, the first five of which form a social-emotional composite and the final three of which form an academic functioning composite: self-awareness, self-management, social awareness, relationship skills, responsible decision making, motivation to learn, reading skills, and mathematics skills. The norm-referenced measure provides a more comprehensive assessment and includes three forms: parent, teacher, and student. For students identified as at-risk based on the screener, practitioners can use these scales as part of a multimethod, multisetting, multirater psychoeducational assessment process. This measure includes the same social-emotional competency domains as those on the screening/progress monitoring scale, which can be used to form the SEL composite. Certain items from each of these domains also form a core skills scale, which can be used as a brief norm-referenced measure aligned with CIP. The teacher form also includes items related to motivation and performance in math and reading that form an Academic Competence Scale. Parent and student forms are available in English and Spanish.
The screening/progress monitoring scale and parent, teacher, and student forms are intended to be used with students in preschool to 12th grade (3:0-18:11), though the student form is only meant for students (8:0-18:11). The screening/progress monitoring scale is expected to take 30 to 45 min to complete for an entire class. The duration to complete the parent/teacher/student form is 10 to 20 min. Comparisons can be made to a gender-specific or combined norm group. Standard scores on this measure have a mean of 100 and standard deviation of 15. Higher scores indicate higher levels of functioning. Both measures can be completed in paper–pencil format or online. Scoring of the norm-referenced measure can only be done online.
The SSIS SEL includes the assessment manual, intervention program manual, screening/progress monitoring forms, and parent and/or teacher and/or student forms. The screening/progress monitoring form contains definitions for each of the competencies assessed along with examples of behaviors and skills indicative of performance level. Each competency is rated on a 5-point scale, where a 3, 4, or 5 suggests adequate or above average skill. A 2 indicates the student’s skill is developing, and a 1 suggests limited skill. There are 58 items on the teacher form, 51 (parent), and 46 (student). For the teacher and parent forms, items are rated on a 4-point scale to indicate frequency of a particular skill: never, seldom, often, and almost always. Students also rate items on a 4-point scale, but each response indicates degree of trueness rather than frequency of exhibition: not true, a little true, a lot true, and very true (Gresham et al., 2017).
Technical Adequacy
Development: Test Construction, Item Analysis, and Standardization Sample
Development of the screening/progress monitoring scale began with the selection of the five domains from CASEL (2018) as well as academic areas from the SSIS Performance Screening Guides (PSG; Elliott & Gresham, 2007). Definitions for each area were developed and behavioral descriptions for each rating determined through extensive review. In a study conducted by Elliott, Davies, Frey, Gresham, and Cooper (2018), teachers were asked to place the 25 descriptors onto the five competency scales and order them from lowest to highest. There was 100% agreement. Development of the parent, teacher, and student forms was based on a reconfiguration of items from the SSIS (Gresham & Elliott, 2008) teacher, parent, and student forms to match the five SEL competencies. Confirmatory factor analysis was used to confirm the placement of items on each SEL competency scale. One item was dropped during the development of the SSIS based on differential item functioning analysis.
Data for the SSIS SEL parent, teacher, and student forms standardization sample were collected from September 2006 to October 2007 as part of the SSIS Rating Scales standardization project. The sample included 4,700 individuals aged 3 to 18 years from 36 states. There were less than 100 individuals at each age level for the teacher form, and several age levels had fewer than 100 individuals for the parent form (5, 13-18) and student form (8-9, 13-18). Comparisons of the norm group to national demographics were based on population survey data collected by the U.S. Census Bureau (2006). Data for the standardization sample were similar in terms of race, mother’s education level, and geographic location—though the West was somewhat underrepresented for the 3- to 5-year age group on the teacher form (13.5% vs. 23.7%). Data were also presented for students with disabilities. Although the normative sample is overall representative for the aforementioned demographic categories, it is unfortunate the authors did not conduct another standardization study. These data are more than 10 years old and, thus, are arguably already approaching the maximum useful life for a norm-referenced test (Salvia, Ysseldyke, & Witmer, 2017).
Reliability
For this review, a criterion of .80 was considered the minimum coefficient to be considered satisfactory for the screening/progress monitoring scale and .90 the minimum for the parent, teacher, and student form (Salvia et al., 2017). To examine the internal consistency of the screening/progress monitoring scale, a study was conducted using Australian students in grades prep through Grade 3 (N = 268; Elliott et al., 2018). The coefficient alphas for the social-emotional and academic functioning composite were both .90 and above. Test–retest data were collected from 12 teachers and 266 students—aged 4 to 8 years. Duration between each data collection was 1 month. Both composites and all but three of the scales demonstrated evidence of stability over time (r ≥ .80). Given that data were not provided for individuals beyond age of 8 years, caution should be exercised when using this scale for older individuals.
Reliability data for the parent and teacher form are presented by age group: 3 to 5 years, 5 to 12 years, and 13 to 18 years; 8 to 12 years and 13 to 18 years (student form). It should be noted that 5-year-olds are included in two groups: 3 to 5 years (preschool) and 5 to 12 years (school age). As children of this age are in both grade levels, the authors included 5-year-olds in both groups to control for the effect of grade level context on social behavior. The coefficient alphas for the parent, teacher, and student form SEL composite were all above .90 for each age band and norm group. Although there is evidence of excellent internal consistency for all three forms at the composite level, some competency areas were below .90 for certain norm groups and age levels across forms. For test–retest reliability, the interval between tests ranged from 2 to 87 days. None of the corrected correlation coefficients for the SEL composites were above .90 for any form. Interrater reliability results showed all corrected correlation coefficients for the teacher and parent forms were moderate to very large (r = .38 to .71).
Validity
The content for the SSIS SEL was based on a previous assessment of social skills and problem behaviors using items from the SSIS. It is therefore conceivable the scale content on the SSIS SEL is representative of the construct as outlined by CASEL (2018), demonstrating evidence of content validity. However, it would have been beneficial for the authors to include data from the confirmatory factor analysis to allow practitioners to assess the model fit.
Using the SSIS PSG (Elliott & Gresham, 2007) subscales as an indicator of socially and academically at-risk functioning, receiver operating characteristic/area under the curve analysis conducted by Elliott et al. (2018) on the screening/progress monitoring scale showed fair to excellent diagnostic accuracy for socially and academically at-risk students for all scales and composites (.70 to .92). There was evidence of adequate specificity and negative predictive power for both types of risk. Positive predictive power was adequate for academically at-risk students. However, sensitivity indices were below .80 for socially and academically at-risk students, and positive predictive power was below .80 for socially at-risk students. Although there is some evidence of predictive ability at the composite level, this measure has lower sensitivity and positive predictive indices than would be considered adequate for a screening measure (Carran & Scott, 1992).
Correlation analysis showed large to very large correlations between the social-emotional subscales and the SSIS PSG (Elliott & Gresham, 2007) prosocial subscale (r = .63 to .76). The correlations between the corresponding academic subscales for math and reading on both measures were each very large (r = .85). Although there is evidence of convergent validity based on these results, additional comparisons with other theoretically related measures would have been beneficial.
Evidence of convergent validity for the norm-referenced measure is provided in the manual. Intercorrelations for the teacher, parent, and student form were moderate to nearly perfect (r = .30 to .92). The correlations between scales on the parent and teacher form ranged from small to moderate (r = .23 to .38). Correlations across teacher and student forms were small to moderate (r = .14 to .30). On the parent and student forms there were small correlations between the scales (r = .17 to .29). The authors also compared scaled scores from the SSIS SEL parent, teacher, and student forms to scores from other behavioral and emotional assessments: SSIS parent, teacher, and student forms (Gresham & Elliott, 2008); Behavior Assessment System for Children–2nd edition (BASC-2; Reynold & Kamphaus, 2004); and Vineland Adaptive Behavior Scales–2nd edition (Vineland II; Sparrow, Cicchetti, & Balla, 2005). The results supported evidence of convergent validity. The authors also describe a special group study in the manual, which showed predicted group differences. Overall, the aforementioned results demonstrate evidence of construct validity.
Commentary
There is evidence to suggest kindergarten students’ social-emotional skills are predictive of future outcomes, both positive and negative (Jones, Greenberg, & Crowley, 2015). Thus, it may be beneficial to assess social-emotional skills and intervene when necessary early in a child’s school career. This positive relationship between social-emotional competencies and positive outcomes, along with an inverse relation to negative outcomes, has also been shown in meta-analytic studies for students K-12 (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). Therefore, the addition of an assessment that was constructed to align with the five social-emotional competencies outlined by CASEL (2018) is arguably of benefit to schools and school professionals looking to screen, progress monitor, and/or comprehensively assess for these skills. Its alignment allows schools to integrate its use with CASEL aligned intervention programs already in use. For those schools not currently using a program, the SSIS SEL provides an aligned intervention program in the form of CIP.
However, there are some issues educational professionals should be cognizant of. The first is there is limited reliability evidence for the screening/progress monitoring scale. The study conducted by Elliott et al. (2018) utilized a small sample size, with a non-U.S. population, and a severely restricted age band compared with the range the measure is purported to be useful for. Second, there is a need for additional evidence of construct validity for the screening/progress monitoring scale. Third, although the normative data are still useful for the parent, teacher, and student form, the lack of recent standardization data limits the useful life span of this measure for normative use. Finally, although there was evidence of excellent internal consistency for the composite across forms and age groups, there was a lack of stability for the parent, teacher, and student forms—even at the composite level. Overall, although the work of the authors to create an assessment aligned with current theory regarding key social-emotional competencies is applauded for its timeliness and importance for student outcomes, the current and longitudinal usefulness of the assessment is arguably hindered due to dated norms and evidence of less than adequate psychometric properties. It would be beneficial for practitioners if more up-to-date normative data and additional reliability data were presented for both the screening/progress monitoring scale and parent, teacher, and student forms. These additions would provide more support for a recommendation of the use of the SSIS SEL for assessing and monitoring social-emotional competence in K-12 students.