Assessment for Effective Intervention

This study examined the predictive validity and classification accuracy of two commonly used universal screening measures relative to a statewide achievement test. Results indicated that second-grade performance on oral reading fluency and the Measures of Academic Progress (MAP), together with special education status, explained 68% of the variance in third-grade standardized test performance (N = 399). Six methods of dichotomous risk classification were also explored to predict whether students would pass or fail the state test. A random sample of half of the participants was used to establish locally appropriate risk cut-scores, holding sensitivity constant at .95; the cut-scores were then cross-validated on the remaining participants. Cross-validation reflected sensitivity of above .90 for all methods, with MAP Language Usage and two multi-screener methods demonstrating the strongest overall technical adequacy for risk classification. The limitations and implications of these findings are discussed.

The amount of research evaluating the technical merits of general outcome measures of science and social studies achievement is growing. This study targeted criterion validity for critical content monitoring. Questions addressed the concurrent criterion validity of alternate presentation formats of critical content monitoring and the measure’s predictive validity. Participants were fifth-grade students (N = 51) who completed five different forms of critical content monitoring probes as well as oral reading fluency and maze probes over three benchmarking periods. Criterion measures were the science and social studies subtests of the online abbreviated Stanford Achievement Test–10th edition. Concurrent correlation magnitudes for critical content monitoring ranged from .47 to .60. Predictive correlations for fall and winter ranged from .23 to .64. In three of four cases, commonality analyses findings favored critical content monitoring over oral reading fluency and maze as benchmarking choices. Study limitations and benchmark assessment framework implications are discussed.

The Child Behavior Checklist for children 6 to 18 (CBCL/6-18) is a widely used, standardized parent rating scale. However, few studies have tested the psychometric properties of this instrument in the Arab world despite the great need for such instruments to support the identification and education of children with emotional, behavioral, and social problems in these countries. In this study, we examined the configural invariance and reliability of the core items of the CBCL present in both of its editions using the Arabic translation of the CBCL/6-18 and a sample of 1,360 elementary school children in Qatar. Results from the confirmatory factor analysis (CFA) supported the factor structure of Achenbach’s eight-syndrome scales of the CBCL/6-18 for our sample in (a) its overall fit, (b) pattern of factor loadings, (c) syndrome correlations, and (d) comparability of results in other societies. Internal consistency was high. Categorization of the sample as normal, borderline, and clinical using standard scoring produces proportions roughly in line with expectations, but points to the need for local norms and validation of the instrument prior to its clinical use in Qatar.

This study investigated the classification accuracy of the Dynamic Indicators of Vocabulary Skills (DIVS) as a preschool vocabulary screening measure. With a sample of 240 preschoolers, fall and winter DIVS scores were used to predict year-end vocabulary risk using the 25th percentile on the Peabody Picture Vocabulary Test–Third Edition (PPVT-III) to denote risk status. Results indicated that DIVS Picture Naming Fluency (PNF) and Reverse Definition Fluency (RDF) demonstrated very good accuracy in classifying students according to year-end vocabulary risk status. The DIVS measures also demonstrated stronger accuracy than demographic characteristics known to be indicators of vocabulary difficulties (socioeconomic status, English learner [EL] status, and sex). Combining PNF and RDF did not result in sufficient improvement in accuracy to justify administering both measures as opposed to just one. Further examination of predictive probability values revealed the potential for DIVS measures to improve the precision of vocabulary risk identification over considering EL status alone. Overall, results supported the use of the DIVS as a brief and inexpensive tool for preschool vocabulary screening.

Numerous research groups have consistently called for increased rigor within the evaluation of social programming to better understand pivotal factors to treatment outcomes. The underwhelming data on the essential features of social competence programs for students with behavior challenges may, in part, be attributed to the manner by which available measures are accessed within applied research. The growing literature highlighting the bidirectionality of executive functioning (EF) and social competence supports a comprehensive evaluation of EF within intervention outcomes. Existing data from a suite of four informant and performance measures frequented within EF literature were evaluated as a case example of the contribution of EF in relation to the impact of the Social Competence Intervention for Adolescents (SCI-A) curriculum on social competence. Analyses explored a more “child-world” approach to EF assessment within a context of general social competence gains. Key findings highlight students’ improvements in broad, informant-based measures of executive function and in performance-based tasks designed to assess specific EF processes. Furthermore, these measures provide unique sources of information. Implications for the use of multisource executive function assessments within social competence interventions are discussed.