Conceptual Replications of the Critical Content Monitoring General Outcome Measure in Science Content

Participants

Participants included 547 Grades 4 to 6 students from seven schools in a rural south Louisiana district of about 4,700 students, approximately 9% of which were students with disabilities. District fourth (n = 158), fifth (n = 247), and sixth graders (n = 142) who completed all benchmark CCM probes and had state achievement scores comprised the sample. The demographic breakdown was 50% female, 68% non-White, 86% lower socioeconomic status (SES), and 7% students with disabilities. Less than 1% of the entire district were English language learner. The mean age was 11.9 (SD = 1.8). At the request of the district, a subset of the sample (N = 84) in two of the district’s low-achieving schools also completed science and reading comprehension subtests of the online abbreviated Stanford Achievement Tests–Tenth Edition (SAT-10; Pearson Education, n.d.). The number was limited to the funds available to finance the cost of test administration. The makeup of the subset, whose average age was 11.6 years (SD = 1.9), was 56% female, 98% African American, 96% low SES, and 4% students with disabilities.

Measures

Three benchmark CCM probes, two criterion measures, and a teacher social validity measure were administered. Both criterion measures were compared to CCM across multiple administrations. The criterion measures for the larger sample were the science and ELA tests of the Louisiana Educational Assessment Program (LEAP; Louisiana Department of Education [LDE]; fourth grade) and the integrated LEAP (iLEAP; fifth and sixth grades). The smaller sample was administered the iLEAP/LEAP science and ELA tests and the SAT-10 science and reading comprehension tests.

Procedures

Students took CCM benchmark assessments in October, January, and May. The assessments were delivered via Qualtrics, an online survey software system. Researchers administered the initial October CCM probe with the teachers over the course of a week to facilitate fidelity of implementation. During subsequent administrations, district teachers administered the tests independently during a testing window. The second author conducted informal observations of testing at each of the schools and reported anecdotal evidence that CCM implementation went as planned. However, no formal fidelity checks were conducted. Teachers administered the statewide accountability test in April of the academic year in accordance with guidelines established by the state department of education. Researchers administered the SAT-10 test shortly before state testing.

Data Analysis

Descriptive statistics for all measures were calculated. To examine mean differences among subpopulations on the criterion measures and CCM probes, each measure was examined separately as a dependent variable with demographic variables as independent variables. Using separate multiple regressions, comparisons were examined for the dichotomous demographic categories gender, education classification (students with disabilities vs. students without disabilities), and SES status (high vs. low). Race was treated as a dichotomous variable limited to White and African American students, as only 16 students were in a different racial category. This group of 16 was excluded from the analysis regarding race but were included in the other analyses. For the correlational analysis, bivariate correlation coefficients were calculated to determine the criterion and predictive validity for CCM with iLEAP/LEAP or SAT.

For the diagnostic accuracy analyses, state test data were provided to the researchers as both a continuous score and an “achievement” string variable. Unsatisfactory and approaching basic were considered failing by the state, whereas basic, mastery, and advanced were deemed passing. This variable was transformed into a dichotomous “Pass/Fail” variable. To determine a cut score for the benchmark CCM scores, a Receiver Operating Characteristic (ROC) curve was calculated for the score that came closest to .80 sensitivity and .20 1-specificity level which was then chosen as the cut score. A new dichotomous variable was computed for each probe, with “1” if a CCM score fell below the cut score and “0” if the score was at or above the cut score.

Interpretation of the area under the curve (AUC) was viewed as an effect size of the diagnostic accuracy of the scale with scores between .50 and .69 considered “low,” .70 - .89 “moderate” and .90–1 “high” (Kilgus, Riley-Tillman, Chafouleas, Christ, & Welsh, 2014). Statistics were calculated for true positive (TP), students who were correctly identified as at risk; false positive (FP), students incorrectly identified as at risk; true negatives (TN), students correctly identified as not at risk; and false negatives (FN), students incorrectly identified as not at risk. Sensitivity (SN) was the proportion of students at risk who were correctly identified (TP / [FN + TP]). Specificity (SP) referred to the proportion of students not at risk who were correctly identified as not at risk (TN / [TN + FP]). Positive predictive power (PPP) was the proportion of students correctly identified as at risk (TP / [TP + FP]). Negative predictive power (NPP) was the proportion of students correctly identified as not at risk (TN / [TN + FN]). The prevalence or base rate referred to the proportion of students who were identified as at risk on the given probe ([TP + FN] / N) and the correct classification (CC) described the proportion of the overall sample who were correctly identified ([TP + TN] / N). Finally, Cohen’s Kappa (K) referred to the proportion of agreement between the test and actual condition of the subjects (at risk vs. not at risk) beyond that accounted for by chance, where K values of ≤ .2 was considered “poor,” .21–.40 “fair,” .41–.60 “moderate,” and ≥.61 “good” (Forstmeier & Maercker, 2007; Kilgus et al., 2014).

For growth, a linear mixed model was conducted to determine if CCM accounted for students’ growth over time and if there were differences in growth rates (Espin et al., 2013). The level one variable was time and the level two variable was individual students. First, an unconditional (null) model was calculated to determine if there was enough variation over the participants to warrant this type of analysis. The null model is the equivalent of a one-way analysis of variance (ANOVA) with no predictors, only a random effect (Shek & Ma, 2011). Once this was established, time was entered into the model as a fixed factor to establish the mean growth rate, and then tested as a random factor to determine if differences in the model could be explained by variability between individuals. Finally, the model was tested to see if there were differences among the grades and the interaction of time and grade.

Descriptive Statistics and Demographic Comparisons

Table 1 provides descriptive test data for the three CCM probes and both the state achievement and SAT-10 tests. Overall mean CCM probe scores increased across benchmark time periods, from 10.8 (SD = 5.1; 95% confidence intervals [10.4, 11.2]) in October to 13.2 (SD = 5.1; [12.8, 13.6]) in January to 15.9 (SD = 5.5; [15.4, 16.4]) in May. Each of the three individual grade levels also showed incremental increases in scores out of 30 questions/points possible from fall to winter to spring. The greater proportion of growth came from winter to spring for fourth and fifth grades and for fall to winter for sixth grade.

OPEN IN VIEWER

Table 1.

Critical Content Monitoring and Criterion Distributions of Means, Standard Deviations, and 95% CI by Grade.

Measure	M	SD	95% CI	Range	Skew	Kurtosis	N
Overall
October	10.8	5.1	[10.4, 11.2]	0–30	.80	0.66	547
January	13.2	5.1	[12.8, 13.6]	1–30	.18	−0.26	547
May	15.9	5.5	[15.4, 16.4]	1–29	−.04	−0.48	547
i/LEAP-S	310.6	45.8	[307, 315]	100–462	−.98	3.5	547
i/LEAP-E	739.9	25.4	[738, 742]	650–822	−.12	0.09	547
SAT-10-S	631.3	29.2	[625, 638]	551–703	−.14	0.01	84
SAT-10-E	621.3	36.7	[613, 629]	545–711	.21	−0.21	84
Fourth grade
October	9.6	4.6	[8.9, 10.4]	2–24	.62	0.21	158
January	10.8	4.9	[9.9, 11.5]	2–24	.53	−0.43	158
May	14.1	5.3	[13.2, 14.9]	3–26	.12	−0.75	158
i/LEAP-S	324.5	45.8	[317, 332]	11–424	−.88	3.0	158
i/LEAP-E	737.9	26.0	[734, 742]	669–803	−.19	−0.18	158
SAT-10-S	616.5	34.2	[603, 630]	551–681	.35	−0.49	29
SAT-10-E	597.9	32.8	[545, 610]	545–674	.61	−0.07	29
Fifth grade
October	12.4	5.4	[13.1, 12.2]	1–27	.38	−0.37	247
January	13.9	4.7	[13.3, 14.5]	0–28	−.03	0.02	247
May	17.6	5.5	[16.9, 18.3	1–30	−.33	−0.19	247
i/LEAP-S	307.7	44.8	[302, 313]	100–462	−.78	2.3	247
i/LEAP-E	742.6	27.8	[739, 746]	650–822	−.10	−0.11	247
SAT-10-S	631.5	21.5	[622, 641]	590–674	−.02	−0.46	23
SAT-10-E	635.0	24.8	[624, 646]	593–689	.26	−0.39	23
Sixth grade
October	9.3	4.2	[8.6, 10.0]	0–30	2.0	7.9	142
January	14.7	5.1	[13.8, 15.5]	4–30	.30	0.01	142
May	15.2	4.9	[14.4, 16.0]	4–29	.16	0.01	142
i/LEAP-S	300.1	43.9	[293, 307]	100–384	−1.8	7.5	142
i/LEAP-E	737.6	19.1	[734,741]	679–781	−.46	0.36	142
SAT-10-S	644.4	22.8	[636, 653]	594–703	.48	0.82	32
SAT-10-E	632.7	37.4	[619, 632]	562–711	.37	−0.19	32

Note. CI = confidence intervals; i/LEAP-S = Louisiana state achievement test—Science; i/LEAP-E = Louisiana state achievement test—English language arts; SAT-10-S = Stanford Achievement Test, Tenth edition—Science; SAT-10-E = Stanford Achievement Test, Tenth edition—English language arts.

Table 2 compares the predictor and criterion tests with respect to subgroup scoring pattern differences. There were more comparable scoring patterns for the state test than the national test. Differences among subgroups for the CCM probe scores matched those of the science state achievement test for each benchmark for gender, disability status, and SES but not race. CCM scoring patterns matched those of the state ELA test in disability and SES but not race or gender. CCM scores across all benchmarks only matched those of SAT-10 science for gender. There were no matches reported between CCM scores and SAT reading comprehension.

OPEN IN VIEWER

Table 2.

Comparison of CCM Benchmark Probes and Criterion Tests With Respect to Differences in Subgroup Mean Scores.

Assessment	B	SEB	β
October CCM
Male—Female	0.48	0.43	0.01
African American—White	−0.49	0.51	−0.05
General education—Special education	−3.14	0.86	−0.15**
High SES—Low SES	−2.26	0.64	−0.15**
January CCM
Male—Female	−0.01	0.43	−0.00
African American—White	−0.42	0.51	−0.04
General education—Special education	−3.96	0.86	−0.19**
High SES—Low SES	−2.90	0.64	−0.19**
May CCM
Male—Female	−0.47	0.46	−0.04
African American—White	−1.03	0.54	−0.09
General education—Special education	−4.92	0.92	−0.22**
High SES—Low SES	−2.53	0.69	−0.16**
i/LEAP (N = 547) SCIENCE
Male—Female	5.40	3.61	0.06
African American—White	−30.0	4.28	−0.29**
General education—Special education	−38.9	7.45	−0.21**
High SES—Low SES	−20.5	5.48	−0.15**
i/LEAP (N = 547) ELA
Male—Female	−4.53	2.01	−0.09**
African American—White	−10.3	2.36	−0.19**
General education—Special education	−25.3	4.07	−0.25**
High SES—Low SES	−15.4	3.04	−0.21**
SAT − 10 (N = 84) SCIENCE
Male—Female	−2.31	6.67	−0.04
General education—Special education	−34.2	18.8	−0.21
High SES—Low SES	−17.3	18.8	−0.11
SAT − 10 (N = 84) ELA
Male—Female	−9.84	8.40	−0.13
General education—Special education	−34.9	23.5	−0.18
High SES—Low SES	−3.60	23.5	−0.02

Note. CCM = critical content monitoring; B = unstandardized coefficient; SE_B = Standard error of the coefficient; β = standardized coefficient; LEAP = Louisiana Educational Assessment Program; SES = socioeconomic status; ELA = English language arts; SAT-10 = Stanford Achievement Tests–Tenth Edition.

*

p < .05. **p < .01.

Concurrent and Predictive Validity

Table 3 displays correlations between the benchmark CCM probes and iLEAP/LEAP and SAT tests for science and ELA. For science, the concurrent correlation between CCM and iLEAP/LEAP was .50, while the concurrent comparison with SAT-10 was .67. Across grades, correlations ranged from .44 in sixth grade to .66 in fourth grade. Predictive overall CCM science correlations ranged from .38 to .42 for the state test and .40 to .58 for the national test. In ELA, the concurrent correlation between CCM and iLEAP/LEAP was .66, while the concurrent comparison with SAT-10 was .75. Across grades, correlations ranged from .55 in sixth grade to .70 in fifth grade. Predictive overall CCM ELA correlations ranged from .51 to .56 for the state test and .41 to .64 for the national test.

OPEN IN VIEWER

Table 3.

Correlations Across CCM Probes and Criterion Measures—Science in Bold in Lower Matrix, ELA in Upper Matrix.

N = 547	LEAP	Oct	Jan	May	N = 84	SAT	Oct	Jan	May
Overall	—	.51	.56	.66	Overall	—	.41	.64	.75
Oct	.42	—	.55	.60	Oct	.40	—	.55	.60
Jan	.38	.55	—	.64	Jan	.58	.55	—	.64
May	.50	.60	.64	—	May	.67	.60	.64	—
Fourth grade	—	.52	.63	.68	Fourth grade	—	.30 a	.47	.68
Oct	.53	—	.73	.68	Oct	.34 a	—	.73	.68
Jan	.56	.73	—	.75	Jan	.57	.73	—	.75
May	.66	.68	.75	—	May	.53	.68	.75	—
Fifth grade	—	.66	.62	.70	Fifth grade	—	.63	.51*	.83
Oct	.59	—	.69	.64	Oct	.57	—	.69	.64
Jan	.51	.68	—	.63	Jan	.57	.73	—	.63
May	.57	.64	.63	—	May	.53	.68	.75	—
Sixth grade	—	−.02 a	.47	.55	Sixth grade	—	.18 a	.56	.71
Oct	.08 a	—	.20*	.28	Oct	.20 a	—	.20*	.28
Jan	.29	.20	—	.53	Jan	.58	.20*	—	.53
May	.44	.28	.53	—	May	.76	.28	.53	—

Note. Science scores in bold in lower left of the matrix, ELA scores in upper right of the matrix. CCM = critical content monitoring; ELA = English language arts; LEAP = Louisiana Educational Assessment Program; SAT = Stanford Achievement Tests; Oct = October CCM, Jan = January CCM, May = May CCM.

a

not significant.

Correlations marked * significant at p < .05; all other correlations significant at p < .01.

Diagnostic Accuracy

Table 4 provides a summary of diagnostic statistics for the benchmark CCM probes with the state achievement test. In science, the May CCM cut score of ≤14 correctly classified 75% of participants as passing or failing the test based on state criterion levels. The cut scores increased for each successive benchmark period along with the correct classification rate. In ELA a similar pattern was indicated. The October ELA cut score was different than that determined for science, but the January and May scores were the same across content areas. Kappa statistics increased as the year progressed with values that ranged from fair to moderate.

OPEN IN VIEWER

Table 4.

Diagnostic Accuracy Statistics for CCM Probes Across Benchmarking Periods for Science and ELA.

CCM Probe	Cut Score	SN	SP	PPP	NPP	CC	BR	K	AUC	95% [CI]
October—Science	9.5	.65	.69	.82	.47	.67	.69	.30	.73	[.69, .78]
October—ELA	8.5	.71	.62	.84	.43	.68	.73	.29	.72	[.67, .77]
January—Science	11.5	.73	.63	.82	.51	.70	.69	.33	.74	[.69, .79]
January—ELA	11.5	.73	.67	.86	.47	.71	.74	.35	.78	[.74, .83]
May—Science	14.5	.73	.63	.82	.51	.75	.69	.34	.78	[74, .82]
May—ELA	14.5	.74	.76	.90	.52	.75	.74	.44	.83	[.79, .86]

Note. CCM = Critical content monitoring; ELA = English language arts; SN = sensitivity; SP = specificity; PPP = positive predictive power; NPP = negative predicting power; CC = correct classification; BR = base rate/prevalence; K = Cohen’s kappa; AUC = area under the curve; 95% [CI] = 95% confidence intervals for AUC.

Growth

The model building process started with an unconditional or “null” model, in which there were no predictor variables and the intercept was allowed to vary randomly. Calculation of the intraclass correlation determined that there was enough variability in individuals to warrant a linear mixed model. Time was entered into the model as a fixed factor to establish the mean growth rate on CCM. Akaike Information Criteria (AIC) was used to evaluate the model at each step. Decreases in magnitude of AIC indicate a better model and guided the model building process (Bell, Ene, Smiley, & Schoeneberger, 2013). The intercept was centered at the fall benchmark assessment, the initial point of data collection, to establish if there were individual differences in students’ CCM scores at the beginning of the study (see Table 5).

OPEN IN VIEWER

Table 5.

Growth as Measured by CCM.

Fixed Effects	Model 1	Model 2	Model 3	Final Model
Intercept	13.3* (.19)	10.7* (0.21)	10.7* (.17)	9.26* (0.39)
Time	—	2.6* (0.10)	2.6* (.15)	2.2* (0.19)
Grade 5	—	—	—	2.8* (0.49)
Grade 6	—	—	—	.83 (07.57)
Grade*5	—	—	—	.37 (0.24)
Grade*6	—	—	—	.76* (0.27)
Random effects
Error variance
Level 1	17.8* (.75)	11.2* (0.48)	11.5* (.61)	11.1* (0.47)
Time	14.3* (1.2)	16.5* (1.2)	—	—
Intercept + Time	—	—	6.2* (.48)	14.8 (1.1)
Model fit
AIC	10,055	9,547	9,726 a	9,497

Note. Intraclass correlation = .45. Entries show parameter estimates and standard errors in parenthesis. Estimation method = maximum likelihood. CCM = critical content monitoring; AIC = Akaike Information Criteria.

a

AIC value increased, indicating worse model.

*

p < .05.

The null model (Model 1) revealed that the intercept was statistically significant with a magnitude of 17.8 at initial time of testing. The intraclass correlation (14.3/[17.5+ 14.3]) indicated that about 45% of the total variation in the CCM scores was due to significant differences between the participants. To determine if CCM was an indicator of average growth for students, the predictor time was added as a fixed effect to the model (Model 2). The result was statistically significant and indicated that for every benchmarking period, students showed an average improvement in CCM scores by 2.6 points. To determine if CCM growth varied across students, time was added as a random effect. Because the AIC increased (Model 3), that indicated that Model 2, with fixed effects for time, was the better model. Finally, to examine if there were differences among the three grades, the main effects for grade and the interaction of time and grade were examined. Model 4 shows the results, with the lowest AIC indicating the better fit. The interaction of time and grade was significant, meaning that there was a difference in sixth graders (specifically) depending on when they were tested. With fourth grade as the reference category, it appears that fifth graders scored 2.8 points higher each marking period compared to the fourth graders, and sixth graders scored .83 higher, depending on the marking period (see Table 6).

OPEN IN VIEWER

Table 6.

Parameter Estimates for Growth Model 4 With Grade and Time as Predictors.

Parameter	B	SE	df	t	p	95% Confidence Interval
Intercept	9.3	0.4	884.1	23.8	.0*	[8.5, 10.0]
Time	2.2	0.2	1094.0	11.8	.0*	[1.8, 2.6]
Fifth grade	2.8	0.5	884.1	5.6	.0*	[1.8, 3.8]
Sixth grade	0.8	0.6	884.1	1.5	.1	[–0.3, 1.9]
Fourth grade	0	0.0
Fifth Grade × Time	0.4	0.2	1094.0	1.6	.1	[–0.1, 0.8]
Sixth Grade × Time	0.7	0.3	1094.0	2.7	.0*	[0.2, 1.3]
Fourth Grade × Time	0	0

Note. Fourth grade reference category.

*

p < .05.

Social Validity

For the 15 responders, the majority regarded CCM as very helpful in communicating student progress to parents while the majority of respondents regarded CCM as helpful in communicating student progress to students and school personnel, establishing goals, measuring progress toward goals, and deciding when to change instruction. Totally, 68% of responders thought the CCM procedure was not very or not at all time consuming, with 25% indicating that it was time consuming.

Limitations

Four limitations likely impact the evaluation of study findings. First, the purposeful sample of predominantly African American and lower SES students, while large, multigrade, and quite likely representative of urban public school populations, was not representative of the larger U.S. population demographically, thereby limiting the potential generalizability of findings. Second, the particular probe administration program we utilized, Qualtrics, did not allow for immediate teacher access to useable data as the assessments were put online by the researchers, which may have affected subsequent CCM scores over the course of the study. Third, our analysis did not control for students’ reading abilities, which did not allow for determination of whether or how reading skill affected performance. Finally, alternate-form reliability analyses were not conducted for the probes used for this study. Therefore, it was not known whether the probes were statistically equivalent or not. This is important because normal development of CCM probes does not guarantee statistical equivalence (Mooney, McCarter, Russo, et al., 2013).

Implications for Research and Practice

Previously, we have suggested that indicators of academic vocabulary such as CCM and vocabulary matching be considered viable measures of content assessment frameworks (Mooney & Lastrapes, 2016; Mooney et al., 2016). That research-informed assertion is bolstered by the present conceptual replication findings. First, there are now three studies that report criterion-related validity correlation magnitudes for CCM with meaningful science criterion measures in the moderate to large range (as specified by Marston, 1989) for a large, multigrades, and more representative sample of public school students that included a small number of students with disabilities. The content area science literature also includes Espin et al. (2013) in which generally strong in magnitude findings were reported for vocabulary matching for an older sample of demographically different students. Moreover, the content literature includes evidence of moderate to strong criterion validity correlations with national- and state-level measures in social studies (Espin Busch, Shin, & Kruschwitz, 2001; Mooney, McCarter, Russo, et al., 2013, 2014) across upper elementary and middle school grades. In addition, present data demonstrate moderate to strong correlations to relevant criterion in ELA and reading comprehension across Grades 4 to 6.

Second, and we believe more importantly, a growing literature is highlighting that, when directly compared, academic language-oriented predictor measures such as CCM and vocabulary matching have consistently provided stronger in magnitude correlations with meaningful criterion than those of other predictors. For example, and likely expectedly, Mooney, McCarter, Schraven, et al. (2013) reported findings for social studies content wherein a measure of social studies vocabulary (i.e., vocabulary matching) produced statistically significantly stronger concurrent validity comparisons than did a reading measure (i.e., traditional maze) and a writing measure (written expression-Curriculum Based Measurement) in content criterion comparisons. More recently, Lastrapes and Mooney (2017), Mooney and Lastrapes (2016), and Mooney et al. (2016) have compared CCM to ORF, traditional maze, sentence verification technique, and written retell for science content, both in concurrent and predictive criterion validity circumstances, and found that the academic vocabulary measure provides descriptively stronger correlation coefficients than other predictors in most all cases. So there is technical adequacy justification, albeit predominantly at the static score level (L. S. Fuchs, 2004).

Finally, the present data strengthened the argument for academic vocabulary measure use by demonstrating predictive, growth, and implementation capabilities. While not powerfully accurate, fall and winter CCM benchmark scores were able to distinguish participants who “passed” and “failed” the state test. CCM benchmark scores also displayed evidence that individual and collective student learning can be captured through repeated administration of timed probes. The diagnostic accuracy and growth analyses came following teacher-directed administration of the probes across professionals, grades, schools, and benchmark periods. Teachers and not solely researchers demonstrated the ability to carry out a year-long assessment implementation process. Collectively, these data suggest that academic vocabulary general outcome measures can be used to systematically document student learning in response to intervention frameworks similar to the more recognizable reading and math measures. Systematic research in the design, implementation, and evaluation of response to intervention frameworks for older age students and in science and social studies content areas is warranted given the variability in system implementation (e.g., D. Fuchs & Fuchs, 2017) and the differences necessary for older students (e.g., L. S. Fuchs, Fuchs, & Compton, 2010).

With the general outcome measurement reading and math history of inquiry as guidepost to future investigation in the science and social studies content areas, the notion of systematically designing and completing conceptual replications in support of generalizability suggested by Coyne, Cook, and Therrien (2016) can also guide development. While significantly more technical adequacy research remains necessary, systematic program inquiry in logistical feasibility and instructional effectiveness (Deno & Fuchs, 1987) by diverse researcher–practitioner groups are logical next steps.

In terms of logistical feasibility, we provided evidence that content and/or secondary school teachers can successfully direct benchmark data collection efforts with researcher guidance and support. Determining how best to collect, summarize, and use benchmark data will be beneficial to the hope of more widespread adoption of preventive assessment and instructional frameworks in the content areas and at the secondary school level. Research in the general outcome measurement for science and social studies content area, and particularly the academic vocabulary measure area remains researcher driven. For such assessments, and the potential data-based decision making and intervention processes that could be associated with their use, the involvement and leadership of practitioners is sorely needed. Diversity in the size and makeup of schools and districts would allow for development of a systematic implementation replications. Input from practitioners might first address how to best incorporate academic vocabulary assessment and intervention into science and social studies courses.

In terms of instructional effectiveness, L. S. Fuchs (2004) has made the point that curriculum-sampled general outcome measures better enable teachers to target skills—in this case, content—for remediation. Academic vocabulary is relevant remediation content given its status as academic currency (Alexander, n.d.), its centrality in classroom discussion and action (Nagy & Townsend, 2012), and its ability to distinguish between stronger- and weaker-performing students. Academic vocabulary use also seems timely as proponents of the Common Core State Standards (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010) in ELA advocate for instructional strategies such as close reading (Brown & Kappes, 2012) to be used with informational text. Few studies (e.g., Boudreaux-Johnson, Mooney, & Lastrapes, 2017) have incorporated academic vocabulary measures as indicators of effectiveness. A body of systematic, conceptual replications would inform stakeholder efforts to improve instruction and/or achievement across the assessments tiers that comprise the suggested or implemented response to intervention frameworks in secondary schools. A starting place may be the application of data-based individualization to students with disabilities in inclusive science and social studies courses.