Developing and Validating the College and Career Readiness Teacher Expectation Survey for Students With Disabilities

Abstract

All students, including those with disabilities, must be college and career ready, which requires high expectations for them (Every Student Succeeds Act, 2015). In this study, we developed and validated the College and Career Readiness Teacher Expectation Survey (CCR-TES), an instrument that measures the postschool expectations that educators have for students with disabilities (SWD). With a sample of 459 educators, we provide initial evidence for strongly correlated factors, highlighting the difficulty in measuring CCR. Results support one general CCR factor, and invariance testing results show the CCR-TES functions similarly for special and general education teachers. Furthermore, results also revealed educators have somewhat low postsecondary expectations for SWD. Implications for teachers, researchers, and policymakers are discussed.

Keywords

college and career readiness transition assessment measurement students with disabilities

Obtaining some form of postsecondary education is a common aspiration for many students with disabilities (SWD); yet, longitudinal studies show many students do not achieve these goals (Lipscomb et al., 2017; U.S. Department of Education, 2017). In addition, people with disabilities have lower rates of paid work experience in high school and poorer employment outcomes once they enter into young adulthood compared to their peers without disabilities (Bureau of Labor Statistics, U.S. Department of Labor, 2021). Federal legislation attempts to close this gap; the most recent of which is the Every Student Succeeds Act (ESSA) of 2015, which emphasizes the importance of preparing all students, including those with disabilities, for college and careers while also highlighting the importance of high expectations and access to a high-quality education.

College and Career Readiness

Before ESSA (2015), many states adopted the Common Core State Standards (CCSS), which prompted discussions about what it meant to be college and career ready (Mishkind, 2014). As of 2014, the majority of states defined college and career readiness (CCR) with one definition for both college and career, and included academic and non-academic components (Mishkind, 2014). While policymakers have worked to develop definitions of CCR, scholars have also hypothesized how CCR may be defined.

Monahan et al. (2020) examined CCR frameworks that were empirically supported. Many studies in this literature review identified academic and non-academic CCR components including transition skills (Conley, 2010; Morningstar et al., 2017) and external factors to success such as student motivation, college culture awareness, and financial literacy (Karp & Hughes, 2008; Leonard, 2013; Molle & Lee, 2017; Welton & Martinez, 2014). The only CCR framework with a focus on SWD was developed by Morningstar and colleagues in 2017 (Morningstar et al., 2017). This framework was condensed from six to five domains: Academic Engagement, Interpersonal Engagement, Process-Oriented Skills, Ownership of Learning, and Transition Competencies (Lombardi, Monahan, & Morningstar, 2020). However, all of these CCR frameworks focus on student skills; they do not address teachers’ expectations of SWD and their postschool goals.

Teacher Expectations

Many general and special education teachers believe that SWD should have different expectations than peers without disabilities related to academic achievement (McGrew & Evans, 2004; Quenemoen & Thurlow, 2019). As Butrymowicz and Mader (2017) highlight, SWD have been negatively affected by low teacher expectations, speaking of “expectations lowered to the point where they do students more harm than good” (para. 8). Studies have shown these lowered expectations extend beyond expectations of what students should learn and also affect what educators expect of students once they exit high school. Career and technical education (CTE) teachers, high school general education teachers, and professionals involved in a student’s transition held lower expectations for employment and postsecondary success for SWD compared to their peers without disabilities, indicating that SWD would be less likely to access postsecondary education, succeed in postsecondary education, and gain and maintain employment (see Harvey, Cotton, & Koch, 2005, 2007; Harvey & Pellock, 2003, 2004; Keel et al., 2018; Levin et al., 1982; Shifrer, 2013; Sinclair et al., 2017). To date, no studies have empirically examined teachers’ postsecondary expectations for SWD with respect to CCR, specifically.

Students with disabilities are protected by the Individuals with Disabilities Education Improvement Act (IDEIA, 2004) and are guaranteed transition services. Transition services are included in a student’s Individualized Education Program (IEP) in order to achieve postsecondary goals related to employment, postsecondary education, and independent living, if applicable (IDEIA, 2004). Many of these components are included in common CCR frameworks (Lombardi, Monahan, & Morningstar, 2020; Monahan et al., 2020). General and special education teachers are required members of the IEP team and they contribute to the conversation related to a student’s transition goals and outcome statements. Educator expectations of what students are capable of may affect how teachers work to develop goals and services for SWD (Butrymowicz & Mader, 2017). Given the evidence cited concerning low expectations for SWD, the mainstream and policy excitement over high expectations, and the deeply important role educators play in IEP development, it is critical that the field empirically study teacher postschool expectations for SWD within the context of CCR.

Purpose Statement

Teachers’ beliefs and expectations for SWD once they leave high school may signify a critical area of study within general and special education research. Currently, there is no instrument that measures the CCR expectations educators have for SWD. Given the poor postschool outcomes for SWD and the importance of teacher expectations, an instrument of this type is necessary for the field. The purpose of this study was to develop and validate the College and Career Readiness Teacher Expectation Survey (CCR-TES), an affective instrument designed to measure CCR expectations of secondary special and general educators. Specifically, we addressed five research questions:

Research Question 1 (RQ1): To what extent do the hypothesized constructs (Academic Engagement, Mindsets, Learning Processes, Critical Thinking, Interpersonal Engagement, and Transition Competencies) explain the pattern of correlations in the responses in the CCR-TES?

Research Question 2 (RQ2): Does the CCR-TES function similarly for general education and special education teachers?

Research Question 3 (RQ3): To what extent, if any, are there differences in CCR expectations for SWD between secondary special educators and general educators?

Research Question 4 (RQ4): To what extent, if any, are there differences between educators’ CCR expectations for SWD and those without?

Research Question 5 (RQ5): Do demographic variables (e.g., school type, years teaching, teaching settings) predict teacher expectations for SWD?

Method

The study took place in two phases: development and validation. The development phase was completed through item review by a panel of experts and a teacher focus group. The validation phase included a series of quantitative analyses using split-sample cross-validation to conduct an exploratory factor analysis (EFA), which informed the a priori model tested in a confirmatory factor analysis (CFA). This study was approved by the university’s Institutional Review Board.

Phase 1: Instrument Development and Content Validity

After a comprehensive review of CCR frameworks and expectations in the literature, we determined that the constructs and items would be grounded in the CCR framework established by Morningstar et al. (2017; Academic Engagement, Mindsets, Learning Processes, Critical Thinking, Interpersonal Engagement, and Transition Competencies). Evidence of content validity was established by aligning the items on the CCR-TES with existing CCR measures designed for students to self-report their perceptions of academic engagement, critical learning processes, mindset, and transition competencies (Lombardi et al., 2022; Lombardi et al., 2018). The CCR-TES is intended for educators, and therefore, not meant to assess individual students. Rather, the CCR-TES is meant to measure teacher expectations of SWD, broadly.

The CCR-TES was designed for use by general and special educators who teach grades 9 through 12 as well as in transition programs for individuals with disabilities through age 21. An initial draft of the instrument contained 54 items. Each item asked about students with and without disabilities separately. The “expectations of students without disabilities” group referred to all items about students who are not on an IEP and do not have documented disabilities through IDEIA (2004). The second group, “expectations of SWD,” contained all items relating to students on an IEP and students on plans based on Section 504 of the Rehabilitation Act of 1973.

Content validation

Content validity refers to the extent to which the items measure what is intended and the different dimensions of the hypothesized factors are well-accounted (Kane, 2009). Following item development, evidence of content validity was obtained through a panel review of nine expert researchers in CCR or related fields and one member of the target population (Kline, 2016). Based on the feedback, we retained, deleted, or revised items. There are no established criteria in the literature; however, McCoach et al. (2013) recommend high agreement, confidence, and relevance by each expert. Therefore, we decided that items with at least 80% agreement among the experts would be retained. Confidence and relevance had to also be high (2.0 or higher; McCoach et al., 2013).

In addition to the items on the instrument, experts commented on the proposed factor descriptions. Based on comments regarding the cross-loading of many items, we reduced the number of factors on the instrument to five by combining the factors related to expectations of critical thinking and learning processes (Critical Learning Processes Expectations, CLPE). This decision reflected the opinions of the content experts and was aligned with the most recent iteration of CCR domains for SWD (Lombardi et al., 2020). The Academic Engagement Expectation (AEE) measures teacher expectations that all students can actively engage with academic content (acquisition of academic content by interacting and engaging with the material; cognitive and behavioral skills). The Mindsets Expectations (ME) construct includes a teacher’s expectation that students will acquire the mentality that they can and will do well academically and in their careers (i.e., sense of belonging, growth mindset, ownership of learning, perseverance, willingness to take academic risks).

The CLPE domain measures teacher expectations that a student can develop critical thinking skills that enable them to engage in the learning process (note-taking and test-taking strategies) and be able to move through the scientific process. The Interpersonal Engagement Expectation (IEE) domain focuses on interactions with other individuals as well as self-awareness. The Transition Competencies Expectations (TCE) measures a teacher’s expectation that students have the skills necessary to be successful after high school (employment, postsecondary education, and independent living).

Focus group

We assessed the usability of the instrument and revised items using data from a focus group with special education and general education teachers. The focus group allowed participants to provide input about the constructs, items, and usability. Three teachers participated in the focus group and were compensated for their time with a $15 gift card. The CCR-TES was sent to participants before the focus group, with instructions to complete the survey. Focus group questions are available as a supplemental file.

Phase 2: Validation Phase

The second phase of the study consisted of an examination of the psychometric properties. An EFA, CFA, reliability, and subsequent analyses were conducted.

Recruitment and participants

Participants were recruited through local state agencies as well as national organizations for transition. In addition, we identified approximately 1.0% of the public school districts in the country (n = 180) via data from the National Center for Education Statistics (NCES, 2021) and acquired email addresses of any secondary educators for each district via their publicly available website. These email addresses were then moved to Qualtrics (n = 4,662) and emailed twice (approximately 1 week apart), which resulted in a 7.4% response rate (n = 343). To increase the likelihood of participation, individuals who completed a survey were entered in a drawing to win one of four $50 Visa gift cards.

Participants (n = 666) identified themselves as certified general or special education teachers. We removed responses that had no demographic data or only demographic data and no responses for items about SWD or students without disabilities (SWOD; n = 116), resulting in 550 responses. Responses that did not include data for SWD were removed (n = 92), resulting in a total of 459 usable responses randomly assigned to either the EFA or CFA (EFA = 193, CFA = 266). Table 1 includes participant demographics for each sample.

Table 1.

Participant Demographics for the Survey Evaluation.

	EFA sample (n = 193)		CFA sample (n = 266)
Demographic Variable	n	%	n	%
Certification
Special Ed	56	29.2	68	25.6
General Ed	90	46.9	115	43.2
Dual Cert	46	24.0	83	31.2
Education
Bachelor’s degree	48	25.0	68	25.6
Master’s degree	119	62.0	159	59.8
6th year or admin	16	8.3	29	10.9
PhD or EdD	9	4.7	9	2.3
Years teaching
<3	8	4.2	24	9.0
3–9	38	19.8	70	26.3
10–20	74	38.5	96	36.1
>20	72	37.5	76	28.6
School setting
Urban	39	20.3	48	18.0
Suburban	68	35.4	107	40.2
Rural	83	43.2	111	41.7
School type
Public	171	89.1	243	91.4
Private	4	2.1	1	0.40
Charter	13	6.8	16	6.0
Student support level
Most support	27	14.1	49	18.4
More support	61	31.8	99	37.2
Little to no support	93	48.4	110	41.4
Other	10	5.2	8	3.0

Note. EFA = exploratory factor analysis; CFA = confirmatory factor analysis; Special Ed = special education; General Ed = general education; Dual Cert = dual certification.

Exploratory factor analysis

Each item was asked twice, once for SWD and once for SWOD. Therefore, we planned to conduct one factor analysis for each set of items. Item response data for the EFA (n = 193) was analyzed in MPlus Version 8 (Muthén & Muthén, 2017). We examined the frequency distributions of item responses by reviewing means and standard deviations, as well as histograms for each item. Due to the lack of variability in the data for SWOD (very high skew and kurtosis, means all between 4.091 and 5.005 with standard deviations below or close to 1.0), an EFA and CFA were not conducted with this dataset (Kline, 2016). As such, from this point forward, the study addressed only responses for SWD. The significance of this phenomenon is addressed in the discussion.

We examined inter-item correlations to identify problematic items. Decisions to retain or remove items based on correlations were made with a focus on theoretical meaning. Next, we conducted an EFA in Mplus using maximum likelihood with robust standard errors and robust test statistics (MLR) and an oblique rotation extracting one to six factors. From there, we decided which model had the best fit and examined the rotated Geomin Loadings (pattern coefficients) for that specific model. Items were retained if they had a pattern coefficient of 0.50 or higher with no cross-loadings (McCoach et al., 2013).

Reliability

To test the internal consistency reliability of the factors determined by the EFA, we calculated the Omega Coefficient in Mplus (Dunn et al., 2014; McDonald, 1999). Omega is superior to the commonly used Cronbach’s alpha, in that it is sensitive to violations of tau-equivalence, which is highly likely in social science data (Zinbarg et al., 2005). After we tested for internal consistency, we then conducted a CFA.

Confirmatory factor analysis

Based on the results of the EFA, the relationships between the items and hypothesized factors were determined, and a CFA was conducted using the remaining sample (n = 266). Using the standardized residuals and modification indices, and aligned with theoretical knowledge, we respecified the model. To answer RQ2, we conducted a multiple group–CFA (MG-CFA). We tested the CCR-TES for configural invariance, metric invariance, and scalar invariance (Van de Schoot et al., 2012). We compared these models by examining the change in comparative fit index (CFI) indices. Models with a less than 0.01 change in CFI were deemed to have adequate measurement invariance (Cheung & Rensvold, 2012).

To answer RQ3, we conducted a latent variance and mean comparison. RQ4 was answered indirectly as we were unable to conduct an EFA on the data about SWOD, and thus, unable to draw a direct comparison of expectations for both groups. Finally, we answered RQ5 using a series of one-way analysis of variance (ANOVA) using the summed score as the dependent variable.

Results

The EFA identified a five-factor model but showed strong evidence of a single-factor and high-factor correlations. The CFA showed poor model fit on the five-factor model, but acceptable fit with a one-factor model. The CCR-TES (one-factor model) functioned similarly for general and special education teachers, and there were no differences in the expectations for SWD between general and special educators. There was initial evidence to suggest that expectations for SWD are different than those without. The level of student support is a predictor of teacher expectations.

Psychometric Properties of the CCR-TES

Through the content validation phase of this study, the six domains were reduced to five: (a) AEE, (b) ME, (c) IEE, (d) TCE, and (e) CLPE. Thus, the focus shifted to exploring the extent to which these five constructs explained the pattern of correlations in the responses in the CCR-TES for SWD.

Exploratory factor analysis

We ran an EFA with a Geomin rotation extracting between one and six factors. Table 2 highlights the model fit indices for each model. The five-factor model had the best fit, with CFI and Tucker–Lewis index (TLI) above .90, root-mean-square error of approximation (RMSEA) below .080, including the confidence interval, and a low standardized root mean squared residual (SRMR). Model fit was not as good for the four-factor model and decreased again slightly with the six-factor model, indicating the five-factor solution was the best fit. This means that in the preliminary factor analysis stage, the data showed five distinct factors or constructs.

Table 2.

EFA Model Fit Indices for Students with Disabilities.

Factors	χ²		CFI	TLI	RMSEA			SRMR
Factors	Value	df	CFI	TLI	Value	95% CI	p	SRMR
1	1,420.986*	665	.862	.854	.077*	[.071, .082]	.000	.046
2	1,209.136*	628	.894	.881	.069*	[.064, .075]	.000	.036
3	1,039.386*	529	.918	.903	.063	[.056, .069]	.001	.029
4	931.192*	557	.932	.914	.059	[.052, .066]	.013	.025
5	822.273*	523	.945	.927	.055	[.047, .062]	.144	.021
6	880.819*	490	.929	.898	.064	[.058, .071]	.000	.020

Note. EFA = exploratory factor analysis; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root-mean-square error of approximation; SRMR = standardized root mean squared residual.

Significant at 0.000.

The first two factors were highly correlated, indicating approximately 74.0% of shared variance between the two, which was corroborated by a review of the factor structure. When two factors are highly correlated in an EFA, it is possibly indicative of a second-order factor, non-distinct factors, or two factors that are measuring the same concept (Rönkkö & Cho, 2022). It was difficult to discern which latent variables these factors may have represented; however, many items in these two factors explicitly mentioned the postschool setting. In addition, the first factor had an eigenvalue of 24.98, indicating that it is a strong factor.

We then reviewed the pattern coefficients for the items in the five-factor solution, retaining any items that had a pattern coefficient of above 0.50 and below 1.0 on a factor (McCoach et al., 2013), which resulted in the removal of five items. Given the complex findings of the EFA, before continuing with the CFA sample, we conducted a one-factor, two-factor, and five-factor CFA with the EFA sample to determine the best model for moving forward. Based on fit indices and data from the EFA sample, the one-factor model was most tenable to use with the CFA sample. Therefore, the CFA analysis was conducted with only one factor. In order to calculate the reliability of the one-factor model, we calculated the omega coefficient in MPlus. The omega value was 0.99, demonstrating strong internal consistency for the one-factor model.

Confirmatory factor analysis

We conducted a one-factor CFA for the retained 33 items pertaining to SWD. The CFA was conducted in Mplus using MLR with standardized residuals and modification indices. Table 3 outlines the tests of model fit. When model fit is poor, it is acceptable to correlate items that are theoretically related to one another and have similar content and wording (Brown, 2015). The fit of the model was poor, so using information from the residuals and modification indices, we correlated two items about interviewing. These items had a high standardized residual (3.58) and a high modification index (37.75). Correlating these errors slightly improved model fit (see Table 3). The next two items with a high residual (4.46) and a high modification index (47.65) were related to empathy and social awareness, respectively. Again, the fit improved slightly (see Table 3). These were the final modifications to the model, as there were no more residuals that made theoretical sense, and changes to the fit indices were minimal. The final items in the CCR-TES can be found in Table 4.

Table 3.

CFA Fit Indices for Original Model and Correlated Errors.

Model	χ²		CFI	TLI	RMSEA			SRMR
Model	Value	df	CFI	TLI	Value	95% CI	p	SRMR
1	1,197.414*	495	.888	.880	.073*	[.068, .078]	.000	.039
2	1,173.718*	494	.891	.884	.072*	[.067, .077]	.000	.039
3	1,142.501*	493	.896	.889	.070	[.065, .076]	.000	.038

Note. CFA = confirmatory factor analysis; Model 1 = original model; Model 2 = 1 correlated error; Model 3 = 2 correlated errors; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root-mean-square error of approximation; SRMR = standardized root mean squared residual.

Significant at 0.000.

Table 4.

Final Items on the CCR-TES.

“I believe that students with disabilities can . . .”
Actively participate in class in order to learn academic content
Adapt to new or unexpected interpersonal situations in school or at work (e.g., upset customer, conflict with professor, new coworker sharing an office)
Advocate for their needs at school or at work
Assume adult roles and responsibilities (e.g., financial literacy, health and wellness, transportation)
Communicate their learning through either written or verbal presentations
Demonstrate persistence through academic tasks
Demonstrate persistence through tasks while at work
Develop a deep level of understanding academic content
Develop an understanding of social awareness (understand and respond to the needs of others)
Develop post-high school (employment, postsecondary education, independent living, etc.) goals tied to interests
Develop the ability to demonstrate empathy (the ability to understand and share the feelings of another)
Develop trusting relationships with members of the school community
Engage in the academic learning process
Exhibit appropriate communication skills across a variety of settings
Exhibit appropriate listening skills in order to engage in learning
Independently use a variety of skills to access academic content
Interview successfully at job interviews
Learn the core academic content necessary to be successful in postsecondary education
Learn to analyze research findings
Learn to make positive changes through failures and mistakes
Learn to use a variety of note-taking strategies
Learn to use a variety of time management strategies
Meet a variety of employer expectations
Persist through difficult non-academic tasks
Succeed in a competitive employment setting with or without supports (job coach, natural supports, etc.)
Successfully apply to postsecondary institutions
Successfully interview at postsecondary institutions
Take ownership of their own actions or learning through self-monitoring
Take ownership of their own learning through goal setting
Understand failures or mistakes are a natural part of learning
Understand how to access campus resources in a postsecondary setting
Understand the different social and academic expectations in a college or employment setting
Work in small groups to accomplish a common goal

Note. CCR-TES = College and Career Readiness Teacher Expectation Survey.

General and Special Education Teachers

Using the one-factor model identified, we conducted individual CFAs with the general education teachers (n = 115) and special education teachers (n = 151; which included dual-certified teachers). Model fit indices for the full CFA sample and the general education and special education samples are in Table 5. Next, we conducted an MG-CFA to test for measurement invariance. We examined the change in CFI between the configural model (unconstrained; CFI = .857) and the metric model (constrained factor loadings; CFI = .585), which showed a calculated change of .001. We then constrained the intercepts (scalar model; CFI = .856). The CFI value changed by .002 between this model and the metric model, indicating there was evidence of measurement invariance (Cheung & Rensvold, 2012). Therefore, we concluded the one-factor model was appropriate for both general and special educators and the data for both groups remained combined in the subsequent analyses.

Table 5.

CFA Fit Indices for Full, General Education, and Special Education Teacher Samples.

Sample	χ ²		CFI	TLI	RMSEA			SRMR
Sample	Value	df	CFI	TLI	Value	95% CI	p	SRMR
Full	1,154.392*	322	.895	.885	.099*	[.092, .105]	.000	.036
General Ed.	782.336*	322	.875	.864	.111*	[.102, .121]	.000	.039
Special Ed.	1,049.130*	322	.843	.829	.122	[.114, .131]	.000	.047

Note. CFA = confirmatory factor analysis; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root-mean-square error of approximation; SRMR = standardized root mean squared residual.

Significant at 0.000.

Expectations differences in general and special education teachers

The overall mean factor CCR score for items pertaining to SWD was 4.089, with a standard error of 0.137. This score indicates that general and special education teachers “slightly agree” that SWD can acquire CCR skills. To identify differences in CCR expectations for SWD between general education and special education teachers, we conducted a latent variance and mean comparison using a chi-square difference test. Model 1 (χ² = 1,893.187, df = 696) allowed for all variances to freely estimate, and Model 2 (χ² = 1,893.193, df = 697) constrained the variances across groups. There was no difference in latent variance between these two models, so we chose to compare mean differences against Model 1 because it was more parsimonious. Finally, Model 3 (χ² = 1,893.187, df = 699) examined fixed the latent mean for both general education and special education teachers equal to 0.0. The findings of this test showed that the CCR expectations for SWD did not vary between general education and special education teachers.

Students With Disabilities Compared to Students Without Disabilities

When screening the data, we determined that items pertaining to SWOD did not have enough variability to be appropriate for factor analysis (high and positive kurtosis and low standard deviations). While the data pertaining to SWD also showed some kurtosis, it was not as pronounced as items for SWOD. The means and standard deviations of the SWD datasets showed much more variability, indicating that educators’ expectations for SWD had more variability than SWOD. These statistics indicate a difference in educator responses regarding postschool expectations for SWD versus SWOD. However, given the EFA was conducted only on the items pertaining to SWD, no definitive statements regarding the differences in CCR expectations can be made.

Teacher Expectation Predictors

Using the one-factor model, we examined demographic variables as predictors of CCR expectations for SWD. Prior to analysis, we calculated the summed score for items about SWD. The scores ranged from 32 to 192 (range of 160). The mean score was 135.1 with a standard deviation of 35.4, and the median score was 141. We then conducted a series of one-way ANOVAs for each variable. Of the variables examined (school setting [urban, suburban, rural], level of student support need, teacher’s highest level of education, and years of experience), only the level of student support need was a statistically significant predictor of CCR expectations. For this item, the response options were as follows: (a) little to no support needs (functions in general education with accommodations), (b) more significant support needs (functions in general education with modifications and accommodations or spends a portion of their day in general education and a portion in self-contained), (c) most significant support needs (spends the majority of their day in special education classes), or (d) other. Table 6 shows the results of the ANOVAs with the Tukey post hoc procedure. There was a 13-point difference in overall CCR scores for teachers who taught students with little to no support needs and teachers who taught students with the most significant support needs (p = .031). Teachers who taught students with little to no support needs had higher CCR expectations than teacher of students with more significant support needs.

Table 6.

ANOVA Results for Demographic Variables for CCR-TES Summed Score.

Variable	F	p
School setting	0.806	.447
Level of student support	2.996	.031*
Highest level of education	1.943	.122
Years teaching	0.241	.868

Note. CCR-TES = College and Career Readiness Teacher Expectation Survey; School setting = urban, suburban, rural; Level of student support = most significant, more significant, little to no support, other; Highest level of education = bachelor’s, master’s, 6th year or administrative certificate, PhD or EdD; Years Teaching = less than 3, 3–9 years, 10–20 years, over 20 years.

Significant at the 0.05 level.

Discussion

In this study, we examined the psychometric properties of the CCR-TES, an instrument that measures educators’ CCR expectations for SWD. Results indicated that (a) the instrument functions similarly for general and special education teachers; (b) teachers had a mean score of 4.09, suggesting that they “slightly agree” that SWD can learn CCR skills; (c) there were fundamental differences in the data for SWD and SWOD, indicating that there may be differences in CCR expectations for these two populations; and (d) the level of student support needs was a significant predictor of CCR expectation scores.

Validity

Before collecting data, the CCR-TES underwent extensive review from content experts. Content experts felt that the factor descriptions were appropriately detailed and items were comprehensive in measuring these factors, providing initial evidence of construct and content validity. During this process, many content experts expressed difficulty in determining a distinct factor structure. After revising the factors and items to establish more discrete constructs, we conducted an EFA, CFA, and MG-CFA. The data showed evidence of a strong first factor (eigenvalue = 24.98, factor correlation between Factor 1 and Factor 2 = 0.86). The fit indices for a five-factor model showed good model fit, but the pattern coefficients did not support that many factors.

Researchers have shown the difficulty in measuring CCR, as it is a complex set of academic and non-academic skills (Lombardi et al., 2018; Monahan & Bellara, 2018), and the results of this study corroborate those findings. Because the CCR-TES also measures expectations, it may be even more complex. Future iterations of the CCR-TES may include fewer items or focus on one domain of CCR, as opposed to multiple domains at once. The final instrument contained 33 items that evenly covered academic skills, interpersonal skills, and transition competencies. Therefore, the CCR-TES measures expectations of a variety of academic and non-academic skills. The remaining research questions rely heavily on good model fit, and therefore, caution should be used when interpreting the findings.

General and Special Education Teachers

Findings from the MG-CFA confirm that the one-factor model for the CCR-TES functions similarly for general education and special education teachers. General and special education teachers in this sample held similar expectations for SWD. The mean CCR factor score for teachers was 4.09, indicating that teachers in this sample “slightly agree” that SWD can learn CCR skills. While there are limitations regarding model fit, the findings from this study suggest that educators slightly agree that SWD can acquire these skills.

Unfortunately, these expectations were not high, which may have negative ramifications for students. We hypothesized that special education teachers would have higher expectations for SWD, as they are typically advocates for this population. If general education and special education teachers both hold low expectations for SWD, it is possible that there is less of a chance that someone in a student’s school is advocating for their postschool goals if they are more ambitious than teachers expect.

Students With and Without Disabilities

As discussed, limitations in the data presented challenges to addressing this question. However, the data for SWOD were fundamentally different from the data for SWD because they were not appropriate for factor analysis. These findings suggest teachers in this sample were not as likely to choose the same responses for SWD and SWOD. Teachers were more likely to report “mostly agree” that SWOD can learn CCR skills than they were for SWD. While we were unable to confirm a factor model for the SWOD data, these differences in expectations between SWOD and SWD warrant further investigation. In future studies, it will be imperative to find ways to compare the CCR expectations for SWOD and SWD statistically. These findings are consistent with existing research that suggests educators hold lower expectations for SWD (Harvey et al., 2005, 2007; Harvey & Pellock, 2003, 2004; Keel et al., 2018; Levin et al., 1982; Shifrer, 2013; Sinclair et al., 2017). Educators must understand that lowered expectations may inadvertently affect a student’s success in achieving their postschool goals, and researchers should explore this possible relationship in future studies.

Teacher Expectation Predictor Variables

The findings for RQ5 showed the level of student support need was a significant predictor of the overall CCR-TES score. This demographic item asked teachers to describe the population of students that they spend the majority of their day with. Teachers who taught students with little to no support needs had overall CCR-TES scores that were, on average, 13 points higher than teachers who taught students with the most significant support needs. Potentially, teachers hold lower expectations for students with more significant support needs because they do not have an understanding of the types of comprehensive resources that are available at some institutions of higher education for this population (Grigal et al., 2018). In order to further understand this finding, future research should include qualitative follow-up studies that explore why teachers who support students with significant needs hold lower expectations.

Limitations

In this study, there is the possibility of sampling bias, as the survey was voluntary. Teachers chose to complete this survey, and the results may have been different if the teachers who chose not to participate had completed the study. In addition, the sample was not demographically representative of the population of special and general education teachers in America (National Center for Education Statistics, 2021). The focus group that was conducted in Phase 1 had limited demographic information available. The CCR-TES is a self-report measure; thus, responses are the perceptions of teachers and not observed behaviors of teachers. This is a limitation in that participants may have responded to align with how they thought they should answer, even if it was not how they feel. This limitation is partially mitigated by the reassurance of anonymity.

There were several limitations with the statistical analysis that must be considered. We were unable to conduct a factor analysis for SWOD due to the lack of variability in the data, and thus, unable to compare teacher CCR expectations for SWD to SWOD. While we were able to conduct a factor analysis with data for SWD, the factor structure was complex and we found poor model fit, which should be taken into consideration when discussing the findings.

Implications for Research

The first step in continuing this line of research is to identify a factor model for the CCR-TES that has good model fit. The first recommendation is to explore model fit using a second-order or bi-factor model. Given the lack of variability in the data, researchers may also consider dichotomizing the data and conducting an analysis using Item Response Theory. Once researchers are able to show better model fit, the research questions should be re-examined with the improved model fit. Future studies should explore ways to separate out possible differences in disability categories or profiles of students with related strengths and weaknesses. This will be important to further understand the relationship between a student’s strength and weakness profile and what a teacher expects of them. Related to this, researchers should explore how the CCR-TES functions when teachers answer questions based on one specific student. A parallel form of the instrument could be developed that allows teachers to describe the student, which may include disability category, postschool goals, and percentage of the day spent in general education classes. Moreover, CCR-TES responses could be compared with student responses on a comparable CCR measure (Lombardi et al., 2022) to better understand similarities and differences between teacher and student perceptions.

Once the CCR-TES has good model fit, it can be used in future studies to measure the effectiveness of professional development related to raising expectations. In addition, the CCR-TES may be used in longitudinal studies to test the hypothesis that CCR expectations are a predictor of postschool success. The CCR-TES may also be used to identify whether teacher expectations are a predictor of a student’s engagement in CTE classes or programs. Finally, researchers should continue to explore the best ways to measure expectations for SWOD in order to compare postschool expectations for SWD versus SWOD statistically.

Implications for Practice

Despite the complexity that exists in measuring the constructs of CCR expectations, the one-factor model of the CCR-TES includes a variety of items that cover academic and non-academic skills. The overall CCR-TES score can be used to gain insight into what educators’ CCR expectations are for SWD. We recommend that states or local education agencies (LEA) consider asking teachers in their schools to complete the CCR-TES once revisions have been made and better model fit is achieved. This will allow states and LEAs to have a better understanding of the overall level of expectations that teachers have for SWD. While there is no score cut-off, teachers who find themselves answering “disagree” or “somewhat agree” should use these scores to reflect on why their expectations of SWD may be low. Future iterations of the CCR-TES may come with a variety of resources that teachers may engage with if they find their expectations are low. Education leaders may use these data to make informed decisions about professional development, resource allotment, or simply more fully understanding the culture of their school. Teachers may use the scores from the CCR-TES to challenge their preconceived notions and expectations for SWD. It is important to consider the fact that SWD are pursuing college at an increasing rate, and the majority of SWD believe they will obtain some form of postsecondary education (Lipscomb et al., 2017; Newman et al., 2011) despite their disability category. If educators do not fully believe that SWD can achieve these goals, there is potential students will not receive the most appropriate education to meet their transition goals. Teachers must consider their score on the CCR-TES, their beliefs about SWD, and the vast array of possibilities for SWD once they exit high school.

Implications for Policy

Legislation and guidelines for CCR and SWD should address academic and non-academic skills and low expectations of educators. Current CCR frameworks for SWD, along with the content validation process in this study, highlight the importance of non-academic skills that students should have to be successful in postsecondary education and employment (Monahan et al., 2020). Policymakers should consider incorporating measures of non-academic skills into their recommendations. This study adds to the body of literature on low expectations by providing evidence that CCR expectations for SWD are low. Additional funding is needed to explore ways to raise these expectations for SWD to combat poor outcomes.

Conclusion

The results of this study show the CCR-TES has preliminary evidence of construct validity. On average, general education and special education teachers “slightly agree” that SWD can acquire CCR skills, which has possible negative ramifications for the level of support SWD receive in achieving their postschool goals. This study offers preliminary evidence that policymakers should consider when revising and creating new policies related to CCR, keeping a specific focus on SWD, academic and non-academic skills, and maintaining high expectations. Practitioners should reflect on their CCR expectations for SWD and learn about the support available for SWD at many universities and employers.

Supplemental Material

sj-docx-1-cde-10.1177_21651434221116311 – Supplemental material for Developing and Validating the College and Career Readiness Teacher Expectation Survey for Students With Disabilities

Supplemental material, sj-docx-1-cde-10.1177_21651434221116311 for Developing and Validating the College and Career Readiness Teacher Expectation Survey for Students With Disabilities by Jessica L. Monahan, Allison Lombardi, Joseph Madaus, Jennifer Freeman and Nicholas Gelbar in Career Development and Transition for Exceptional Individuals

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This study was partially funded with a scholarship through the Division on Career Development and Transition.

ORCID iDs

Jessica L. Monahan

Allison Lombardi

Joseph Madaus

Nicholas Gelbar

Supplemental Material

Supplementary material for this article is available on the Career Development and Transition for Exceptional Individuals website with the online version of this article.

References

Brown

T. A.

(2015). Methodology in the social sciences: Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.

Bureau of Labor Statistics, U.S. Department of Labor. (2021, August 6). The employment situation—July 2021. https://www.bls.gov/news.release/pdf/empsit.pdf

Butrymowicz

Mader

(2017, November 6). Almost all students with disabilities are capable of graduating. Here’s why they don’t (Hechinger Report). https://www.huffingtonpost.com/entry/special-education-series_us_59fb588ae4b0415a420a55a6?88

Cheung

G. W.

Rensvold

R. B.

(2012). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

Conley

D. T.

(2010). College and career ready: Helping all students succeed beyond high school. Jossey-Bass.

Dunn

T. J.

Baguley

Brunsden

(2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046

Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 114 Stat. 1177 (2015-2016).

Grigal

Hart

Papay

(2018). Inclusive higher education for people with intellectual disability in the United States. In O’Brien

Bonati

M. L.

Gadow

Slee

(Eds.), People with intellectual disability experiencing university life (pp. 69–97). Brill. https://doi.org/10.1163/9789004394551_005

Harvey

M. W.

Cotton

S. E.

Koch

K. R.

(2005). Indiana career and technical instructors’ perceptions of program expectations for students with and without disabilities. The Journal of Vocational Special Needs Education, 28(1), 4–22. https://eric.ed.gov/?id=EJ854363

10.

Harvey

M. W.

Cotton

S. E.

Koch

K. R.

(2007). Indiana secondary CTE instructors’ perceptions of program expectations, modifications, accommodations, and postsecondary outcomes for students with disabilities. The Journal of Vocational Special Needs Education, 29(2), 16–32. https://files.eric.ed.gov/fulltext/EJ841376.pdf

11.

Harvey

M. W.

Pellock

(2003). Influences of student educational labels, behaviors, and learning characteristics as perceived by Pennsylvania’s secondary career and technical educators. The Journal of Vocational Special Needs Education, 25(3), 30–41. https://files.eric.ed.gov/fulltext/EJ854905.pdf

12.

Harvey

M. W.

Pellock

(2004). The relationship between Pennsylvania career and technical instructors’ demographic characteristics and program expectations for students with and without disabilities. The Journal of Vocational Special Needs Education, 26(3), 4–22. https://eric.ed.gov/?id=EJ854917

13.

Individuals With Disabilities Education Improvement Act, 20 U.S.C. § 1400 et seq. (2004).

14.

Kane

M. T.

(2009). Validating the interpretations and uses of test scores. In Lissitz

R. W.

(Ed.), The concept of validity: Revisions, new directions, and applications (pp. 39–64). Information Age Publishing.

15.

Karp

M. M.

Hughes

K. L.

(2008). Supporting college transitions through collaborative programming: A conceptual model for guiding policy. Teachers College Record, 110, 838–866. https://ccrc.tc.columbia.edu/media/k2/attachments/supporting-college-transitions.pdf

16.

Keel

J. M.

Cushing

L. S.

Awsumb

J. M.

(2018). Post-school visions and expectations of Latino students with learning disabilities, their parents, and teachers. Career Development and Transition for Exceptional Individuals, 41(2), 88–98. https://doi.org/10.1177/2165143417708997

17.

Kline

R. B.

(2016). Principles and practice of structural equation modeling (4th ed.). The Guilford Press.

18.

Leonard

(2013). Funding early college high school: Hold harmless or shared commitment. Education Policy Analysis Archives, 21, 46. https://doi.org/10.14507/epaa.v21n46.2013

19.

Levin

Arluke

Smith

(1982). The effects of labeling students upon teachers’ expectations and intentions. The Journal of Social Psychology, 118, 207–212. https://doi.org/10.1080/00224545.1982.9922799

20.

Lipscomb

Haimson

Liu

A. Y.

Burghardt

Johnson

D. R.

Thurlow

M. L.

(2017). Preparing for life after high school: The characteristics and experiences of youth in special education. Findings from the National Longitudinal Transition Study 2012. Volume 1: Comparisons with other youth: Full report (NCEE 2017-4016). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.

21.

Lombardi

Freeman

Rifenbark

(2018). Modeling college and career readiness for adolescents with and without disabilities: A bifactor approach. Exceptional Children, 84(2), 159–176. https://doi.org/10.1177/0014402917731557

22.

Lombardi

A. R.

Monahan

J. L.

Morningstar

M. E.

(2020). Integrating college and career readiness into transition education. In Wehmeyer

Shogren

K. A.

(Eds.), The handbook of adolescent transition education for youth with disabilities (2nd ed., pp. 14). Routledge.

23.

Lombardi

A. R.

Rifenbark

G. G.

Rogers

H. J.

Swaminathan

Taconet

Mazzotti

V. L.

Morningstar

Langdon

(2022). Establishing construct validity of a measure of adolescent perceptions of college and career readiness. Career Development and Transition for Exceptional Individuals. Advance online publication. https://doi.org/10.1177/21651434221081229

24.

McCoach

D. B.

Gable

R. K.

Madura

J. P.

(2013). Instrument development in the affective domain (3rd ed.). Springer.

25.

McDonald

R. P.

(1999). Test theory: A unified treatment. Lawrence Erlbaum Associates.

26.

McGrew

K. S.

Evans

(2004). Expectations for students with cognitive disabilities: Is the cup half empty or half full? Can the cup flow over? (Synthesis Report 54). National Center on Educational Outcomes. https://nceo.info/Resources/publications/OnlinePubs/Synthesis55.html

27.

Mishkind

(2014). Definitions of college and career readiness: An analysis by state. College and Career Readiness and Success Center, American Institutes for Research.

28.

Molle

Lee

(2017). Opportunities for academic language and literacy development for emergent bilingual students during group work. International Journal of Bilingual Education and Bilingualism, 20(5), 584–601. https://doi.org/10.1080/13670050.2015.1103206

29.

Monahan

Bellara

A. P.

(2018). College and career readiness: An examination of secondary special educators’ self-efficacy [Poster presentation]. Division on Career Development and Transition Conference, Cedar Rapids, IA, United States.

30.

Monahan

J. L.

Lombardi

A. R.

Madaus

Carlson

S. R.

Freeman

Gelbar

(2020). A systematic literature review of college and career readiness frameworks for students with disabilities. Journal of Disability Policy Studies, 30(3), 131–140. https://doi.org/10.1177/1044207320906816

31.

Morningstar

M. E.

Lombardi

Fowler

C. H.

Test

D. W.

(2017). A college and career readiness framework for secondary students with disabilities. Career Development and Transition for Exceptional Individuals, 40(2), 79–91. https://doi.org/10.1177/2165143415589926

32.

Muthén

L. K.

Muthén

B. O.

(2017). Mplus: Statistical analysis with latent variables. User’s guide (8th ed.). Muthén & Muthén.

33.

National Center for Education Statistics, U.S. Department of Education. (2021). Digest of education statistics, 2019 (2021-009), Chapter 3. https://nces.ed.gov/fastfacts/display.asp?id=60

34.

Newman

Wagner

Knokey

A. M.

Marder

Nagle

Shaver

Wei

Cameto

Contreras

Ferguson

Greene

Schwarting

(2011). The post-high school outcomes of young adults with disabilities up to 8 years after high school. National Longitudinal Transition Study-2 (NLTS2) (NCSER 2011-3005). SRI International. https://nlts2.sri.com/reports/

35.

Quenemoen

R. F.

Thurlow

M. L.

(2019). Students with disabilities in educational policy, practice, and professional judgment: What should we expect? (NCEO Report 413). National Center on Educational Outcomes. https://nceo.info/Resources/publications/OnlinePubs/report413/default.html

36.

Rönkkö

Cho

(2022). An updated guideline for assessing discriminant validity. Organizational Research Methods, 25(1), 6–14. https://doi.org/10.1177/1094428120968614

37.

Shifrer

(2013). Stigma of a label: Educational expectations for high school students labeled with learning disabilities. Journal of Health and Social Behavior, 54, 462–480. https://doi.org/10.1177/0022146513503346

38.

Sinclair

J. S.

Unruh

D. K.

Griller Clark

Waintrup

M. G.

(2017). School personnel perceptions of youth with disabilities returning to high school from the juvenile justice system. The Journal of Special Education, 51, 95–105. https://doi.org/10.1177/0022466916676089

39.

U.S. Department of Education. (2017). A transition guide to postsecondary education and employment for students and youth with disabilities. Office of Special Education and Rehabilitative Services. https://www2.ed.gov/about/offices/list/osers/transition/products/postsecondary-transition-guide-2017.pdf

40.

Van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492. https://doi.org/10.1080/17405629.2012.686740

41.

Welton

Martinez

(2014). Coloring the college pathway: A more culturally responsive approach to college readiness and access for students of color in secondary schools. Urban Review, 46(2), 197–223. https://dx-doi-org.web.bisu.edu.cn/10.1007/s11256-014-0283-8

42.

Zinbarg

R. E.

Revelle

Yovel

(2005). Cronbach’s α, Revelle’s β, and McDonald’s ωH: Their relations with each other and two alternate conceptualizations of reliability. Psychometrika, 70(1), 123–133. https://doi.org/10.1007/s11336-003-0974-7

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB