A Test and Cross-Validation of the Factor Structure of the Engagement Versus Disaffection With Learning Instrument Among Middle School Students

Abstract

Factor analytic procedures were used to gather validity evidence on the internal structure of Skinner, Kinderman, and Furrer’s Engagement versus Disaffection With Learning measure to operationalize students’ classroom engagement. Independent samples of middle school students (Sample 1 = 618, Sample 2 = 493) were used to test the scale’s factor structure and invariance of measurement model parameters. Factor analytic results supported an alternative three-factor model of the scale’s previously reported four-factor solution. Cross-validation was based on the use of multisample confirmatory factor analysis, which supported strict factorial invariance of the scale’s measurement model parameters (e.g., factor loadings) across samples. Subscale reliability estimates, based on coefficient omega, exceeded 0.74. Implications of findings for research and practice are discussed.

Keywords

student engagement factor structure measurement invariance cross-validation

Theoretical Framework

Student engagement is considered a key determinant of students’ academic and behavioral outcomes (Appleton, Christenson, & Furlong, 2008; Fredricks, Blumenfeld, & Paris, 2004). Broadly, it is the “quality of a student’s connection or involvement with the endeavor of schooling and hence with the people, activities, goals, values, and place that compose it” (Skinner, Kindermann, & Furrer, 2009, p. 494). It is posited to influence classroom participation and affected by the learning environment, such that it is found to decline with schooling (Hughes & Kwok, 2007; Skinner, Kindermann, Connell, & Wellborn, 2009). At the middle school level, research has focused on student engagement as a key factor in student learning (Abbott, 2017; Orthner, Jones-Sanpei, Akos, & Rose, 2013; Wang & Holcombe, 2010). This includes the development of classroom-based surveys to operationalize student engagement (Fredricks & McColskey, 2012; Fredricks et al., 2011). The aim of this study was to expand the validity evidence of scores of the classroom-administered Engagement versus Disaffection With Learning (EvD; Skinner, Kindermann, & Furrer, 2009) survey to assess students’ classroom engagement (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014).

The conceptualization and measurement of student engagement vary considerably in the literature (Appleton et al., 2008). Although conceptualized as a multidimensional construct, it has been operationalized in terms of a two-factor model with behavioral and affective components (Willms, 2003) and, more broadly, as a four-factor model with additional cognitive and psychological components (Reschly & Christenson, 2006). Within academic settings, engagement behaviors include persistence and classroom participation, whereas emotional engagement includes interest and enjoyment. Within a motivational framework, Skinner, Kindermann, & Furrer (2009) advanced a four-factor model of engagement with distinct engagement and disaffection, as well as behavioral and emotional, dimensions. Uniquely, their engagement model also considered its opposite, disaffection, derived from behaviors, mental states, and emotions that stem from disengagement. Disaffected behaviors include passivity and a lack of attention, whereas disaffected emotions include frustration and tiredness. The model posits that engagement and disaffection are observable behaviors indicative of classroom behavior and related to various student and environmental factors.

The EvD (Skinner, Kindermann, & Furrer, 2009) was developed as a multidimensional measure of students’ classroom engagement with four subscales: Behavioral Engagement (BE), Behavior Disaffection (BD), Emotional Engagement (EE), and Emotional Disaffection (ED). It was developed and validated based on data obtained on a large sample of third to sixth graders. Reported internal consistency estimates of subscale scores ranged from 0.61 to 0.85 and test–retest estimates ranged from 0.53 to 0.68. Factor analysis supported the unidimensionality of subscales, and alternative two-factor models supported distinct behavioral, emotional, engagement, and disaffection factors. A four-factor solution reported similar model-data fit as the two-factor models. Student and teacher reports were moderately correlated, establishing preliminary evidence for interrater reliability. Convergent validity was based on the correlation of scores to theoretically related variables (e.g., perceived control). Teacher reports were also correlated with these measures but to a weaker extent. Classroom observations of student engagement were found to correlate with teacher reports but not with student reports. Skinner, Kindermann, & Furrer (2009) reported that subscales can be used separately or aggregated to report students’ engagement versus disaffection. Although the psychometrics properties of the EvD are comparable to other engagement measures (Fredricks et al., 2011), further research is needed on the measurement of student engagement to support the use of engagement scale scores for decision-making purposes (Appleton et al., 2008).

In response, the purpose of this study was to gather additional validity evidence on the internal structure of the EvD among middle school students. Although Skinner, Kindermann, & Furrer (2009) also developed a teacher version of the instrument, this study focused on the student version. Situated in the Standards (e.g., Standard 1.13; AERA et al., 2014), this study contributes to ongoing validity studies to substantiate score interpretation and use to guide decisions pertaining to middle school students’ classroom engagement.

Method

Participants

Data were based on independent samples (N_{Sample 1} = 618, N_{Sample 2} = 493) of Grade 5 to 8 middle school students from 17 middle schools with 48 teachers participating in regional professional development on the implementation of project-based mathematics learning. Sample 1 grade levels included 22.3% Grade 6, 49.1% Grade 7, and 28.6% Grade 8. Sample 2 grade levels included 4.2% Grade 5, 39% Grade 6, 16.2% Grade 7, and 21.6% Grade 8 (19.1% missing). Due to confidentiality, individual student demographics were not gathered; however, the average school enrollment was 563.59 (SD = 215.12), 87% White, and 50.37% qualified for free/reduced lunch (SD = 13.57). On average, the percent of students scoring at or above proficiency in reading and mathematics were 57.53% (SD = 11.14) and 45.66% (SD = 16.55), respectively. The surveys, administered based on passive consent by the researchers in individual classrooms, took approximately 30 minutes per classroom. The average number of students within classrooms was 23 (SD = 8.68).

Instrumentation

The 24-item EvD measures students’ behavioral and emotional classroom engagement and disaffection, with responses recorded on a 4-point scale (1 = not at all true, 2 = rarely true, 3 = somewhat true, and 4 = very true). The four subscales are as follows: BE (five items), BD (five items), EE (five items), and ED (nine items). Notably, the ED subscale includes an additional three items to “tap the more differentiated disaffected emotions” (Skinner, Kindermann, & Furrer, 2009, 520), included in the present study. Scale validation was based on data obtained from 1,018 students in Grades 3 to 6 from a 4-year longitudinal study. Correlations among components indicated that engagement scores were strongly correlated, whereas disaffection scores were moderately to strongly correlated. With the exception of the ED subscale, subscales were found to be unidimensional. Competing confirmatory factor analysis (CFA) models supported the fit of a four-factor solution for the entire instrument with distinct BE, EE, BD, and ED factors. Notably, an alternative two-factor solution was also found to be acceptable. Subscale score internal consistency estimates ranged from 0.61 (BE) to 0.85 (ED), whereas combined scale scores (e.g., BE, EE) exceeded 0.73. Test–retest reliabilities over the academic year ranged from 0.53 to 0.73. Additional information is provided in Skinner, Kindermann, & Furrer (2009) study.

Data Analysis

Item analyses were used to examine the functioning of scale items, with designated items reverse scored. Alternative CFA models were used to test the scale’s factor structure based on Sample 1 data, with multisample CFA (Cheung & Rensvold, 2002) used to cross-validate its factor structure with Sample 2 data. Sample 1 data were randomly divided in half (Sample 1A, Sample 1B) such that if model-data fit of the CFA model was poor, an exploratory factor analysis (EFA) was used to identify the empirical factors, based on Sample 1B data. Factor retention was based upon eigenvalues >1.0, parallel analysis (PA), scree plot, and factor interpretability (Henson & Roberts, 2006), with loadings above 0.40 to identify salient loadings.

CFA was based upon robust weighted least squares (WLSMV; Muthén et al., 1997) for parameter estimation using Mplus 8.0 (Muthén & Muthén, 1998-2017) and reported to yield accurate parameter estimates for ordinal variables (Flora & Curran, 2004). Evaluation of model-data fit included chi-square statistic, root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis Index (TLI). Due to the absence of cutoff criteria for fit statistics with the WLSMV estimator, Hu and Bentler’s (1999) empirically-based cutoffs (based on maximum likelihood) were used to evaluate model-data fit: RMSEA values less than 0.05 were used to indicate good model fit and those less than 0.08 suggested reasonable fit. CFI and TLI values above 0.95 were deemed acceptable. The COMPLEX command was used to compute standard errors and model chi-square statistics to account for nonindependence of observations (students nested in classrooms). Measurement model parameters of interest included factor loadings, thresholds, and residuals to determine level of invariance (Meredith, 1993).

As the chi-square difference statistic (χ²_Difference) is known to reject the null hypothesis of equivalent model parameters in invariance testing based on trivial differences in large sample sizes, the incremental changes of the CFI and RMSEA values also were used in the tests of measurement invariance (i.e., <0.01; Cheung & Rensvold, 2002). Furthermore, WLSMV (DIFFTEST) as well as the theta parameterization option in Mplus were used to test error variance equality (Muthén & Muthén, 1998-2017). As a factor analytic estimate of reliability, coefficient omega was used to estimate internal consistency of scores (ω; Revelle & Zinbarg, 2009).

Results and Discussion

Descriptive Statistics

Table 1 reports item means ranging from 2.33 (Item 27) to 3.70 (Item 1) and standard deviations ranging from 0.51 (Item 1) to 1.14 (Item 27). Across items, the range was 3, indicating use of the entire response continuum. The average item-total correlation for Sample 1 was 0.58 (SD = 0.10; range = 0.29 [Item 3] to 0.74 [Item 12]) and for Sample 2 was 0.60 (SD = 0.07; range = 0.35 [Item 3] to 0.72 [Item 7]), indicating moderate relationships between items and the total score.

Table 1.

Item Descriptive Statistics of the EvD Instrument Across Samples 1 and 2.

Item	M	SD	Median
1	3.70 (3.67)	0.51 (0.57)	4 (4)
2	3.47 (3.53)	0.55 (0.58)	3 (4)
3	3.20 (3.19)	0.71 (0.76)	3 (3)
4	3.46 (3.44)	0.60 (0.64)	4 (4)
5	3.30 (3.29)	0.59 (0.67)	3 (3)
6	3.05 (2.97)	0.83 (0.90)	3 (3)
7	2.95 (2.90)	0.76 (0.85)	3 (3)
8	2.81 (2.78)	0.90 (0.99)	3 (3)
9	3.21 (3.13)	0.79 (0.84)	3 (3)
10	3.37 (3.29)	0.68 (0.76)	3 (3)
11	3.32 (3.44)	0.85 (0.85)	4 (4)
12	3.64 (3.57)	0.65 (0.80)	4 (4)
13	3.19 (3.30)	0.93 (0.94)	3 (4)
14	2.53 (2.69)	0.82 (0.95)	3 (3)
15	2.49 (2.67)	0.88 (0.94)	3 (3)
16	2.54 (2.73)	0.89 (0.98)	3 (3)
17	2.59 (2.76)	0.92 (1.01)	3 (3)
18	2.88 (2.97)	0.96 (0.96)	3 (3)
19	3.20 (3.24)	0.92 (0.95)	3 (4)
20	2.96 (3.06)	0.95 (1.00)	3 (3)
21	2.80 (2.93)	1.01 (0.99)	3 (3)
22	3.37 (3.38)	0.81 (0.89)	4 (4)
23	2.86 (3.01)	1.04 (1.04)	3 (3)
24	3.48 (3.49)	0.75 (0.81)	4 (4)
25	3.47 (3.57)	0.79 (0.80)	4 (4)
26	2.52 (2.65)	1.02 (1.06)	3 (3)
27	2.33 (2.63)	1.05 (1.14)	2 (3)

Note. Values in parenthesis. Minimum and maximum values were 1 and 4 across samples. EvD = Engagement Versus Disaffection With Learning.

Model-data fit of alternative two-factor CFA models with distinct Engagement and Disaffection—χ²(323) = 989.51, RMSEA = 0.083 (90% confidence interval [CI] = [0.077, 0.089]), CFI = 0.854, TLI = 0.841—or Behavior and Emotional factors—χ²(323) = 971.53, RMSEA = 0.082 (90% CI = [0.076, 0.088]), CFI = 0.857, TLI = 0.845—based on Sample 1A were not acceptable across fit statistics. These findings are in contrast with Skinner, Kindermann, & Furrer (2009) findings of acceptable model-data fit of two-factor models with distinct Engagement and Disaffection and Behavioral and Emotional factors. Subsequently, four- and six-factor models with separate BE, BD, and alternative emotional factors were fit to the data, resulting in improper solutions (i.e., not positive definite matrix for the latent variable covariance matrix—correlations). Inspection of parameter estimates indicated that the correlation between the BE and BD factors was close to unity, indicating indistinguishable factors. Instead of imposing parameter constraints, an EFA was conducted to further examine the scale’s factor structure.

Based on Sample 1B data, a principal axis EFA with promax rotation (kappa = 4) supported a three-factor solution, with real-data eigenvalues of 9.03, 3.17, and 1.66 (PA = 1.66, 1.55, and 1.48). The factors resembled those of the original scale, with the emotional engagement and disaffection items generally falling on distinct factors and behavioral items loading onto Factor 3. Table 2 reports factor loadings, structure coefficients, and communalities. Factor correlations were moderate (range = 0.36 [BE and ED] to 0.68 [EE and ED]). Factor 1 included eight items and was labeled Emotional Engagement (EE), Factor 2 included seven items and was labeled Emotional Disaffection (ED), and Factor 3 included seven items and was labeled Behavioral Engagement (BE). Five items were eliminated due to a lack of a salient loading (<0.40). The lack of distinct behavioral engagement and disaffection factors suggested that students did not differentiate between these engagement types. Therefore, EFA findings generally support Skinner, Kindermann, & Furrer’s (2009) proposition of distinct emotional engagement and disaffection dimensions but differ in terms of a single unified behavioral dimension.

Table 2.

Exploratory Factor Analysis Pattern Coefficients, Structure Coefficients, and Communality Estimates.

Item	Factor 1		Factor 2		Factor 3		h ²
Item	P	S	P	S	P	S	h ²
1	−0.035	0.424	−0.024	0.210	0.690	0.658	0.43
2	0.059	0.387	−0.204	0.044	0.616	0.582	0.37
3	0.087	0.313	0.158	0.279	0.228	0.344	0.15
4	0.182	0.519	−0.110	0.176	0.567	0.652	0.45
5	0.298	0.498	−0.142	0.130	0.386	0.538	0.34
6	0.533	0.607	0.083	0.340	0.055	0.447	0.38
7	0.725	0.692	−0.044	0.272	−0.020	0.458	0.48
8	0.902	0.782	−0.141	0.230	−0.083	0.480	0.63
9	0.656	0.631	−0.116	0.190	0.039	0.443	0.41
10	0.195	0.428	0.018	0.224	0.331	0.470	0.24
11	−0.176	0.305	−0.038	0.148	0.732	0.598	0.38
12	−0.002	0.473	−0.033	0.226	0.720	0.706	0.50
13	−0.041	0.419	0.164	0.351	0.567	0.599	0.38
14	0.113	0.476	0.155	0.361	0.433	0.566	0.36
15	0.230	0.548	0.135	0.374	0.378	0.583	0.40
16	0.785	0.811	0.037	0.391	0.015	0.562	0.66
17	0.847	0.830	0.049	0.405	−.0056	0.537	0.69
18	0.647	0.743	0.078	0.398	0.090	0.559	0.56
19	−0.121	0.219	0.723	0.679	0.026	0.205	0.47
20	0.024	0.319	0.735	0.729	−0.046	0.235	0.53
21	−0.010	0.274	0.844	0.791	−0.135	0.164	0.64
22	−0.04	0.375	0.655	0.703	0.175	0.387	0.52
23	0.77	0.814	0.114	0.453	−0.010	0.555	0.67
24	0.13	0.452	0.469	0.587	0.167	0.425	0.41
25	0.07	0.429	0.381	0.513	0.285	0.467	0.36
26	−0.06	0.178	0.749	0.672	−0.135	0.092	0.48
27	0.08	0.306	0.668	0.666	−0.105	0.191	0.45

Note. Bold values indicate salient factor loadings greater than |0.40|.P = Pattern coefficients; S = Structure coefficients.

Based on Sample 2 data, model-data fit of the EFA-based factor structure was acceptable—χ²(206) = 736.09, RMSEA = 0.072 (90% CI = [0.067, 0.078]), CFI = 0.953, TLI = 0.947. Table 3 reports pattern and structure coefficients and error variances. Factor correlations were low to moderate (range = 0.34 [EE and ED] to 0.69 [BE and EE]). Configural invariance was supported with acceptable model-data fit of the three-factor solution across Samples 1 and 2 data—χ²(412) = 1,383.863, RMSEA = 0.065 (90% CI = [0.061, 0.069]), CFI = 0.958, TLI = 0.953. Although a subsequent test of invariant factor loadings reported a statistically significant chi-square difference statistic—χ²_Difference(19) = 29.09, p < .05—changes in RMSEA and CFI were less than 0.01, thus supporting their invariance. Next, the invariance of thresholds was tested which were deemed equal across groups due to changes in the RMSEA and CFI values less than 0.01, despite a significant chi-square difference test—χ²_Difference(41) = 75.12, p < .01. Last, strict factorial invariance was met with the finding of invariant error variances—χ²_Difference(22) = 95.07, p < .01; ΔRMSEA and ΔCFI < 0.01—suggesting similar reliability across distinct samples. Coefficient omega for the BE, EE, and ED subscales were 0.74, 0.81, and 0.77,

Table 3.

Factor Loadings, Structure Coefficients, and Error Variance Across Samples 1 and 2.

Item	Behavioral engagement		Emotional engagement		Emotional disaffection		Error variance
Item	P	S	P	S	P	S	Error variance
1	0.76 (0.78)	0.76 (0.78)		0.52 (0.52)		0.31 (0.40)	0.43 (0.39)
2	0.67 (0.83)	0.67 (0.83)		0.46 (0.55)		0.27 (0.42)	0.56 (0.32)
4	0.70 (0.79)	0.70 (0.79)		0.49 (0.53)		0.29 (0.40)	0.51 (0.38)
11	0.57 (0.72)	0.57 (0.72)		0.40 (0.48)		0.23 (0.37)	0.67 (0.48)
12	0.86 (0.82)	0.86 (0.82)		0.60 (0.55)		0.35 (0.42)	0.25 (0.33)
13	0.70 (0.73)	0.70 (0.73)		0.49 (0.49)		0.29 (0.37)	0.51 (0.47)
14	0.74 (0.75)	0.74 (0.75)		0.51 (0.50)		0.30 (0.38)	0.45 (0.44)
6		0.48 (0.45)	0.69 (0.67)	0.69 (0.67)		0.23 (0.36)	0.52 (0.55)
7		0.47 (0.47)	0.74 (0.70)	0.74 (0.70)		0.25 (0.38)	0.46 (0.51)
8		0.51 (0.54)	0.72 (0.81)	0.72 (0.81)		0.24 (0.43)	0.48 (0.35)
9		0.50 (0.49)	0.67 (0.73)	0.67 (0.73)		0.23 (0.39)	0.55 (0.47)
16		0.47 0.60)	0.91 (0.90)	0.91 (0.90)		0.30 (0.48)	0.18 (0.19)
17		0.63 (0.60)	0.89 (0.90)	0.89 (0.90)		0.30 (0.48)	0.22 (0.18)
18		0.61 (0.49)	0.78 (0.73)	0.78 (0.73)		0.26 (0.39)	0.39 (0.47)
23		0.54 (0.51)	0.85 (0.77)	0.85 (0.77)		0.29 (0.41)	0.28 (0.41)
19		0.59 (0.36)		0.25 (0.38)	0.76 (0.71)	0.76 (0.71)	0.43 (0.50)
20		0.31 (0.40)		0.27 (0.42)	0.80 (0.79)	0.80 (0.79)	0.36 (0.38)
21		0.36 (0.38)		0.29 (0.39)	0.87 (0.74)	0.87 (0.74)	0.24 (0.46)
22		0.34 (0.42)		0.28 (0.44)	0.82 (0.82)	0.82 (0.82)	0.32 (0.33)
24		0.33 (0.44)		0.27 (0.46)	0.80 (0.87)	0.80 (0.87)	0.35 (0.24)
26		0.29 (0.36)		0.23 (0.37)	0.70 (0.70)	0.70 (0.70)	0.51 (0.51)
27		0.30 (0.35)		0.25 (0.36)	0.74 (0.68)	0.74 (0.68)	0.45 (0.54)

Note. Parameter estimates in parenthesis. Standardized solution reported.P = Pattern coefficients; S = Structure coefficients.

Table 4 reports descriptive statistics of scale scores in which students reported slightly higher than somewhat true in their behavioral engagement and slightly lower than somewhat true for their emotional engagement and disaffection. Subscale score coefficient alpha estimates ranged from .79 to .90 (EE), higher than previously reported estimates. BE scores were moderately correlated with EE scores (r = .59, p < .01) and low with ED scores (r = .29, p < .01), whereas EE and ED scores were moderately correlated (r = .40, p < .01). Although Skinner, Kindermann, & Furrer (2009) suggested that subscales could be used as distinct measures or combined to create aggregate scores, continued research is needed to further substantiate the value added by using scores in this manner. Nonetheless, combined with acceptable internal consistency reliability estimates, the scale seems suitable for use in research and educational settings. These findings further support the multidimensional structure of student engagement and the need for continued research to examine the structure of student engagement scores across diverse settings, students and their relationships to external variables.

Table 4.

Descriptive Information for Samples 1 and 2.

Subscale/scale	Item numbers	M	SD	α (95% CIs)
Subscale/scale	Item numbers	M	SD	Sample 1	Sample 2
Behavioral Engagement	1, 2, 4, 11, 12, 13, 14	3.33 (3.38)	0.48 (0.55)	.79 [0.76, 0.82]	.83 [0.81, 0.86]
Emotional Engagement	6, 7, 8, 9, 16, 17, 18, 23	2.91 (2.93)	0.72 (0.71)	.90 [0.89, 0.91]	.90 [0.88, 0.91]
Emotional Disaffection	19, 20, 21, 22, 24, 26, 27	2.95 (3.05)	0.69 (0.71)	.86 [0.84, 0.88]	.85 [0.83, 0.87]

Note. Values in parenthesis unless otherwise specified.α = Cronbach’s coefficient alpha; CI = confidence interval.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Abbott

A. L.

(2017). Fostering student interest development: An engagement intervention. Middle School Journal, 48, 34-45. doi:10.1080/00940771.2017.1297666

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Appleton

J. J.

Christenson

S. L.

Furlong

M. J.

(2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45, 369-386. doi:10.1002/pits.20303

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.

Flora

D. B.

Curran

P. J.

(2004). An empirical evaluation of alternative methods for estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466-491. doi:10.1037/1082-989X.9.4.466

Fredricks

J. A.

Blumenfeld

P. C.

Paris

A. H.

(2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74, 59-109.

Fredricks

J. A.

McColskey

(2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In Christenson

S. L.

Reschly

A. L.

Wylie

(Eds.), Handbook of research on student engagement (pp. 763-782). New York, NY: Springer Science+Business Media.

Fredricks

J. A.

McColskey

Meli

Mordica

Montrosse

Mooney

(2011). Measuring student engagement in upper elementary through high school: A description of 21 Instruments (Issues & answers, REL 2011-No. 098). Regional Educational Laboratory Southeast.

Henson

R. K.

Roberts

J. K.

(2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66, 393-416.

10.

Hughes

Kwok

(2007). Influence of student-teacher and parent-teacher relationships on lower achieving readers’ engagement and achievement in the primary grades. Journal of Educational Psychology, 99, 39-51.

11.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

12.

Meredith

(1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543.

13.

Muthen

du Toit

S. H. C.

Spisic

(1997). Robust inference using weighted least squares and quadratic estimating equation in latent variable modeling with categorical and continuous outcomes. Unpublished manuscript.

14.

Muthén

L. K.

Muthén

B. O.

(1998-2017). Mplus user’s guide (8th ed.). Los Angeles, CA: Author.

15.

Orthner

D. K.

Jones-Sanpei

Akos

Rose

R. A.

(2013). Improving middle school student engagement through career-relevant instruction in the core curriculum. The Journal of Educational Research, 106, 27-38. doi:10.1080/00220671.2012.658454

16.

Reschly

Christenson

S. L.

(2006). Prediction of dropout among students with mild disabilities: A case for the inclusion of student engagement variables. Remedial and Special Education, 27, 276-292.

17.

Revelle

Zinbarg

R. E.

(2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74, 145-154.

18.

Skinner

E. A.

Kindermann

T. A.

Connell

J. P.

Wellborn

J. G.

(2009). Engagement and disaffection as organizational constructs in the dynamics of motivational development. In Wenzel

K. R.

Wigfield

(Eds.), Handbook of motivation at school (pp. 223-245). New York, NY: Routledge.

19.

Skinner

E. A.

Kindermann

T. A.

Furrer

C. J.

(2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69, 493-525. doi:10.1177/0013164408323233

20.

Wang

Holcombe

(2010). Adolescents’ perceptions of school environment, engagement, and academic achievement in middle school. American Educational Research Journal, 47, 633-662.

21.

Willms

J. D.

(2003). Student engagement at school: A sense of belonging and participation. Paris, France: Organisation for Economic Co-Operation and Development.