A Comparison of Reliability Estimation Based on Confirmatory Factor Analysis and Exploratory Structural Equation Models

Abstract

Composite reliability, or coefficient omega, can be estimated using structural equation modeling. Composite reliability is usually estimated under the basic independent clusters model of confirmatory factor analysis (ICM-CFA). However, due to the existence of cross-loadings, the model fit of the exploratory structural equation model (ESEM) is often found to be substantially better than that of ICM-CFA. The present study first illustrated the method used to estimate composite reliability under ESEM and then compared the difference between ESEM and ICM-CFA in terms of composite reliability estimation under various indicators per factor, target factor loadings, cross-loadings, and sample sizes. The results showed no apparent difference in using ESEM or ICM-CFA for estimating composite reliability, and the rotation type did not affect the composite reliability estimates generated by ESEM. An empirical example was given as further proof of the results of the simulation studies. Based on the present study, we suggest that if the model fit of ESEM (regardless of the utilized rotation criteria) is acceptable but that of ICM-CFA is not, the composite reliability estimates based on the above two models should be similar. If the target factor loadings are relatively small, researchers should increase the number of indicators per factor or increase the sample size.

Keywords

confirmatory factor analysis composite reliability exploratory structural equation modeling

Under classical testing theory, the reliability coefficient was defined as the ratio of the true variance to the observed variance, and it is independent of the underlying dimensionality of the composite score (Lord & Novick, 1968; McDonald, 1985; Raykov & Shrout, 2002). Because the true variance is typically unknown, reliability is most frequently estimated by computing Cronbach’s coefficient alpha (Hogan et al., 2000). Coefficient alpha is equal to the reliability coefficient when (a) the measurement errors are uncorrelated and (b) the test is essentially tau equivalent. Essential tau equivalence assumes that each item measures the same latent variable on the same scale but with different precision rates (see Graham, 2006; Raykov, 1997). However, the above two assumptions are often violated in applied research, especially the second assumption. If assumption (b) is violated, coefficient alpha tends to underestimate reliability (Green & Yang, 2009; Novick & Lewis, 1967; Sijtsma, 2009; Zinbarg et al., 2005). On the other hand, the coefficient usually overestimates reliability if assumption (a) is violated (Zimmerman et al., 1993) but might also underestimate reliability (Zimmerman et al., 1993). We cannot stress enough that coefficient alpha is still valuable for applied research since it is easy to calculate and is a lower bound of reliability when the obtained measurement errors are not correlated (Raykov & Marcoulides, 2019). Nonetheless, it is safe to say that coefficient alpha is not an accurate estimator of reliability, especially under the condition of correlated errors. The comparison of the coefficient alpha and the population reliability under the conditions of the present experiment can be retrieved in the Supplemental Material (available online).

Alternatively, researchers have proposed a model-based approach to estimate the reliability of the composite score (e.g., Bentler, 2009; Raykov, 1998, 2002). This approach does not require the assumptions of essential tau equivalency and uncorrelated errors (Yang & Green, 2010); instead, it is based on an appropriately constrained covariance structure model that accounts for the possibility of multiple sources of latent variability (Raykov & Shrout, 2002). The model uses a specific parameterization to estimate the true and observed composite variances, the two fundamental elements of the scale reliability coefficient. The ratio of the true and observed variance furnishes an estimate of the composite score reliability. Thus, the estimation of the reliability coefficient under classical testing theory is realized. This ratio is the coefficient omega. As stated by McDonald (1985), the coefficient omega “captures the notion of the reliability of a test score” (p. 90). In this article, we use composite reliability to represent the coefficient omega.

The independent clusters model of confirmatory factor analysis (ICM-CFA) is commonly used to estimate composite reliability. It typically assumes the latent variables have concise factor structures without cross-factor loadings. A serious defect of this modeling approach is that ICM-CFA tends to be overly restrictive and often fails to provide a satisfactory model fit (e.g., Marsh, 2007; Morin et al., 2013); therefore, it causes biased reliability estimations (Yang & Green, 2010). An alternative is the exploratory structural equation modeling (ESEM) proposed by Asparouhov and Muthén (2009). ESEM is much more flexible than ICM-CFA in accommodating cross-loading specifications and is likely to produce a better model fit. ESEM is frequently selected as the best-suited measurement model (e.g., Marsh et al., 2009; Marsh et al., 2010; Tóth-Király et al., 2018), yet few studies have estimated reliability based on ESEM. Moreover, few studies have compared the performances of ICM-CFA and ESEM in terms of composite reliability estimation. The present study is among the first to provide a comprehensive comparison between ICM-CFA and ESEM approaches concerning reliability estimation based on simulated data.

Similar to the exploratory factor analysis (EFA) model, ESEM can perform factor rotation, which might be considered a severe disadvantage since the patterns of factor loadings and the sizes of estimated factor correlations vary with specific rotations (Marsh et al., 2014). Additionally, EFA gives researchers little control if they have a priori models in mind. Marsh et al. (2013) argued that this problem is circumvented, at least to some extent, by target rotation in ESEM. Target rotation allows researchers to set the factor model in advance, for example, the factor number and item’s target factor, as in ICM-CFA. The difference between ICM-CFA and target rotation in ESEM is that target rotation does not need nontarget factor loadings (or cross-factor loadings) to be fixed to zero. The cross-factor loadings are made to be as close to zero as possible, but they are not constrained to zero (Asparouhov & Muthén, 2009; Marsh et al., 2013). Target rotation gives researchers greater control in specifying the model and facilitates the interpretation of the obtained results (Marsh et al., 2014).

Nevertheless, there are various rotation methods in ESEM in addition to target rotation. In particular, what if there is no a priori model to specify? Geomin rotation might be the answer. Geomin rotation in ESEM is similar to EFA in that it gives only the number of factors but no a priori information regarding the factor structure of the model. However, how can we choose between geomin rotation and target rotation? Since factor rotations are independent of goodness of fit, different kinds of factor rotations result in the same model fit index. Therefore, the goodness-of-fit indices provide no evidence for choosing the best rotation (Marsh et al., 2014; Sass & Schmitt, 2010; Schmitt & Sass, 2011). A simulation study showed that geomin rotation is the most promising rotation criterion when little is known about the true loading structure or when the factor structure is rather simple (Asparouhov & Muthén, 2009; Marsh et al., 2014). However, suppose the factor structure is rather complicated (i.e., the loading matrix structures involve three or more factors and each variable has three or more nonzero loadings). In that case, the target rotation criterion leads to better results (Asparouhov & Muthén, 2009). Therefore, the present study also discusses the performance of target rotation when geomin rotation is not accurate for reliability estimation.

Reliability Estimation Based on Exploratory Structural Equation Model

A hidden assumption when using ESEM as a measurement model is multidimensionality. As long as there is only one factor, the measurement model of ESEM is the same as that of ICM-CFA due to the absence of cross-loadings. Assume there are p items $x_{1}, x_{2}, \dots, x_{p}$ for measuring n latent variables $ξ_{1}, ξ_{2}, \dots, ξ_{n}$ , where $δ_{1}, δ_{2}, \dots, δ_{p}$ are measurement errors. The measurement equations for ICM-CFA are (e.g., Jöreskog, 1971)

x_{j} = \sum_{m = 1}^{n} λ_{jm} ξ_{m} + δ_{j}, j = 1, 2, \dots, p,

(1)

where $λ_{jm}$ is the factor loading of item j on $ξ_{m}$ , and it is zero when item j is not the indicator of $ξ_{m}$ . The reliability of the composite score (suppose the item scores can be added together) $X = x_{1} + x_{2} + \dots + x_{p}$ is (McDonald, 1985; Raykov & Shrout, 2002)

ρ_{com} = \frac{var (\sum_{j = 1}^{p} \sum_{m = 1}^{n} λ_{jm} ξ_{m})}{var (\sum_{j = 1}^{p} \sum_{m = 1}^{n} λ_{jm} ξ_{m}) + \sum_{j = 1}^{p} var (δ_{j})} .

(2)

If cross-factor loadings exist, then the measurement equations are the same as in Equation 1, but there is no constraint on $λ_{jm}$ . Thus, the reliability of the composite score $X = x_{1} + x_{2} + \dots + x_{p}$ is the same as Equation 2 under ESEM structure. The difference between ICM-CFA and ESEM when estimating composite reliability is the existence of cross-factor loadings in the latter. For example, suppose the target factor of the first item is $ξ_{1}$ and the target factor of the fourth item is $ξ_{2}$ . In ICM-CFA, $λ_{12} = λ_{41} = 0$ , while in ESEM, $λ_{12} \neq 0$ and $λ_{41} \neq 0$ .

Here, we illustrate the estimation of composite reliability based on ESEM using a simulated data set. To this end, multinormal data were generated for N = 1,000 cases of p = 6 items, $x_{1}$ to $x_{6}$ , in Mplus 8.3 according to the following model:

\begin{matrix} x_{1} = 0.5 ξ_{1} + δ_{1}, \\ x_{2} = 0.6 ξ_{1} + δ_{2}, \\ x_{3} = 0.4 ξ_{1} + 0.3 ξ_{2} + δ_{3}, \\ x_{4} = 0.2 ξ_{1} + 0.55 ξ_{2} + δ_{4}, \\ x_{5} = 0.5 ξ_{2} + δ_{5}, \\ x_{6} = 0.4 ξ_{2} + δ_{6}, \end{matrix}

(3)

where the two latent constructs $ξ_{1}$ and $ξ_{2}$ evaluated by the battery of six items were simulated to have unitary variances and a correlation of $ρ = corr (ξ_{1}, ξ_{2}) = 0.3$ . The standard error (SE) deviations were computed as follows (Marsh et al., 2013):

var (δ_{i}) = 1 - [λ_{i 1}^{2} + λ_{i 2}^{2} + 2 (λ_{i 1} \times λ_{i 2} \times ρ)] .

(4)

Thus, the values of $var (δ_{1}), var (δ_{2}), \dots, var (δ_{6})$ were 0.75, 0.64, 0.68, 0.59, 0.75, and 0.84, respectively. The composite reliability of these six items according to Equation 2 is

\begin{matrix} ρ_{com_ESEM} = var [(0.5 + 0.6 + 0.4 + 0.2) ξ_{1} + (0.3 + 0.55 + 0.5 + 0.4) ξ_{2}] / \\ {var [(0.5 + 0.6 + 0.4 + 0.2) ξ_{1} + (0.3 + 0.55 + 0.5 + 0.4) ξ_{2}] \\ + 0.75 + 0.64 + 0.68 + 0.59 + 0.75 + 0.84} \end{matrix}

\begin{matrix} = (1 . 7^{2} + 1 . 75^{2} + 2 \times 0.3 \times 1.7 \times 1.75) / \\ [(1 . 7^{2} + 1 . 75^{2} + 2 \times 0.3 \times 1.7 \times 1.75) + 4.25] \\ \approx 0.645 \end{matrix}

If ESEM adopts geomin rotation, we are able to estimate the composite reliability in Mplus (see the appendix). We estimated the composite reliability on the simulated data set, and the outcome was close to the population value. For target rotation, the composite reliability can be computed manually based on the Mplus output parameters.

Design of Simulation 1

A Monte Carlo simulation was used to compare composite reliability estimations based on ICM-CFA and ESEM with geomin rotation (ESEM_Geo). The population model contained two factors ( $F_{1}$ , $F_{2}$ ), and the factor correlation was 0.3. The mean and variance of each factor were set to be 0 and 1, respectively. The variance of the measurement error was calculated using Equation 4. The measurement errors were not correlated with the factors or with each other. The population model did not contain any intercepts. For each factor, two thirds of the target items have cross-loadings on the other factor. For example, if factor $F_{1}$ had three target items, then two items had cross-loadings on $F_{2}$ . Likewise, factor $F_{1}$ had six target items then four items had cross-loadings on $F_{2}$ .

The simulation experiment had a 2 × 3 × 3 × 3 factorial design with four design conditions. The four design conditions were fully crossed, creating 54 (= 2 × 3 × 3 × 3) unique cell conditions. Within each of the 54 unique cell conditions, two models were fitted to each simulated sample data set: ICM-CFA model and ESEM model based on geomin rotation. The details of these design conditions and their levels are described as follows:

Number of items per factor (NI). The number of items with the same target factor was either three or six. Therefore, the number of cross-loadings per factor was either two or four.

The value of the item loadings on the target factor (Lt). The target factor loadings were set to be above the threshold value of 0.3. The standardized values of the target factor loadings were 0.4, 0.6, and 0.8, representing low, medium, and high target factor loadings, respectively.

The value of the cross-factor loadings (Lc). One of the advantages of ESEM is that small cross-loadings do not need to be eliminated from the model (Muthén & Asparouhov, 2012). In the present study, the standardized values of the cross-loadings were 0.01, 0.1, and 0.25.

The simulated sample size (N) conditions were 100, 300, and 500. We used 100 as the minimum sample size following the suggestion by Boomsma (1982) for latent variable models.

Within each unique design condition, 500 sample data sets were generated based on a set of specified population parameters. Each simulated sample data set was fitted into two models: ICM-CFA and ESEM_Geo. We used geomin rotation but not target rotation in the comparison because the Mplus software could estimate the composite reliability directly based on geomin rotation. If the performance of ESEM_Geo was not good, we compared the composite reliability estimations of ICM-CFA and ESEM based on target rotation. When using target rotation, researchers should manually compute the composite reliability based on the Mplus output.

For each unique combination of the four design conditions (1-4 as previously outlined), Mplus 8.3 was used to generate 500 sample data sets under the population model. For each sample data set generated under the population model, both ICM-CFA and ESEM_Geo were fitted to the sample data. The R program (R 3.5.1) was used to extract the results provided by the Mplus output files.

Results of Simulation 1

The estimation of composite reliability was based on the chosen model parameters. Therefore, we needed the model to converge with proper solutions (e.g., no estimated variance term could be negative) and to fit the data properly. In the present study, three criteria were employed for comparison purposes: the rate of model convergence with proper solutions, the goodness of fit, and the relative bias of the composite reliability estimates.

Fully Proper Solutions

First, we evaluated the proportion of fully proper solutions for each cell of the design (the estimation procedure converged to a proper solution such that no estimated variance term and no SE of a parameter were negative). The fully proper solution rate (% proper) was calculated as the number of convergent replications having proper solutions divided by the total number of replications. Only the fully proper solutions were considered in evaluating the goodness of fit and reliability estimates.

The results showed that the fully proper solutions of ICM-CFA were generally higher than those of ESEM_Geo as depicted in Table 1. If the target factor loading was low (Lt = 0.4) and the cross-loading was high (Lc = 0.25), the fully proper solutions of both models were somewhat low. For example, when NI = 3, Lt = 0.4, Lc = 0.25, and N = 500, the fully proper solution rate of ICM-CFA was 0.894 and that of ESEM_Geo was only 0.648.

Table 1.

Simulation 1: Convergence-With-Proper-Solution and Acceptable Fit Rate and Goodness of Fit.

NI	N	Lt	Lc	ESEM_Geo					ICM-CFA
NI	N	Lt	Lc	% Proper	Acceptable Fit	CFI	TLI	RMSEA	% Proper	Acceptable Fit	CFI	TLI	RMSEA
3	100	0.4	0.01	0.448	0.416	0.992	2.049	0.007	0.544	0.400	0.953	1.563	0.019
		0.4	0.1	0.426	0.522	0.992	1.411	0.008	0.726	0.378	0.950	1.186	0.024
		0.4	0.25	0.410	0.510	0.994	1.143	0.009	0.698	0.370	0.961	0.993	0.032
		0.6	0.01	0.906	0.786	0.989	1.028	0.024	0.964	0.706	0.978	1.003	0.028
		0.6	0.1	0.914	0.798	0.991	1.021	0.025	0.988	0.714	0.976	0.983	0.035
		0.6	0.25	0.824	0.656	0.995	1.016	0.023	0.996	0.700	0.972	0.957	0.055
		0.8	0.01	1.000	0.832	0.995	1.000	0.031	1.000	0.824	0.992	0.997	0.030
		0.8	0.1	1.000	0.706	0.996	1.000	0.031	1.000	0.830	0.988	0.982	0.051
		0.8	0.25	0.998	0.130	0.997	1.000	0.030	0.998	0.822	0.967	0.939	0.127
	300	0.4	0.01	0.836	0.786	0.990	1.094	0.009	0.954	0.670	0.976	1.033	0.012
		0.4	0.1	0.748	0.824	0.994	1.062	0.009	0.986	0.648	0.975	0.992	0.017
		0.4	0.25	0.546	0.780	0.998	1.035	0.007	0.858	0.514	0.982	0.981	0.023
		0.6	0.01	1.000	0.988	0.995	1.001	0.016	1.000	0.872	0.993	1.000	0.014
		0.6	0.1	1.000	0.980	0.996	1.001	0.016	1.000	0.894	0.989	0.984	0.027
		0.6	0.25	0.994	0.798	0.998	1.002	0.016	1.000	0.958	0.979	0.960	0.057
		0.8	0.01	1.000	1.000	0.999	1.000	0.017	1.000	0.976	0.998	1.000	0.015
		0.8	0.1	1.000	0.882	0.999	1.000	0.017	1.000	0.974	0.991	0.984	0.050
		0.8	0.25	1.000	0.012	0.999	1.000	0.017	1.000	0.976	0.968	0.941	0.131
	500	0.4	0.01	0.940	0.854	0.990	1.021	0.011	0.996	0.708	0.980	1.002	0.012
		0.4	0.1	0.886	0.858	0.994	1.022	0.010	0.998	0.742	0.979	0.980	0.017
		0.4	0.25	0.648	0.842	0.998	1.018	0.007	0.894	0.616	0.986	0.980	0.022
		0.6	0.01	1.000	1.000	0.997	1.000	0.014	1.000	0.928	0.995	0.999	0.013
		0.6	0.1	1.000	0.998	0.998	1.000	0.014	1.000	0.962	0.989	0.982	0.028
		0.6	0.25	0.998	0.858	0.999	1.000	0.013	1.000	0.994	0.979	0.960	0.060
		0.8	0.01	1.000	1.000	0.999	1.000	0.014	1.000	1.000	0.999	1.000	0.013
		0.8	0.1	1.000	0.918	0.999	1.000	0.014	1.000	0.998	0.991	0.984	0.054
		0.8	0.25	1.000	0.000	1.000	1.000	0.014	1.000	0.998	0.968	0.940	0.132
6	100	0.4	0.01	0.852	0.620	0.947	1.036	0.019	0.980	0.508	0.924	0.988	0.022
		0.4	0.1	0.770	0.620	0.965	1.031	0.017	0.992	0.514	0.930	0.964	0.025
		0.4	0.25	0.634	0.632	0.984	1.030	0.013	0.850	0.508	0.951	0.958	0.030
		0.6	0.01	1.000	0.938	0.979	0.985	0.025	1.000	0.724	0.977	0.986	0.024
		0.6	0.1	1.000	0.950	0.982	0.987	0.025	1.000	0.778	0.975	0.977	0.029
		0.6	0.25	0.996	0.934	0.988	0.991	0.024	1.000	0.876	0.966	0.960	0.046
		0.8	0.01	1.000	0.988	0.993	0.994	0.025	1.000	0.974	0.992	0.995	0.024
		0.8	0.1	1.000	0.974	0.994	0.995	0.025	1.000	0.988	0.988	0.987	0.036
		0.8	0.25	1.000	0.424	0.995	0.996	0.025	1.000	0.986	0.969	0.962	0.082
	300	0.4	0.01	0.998	0.890	0.980	0.999	0.011	1.000	0.742	0.975	0.994	0.012
		0.4	0.1	0.978	0.918	0.986	1.002	0.010	1.000	0.812	0.974	0.981	0.015
		0.4	0.25	0.858	0.958	0.994	1.006	0.008	0.972	0.832	0.979	0.978	0.020
		0.6	0.01	1.000	1.000	0.994	0.998	0.012	1.000	0.986	0.993	0.997	0.012
		0.6	0.1	1.000	1.000	0.995	0.998	0.012	1.000	0.992	0.989	0.988	0.020
		0.6	0.25	1.000	1.000	0.997	0.999	0.012	1.000	0.998	0.974	0.967	0.045
		0.8	0.01	1.000	1.000	0.998	0.999	0.012	1.000	1.000	0.998	0.999	0.012
		0.8	0.1	1.000	1.000	0.998	0.999	0.012	1.000	1.000	0.993	0.991	0.032
		0.8	0.25	1.000	0.462	0.999	0.999	0.012	1.000	1.000	0.972	0.965	0.080
	500	0.4	0.01	1.000	0.984	0.988	0.998	0.009	1.000	0.850	0.985	0.996	0.009
		0.4	0.1	0.998	0.992	0.991	0.999	0.008	1.000	0.916	0.982	0.983	0.013
		0.4	0.25	0.946	0.994	0.996	1.001	0.008	0.994	0.940	0.982	0.979	0.020
		0.6	0.01	1.000	1.000	0.997	0.999	0.009	1.000	1.000	0.996	0.999	0.009
		0.6	0.1	1.000	1.000	0.997	0.999	0.009	1.000	1.000	0.991	0.989	0.020
		0.6	0.25	1.000	1.000	0.998	0.999	0.009	1.000	1.000	0.975	0.968	0.045
		0.8	0.01	1.000	1.000	0.999	1.000	0.009	1.000	1.000	0.999	0.999	0.009
		0.8	0.1	1.000	1.000	0.999	1.000	0.009	1.000	1.000	0.993	0.991	0.033
		0.8	0.25	1.000	0.440	0.999	1.000	0.009	1.000	1.000	0.972	0.965	0.080

Note. NI = number of items per factor; N = sample size; Lt = target factor loadings; Lc = cross-factor loadings; ESEM_Geo = exploratory structural equation model with geomin rotation; ICM-CFA = independent clusters model of confirmatory factor analysis; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation.

Furthermore, the fully proper solutions of both models increased as the sample size increased. If the sample size was small (N = 100), the fully proper solutions of both models were rather low, with a small number of items (NI = 3) and a low target factor loading (Lt = 0.4). For example, when NI = 3, Lt = 0.4, Lc = 0.01, and N = 100, the fully proper solution rate of ICM-CFA was 0.544 and that of ESEM_Geo was 0.448.

Goodness of Fit

The indices for goodness of fit used in the present study included widely used model fit indices, such as the comparative fit index (CFI), Tucker–Lewis index (TLI), and root mean square error of approximation (RMSEA). Following the decision rules of Hu and Bentler (1999) and Marsh et al. (2004), model fitness was considered acceptable when CFI > 0.9, TLI > 0.9, and RMSEA < 0.08 and good when CFI > 0.95, TLI > 0.95, and RMSEA < 0.06. Furthermore, we used the above indices to compare the performances of two models under the same design conditions, since ICM-CFA was nested within ESEM_Geo.

As depicted in Table 1, ESEM_Geo performed better than ICM-CFA in terms of model fit. More specifically, ESEM_Geo fitted the data well in all conditions, with the CFI and TLI being greater than 0.95 and the RMSEA being lower than 0.04. In contrast, the model fit of ICM-CFA was mostly acceptable. If the target factor loading and cross-loading were high (e.g., Lt = 0.8 and Lc = 0.25), the RMSEA of ICM-CFA was usually larger than 0.08.

It is worth noting that the model fits of ICM-CFA and ESEM were noticeably different, even though the mean values of the model fitness indices appeared to be very similar. “Acceptable fit” in Table 1 was calculated by the number of the model fitness values that reached the standards of CFI > 0.9, TLI > 0.9, and RMSEA < 0.08 divided by 500. We calculated the composite reliability as long as the model converged to a proper solution, regardless of model fitness. The numbers of models that reached the level of acceptable fit were substantially different between the two model types. For example, when NI = 3, N = 100, Lt = 0.8, and Lc = 0.25, the proportion of models that reached acceptable fit for ICM-CFA was 0.130 while that of ESEM was 0.822.

Relative Bias of the Composite Reliability Estimates

The relative bias of each estimation was calculated by subtracting the true value from the average of estimates and then dividing the result by the true value:

Bias (\hat{θ}) = \frac{\bar{\hat{θ}} - θ}{θ},

where $\bar{\hat{θ}}$ represents the average of the parameter estimates in each condition and $θ$ is the population parameter (Gu et al., 2017). A relative bias, that is, $Bias (\hat{θ})$ , less than 5% could be considered negligible (Hoogland & Boomsma, 1998), and a bias less than 10% could be acceptable (Bandalos, 2002; Reise et al., 2013).

Table 2 reports the relative biases of the composite reliability estimates based on the two models and their mean differences when cross-loadings existed. In general, the relative biases of the composite reliability estimates were less than 5% under most conditions, especially when the target factor loading was rather high (e.g., Lt = 0.6 or 0.8). However, when the target factor loading and cross-loading were low (Lt = 0.4 and Lc = 0.01) and the sample size was small (N = 100), the relative biases of the composite reliability estimates based on both models were positive, indicating overestimation.

Table 2.

Simulation 1: Relative Bias of Composite Reliability and Mean Differences Between Two Models.

N	Lt	Lc	NI = 3				NI = 6
N	Lt	Lc	CR_P	Bias_ESEM_Geo	Bias_ICM-CFA	µ	CR_P	Bias_ESEM_Geo	Bias_ICM-CFA	µ
100	0.4	0.01	0.435	14.3	12.7	0.007	0.602	2.1	0.1	0.012
	0.4	0.1	0.510	10.0	5.3	0.024	0.642	6.3	4.6	0.011
	0.4	0.25	0.623	6.3	0.8	0.034	0.706	9.7	8.1	0.011
	0.6	0.01	0.692	1.9	0.7	0.009	0.817	0.4	0.0	0.003
	0.6	0.1	0.740	1.5	0.1	0.010	0.837	1.7	1.4	0.002
	0.6	0.25	0.810	1.1	−0.5	0.013	0.871	2.8	2.4	0.003
	0.8	0.01	0.877	0.0	−0.2	0.002	0.934	0.1	0.0	0.001
	0.8	0.1	0.901	0.0	−0.2	0.002	0.944	0.4	0.4	0.001
	0.8	0.25	0.939	0.0	−0.4	0.003	0.962	0.6	0.5	0.001
300	0.4	0.01	0.435	5.4	2.7	0.012	0.602	1.5	1.0	0.003
	0.4	0.1	0.510	4.5	0.8	0.019	0.642	6.0	5.4	0.003
	0.4	0.25	0.623	3.6	0.0	0.023	0.706	9.4	8.5	0.006
	0.6	0.01	0.692	0.5	0.1	0.002	0.817	0.4	0.3	0.001
	0.6	0.1	0.740	0.4	0.1	0.002	0.837	1.7	1.7	0.001
	0.6	0.25	0.810	0.4	−0.3	0.005	0.871	2.8	2.6	0.002
	0.8	0.01	0.877	0.1	0.0	0.001	0.934	0.1	0.1	0.000
	0.8	0.1	0.901	0.1	0.0	0.001	0.944	0.4	0.4	0.000
	0.8	0.25	0.939	0.0	−0.3	0.003	0.962	0.6	0.5	0.001
500	0.4	0.01	0.435	3.5	1.4	0.010	0.602	1.3	1.0	0.002
	0.4	0.1	0.510	2.9	0.6	0.012	0.642	5.7	5.4	0.002
	0.4	0.25	0.623	2.3	−0.2	0.016	0.706	9.2	8.5	0.005
	0.6	0.01	0.692	0.4	0.2	0.001	0.817	0.3	0.3	0.000
	0.6	0.1	0.740	0.3	0.1	0.001	0.837	1.7	1.7	0.000
	0.6	0.25	0.810	0.3	−0.3	0.004	0.871	2.8	2.6	0.001
	0.8	0.01	0.877	0.1	0.0	0.000	0.934	0.1	0.1	0.000
	0.8	0.1	0.901	0.1	0.0	0.001	0.944	0.4	0.4	0.000
	0.8	0.25	0.939	0.0	−0.3	0.003	0.962	0.6	0.5	0.001

Note. NI = number of items per factor; N = sample size; Lt = target factor loadings; Lc = cross-factor loadings; ICM-CFA = independent clusters model of confirmatory factor analysis; ESEM_Geo = exploratory structural equation model with geomin rotation; ESEM_Tgt = Target rotation in ESEM

Under the same conditions, the mean differences in composite reliability (see $μ$ in Table 2) fell in the interval between 0.000 and 0.034, and this meant that the differences were negligible.

Standard Error and Mean Square Error

The SE was calculated by $SE = \sqrt{E {(\hat{θ} - \bar{\hat{θ}})}^{2}}$ , where $\hat{θ}$ represents the estimated value within each replication. The mean square error (MSE) was calculated by $MSE = E (\hat{θ} - θ)^{2}$ . The smaller the SE and the MSE the better (Yuan & MacKinnon, 2009). Table 3 shows the difference of the SE (×10,000) and the MSE (×10,000). First, the SE and the MSE of both models are very small, which shows the estimation of composite reliability based on both models is rather stable. The SE and the MSE decrease as sample size, number of indicators, target factor loadings, and cross-factor loadings increase. Under the condition of low target factor loading (Lt = 0.4) and small sample size (N = 100 and 300), the SE and the MSE are relatively high. This pattern is similar to the relative bias of composite reliability estimation.

Table 3.

The Standard Errors (×10,000) and Mean Square Errors (×10,000) of Stimulation Study 1.

N	Lt	Lc	NI = 3				NI = 6
			CFA		ESEM_Geo		CFA		ESEM_Geo
			SE	MSE	SE	MSE	SE	MSE	SE	MSE
100	0.4	0.01	33.5	86.5	33.4	94.0	26.5	35.2	25.9	34.3
	0.4	0.1	29.0	49.3	30.6	72.6	21.9	24.1	20.6	21.7
	0.4	0.25	25.8	33.3	21.7	39.0	15.0	11.4	14.7	11.2
	0.6	0.01	20.6	21.5	20.5	22.8	11.7	6.8	11.5	6.6
	0.6	0.1	18.1	16.3	17.6	16.6	9.6	4.6	9.5	4.5
	0.6	0.25	13.7	9.6	13.2	9.4	6.7	2.3	6.7	2.3
	0.8	0.01	9.1	4.2	9.0	4.0	4.2	0.9	4.1	0.9
	0.8	0.1	7.4	2.7	7.3	2.6	3.3	0.5	3.3	0.5
	0.8	0.25	4.7	1.3	4.6	1.1	2.0	0.2	2.0	0.2
300	0.4	0.01	20.9	23.3	21.4	28.5	14.7	10.8	14.5	10.8
	0.4	0.1	19.0	18.2	18.5	22.4	11.9	7.1	11.8	7.2
	0.4	0.25	14.7	10.8	14.2	15.1	8.5	3.6	8.2	3.6
	0.6	0.01	12.2	7.4	12.0	7.3	6.8	2.3	6.8	2.3
	0.6	0.1	10.3	5.3	10.2	5.3	5.5	1.5	5.5	1.5
	0.6	0.25	7.7	3.0	7.5	2.9	3.8	0.7	3.8	0.7
	0.8	0.01	4.9	1.2	4.9	1.2	2.5	0.3	2.5	0.3
	0.8	0.1	4.0	0.8	4.0	0.8	1.9	0.2	1.9	0.2
	0.8	0.25	2.6	0.4	2.5	0.3	1.2	0.1	1.2	0.1
500	0.4	0.01	17.3	15.3	17.8	18.2	11.3	6.4	11.3	6.5
	0.4	0.1	15.2	11.6	15.7	14.5	9.4	4.4	9.3	4.5
	0.4	0.25	11.8	7.0	12.2	9.5	6.8	2.3	6.7	2.4
	0.6	0.01	9.7	4.7	9.7	4.7	5.2	1.3	5.2	1.4
	0.6	0.1	8.2	3.4	8.2	3.4	4.3	0.9	4.3	0.9
	0.6	0.25	6.1	1.9	6.0	1.9	3.0	0.5	3.0	0.5
	0.8	0.01	4.0	0.8	4.0	0.8	1.9	0.2	1.9	0.2
	0.8	0.1	3.2	0.5	3.2	0.5	1.5	0.1	1.5	0.1
	0.8	0.25	2.0	0.3	2.0	0.2	0.9	0.0	0.9	0.0

Note. NI = number of items per factor; N = sample size; Lt = target factor loadings; Lc = cross-factor loadings. CFA = confirmatory factor analysis; SE = standard error; MSE = mean square error; ESEM_Geo = exploratory structural equation model with geomin rotation.

Design of Simulation 2

The results of Simulation 1 showed that the performance of composite reliability estimation based on ESEM_Geo was unsatisfactory when the target factor loading was low (e.g., Lt = 0.4) and the sample size was small (e.g., N = 100, 300). To be precise, both ICM-CFA and ESEM_Geo tend to overestimate reliability under the above conditions. Target rotation in ESEM (ESEM_Tgt) provides a more robust a priori model, gives the researcher greater control in specifying the model, and facilitates the interpretation of the results (Marsh et al., 2014). Moreover, different rotation methods lead to different factor loading matrices and factor correlation matrices (e.g., Asparouhov & Muthén, 2009; Schmitt & Sass, 2011), which are critical elements for estimating composite reliability. Simulation 2 investigated whether the estimation of composite reliability based on ESEM_Tgt would be more precise than that based on ESEM_Geo with a low target factor loading (e.g., Lt = 0.4) and a small sample size (e.g., N = 100, 300).

Simulation 2 was a follow-up study of Simulation 1, so their designs and conditions were similar. Suppose the population model contained two factors ( $F_{1}$ , $F_{2}$ ) and the factor correlation was 0.3. We replicated the mean and variance settings for each factor and the measurement error from their values in Simulation Study 1. The target factor loadings were all set to 0.4. Within each factor, the targeted items had items totaling two thirds of the number of targeted items cross-loaded on the other factor.

The simulation experiment had a 2 × 3 × 2 factorial design with three design conditions. The three design conditions were fully crossed, creating 12 (2 × 3 × 2 = 12) unique cell conditions. Within each of the 12 unique cell conditions, two models were fitted to each simulated sample data set: ESEM_Geo and ESEM_Tgt. The details of these design conditions and their levels are described below.

Number of items per factor (NI). The number of items with the same target factor was either 3 or 6.

The value of the cross-factor loadings (Lc). The standardized values of the cross-loadings were 0.01, 0.1, and 0.25.

The simulated sample size (N) conditions were 100 and 300.

Within each unique design condition, 500 sample data sets were generated based on a set of specified population parameters. Each simulated sample data set was fitted by different rotations in ESEM: ESEM_Geo and ESEM_Tgt. As in Simulation 1, Mplus 8.3 and the R program (R 3.5.1) were used for data generation and analysis.

Results of Simulation 2

Fully Proper Solutions and Goodness of Fit

For the convenience of comparison, we show ESEM_Geo portion of the results from Simulation 1 in Tables 4 and 5. As depicted in Table 4, the fully proper solutions of ESEM_Tgt were slightly lower than those of ESEM_Geo. As expected, the goodness-of-fit indices were almost the same for the geomin and target rotations. In other words, the different rotations fit the data equally well.

Table 4.

Simulation 2: Convergence-With-Proper-Solution Rate and Goodness of Fit.

NI	N	Lt	Lc	ESEM_Geo				ESEM_Tgt
NI	N	Lt	Lc	% Proper	CFI	TLI	RMSEA	% Proper	CFI	TLI	RMSEA
3	100	0.4	0.01	0.448	0.992	2.049	0.007	0.442	0.992	2.059	0.007
		0.4	0.1	0.426	0.992	1.411	0.008	0.416	0.992	1.417	0.008
		0.4	0.25	0.410	0.994	1.143	0.009	0.404	0.994	1.140	0.009
	300	0.4	0.01	0.836	0.990	1.094	0.009	0.828	0.990	1.089	0.010
		0.4	0.1	0.748	0.994	1.062	0.009	0.748	0.994	1.062	0.009
		0.4	0.25	0.546	0.998	1.035	0.007	0.542	0.998	1.035	0.007
6	100	0.4	0.01	0.852	0.947	1.036	0.019	0.852	0.947	1.036	0.019
		0.4	0.1	0.770	0.965	1.031	0.017	0.770	0.965	1.031	0.017
		0.4	0.25	0.634	0.984	1.030	0.013	0.628	0.985	1.031	0.013
	300	0.4	0.01	0.998	0.980	0.999	0.011	0.998	0.980	0.999	0.011
		0.4	0.1	0.978	0.986	1.002	0.010	0.978	0.986	1.002	0.010
		0.4	0.25	0.858	0.994	1.006	0.008	0.860	0.995	1.006	0.008

Note. NI = number of items per factor; N = sample size; Lt = target factor loadings; Lc = cross-factor loadings; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; ESEM_Geo = exploratory structural equation model with geomin rotation; ESEM_Tgt = Target rotation in ESEM.

Table 5.

Simulation 2: Relative Bias of Composite Reliability, Standard Error (×10,000) and Mean Square Error (×10,000) Based on ESEM_Geo and ESEM_Tgt.

N	Lt	Lc	NI = 3						NI = 6
			ESEM_Geo			ESEM_Tgt			ESEM_Geo			ESEM_Tgt
			Bias	SE	MSE	Bias	SE	MSE	Bias	SE	MSE	Bias	SE	MSE
100	0.4	0.01	14.3	33.4	94.0	14.3	33.5	94.5	2.1	25.9	34.3	2.1	25.9	34.2
	0.4	0.1	10.0	30.6	72.6	9.9	30.3	71.1	6.3	20.6	21.7	6.3	20.6	21.7
	0.4	0.25	6.3	21.7	39.0	6.5	21.3	39.3	9.7	14.7	11.2	9.6	14.7	11.2
300	0.4	0.01	5.4	21.4	28.5	5.5	21.3	28.4	1.5	14.5	10.8	1.5	14.5	10.8
	0.4	0.1	4.5	18.5	22.4	4.5	18.5	22.4	6.0	11.8	7.2	6.0	11.8	7.2
	0.4	0.25	3.6	14.2	15.1	3.6	14.2	15.0	9.4	8.2	3.6	9.4	8.2	3.6

Note. NI = number of items per factor; N = sample size; Lt = target factor loadings; Lc = cross-factor loadings; ICM-CFA = independent clusters model of confirmatory factor analysis; MSE = mean square error; ESEM_Geo = exploratory structural equation model with geomin rotation; ESEM_Tgt = Target rotation in ESEM.

Relative Bias of the Composite Reliability Estimates, Standard Error, and Mean Square Error

The patterns of the relative biases of the composite reliability estimates, the SE and the MSE were similar between different rotation methods under all research conditions, as shown in Table 5. When the target factor loadings were low and sample sizes were small, compared with geomin rotation, target rotation neither improved nor deteriorated the precision of the composite reliability estimates.

An Empirical Example

An example is analyzed to illustrate the difference between ICM-CFA and ESEM regarding model fit and composite reliability estimation. The example concerns a child-rated measurement regarding their parents’ psychological control (Wang et al., 2007). The Parents’ Psychological Control Scale has three dimensions: (1) guilt induction (10 items, e.g., “My parents tell me that I should feel guilty when I do not meet their expectations”); (2) love withdrawal (five items, e.g., “My parents act cold and unfriendly if I do something they do not like”); and (3) authority assertion (three items, e.g., “My parents tell me that what they want me to do is the best for me and I should not question it.’”). The above 18 items were rated between 1 (not at all true) and 5 (very true). A total of 2,084 children (including 1,015 boys, 1,018 girls, and 51 unidentified by gender) were asked to complete the questionnaire in a junior high school in China. We first fitted the data with both ICM-CFA and ESEM models and then calculated the composite reliability using the Mplus software (except for the composite reliability of ESEM_Tgt). The results are shown in Table 6.

Table 6.

The Model Fitness and Their Composite Reliability of The Example.

Model	χ²/df	CFI	TLI	RMSEA	Composite reliability
ICM-CFA	20.511	0.862	0.836	0.097	.937
ESEM_Geo	12.829	0.934	0.900	0.075	.937
ESEM_Tgt	12.829	0.934	0.900	0.075	.937

Note. ICM-CFA = independent clusters model of confirmatory factor analysis; ESEM_Geo = exploratory structural equation model with geomin rotation; ESEM_Tgt = Target rotation in ESEM; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; df = degrees of freedom.

ICM-CFA model did not fit the data well, as the CFI and TLI were below 0.9 but above 0.8, and the RMSEA was above 0.08. On the other hand, the model fit of ESEM reached an acceptable level, as the CFI and TLI were above 0.9, and the RMSEA was below 0.08. As expected, the rotation method did not affect the model fit for ESEM. Although the model fit of ICM-CFA and ESEM were different, the composite reliability estimates based on these two models were the same.

Discussion

ESEM has been widely used in applied research in recent years (e.g., Chen et al., 2018; Tóth-Király et al., 2017). The main goals of the present study are (a) to provide a clear estimation process for composite reliability based on ESEM and (b) to demonstrate the differences in the composite reliability estimations yielded by ESEM and ICM-CFA when cross-loadings exist. We achieved the first goal by providing applied researchers with the Mplus codes to estimate reliability under ESEM_Geo and the computation process for ESEM_Tgt at the beginning of this article.

Then, we compared ESEM_Geo and ICM-CFA with respect to composite reliability estimation in the first simulation study. The results showed that the differences were negligible, and this was beyond our expectations since cross-loadings were taken into account in ESEM but were not in ICM-CFA. One of the possible reasons for this is that SEM tended to explain the data variances maximally. For example, one simulation study showed that ICM-CFA often overestimates target factor loadings and factor correlations when cross-loadings exist in a population model (Xiao et al., 2019). These inflated target factor loadings and factor correlations might account for the variance of the cross-loadings; thus, the difference between the composite reliability estimations based on ESEM and ICM-CFA was very small.

Moreover, we found that under the conditions of low target factor loadings and cross-loadings (e.g., Lt≤ 0.4 and Lc≤ 0.01), a small number of indicators per factor (e.g., NI = 3), and small sample sizes (e.g., N = 100), ESEM_Geo and ICM-CFA tended to overestimate composite reliability. Increasing the sample size might be a solution to this problem. For instance, under the same conditions as mentioned above except for the use of a larger sample size (e.g., N≥ 300), the relative bias of the composite reliability estimate was acceptable. Low target factor loadings usually imply high measurement errors. Therefore, large sample size is needed to correct for measurement error.

Although the mean values of the models’ goodness-of-fit indices in this study all (except 4) reached acceptable levels, the model fits of ICM-CFA and ESEM were different. More of ICM-CFA-based models failed to reach acceptable level than ESEM-based models. To our surprise, model fitness did not affect the accuracy of the composite reliability estimation. The empirical example suggested the same conclusion. Hence, if the model fit of ESEM is acceptable but that of ICM-CFA is not, the difference in the composite reliability estimates obtained based on the two models is ignorable as long as the primary model structures are the same.

Target rotation is always considered an advantage of ESEM since it provides a priori model and gives the researcher control in specifying the model, therefore facilitating the interpretation of the results. However, Mplus does not provide composite reliability based on target rotation. The results of Simulation 2 showed no difference in the estimation of composite reliability between geomin rotation and target rotation in ESEM. In other words, both rotation methods lead to the same reliability estimation results, thereby demonstrating that the reliability estimation of ESEM is, to some extent, consistent.

In conclusion, we recommend that researchers first try to fit their data into an ICM-CFA model regarding composite reliability estimation. If the model fitness is not acceptable, ESEM_Geo should be used. These two methods allow researchers to estimate composite reliability directly with the Mplus software. When ESEM_Tgt is adopted, the composite reliability can be computed based on the output model parameters.

Supplemental Material

sj-pdf-1-epm-10.1177_00131644211008953 – Supplemental material for A Comparison of Reliability Estimation Based on Confirmatory Factor Analysis and Exploratory Structural Equation Models

Supplemental material, sj-pdf-1-epm-10.1177_00131644211008953 for A Comparison of Reliability Estimation Based on Confirmatory Factor Analysis and Exploratory Structural Equation Models by Yuanshu Fu, Zhonglin Wen and Yang Wang in Educational and Psychological Measurement

Footnotes

Appendix

Title: Calculate composite reliability under ESEM geomin rotation;

Data: File is S1_exp.dat;

Variable: Names are x1-x6;

Analysis: Rotation = geomin;

Model:

f1-f2 by x1-x6(*1);

f1 by x1-x6 *(la1-la6);

f2 by x1-x6 * (lb1-lb6);

x1-x6(v1-v6);

f1 WITH f2(cor);

Model constraint: NEW(C1 C2 CR);

c1 = la1 + la2 + la3 + la4 + la5 + la6;

c2 = lb1 + lb2 + lb3 + lb4 + lb5 + lb6;

CR = (c1**2 + c2**2 + 2*c1*c2*cor)/

(c1**2 + c2**2 + 2*c1*c2*cor + v1 + v2 + v3 + v4 + v5 + v6);

Note. This Mplus code was modified from Raykov and Marcoulides (2011, p. 173) where the authors provided the Mplus code for composite reliability estimation based on an independent clusters model of confirmatory factor analysis model. For EQS and LISREL input of reliability estimation based on a general structure (with cross-factor loadings), please refer to Raykov and Shrout (2002).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by grants from the National Natural Science Foundation of China [31771245].

ORCID iD

Yuanshu Fu

Supplemental Material

Supplemental material for this article is available online.

References

Asparouhov

Muthén

(2009). Exploratory structural equation modeling. Structural Equation Modeling, 16(3), 397-438. https://doi.org/10.1080/10705510903008204

Bandalos

D. L.

(2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling, 9(1), 78-102. https://doi.org/10.1207/S15328007SEM0901_5

Bentler

P. M.

(2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137-143. https://doi.org/10.1007/s11336-008-9100-1

Boomsma

(1982). On the robustness of LISREL against small sample size in facto analysis models. North-Holland.

Chen

B. B.

Wiium

Dimitrova

(2018). Factor structure of positive youth development: Contributions of exploratory structural equation modeling. Personality and Individual Differences, 124, 12-15. https://doi.org/10.1016/j.paid.2017.11.039

Graham

J. M.

(2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66(6), 930-944. https://doi.org/10.1177/0013164406288165

Green

S. B.

Yang

(2009). Commentary on coefficient alpha: A cautionary tale. Psychometrika, 74(1), 121-135. https://doi.org/10.1007/s11336-008-9098-4

Wen

Fan

(2017). Examining and controlling for wording effect in a self-report measure: A Monte Carlo simulation study. Structural Equation Modeling, 24(4), 1-11. https://doi.org/10.1080/10705511.2017.1286228

Hogan

T. P.

Benjamin

Brezinski

K. L.

(2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60(4), 523-531. https://doi.org/10.1177/00131640021970691

10.

Hoogland

J. J.

Boomsma

(1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods & Research, 26(3), 329-368. https://doi.org/10.1177/0049124198026003003

11.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

12.

Jöreskog

K. G.

(1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109-133. https://doi.org/10.1007/BF02291393

13.

Lord

F. M.

Novick

M. R.

(1968). Statistical theories of mental test scores. Addison-Wesley.

14.

Marsh

H. W.

(2007). Application of confirmatory factor analysis and structural equation modeling in sport/exercise psychology. In Tenenbaum

Eklund

R. C.

(Eds.), Handbook of sport psychology (3rd ed., pp. 774-798). Wiley.

15.

Marsh

H. W.

Hau

K. T.

Wen

(2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320-341. https://doi.org/10.1207/s15328007sem1103_2

16.

Marsh

H. W.

Lüdtke

Muthén

Asparouhov

Morin

A. J. S.

Trautwein

Nagengast

(2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22(3), 471-491. https://doi.org/10.1037/a0019227

17.

Marsh

H. W.

Lüdtke

Nagengast

Morin

A. J. S.

Von Davier

(2013). Why item parcels are (almost) never appropriate: Two wrongs do not make a right: Camouflaging misspecification with item parcels in CFA models. Psychological Methods, 18(3), 257-284. https://doi.org/10.1037/a0032773

18.

Marsh

H. W.

Morin

A. J.

Parker

P. D.

Kaur

(2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85-110. https://doi.org/10.1146/annurev-clinpsy-032813-153700

19.

Marsh

H. W.

Muthén

Asparouhov

Lüdtke

Robitzsch

Morin

A. J. S.

Trautwein

(2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16(3), 439-476. https://doi.org/10.1080/10705510903008220

20.

McDonald

R. P.

(1985). Factor analysis and related methods. Lawrence Erlbaum.

21.

Morin

A. J. S.

Marsh

H. W.

Nagengast

(2013). Exploratory structural equation modeling: An introduction. In Hancock

G. R.

Mueller

R. O.

(Eds.), Structural equation modeling: A second course (2nd ed., pp. 395-436). IAP.

22.

Muthén

Asparouhov

(2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313-335. https://doi.org/10.1037/a0026802

23.

Novick

M. R.

Lewis

(1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32(1), 1-13. https://doi.org/10.1007/BF02289400

24.

Raykov

(1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173-184. https://doi.org/10.1177/01466216970212006

25.

Raykov

(1998). A method for obtaining standard errors and confidence intervals of composite reliability for congeneric items. Applied Psychological Measurement, 22(4), 369-374. https://doi.org/10.1177/014662169802200406

26.

Raykov

(2002). Analytic estimation of standard error and confidence interval for scale reliability. Multivariate Behavioral Research, 37(1), 89-103. https://doi.org/10.1207/S15327906MBR3701_04

27.

Raykov

Marcoulides

G. A.

(2011). Introduction to psychometric theory. Routledge.

28.

Raykov

Marcoulides

G. A.

(2019). Thanks coefficient alpha, we still need you! Educational and Psychological Measurement, 79(1), 200-210. https://doi.org/10.1177/0013164417725127

29.

Raykov

Shrout

P. E.

(2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9(2), 195-212. https://doi.org/10.1207/S15328007SEM0902_3

30.

Reise

S. P.

Scheines

Widaman

K. F.

Haviland

M. G.

(2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educational and Psychological Measurement, 73(1), 5-26. https://doi.org/10.1177/0013164412449831

31.

Sass

D. A.

Schmitt

T. A.

(2010). A comparative investigation of rotation criteria within exploratory factor analysis. Multivariate Behavior Research, 45(1), 1-33. https://doi.org/10.1080/00273170903504810

32.

Schmitt

T. A.

Sass

D. A.

(2011). Rotation criteria and hypothesis testing for exploratory factor analysis: Implications for factor pattern loadings and interfactor correlations. Educational and Psychological Measurement, 71(1), 95-113. https://doi.org/10.1177/0013164410387348

33.

Sijtsma

(2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107-120. https://doi.org/10.1007/s11336-008-9101-0

34.

Tóth-Király

Bőthe

Rigó

Orosz

(2017). An illustration of the exploratory structural equation modeling (ESEM) framework on the Passion Scale. Frontiers in Psychology, 8, Article 1968. https://doi.org/10.3389/fpsyg.2017.01968

35.

Tóth-Király

Morin

A. J. S.

Bőthe

Orosz

Rigó

(2018). Investigating the multidimensionality of need fulfillment: A bifactor exploratory structural equation modeling representation. Structural Equation Modeling, 25(2), 1-20. https://doi.org/10.1080/10705511.2017.1374867

36.

Wang

Pomerantz

E. M.

Chen

(2007). The role of parents’ control in early adolescents’ psychological functioning: A longitudinal investigation in the United States and China. Child Development, 78(5), 1592-1601. https://doi.org/10.1111/j.1467-8624.2007.01085.x

37.

Xiao

Liu

Hau

K. T.

(2019). A comparison of CFA, ESEM, and BSEM in test structure analysis. Structural Equation Modeling, 26(5), 665-677. https://doi.org/10.1080/10705511.2018.1562928

38.

Yang

Green

S. B.

(2010). A note on structural equation modeling estimates of reliability. Structural Equation Modeling, 17(1), 66-81. https://doi.org/10.1080/10705510903438963

39.

Yuan

MacKinnon

D. P.

(2009). Bayesian mediation analysis. Psychological Methods, 14(4), 301-322. https://doi.org/10.1037/a0016972

40.

Zimmerman

D. W.

Zumbo

B. D.

Lalonde

(1993). Coefficient alpha as an estimate of test reliability under violation of two assumptions. Educational and Psychological Measurement, 53(1), 33-49. https://doi.org/10.1177/0013164493053001003

41.

Zinbarg

R. E.

Revelle

Yovel

(2005). Cronbach’s α, Revelle’s β, and McDonald’s ω_h: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133. https://doi.org/10.1007/s11336-003-0974-7

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.13 MB