Essential Unidimensionality Examination for Multicomponent Scales

Abstract

A procedure for examining essential unidimensionality in multicomponent measuring instruments is discussed. The method is based on an application of latent variable modeling and is concerned with the extent to which a common factor for all components of a given scale accounts for their correlations. The approach provides point and interval estimates of the proportion of observed measures interrelationships that is explained by the common factor, the proportion explained by possible additional factors, and of the difference in these two proportions. The procedure is readily used in empirical research using the popular software Mplus and is illustrated with a numerical example.

Keywords

common factor correlation essential unidimensionality interval estimation proportion explained correlations

Educational, behavioral, social, and biomedical scientists involved in scale construction and development often face the question whether a given measuring instrument is essentially unidimensional. This would be the case if the scale is found not to be strictly unidimensional (homogeneous) but rather measuring predominantly a factor common to all its components, along with several additional (local) factors shared each by some subsets of components (see, e.g., Raykov & Pohl, 2012). For example, intelligence scholars are frequently concerned with the question whether a test battery assesses mostly a general factor of intelligence like g (Spearman, 1904), in addition to several narrow task-specific factors that are shared each by groups of subtests (e.g., Stankov & Raykov, 1995). A wisdom researcher may well be interested in the question whether a set of life-review situations are dominated by an overall wisdom trait, in addition to task-related factors that are common to some situations (e.g., Staudinger & Glück, 2011). An educational scientist is often concerned with the query whether a test measures primarily a single construct, such as a main ability of interest, and in addition one or more latent variables (minor abilities) that pertain each to some groups of its elements. A clinician may similarly wish to evaluate the extent to which a scale under consideration taps chiefly into a construct of main concern, like anxiety, alongside one or more factors of less interest and potency that are shared by some of its items. A medical scientist developing a scale for patient literacy in dealing with arthritis may well be concerned with evaluating the degree to which the scale operates approximately as homogeneous and to what extent it captures also familiarity with medical terms that may be a factor of lesser importance for his/her goals.

The common characteristic of these and many similar circumstances in behavioral and social research is the concern with the extent to which a measuring instrument under consideration that is not really homogeneous may be operating in an essentially unidimensional way for many empirical aims. Tests and scales of interest in the educational and psychological disciplines oftentimes inadvertently tap into relatively minor abilities along main constructs of focal interest. In these cases, scholars are typically interested in the degree to which these instruments (and their sum scores) can still be treated in applied research as basically unidimensional. Thereby, of particular relevance would be the answer to this question with regard to the population for which the instrument has been or is being developed rather than with respect only to a given sample of subjects.

This essential unidimensionality query has attracted some attention in the literature, but the treatments it has received have been mostly of descriptive nature and focusing primarily on observed variance partition for individual components in available samples. Recently, Raykov and Pohl (2012) described a method allowing one to interval estimate the proportion of observed scale variance explained by a common factor and by additional local factors, as well as of their difference. However, explained variance is only one aspect in which essential unidimensionality could manifest itself. Indeed, as is well known, within the framework of factor analysis one of the characteristic features of common factors is their ability to explain interrelationships between observed variables (e.g., Bartholomew, Knott, & Moustaki, 2011; Raykov & Marcoulides, 2008, 2011). Therefore, in the essential unidimensionality context of interest in this article, a related scale property that may be considered at least as important as manifest variance explanation is the extent to which observed interrelationships are accounted for by a common factor and to what degree by local (additional, minor) factors in a studied population. This is because decomposition of interrelationships between collected measures (scale components) represents another essential aspect of scale dimensionality and adds further important information on the issue of approximate homogeneity.

The present article is concerned with this question of essential unidimensionality that to our knowledge has not yet found a satisfactory and comprehensive resolution in the literature. The remainder of this discussion approaches the question from the above standpoint or criterion of explaining observed correlations. Below we describe a readily applicable procedure for point and interval estimation of (a) the proportion of observed measure interrelationship (or, equivalently, average correlation) that is explained by a factor common to all instrument components, (b) the proportion of this interrelationship that is captured by the local factors, and (c) the difference in these two proportions. The method outlined next provides important unique insights into the query as to whether a considered multicomponent measuring instrument, while not strictly homogeneous, could still be treated as essentially unidimensional for many goals in empirical behavioral and social research. The approach complements earlier research on explained observed scale variance as another related yet distinct aspect of approximate homogeneity (see Raykov & Pohl, 2012, and references therein), and provides further information that bears on the query of essential unidimensionality of a scale under consideration (see also discussion section).

Background and Assumptions

For the purposes of this article, we assume that a set of continuous scale components y₁, y₂, . . . , y_p are given (p > 2), which represents a multicomponent behavioral measuring instrument of concern (see Raykov & Pohl, 2012; the assumption of continuity is adopted here for simplicity and relaxed later; see Conclusion section). All these components are assumed to load on a common (global) factor denoted g, and on one or more local (additional) factors designated f₁, . . . , f_q (q > 0, p > q). The interpretation of the factors f₁, . . . , f_q is as usual in terms of such sources of common variance in some groups of components, which are not captured by the common factor g reflecting the shared variability and covariability by all p measures. There is no limit on the number of components that load on a local factor as long as it is at least 2, and no further restrictions are imposed on the relation of these factors to observed variables beyond such needed for overall model identification (see below). The common factor g is also assumed to be uncorrelated with any of the local factors f₁, . . . , f_q, while their own covariance matrix is assumed diagonal owing to the fact that the commonality of the observed variables y₁, . . . , y_p is accounted for by the common factor.¹ That is, formally we assume that the following model holds in a studied population:

\underline{y} = \underline{μ} + Λ \underline{η} + \underline{ε},

where $\underline{y}$ = (y₁, . . . , y_p)′ symbolizes the p × 1 vector of observed measures (instrument/scale components), Λ = [λ_ij] is the p × (q + 1) factor loading matrix, $\underline{η}$ = (g, f₁, . . . , f_q)′ is the (q + 1) × 1 vector of the common and local factors that are all assumed uncorrelated among themselves, $\underline{µ}$ is a p × 1 vector of intercepts (e.g., Millsap, 2011), and $\underline{ε}$ is the p × 1 residual term vector with mean 0 that is assumed uncorrelated with η and with a diagonal covariance matrix (i = 1, . . . , p, j = 1, . . . , q; see also Discussion and Conclusion section; priming designates transposition in this article). Furthermore, all q + 1 factors are assumed to have unit variances, and in the second through last columns of Λ corresponding to f₁ through f_q there are some pertinent free parameters and zeros as remaining elements, with the overall model assumed to be identified. Model (1) may have been developed based on prior research and/or existing theory in a given substantive domain, or derived through exploratory factor analysis carried out on an independent sample from the same population, or may reflect a hypothesis of substantive interest that is to be tested. As an example, Model (1) with p = 9 scale components and q = 3 local factors that is used in the illustration section is displayed in Figure 1 (Raykov & Pohl, 2012).

Figure 1.

Example Model (1) with p = 9 instrument components and q = 3 local factors

The remainder of this article outlines a latent variable modeling (LVM) approach to point and interval estimation of (a) the proportion of covariability (average correlation) in the measures y₁ through y_p, which is accounted for by the common factor g; (b) the proportion of their covariability (average correlation) that is explained by the local factors f₁ through f_p; and (c) the difference in these two proportions. The quantities in (a), (b), and (c) may be considered indexes of the degree to which the scale under consideration may be viewed as essentially unidimensional in empirical research.

Evaluation of Proportion Observed Covariability Accounted for by the Common and by the Local Factors

Let us denote the loadings of the observed measures (scale components) on the common factor by λ_jg, and by λ_js the loadings of the jth manifest variable on the sth local factor f_s (j = 1, . . . , p, s = 1, . . . , q). To achieve more readily the aims of this article, we reparameterize in a first step Model (1) by introducing p (phantom or pseudo) latent variables, ξ₁ through ξ_p, whereby each observed measure loads on a corresponding one of these phantom variables and is formally associated with zero error variance (and thus vanishing error term):

y_{j} = γ_{j} . ξ_{j},

with γ_j being an unknown population parameter (see below for its specific interpretation; j = 1, . . . , p; e.g., Jöreskog & Sörbom, 1996; Raykov, 2001).

As a consequence of this reparameterization, the correlations of the observed measures become equal to the correlations of the respective phantom/pseudo latent variables, that is,

C o r r (\underline{ξ}) = C o r r (\underline{y}),

where Corr(.) denotes correlation matrix of the vector in parentheses and underlining symbolizes vector.

In a second step, we impose the constraints of all variances of the phantom latent variables in $\underline{ξ}$ to be 1, that is,

d i a g (C o v (\underline{ξ}) = {(1, 1, \dots, 1)}^{'},

where Cov(.) denotes the covariance matrix of the vector in parentheses, diag(.) stands for the diagonal of the matrix, and there are p ones in the right-hand side of Equation (4). The introduction of the constraints (4) is accomplished by a set of p nonlinear restrictions that constrain the variance of each measure—as implied by Model (1)—to be 1:

λ_{j g}^{2} + λ_{j s}^{2} + θ_{j} = 1 = v a r (ξ_{j}),

where var(.) denotes variance of the variable in parentheses and θ_j is the variance of the jth residual term (j = 1, . . . , p, s = 1, . . . , q; see the appendix for software implementation). With restrictions (4), from Equations (2) and (3) it follows that

γ_{j} = s d (y_{j}),

with sd(.) denoting standard deviation of the variable in parentheses (j = 1, . . . , p).

Based on this discussion, we can make the following important observation. The decomposition of the observed correlations (average correlation) into (a) a part accounted for by the common factor g and (b) a part explained by the local factors f₁ through f_q is tantamount to decomposing the correlations among the phantom latent variables ξ₁ through ξ_p into their pertinent two sources (a) and (b; see next paragraph).

To be more concrete, let y_j and y_k be two arbitrarily chosen measures (1 ≤ j ≠ k ≤ p). Within the current model parameterization, their correlation is obviously:

c o r r (y_{j}, y_{k}) = c o r r (ξ_{j}, ξ_{k}) = c o v (ξ_{j}, ξ_{k}),

where corr(.,.) and cov(.,.) denote correlation and covariance of the pair of variables in parentheses. Given the last equality, the correlation decomposition for any two observed measures that is of interest in this section, amounts to the covariance decomposition for the phantom latent variables ξ₁ through ξ_p into a part explained by the common factor and a part explained by the local factors.

Because of the specifics of Model (1), and in particular the lack of relationship between the common factor g and any of the local factors, we need to differentiate next between two cases.

Case 1. The measures y_j and y_k have loadings on the same local factor, say the sth (1 ≤ s ≤ q). Then their correlation is (see Equation 1):

c o r r (y_{j}, y_{k}) = c o r r (ξ_{j}, ξ_{k}) = c o v (ξ_{j}, ξ_{k}) = λ_{j g} λ_{k g} + λ_{j s} λ_{k s} .

Therefore, the part of the correlation between these two measures, which is accounted for by the common factor g, is in this case λ_jgλ_kg, while the part accounted for by the local factors is λ_jsλ_ks.

Case 2. The measures y_j and y_k have loadings on different local factors, say the sth and tth (1 ≤ s < t ≤ q). Then their covariance, and thus correlation, is (see Equation 1 and associated assumptions):

c o r r (y_{j}, y_{k}) = c o r r (ξ_{j}, ξ_{k}) = c o v (ξ_{j}, ξ_{k}) = λ_{j g} λ_{k g} .

Hence, the part of the correlation between these two measures that is accounted for by the common factor is in this case still λ_jgλ_kg, but the part accounted for by the local factors is 0.

The right-hand sides of Equations (7) and (8) are correspondingly extended by adding the loading products of the two involved observed variables on any additional local factor that they may also be loading on beyond the sth, or sth and tth (in Case 1 or Case 2, respectively), depending on a particular empirical setting of relevance and the specific form of Model (1). We do not consider explicitly in the remainder this case of additional local factor loadings, but stress that it is handled straightforwardly in complete analogy to the following developments.

Therefore, summing over all possible observed variable correlations we can make the following observation. The part α of their sum (or, equivalently, of their average correlation), which is explained by the common factor g, is the ratio of (a) the sum of all possible products of pairs of factor loadings λ_jg on that factor, to (b) that same sum plus the sum of all possible products of pairs of factor loadings on each of the local factors f_s (j = 1, . . . , p; s = 1, . . . , q). That is,

α = \frac{\sum_{1 \leq j < k \leq p} λ_{j g} λ_{k g}}{\sum_{1 \leq j < k \leq p} λ_{j g} λ_{k g} + \sum_{\binom{1 \leq s \leq q,}{1 \leq j < k \leq p}} λ_{j s} λ_{k s}} .

Similarly, it is found that part β of the sum of all possible measure correlations (average correlation) that is explained by the local factors is

β = \frac{\sum_{1 \leq s \leq q} λ_{j s} λ_{k s}}{\sum_{1 \leq j < k \leq p} λ_{j g} λ_{k g} + \sum_{\binom{1 \leq s \leq q,}{1 \leq j < k \leq p}} λ_{js} λ_{ks}} = 1 - α .

The discussion so far was developed at the population level. We next attend to estimation issues for the quantities of interest in available samples, in particular for the proportions α and β and their difference ω = α − β.²

Point Estimation of Proportions Explained Correlation and Their Difference

After Model (1) is fitted to the analyzed data using a method appropriate for the variable distributions (see also Discussion and Conclusion section), and found plausible, the above (average) correlation proportions α and β are estimated correspondingly by the following:

\hat{α} = \frac{\sum_{1 \leq j < k \leq p} {\hat{λ}}_{j g} {\hat{λ}}_{k g}}{\sum_{1 \leq j < k \leq p} {\hat{λ}}_{jg} {\hat{λ}}_{k g} + \sum_{\binom{1 \leq s \leq q}{1 \leq j < k \leq q}} {\hat{λ}}_{j s} {\hat{λ}}_{k s}}

and

\hat{β} = \frac{\sum_{1 \leq s \leq q} {\hat{λ}}_{j s} {\hat{λ}}_{k s}}{\sum_{1 \leq j < k \leq p} {\hat{λ}}_{j g} {\hat{λ}}_{j g} + \sum_{\binom{1 \leq s \leq q}{1 \leq j < k \leq q}} {\hat{λ}}_{j s} {\hat{λ}}_{k s}},

where a hat denotes estimate of the quantity underneath. Once these two proportions are estimated, their difference is estimated by

\hat{ω} = \hat{α} - \hat{β} .

For example, if the measures y₁ through y_p are multivariate normal, use of the maximum likelihood (ML) method to fit Model (1) is justified, which will then furnish the ML estimates (11) through (13) of the explained (average) correlation proportions and their difference. As is well known, these estimates have a number of desirable statistical properties—viz., they are asymptotically unbiased, consistent, normal, and efficient (e.g., Bollen, 1989; see also Conclusion section).

Interval Estimation

The point estimates of the explained correlation quantities (11) through (13) contain rather limited information about their values in a studied population that are typically of concern to the behavioral or social scientist involved in instrument construction and development. Confidence intervals (CIs) of these three parameters provide him or her in addition with ranges of plausible values for these quantities (parameters) in the population.

The proportion difference estimate in (13) is a function of the parameter estimates in Model (1) and reflects the degree to which the common factor outperforms the local factors in terms of accounting for observed measure interrelationships (average correlation). Hence, a large-sample standard error can be obtained for this difference ω using the popular delta method (e.g., Raykov & Marcoulides, 2004), which is implemented in the widely circulated LVM program Mplus (Muthén & Muthén, 2012; see the appendix for the source code). Denoting this standard error $S E (\hat{ω})$ , an approximate confidence interval for the difference in proportions explained observed covariability (average correlation) by the common and local factors is then rendered as

(\hat{ω} - z_{α / 2} S E (\hat{ω}), \hat{ω} + z_{α / 2} S E (\hat{ω})),

where z_α/2 denotes the (1 − α/2)100th quantile of the standard normal distribution. The CI (14) is provided by the cited software at the 90%, 95%, and 99% confidence levels (Muthén & Muthén, 2012).

To obtain CIs for the variance proportions (9) and (10), of particular interest in this article, we observe first that they represent bounded parameters; indeed, they are each limited from below by 0 and from above by 1 (cf. Raykov & Marcoulides, 2011). Hence, a CI at any given confidence level for either of the parameter (9) or (10) must completely reside within the interval (0, 1), because otherwise the CI would contain meaningless values. To ensure this, it is appropriate to obtain initially a CI of a suitable monotone increasing transformation of each of these proportions, which is unbounded (e.g., Browne, 1982). As for such a transformation, one can choose the logit function, and then through inversion using the logistic function obtain the CI of the parameters (9) and (10) as

((1 / (1 + e^{- κ_{lo}}), 1 / (1 + e^{- κ_{up}})),

where κ_lo and κ_up are correspondingly the lower and upper endpoints of the CI of the logit of the parameter in question. The CI (15) is readily furnished in empirical research by the R-function “ci.rel” from Raykov and Marcoulides (2011) employing the popular software R (Venables, Smith, and The Core R Development Team, 2012), once the delta method-based standard error for each of these proportions is provided by Mplus (see second part of the appendix for a minor modification of that function accomplishing this aim).

Having obtained the CIs of the proportions of measure correlation (average correlation) accounted for by the common and by the local factors, as well as of the difference in these two proportions, a researcher is in a better position to make a judgment as to whether a given multiple component measuring instrument could be considered essentially unidimensional in particular applications. To our knowledge, no rule-of-thumb would currently appear directly applicable, but one might conjecture that if (a) the proportion of measure covariability accounted for by the common factor is associated with a CI say at the conventional 95% CI that is entirely above .90, whereas (b) the CI of the proportion of variance accounted for by the additional factors is wholly below .10, then one might consider plausible a statement that the instrument in question is essentially unidimensional for many practical purposes.³ We strongly encourage and defer to future research into possible specific guidelines, as far as interpretation is concerned of the estimated proportions (11), (12), their difference (13), and associated confidence intervals.

Illustration on Data

In this section, we demonstrate the utility and applicability of the described estimation procedure using a numerical example. For our aims, we employ a data set simulated for n = 1,000 cases and p = 9 observed variables y₁ through y₉ according to the following model (see Figure 1, and Raykov & Pohl, 2012):

\begin{array}{l} y_{1} = 1.5 g + . 9 f_{1} + ε_{1}, \\ y_{2} = 1.5 g + . 9 f_{1} + ε_{2}, \\ y_{3} = 1.5 g + . 9 f_{1} + ε_{3}, \\ y_{4} = 1.5 g + . 8 f_{2} + ε_{4}, \\ y_{5} = 1.5 g + . 8 f_{2} + ε_{5}, \\ y_{6} = 1.5 g + . 8 f_{2} + ε_{6}, \\ y_{7} = 1.5 g + . 9 f_{3} + ε_{7}, \\ y_{8} = 1.5 g + . 9 f_{3} + ε_{8}, \\ y_{9} = 1.5 g + . 9 f_{3} + ε_{9}, \end{array}

where the common factor g and the local factors f₁ through f₃ were all standard normal and uncorrelated, while the residual terms were independent normal variables with zero mean and variance 1. The covariance matrix of the so-generated data for the nine components y₁ through y₉ is presented in Table 1.

Table 1.

Covariance Matrix for Data Set Used in Illustration Section

Var.:	y ₁	y ₂	y ₃	y ₄	y ₅	y ₆	y ₇	y ₈	y ₉
y ₁	2.938
y ₂	2.543	4.430
y ₃	2.363	3.323	4.299
y ₄	1.574	2.481	2.405	4.327
y ₅	1.652	2.563	2.459	3.158	4.013
y ₆	1.651	2.480	2.442	3.067	2.962	3.859
y ₇	1.754	2.653	2.552	2.533	2.456	2.514	4.399
y ₈	1.708	2.658	2.520	2.473	2.435	2.414	3.184	4.010
y ₉	1.761	2.766	2.592	2.477	2.472	2.533	3.098	3.085	4.124

Note. Var. = variable.

Fitting first a single factor model to the resulting data, whose latent structure is represented by only one factor η that is common to all 9 components y₁ through y₉, is associated with unacceptable fit indexes: chi-square (χ²) = 997.700, degrees of freedom (df) = 27, p value (p) = 0, and root mean square error of approximation (RMSEA) = .190 with a 90% CI (.180, .200). These indexes suggest rejection of the single-factor model as a possible means of data description and explanation. Hence, the (perfect) unidimensionality hypothesis of the measures y₁ through y₉ is rejected. This conclusion is correct, given that the data were generated by a model having a more complex structure than that of (strict) unidimensionality, viz., a common plus 3 local sources of shared variance within consecutive triples of observed measures.

Fitting then to these data Model (1) (see Figure 1) yielded the following fit indexes: χ² = 14.313, df = 18, p = .708, and RMSEA = 0 with a 90% CI (0, .022), which indicated tenable model fit. (See the appendix for source code to fit the model.) The resulting estimates with standard errors and related statistics for the model parameters of relevance here are presented in Table 2.

Table 2.

Parameter Estimates, Standard Errors, t Values, and p Values Associated With Model (1) (Software Output Format)

	Estimate	SE	Estimate/SE	Two-Tailed p Value
Factor loadings
G BY
LY1	0.626	0.022	28.022	.000
LY2	0.787	0.015	50.831	.000
LY3	0.765	0.017	46.242	.000
LY4	0.732	0.018	40.686	.000
LY5	0.762	0.017	45.651	.000
LY6	0.777	0.016	48.735	.000
LY7	0.773	0.017	45.839	.000
LY8	0.797	0.016	50.549	.000
LY9	0.811	0.015	53.405	.000
F1 BY
LY1	0.498	0.034	14.828	.000
LY2	0.427	0.030	14.382	.000
LY3	0.375	0.030	12.663	.000
F2 BY
LY4	0.477	0.031	15.571	.000
LY5	0.420	0.030	14.049	.000
LY6	0.382	0.029	13.104	.000
F3 BY
LY7	0.356	0.038	9.385	.000
LY8	0.398	0.038	10.547	.000
LY9	0.283	0.035	8.007	.000
Residual variances
LY1	0.360	0.028	12.863	.000
LY2	0.199	0.019	10.606	.000
LY3	0.275	0.019	14.532	.000
LY4	0.237	0.022	10.913	.000
LY5	0.243	0.019	12.913	.000
LY6	0.250	0.018	14.175	.000
LY7	0.275	0.021	12.914	.000
LY8	0.206	0.023	9.127	.000
LY9	0.263	0.018	14.935	.000
New/additional parameters
ALPHA	0.916	0.006	158.536	.000
BETA	0.084	0.006	14.568	.000
OMEGA	0.832	0.012	71.984	.000

As can be seen from Table 2, the part of observed measure covariability accounted for by the common factor g as well as by the three local factors f₁ through f₃ were estimated correspondingly as follows (delta method-based standard errors provided by the software are given in parentheses):

\begin{array}{l} \hat{α} = . 934 (. 005), \\ \hat{β} = . 066 (. 005), and \\ \hat{ω} = . 869 (. 009), with a 95 % CI (. 851, . 886) . \end{array}

With the estimates and standard errors for the first two proportions in Equations (17), the R-function “ci.rel” in Raykov and Marcoulides (2011) yields their following approximate confidence intervals (see second part of the appendix):

\begin{array}{l} for α : (. 923.943), \\ for β : (. 057, . 076) . \end{array}

The last two confidence intervals suggest that with high degree of confidence we may consider as plausible the statement that the common factor accounts for at least 92% of the interrelationships among the observed measures (average correlation) and as high as 94% of them in the population. At the same time, all additional, local factors explain between 6% and 8% of the interrelationships among the manifest measures at large. Relatedly, the CI of the difference ω in these proportions (see last part of Equations 17) suggests that the population difference in these two proportions is in the mid to high 80s (in percentages). Moreover, the proportion π_g of observed variance explained by the common factor, as described in Raykov and Pohl (2012), is obtained here as ${\hat{π}}_{g} = . 578$ with a 95% CI of (.550, .605); this finding indicates that well over half of the observed variance in the scale is due to common factor variance (see also Note 4 and below). Similarly, the proportion π_l of observed variance explained by the local factors, as outlined in Raykov and Pohl (2012), is found to be rather limited, viz., ${\hat{π}}_{l} = . 165$ with a 95% CI of (.148, .183). Based on all these results, one may suggest that the scale consisting of the 9 measures y₁ through y₉ represents a multicomponent measuring instrument that could be treated as essentially unidimensional for many empirical research purposes.

Discussion and Conclusion

Empirical educational, behavioral, social, and biomedical scientists are oftentimes confronted with the question whether a scale for which the unidimensionality hypothesis is not plausible is actually measuring predominantly a single construct of potentially high interest while tapping also into several additional factors of possibly much less relevance. The present article addressed this query by providing a means for examining the degree to which a factor common to all instrument components explains the interrelationships indexes (average correlation) among the observed variables.

The proportions of explained observed correlations of concern here furnish an important aid to researchers when judging if a multicomponent scale under consideration could be viewed as effectively homogeneous in many empirical situations. Using this approach, scholars can acquire relevant and unique pieces of information that bear on the essential unidimensionality query. In particular, focusing on explained (average) correlation by a common factor underlying a measuring instrument renders a distinct approach to examining approximate homogeneity than alternative recent research in Raykov and Pohl (2012).

Specifically the aforementioned proportion α of explained observed correlations by the common factor on the one hand (see Equation 9), and the proportion of explained manifest variance by that factor on the other hand, denoted π_g in Raykov and Pohl (2012), are two distinct—even though related—nonlinear functions of model parameters. Indeed, in the reparameterization of relevance to this article, because of manifest variances being set at 1 (see Equations 5), that proportion explained observed variance by the common factor g is

π_{g} = \frac{1}{p} \sum_{j = 1}^{p} λ_{j g}^{2},

At the same time, the proportion of explained variance by the local factors, denoted π_l in Raykov and Pohl (2012), is

π_{l} = \frac{1}{p} \sum_{j, s = 1}^{p} λ_{j s}^{2} .

A comparison of the right-hand sides of Equations (19) and (20) correspondingly with the right-hand sides of the above defining Equations (9) and (10) for the proportions α and β reveals their clear differences as distinct functions of model parameters, that is, α ≠ π_g and β ≠ π_l.⁴

Although a common factor explaining (part of) observed variance in at least two measures obviously also explains (part of) their observed correlations, this does not mean that there is any deterministic relation between (a) the proportion π_g of explained variance by the common factor, on the one hand, and the proportion α of explained observed correlations (average correlation) by this factor, on the other. (The term “deterministic relation” is used here in the sense of a formal relationship between α and π_g, in which there is no stochastic term.) Similarly, there is no deterministic relation between (b) the proportion π_l of explained variance by the local factors, and the proportion β of explained correlations (average correlation) by these factors. This lack of deterministic relations between the parameters mentioned in (a), as well as between those in (b), is also readily seen by comparing the right-hand side of Equation (19) with that of (9) (as mentioned above), and the right-hand side of Equation (20) with that of (10), respectively. (As is well known, twice the product of any two real numbers—factor loadings here—is only a lower bound of the sum of their squares, and the former equals the latter merely in the special case when these numbers are equal.)

This discussion implies that there is no generally applicable means of deriving the value of the proportion α of explained correlations (only) from knowledge of the proportion π_g of explained variance by the common factor, or vice versa, and similarly for the proportions β and π_l. In this sense, the two proportions α and π_g are two distinct measures of strength of the common factor, that is, two distinct indexes of essential unidimensionality (and similarly for the proportions β and π_l as distinct measures for the strength of the local factors, considered together). Since knowledge (only) of the value of either one of these proportions does not determine the value of the other proportion, each of them provides complementary information to that contained in the other proportion about the potency of the common factor. Hence, either proportion—α or π_g—provides a unique piece of information pertaining to possible essential unidimensionality (or lack thereof) for a given scale, test or measuring instrument, which information is complementary to that contained in the other proportion (π_g or α, respectively). It is further conceivable that one of these two proportions, α or π_g, may be more relevant for a particular research question and empirical setting, an issue that is best deferred to a substantive researcher in a given study. Moreover, in case of what may be deemed to be a substantial discrepancy between α and π_g, a researcher may elect not to resolve unequivocally the query whether to consider a scale as essentially unidimensional or not, but rather to model its scores and structure with all its sources of unobserved variability (common and local factors, and if need be correlated uniquenesses) in his/her subsequent analytic efforts of a studied phenomenon. Last but not least, given the point and interval estimates of α and β (and possibly of π_g and π_l), it may be the case that for some empirical aims and/or studied phenomena an instrument could be considered essentially unidimensional, while not so for others (see also next).

The value of the proportion α of explained correlations by the common factor may tend to be high when that of the proportion π_g of explained variance by that factor is high as well, and conversely. When both proportions are high, one could conclude that the instrument can be considered essentially unidimensional for many empirical purposes. When one of the two proportions is found to be high but the other is deemed not to be high, we submit that the substantive researcher would be in the best position to conclude whether or not to consider the instrument in question as essentially unidimensional for his/her particular empirical goal in a given empirical setting. In those cases, looking also at the ratio of explained variances by the common and by the local factors, π_g/π_l, as well as at the ratio α/β of the proportions of correlations explained by them, along with their confidence intervals, may well be informative. The reason is that either of these two ratios is free from the error variance that may contribute to a possible deflation of the proportion π_g of common factor explained variance (see Note 3, and Raykov & Pohl, 2012). Last but not least, when α is impressively high unlike π_g, a possible reason may be the presence of a considerable error effect in the scale score (as in the example of the preceding section, where the proportion of observed variance due to error is obviously .257 = 1 − .578 − .165).

We also note that while outlined for continuous scale components earlier, the approach of this article is also directly applicable with nonnormal (approximately) continuous scale components y₁ through y_p—with no floor or ceiling effect—by employing the robust maximum likelihood estimation method at the model fitting stage (Muthén & Muthén, 2012). Similarly, the procedure is applicable with discrete components, by fitting then Model (1) to the correlation matrix of the assumed normal variables $y_{1}^{*}$ through $y_{1}^{*}$ underlying the measures y₁ through y_p, respectively (e.g., Raykov, 2012; in that case, one does not need to constrain the variances of these underlying normal variables to 1, as the latter is already accomplished by the weighted least squares method of model fitting if used then; e.g., Muthén, 1984). Furthermore, as indicated above, the method of this note is directly extended to the case of multiple loadings on specific factors for one or more scale components, so long as the overall model is identified, as assumed throughout the article. Also, this interval estimation procedure is readily employed in the presence of missing data, under the assumption of missing at random (e.g., Little & Rubin, 2002). Last but not least, the proposed approach can be used in two-level settings by using robust standard errors that are obtainable then with the software used (Muthén & Muthén, 2012).⁵

We stress that the outlined procedure is applicable if Model (1) is tenable in a given empirical setting (as assumed throughout the article). If this model is deemed not to be fitting sufficiently well an available data set, while additional correlation(s) between uniquenesses (see correlated-trait-correlated-uniqueness model in Raykov & Pohl, 2012) are found to be desirable to include in order for (1) then to become plausible, these correlated uniquenesses likely reflect another form of multidimensionality (see also Kenny & Kashy, 1992; Marsh, 1989; Marsh & Bailey, 1991; Marsh, Byrne, & Craven, 1992). In that case, one may include in Model (1) another local factor(s) for these correlated uniquenesses, and then the above formulas for α and β in Equations (9) and (10) need to be extended by the uniqueness-correlation(s) (and correspondingly the source code in the appendix should reflect this extension; see next). Specifically, this correlation(s) can be considered part of the observed (average) correlation explained by the local factors, and thus needs to be added in the numerator of β and in the denominators of both α and β.

Another limitation of the method described in this article is the requirement for large samples. The reason is that it is based on an application of the LVM methodology, which is grounded in an asymptotic statistical theory (e.g., Muthén, 2002). We are not aware of specific guidelines that can be given for determining sufficient sample size, because of the complexity of the issues involved and empirical setting features possibly contributing to it (such as, for instance, number of observed variables and of local factors, reliability of individual components, and fractions of missing data if present). In the absence of such guidelines, it may be conjectured that the results obtained with this procedure will be more trustworthy with samples of at least several hundred (and possibly over a thousand with categorical items).

In conclusion, this note complements previous research on common factor variance evaluation in multiple component measuring instruments (Raykov & Pohl, 2012, and references therein), by providing an additional means for studying essential unidimensionality via a decomposition of observed measure interrelationships (covariability or average correlation) into common factor and local factors’ contributions. The procedure can be particularly helpful in settings where scales, instruments, or tests of interest are found not to be (perfectly) unidimensional, and educational and behavioral researchers are concerned with the important query of whether they could still be treated as essentially unidimensional for many empirical research purposes.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was in part supported by the German Federal Ministry of Education and Research (Project “National Educational Panel Study”).

Notes

Acknowledgements

Thanks are due to M. Eid and U. Staudinger for valuable discussions on multiple trait and method models, as well as to the editor and two anonymous referees whose comments on an earlier version contributed substantially to its improvement.

References

Bartholomew

D. J.

Knott

Moustaki

(2011). Latent variable models and factor analysis. A unified approach. New York, NY: Wiley.

Browne

M. W.

(1982). Covariance structures. In Hawkins

D. M.

(Ed.), Topics in applied multivariate analysis (pp. 72-141). Cambridge, England: Cambridge University Press.

Bollen

K. A.

(1989). Structural equations with latent variables. New York, NY: Wiley.

Jöreskog

K. G.

Sörbom

(1996). LISREL 8 User's Reference Guide. Chicago: Scientific Software International.

Kenny

D. A.

Kashy

D. A.

(1992). Analysis of multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin, 112, 165-172.

Little

R. J.

Rubin

D. B.

(2002). Statistical analysis with missing data. New York, NY: Wiley.

Marsh

H. W.

(1989). Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335-361.

Marsh

H. W.

Bailey

(1991). Confirmatory factor analyses of multitrait-multimethod data: A comparison of alternative models. Applied Psychological Measurement, 15, 47-70.

Marsh

H. W.

Byrne

B. M.

Craven

(1992). Overcoming problems in confirmatory factor analyses of MTMM data: The correlated uniqueness model and factorial invariance. Multivariate Behavioral Research, 27, 489-507.

10.

Millsap

R. E.

(2011). Statistical approaches to measurement invariance. New York, NY: Taylor & Francis.

11.

Muthén

(1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132.

12.

Muthén

B. O.

(2002). Beyond SEM: General latent variable modeling. Behaviormetrika, 29, 87-117.

13.

Muthén

L. K.

Muthén

(2010). Mplus user’s guide. Los Angeles, CA: Muthén & Muthén.

14.

Raykov

(2001). Testing multivariable covariance structure and means hypotheses via structural equation modeling. Structural Equation Modeling, 8, 224-257.

15.

Raykov

(2012). Scale construction and development using structural equation modeling. In Hoyle

(Ed.), Handbook of structural equation modeling (pp. 472-492). New York, NY: Guilford.

16.

Raykov

Marcoulides

G. A.

(2004). Using the delta method for approximate interval estimation of parametric functions in covariance structure models. Structural Equation Modeling, 11, 659-675.

17.

Raykov

Marcoulides

G. A.

(2006). A first course in structural equation modeling. Mahwah, NJ: Lawrence Erlbaum.

18.

Raykov

Marcoulides

G. A.

(2008). An introduction to applied multivariate analysis. New York, NY: Taylor & Francis.

19.

Raykov

Marcoulides

G. A.

(2011). Introduction to psychometric theory. New York, NY: Taylor & Francis.

20.

Raykov

Marcoulides

G. A.

(2012). Basic statistics. An introduction with R. New York, NY: Rowman & Littlefield.

21.

Raykov

Pohl

(2012). On studying common factor variance in multiple component measuring instruments. Educational and Psychological Measurement. Advance online publication. doi:10.1177/0013164412458673

22.

Spearman

(1904). “General intelligence” objectively determined and measured. American Journal of Psychology, 5, 201-293.

23.

Stankov

Raykov

(1995). Modeling complexity and difficulty in fluid intelligence tasks. Structural Equation Modeling, 2, 335-366.

24.

Staudinger

U. M.

Glück

(2011). Psychological wisdom research: Commonalities and differences in a growing field. Annual Review of Psychology, 62, 215-241.

25.

Venables

W. N.

Smith

D. M.

, & The R Development Core Team. (2012). An introduction to R. Bristol, England: Network Theory.