Taking the Error Term of the Factor Model Into Account

Abstract

The problem of factor score indeterminacy implies that the factor and the error scores cannot be completely disentangled in the factor model. It is therefore proposed to compute Harman’s factor score predictor that contains an additive combination of factor and error variance. This additive combination is discussed in the framework of classical test theory. On this basis, a definition of reliability, standard error of measurement, and confidence intervals for the factor score predictor are proposed. It is argued that factor score predictor intervals should be used instead of single score predictors to account for the error term in the factor model. The calculation of reliabilities and factor score predictor intervals is illustrated by means of a small simulation study and an empirical example.

Keywords

factor analysis factor scores factor score predictors

In several applied settings and especially in psychological assessment, individual factor scores are of interest. It has been pointed out that factor scores cannot be determined precisely, as the number of common and unique factor scores exceeds the number of observed variables (McDonald & Burr, 1967). This problem has been termed factor score indeterminacy, a problem with a long history of debate (e.g., Grice, 2001; Guttman, 1955; Lovie & Lovie, 1995). It has also been noted that the definition of the factors in the factor model precludes the possibility of expressing them as linear combinations of the observed variables (Guttman, 1955). Moreover, Schönemann and Steiger (1976) proved that factor score predictors that are constructed as weighted composites of the observed variables cannot be factors of the common factor model. Nevertheless, linear combinations have been proposed as substitutes or predictors for the factors, because there is a need for some individual scores in applied settings (e.g., Anderson & Rubin, 1956; Bartlett, 1937; Harman, 1976; Krijnen, Wansbeek, & Ten Berge, 1996; McDonald, 1981; Thurstone, 1935). The properties of some of these factor score predictors have been compared and discussed (Grice, 2001; Krijnen, 2006; McDonald & Burr, 1967). Another aspect that introduces indeterminacy into the factor model is factor rotation. Several methods of factor rotation are available and there is a paucity of literature offering recommendations for different purposes (e.g., Browne, 2001; Sass & Schmitt, 2010). Although the focus of the present article is on factor score indeterminacy, effects of factor rotation on factor score predictors are briefly considered in a small simulation study illustrating the main arguments presented here.

The present article starts from the well-known observation that linear combinations of observed variables as they have been proposed cannot be compatible with the basic equations of the common factor model. It is concluded that the only type of factor score predictor that would be in line with the basic equations of the common factor model is a factor score predictor that does not result in a single score for each participant, but in a score probability interval.

First, the definitions of the common factor model are presented. Second, the incompatibility of linear combinations of observed variables with the basic equations of the common factor model is shown. Third, Harman’s ideal variable factor score predictor is shown to combine factor and error variance. Fourth, this combined factor and error score predictor is considered under the assumptions of classical test theory. On this basis, a definition of reliability, standard error of measurement, and factor score predictor intervals are proposed. Finally, a simulation study and an applied example are given to illustrate the effect of considering the reliability and the factor score predictor intervals when factor score predictors are computed.

Definitions

The defining equation of the common factor model is

x = Λ f + e,

where x is the random vector of observations of order p, f is the random vector of factor scores of order q, e is the unobservable random error vector of order p, and Λ is the factor pattern matrix of order p by q. The common factors f, and the unique factors or error vectors e are assumed to have an expectation 0 (ε[x] = 0, ε[f] = 0, ε[e] = 0). The expected covariance between the factors and the error scores is assumed to be 0 (Cov[f, e] = ε[fe′] = 0). The expected covariance matrix of observed variables Σ can be decomposed into

\sum = Λ Φ Λ' + Ψ^{2},

where Φ = ff′ represents the q by q factor covariance matrix and Ψ² is a p by p diagonal matrix representing the expected covariance of the error scores e (Cov[e, e] = ε[ee′] = Ψ²). Equation 2 has been called the fundamental factor theorem (Harman, 1976; Thurstone, 1935). Moreover, postmultiplication of Equation 1 with e′ shows that the expected covariance of the error scores with the observed variables is Cov[e, x] = ε[ex′] = Ψ², because ε[fe′] = 0. It is assumed that the diagonal of Ψ² contains only positive values and that the expectation of the nondiagonal elements is 0.

Results

Factor Score Predictors and the Equations of the Factor Model

The available factor score predictors based on linear combinations of observed variables (e.g., Anderson & Rubin, 1956; Bartlett, 1937; Harman, 1976; McDonald, 1981; Thurstone, 1935) can be described as $\hat{f}$ = B′x, with different weights in B according to their different properties. First, the consequences of using $\hat{f}$ = B′x instead of f for the basic equations of factor analysis are investigated. Lemma 1 proves that the weights of Thurstone’s regression score predictor are compatible with the defining equation of the common factor model. Thus, B = Σ⁻¹ΛΦ results from replacing f by $\hat{f}$ = B′x in the defining equation of the factor model (Equation 1).

Lemma 1. If $x = Λ \hat{f} + e$ and $\hat{f}$ = B′x then B = Σ⁻¹ΛΦ.

Proof. Replacing f by $\hat{f}$ = B′x in Equation 1 yields

x = Λ B' x + e .

Postmultiplying Equation 3 with x′ yields

\sum = Λ B' \sum + Ψ^{2} .

Solving Equation 4 for B′ yields

B^{'} = {(Λ^{'} Λ)}^{- 1} Λ^{'} (\sum - Ψ^{2}) \sum^{- 1} = {(Λ^{'} Λ)}^{- 1} Λ^{'} (Λ Φ Λ^{'}) \sum^{- 1} = Φ Λ^{'} \sum^{- 1} .

This completes the proof.

Moreover, entering the weights given on the right-hand side of Equation 5 for B′ into Equation 4 yields the fundamental factor theorem (Equation 2). Thus, Lemma 1 shows that B = Σ⁻¹ΛΦ contains optimal weights when factor score predictors are entered into the defining equation of factor analysis (Equation 1). In contrast, Lemma 2 shows that computing factor score predictors as $\hat{f}$ = ΦΛ′Σ⁻¹x is incompatible with the fundamental factor theorem (Equation 2).

Lemma 2. If $\hat{f} = Φ Λ^{'} \sum^{- 1} x$ then $Λ Φ Λ^{'} + Ψ^{2} = Λ f f^{'} Λ^{'} + Ψ^{2} \neq Λ \hat{f} {\hat{f}}^{'} Λ^{'} Λ^{'} + Ψ^{2}$ .

Proof. Replacing Φ by $\hat{f} {\hat{f}}^{'}$ with $\hat{f}$ = ΦΛ′Σ⁻¹x in Equation 2 yields

Λ Φ Λ' \sum^{- 1} Λ Φ Λ' + Ψ^{2} \neq Λ Φ Λ' + Ψ^{2} .

This completes the proof.

Thus, Thurstone’s regression score predictor is compatible with the defining equation of the factor model, but it is incompatible with the fundamental factor theorem. It follows from Lemmas 1 and 2 that factor score predictors of the form $\hat{f}$ = B′x cannot be compatible both with the defining factor equation and the fundamental factor theorem. Moreover, according to Equation 1, x contains a sum of the factor f and the error e, there is no premultiplication of x with any weight that would allow for f to remain greater than 0 and for e to become 0 in the same time. However, this would be necessary when a linear combination of x is proposed as a perfect measure of f.

The Combined Factor–Error Score Predictor

According to Equation 1, the factor scores can be computed as

(Λ' Λ)^{- 1} Λ' (x - e) = f .

Inserting this factor score into Equation 2 is, of course, compatible with the fundamental factor theorem, as can be seen from

\begin{matrix} \sum = Λ (Λ' Λ)^{- 1} Λ' (x - e) (x - e)' Λ (Λ' Λ)^{- 1} Λ' + Ψ^{2} \\ = Λ (Λ' Λ)^{- 1} Λ' Λ Φ Λ' Λ (Λ' Λ)^{- 1} Λ' + Ψ^{2} = Λ Φ Λ' + Ψ^{2} . \end{matrix}

However, both f and e are unknown, which is the basis for factor score indeterminacy and probably the main cause of the long debate on factor score predictors. In the following, the meaning of e might be considered more closely. When Equation 1 is considered as the defining equation of the factor model in a population of individuals, e is a measurement error score in this population. This means that e does not represent sampling error with respect to a sample of individuals, just because it is the measurement error score in the population of individuals. Accordingly, e contains the unique parts of the measured variables x (e.g., items) in the population of individuals. However, if e is unique, this means that for each measured variable x, there will be another e. In consequence, e can be regarded as a sampling error with respect to the sampling of measurements x out of the (infinite) population of measured variables. Although e is a sampling error for the measurements, it is a population score with respect to the population of individuals. It is therefore proposed here to consider that even the error scores of the population of interindividual differences are only single manifestations (a sample) e_i out of a normally distributed population of intraindividual error scores from a population of measured variables x with ε[e_i] = 0. Thus, it is considered that for each individual measurement that is taken as a single case from the population of interindividual differences, there is also a population of intraindividual error scores. Moreover, the error scores e_i are assumed to be compatible with the assumptions of the factor model (ε[fe_i′] = 0; ε[e_ix′] = Ψ²; ε[e_ie_i′] = Ψ²). This yields

(Λ' Λ)^{- 1} Λ' (x - e_{i}) = f .

Equation 9 can be transformed into

{\hat{f}}^{*} = {(Λ' Λ)}^{- 1} Λ' x = f + {(Λ' Λ)}^{- 1} Λ' e_{i},

which gives the combined factor–error score predictor $\hat{f} *$ . In analogy to classical test theory (Lord & Novick, 1968) f represents the true score and (Λ′Λ)⁻¹Λ′e_i the error score. As in classical test theory, the factor and the error are uncorrelated and the errors are uncorrelated. The expected variance of $\hat{f} *$ is therefore

Var ({\hat{f}}^{*}) = diag (Φ + (Λ' Λ)^{- 1} Λ' Ψ^{2} Λ (Λ' Λ)^{- 1}) .

As Var(f) = diag(Φ), it follows that Var ( $\hat{f} *$ ) ≥ Var(f). In analogy to classical test theory, Var(f) represents the true-score variance and Var( $\hat{f} *$ ) represents the total variance (including error variance). Accordingly, the reliability of $\hat{f} *$ can be calculated as

{R_{\hat{f}}}_{*} = Var (f) / Var ({\hat{f}}^{*}) = diag (Φ) diag (Φ + (Λ' Λ)^{- 1} Λ' Ψ^{2} Λ (Λ' Λ)^{- 1})^{- 1} .

Although ε[e_i] = 0 for the intraindividual population of measurements, e_i is not 0 for any given x unless Ψ² = 0. It is therefore impossible to ignore the error term (Λ′Λ)⁻¹Λ′e_i, which would result in Harman’s (1976) ideal variable factor score predictor. Moreover, the ideal variable factor score predictor can be written as B′x = $ε [\hat{f}]$ , and it follows from Lemmas 1 and 2 that it cannot be compatible with the basic equations of the factor model. One can see from Equation 10 that Harman’s ideal variable factor score predictor is the aggregate of the factor score and the weighted error score. This does not mean that Harman’s ideal factor score predictor is problematic but that it is simply a combined factor and error score. It is therefore essential to consider the intraindividual distribution of the error scores on the right-hand side of Equation 10 and the reliability as calculated according to Equation 12.

Factor Score Predictor Intervals

It is proposed to take this reliability explicitly into account and to compute the standard error of measurement and intervals for the combined factor–error score predictor $\hat{f} *$ . Following classical test theory (Lord & Novick, 1968), the standard error of measurement for $\hat{f} *$ can be defined as

S_{\hat{f} *} = Var ({\hat{f}}^{*}) (I - {R_{\hat{f}}}_{*})^{1 / 2} .

Although the following interval is conceived in analogy to classical test theory, it is termed factor score predictor interval in the present case to make clear that it refers to the factor and error score as defined in the factor model. The corresponding 90% “factor score predictor interval” containing f is

{\hat{f}}^{*} - 1.65 S_{\hat{f} *} < f < {\hat{f}}^{*} + 1.65 S_{\hat{f} *},

where 1.65 is taken from the standard normal distribution. The corresponding value for a 95% “factor score predictor interval” would be 1.96. To be compatible with the assumptions of the factor model, it is necessary to report the factor score predictor interval instead of reporting the single value for $\hat{f} *$ unless Ψ² = 0. The distribution of $\hat{f} *$ that is marked by the confidence intervals contains the variability that is due to (Λ′Λ)⁻¹Λ′e_i and therefore gives an account for factor score indeterminacy. If, for example, the 95% probability intervals of the score predictors of two persons do not overlap, their factor–error score predictors might be regarded as different.

Because the factor loadings are essential for the calculation of the reliability of $\hat{f} *$ (see Equation 12), it is obvious that the reliability of the combined factor–error score predictor depends on factor rotation. A small simulation study was conducted to investigate the effects of factor rotation on the reliability of $\hat{f} *$ and on the magnitude of the respective factor score predictor intervals.

Simulation Study

A simulation study was conducted with SPSS 20 to investigate the consequences of the use of factor score predictor intervals in applied settings. The simulation study allows for an evaluation of the effect of sampling error on the reliability estimates for the factor–error score predictor and on the factor score predictor interval. The study comprises three sample sizes (150, 300, 900 cases), three levels of salient loadings (.40, .60, .80; there were five salient loadings per factor and the nonsalient loadings were fixed to 0), two numbers of factors (3, 6) and two levels of interfactor correlations (orthogonal vs. oblique). In the oblique population models, the correlation between the factors was .30. Table 1 contains the orthogonal and oblique population models based on three factors with salient loadings of .40. As the population models differ only with respect to the salient loading size and the number of factors from the models presented in Table 1, further tables for the description of the population models would have been redundant. Overall, there were 36 conditions (3 sample sizes × 3 salient loading sizes × 2 numbers of factors × 2 levels of interfactor correlations), and for each condition, 1,000 samples were generated. Moreover, two common methods of orthogonal factor rotation were investigated for the data sets based on orthogonal population factors: Varimax-rotation (Kaiser, 1958) and Equamax-rotation (Saunders, 1962). Moreover, two common methods of oblique factor rotation were investigated for the data sets based on oblique population factors: Promax-rotation with kappa = 4 (Hendrickson & White, 1964) and Oblimin with delta = 0 (Jennrich & Sampson, 1966). Accordingly, the simulation study was based on 72,000 factor analyses. Maximum likelihood estimation was used as an extraction method and the number of factors for the sample analyses was specified according to the population models (3 or 6). As the purpose of the simulation study was the illustration of the use of factor score predictor intervals in applied settings, it was not regarded as necessary to cover different methods of factor extraction or estimation.

Table 1.

Orthogonal and Oblique Three-Factor Population Models Based on Salient Loadings of .40.

	Orthogonal			Oblique
Variables	F1	F2	F3	F1	F2	F3
x ₁	.40	.00	.00	.40	.00	.00
x ₂	.40	.00	.00	.40	.00	.00
x ₃	.40	.00	.00	.40	.00	.00
x ₄	.40	.00	.00	.40	.00	.00
x ₅	.40	.00	.00	.40	.00	.00
x ₆	.00	.40	.00	.00	.40	.00
x ₇	.00	.40	.00	.00	.40	.00
x ₈	.00	.40	.00	.00	.40	.00
x ₉	.00	.40	.00	.00	.40	.00
x ₁₀	.00	.40	.00	.00	.40	.00
x ₁₁	.00	.00	.40	.00	.00	.40
x ₁₂	.00	.00	.40	.00	.00	.40
x ₁₃	.00	.00	.40	.00	.00	.40
x ₁₄	.00	.00	.40	.00	.00	.40
x ₁₅	.00	.00	.40	.00	.00	.40
Interfactor correlations
F1	1.00			1.00
F2	.00	1.00		.30	1.00
F3	.00	.00	1.00	.30	.30	1.00

For each factor analysis, factor–error score predictors were computed according to Equation 10 and the corresponding 90% factor score predictor intervals were computed according to Equation 14. The corresponding intervals were used to calculate the upper and lower interval limit for each individual score on each factor. In the next step, the percentage of score discrimination per factor–error score predictor was calculated. Therefore, it was counted how many times an upper limit of an individual score interval was smaller than a lower limit of another individual score interval on the same factor–error score predictor. This indicates whether two scores can be discriminated according to the factor score predictor interval. This number was divided by the number of different individual scores for the same factor–error score predictor in the sample when no interval is applied and the result was multiplied by 100. Thus, the percentage of score discrimination per factor–error score predictor indicates the amount of overdiscrimination that occurs when factor score predictor intervals are not taken into account. Only when the error terms in a factor model are 0, the index will reach 100%. An index of 50% would mean that only 50% of all the individual differences between scores on the factor–error score predictor can be interpreted when the 90% factor score predictor interval is applied. The index was averaged across the factors for the solution of each sample to give an overall index. Of course, larger factor score predictor intervals can be computed (e.g., 95% or 99%). However, for the purpose of the illustration of the effects of number of cases, salient loading size, number of factors, and obliqueness, the 90% interval was regarded as sufficient. Reliabilities were calculated according to Equation 12.

As a first result of the simulation study, the reliabilities of the factor–error score predictors are presented (see Table 2). The effect of loading sizes on reliability is substantial. A comparison of the reliabilities from the samples of the simulation study with the reliabilities from the corresponding population model reveals that there is a considerable overestimation of reliabilities when sample sizes are small. However, even with a sample size of 300 cases, which might be regarded as sufficient for several applications, there was a reliability larger than .60 in the six-factor solutions, whereas the corresponding population reliability was only .49. Nevertheless, with a sample size of 900 cases, the reliabilities from the sample came reasonably close to the population reliabilities. The effect of overestimation of reliability was slightly more pronounced for oblique than for orthogonal models. There were no relevant differences between Varimax- and Equamax-rotation for the orthogonal solutions and between Promax- and Oblimin-rotation for the oblique solutions.

Table 2.

Reliabilities of Factor–Error Score Predictor.

		Orthogonal solutions
		Three factors				Six factors
		Varimax		Equamax		Varimax		Equamax
Salient loading	n	M	SD	M	SD	M	SD	M	SD
.40	150	.66	.11	.66	.11	.78	.12	.79	.11
	300	.56	.05	.56	.05	.64	.04	.64	.04
	900	.51	.03	.51	.03	.53	.02	.53	.02
	Pop.	.49				.49
.60	150	.78	.05	.78	.05	.82	.04	.82	.04
	300	.76	.04	.76	.04	.77	.03	.78	.03
	900	.74	.02	.74	.02	.75	.02	.75	.02
	Pop.	.74				.74
.80	150	.90	.03	.90	.03	.90	.03	.90	.03
	300	.90	.02	.90	.02	.90	.02	.90	.02
	900	.90	.01	.90	.01	.90	.01	.90	.01
	Pop.	.90				.90
		Oblique solutions
		Three factors				Six factors
		Promax		Oblimin		Promax		Oblimin
Salient loading	n	M	SD	M	SD	M	SD	M	SD
.40	150	.68	.14	.66	.15	.79	.13	.79	.13
	300	.58	.07	.58	.07	.67	.08	.66	.08
	900	.52	.03	.51	.03	.55	.02	.53	.02
	Pop.	.49				.49
.60	150	.80	.05	.79	.05	.86	.04	.85	.04
	300	.77	.04	.76	.04	.80	.03	.79	.03
	900	.75	.02	.75	.02	.76	.02	.75	.02
	Pop.	.74				.74
.80	150	.91	.03	.91	.03	.93	.02	.93	.02
	300	.91	.02	.91	.02	.92	.02	.92	.02
	900	.90	.01	.90	.01	.91	.01	.91	.01
	Pop.	.90				.90

Note: Pop. = population reliability.

The percentage of the population factor score being within the 90% factor score predictor interval (coverage rate) was also calculated (see Table 3). Under ideal conditions, 90% of the population factor scores should be found within the 90% factor score predictor interval. However, as becomes apparent from Table 2, the reliabilities were considerably overestimated when sample sizes were small and medium. In consequence, the size of the factor score predictor intervals was underestimated when sample sizes were small. The underestimation of the size of the factor score predictor intervals implies that less than 90% of the population factor scores should be found within the 90% interval with small and medium samples. It was found that the percentage of population factor scores comes close to 90% for the sample size of 900 cases. The overall positive effect of loading size on coverage was present, but rather small. Moreover, coverage rates were more pronounced for orthogonal than for oblique solutions, whereas the number of factors had only minimal effects on coverage rates.

Table 3.

Coverage Rates: Percentage of Population Factor Scores Within Factor Score Predictor Intervals.

		Orthogonal solutions
		Three factors				Six factors
		Varimax		Equamax		Varimax		Equamax
Salient loading	n	M	SD	M	SD	M	SD	M	SD
.40	150	70.94	8.11	71.00	8.04	53.89	7.53	53.58	7.64
	300	81.74	3.27	81.74	3.27	75.09	3.70	75.07	3.77
	900	85.66	1.36	85.66	1.36	84.36	1.08	84.36	1.08
.60	150	81.71	7.35	81.71	7.35	76.01	6.21	75.81	6.13
	300	85.54	4.33	85.54	4.33	83.05	3.72	82.94	3.87
	900	87.85	1.97	87.85	1.97	87.22	1.53	87.22	1.53
.80	150	84.60	8.59	84.60	8.58	83.41	6.52	83.41	6.57
	300	86.36	6.11	86.36	6.11	85.54	4.90	85.56	4.99
	900	88.55	2.97	88.55	2.97	88.36	2.24	88.36	2.24
		Oblique solutions
		Three factors				Six factors
		Promax		Oblimin		Promax		Oblimin
Salient loading	n	M	SD	M	SD	M	SD	M	SD
.40	150	69.38	8.34	69.98	7.92	51.60	8.21	50.95	8.31
	300	80.51	4.42	80.47	4.55	69.52	4.90	69.56	5.13
	900	85.39	1.36	85.36	1.33	83.38	1.18	83.30	1.12
.60	150	80.67	7.58	80.73	7.69	70.57	7.08	70.82	6.53
	300	85.95	3.90	85.93	3.86	80.43	4.40	80.79	3.84
	900	87.79	1.89	87.79	1.88	86.43	1.76	86.44	1.69
.80	150	82.67	10.02	82.32	10.26	78.35	7.73	78.16	7.23
	300	86.80	6.88	86.82	6.83	82.23	5.64	82.28	5.54
	900	88.54	2.99	88.54	2.98	87.00	2.80	87.04	2.73

Score discrimination was calculated for the factor–error score predictor in each sample. It was the number of times that an upper limit of an individual score interval was smaller than a lower limit of another individual score interval divided by the total number of different scores of the respective factor–error score predictor. This discrimination ratio was presented as a percentage in Table 4. The effect of overestimation of reliabilities in small and medium samples had also effects on score discrimination. Therefore, the percentage of score discrimination was larger for smaller samples. Moreover, the effect of loading size on discrimination was substantial. It should, however, be noted that even with factor loadings of about .80, only about 50% of the discriminations between individual scores were relevant when a 90% interval is used. This means that about 50% of the discriminations that might be interpreted when the scores are used without factor score predictor intervals were irrelevant.

Table 4.

Percentage of Differences for Factor Score Predictor Intervals.

		Orthogonal solutions
		Three factors				Six factors
		Varimax		Equamax		Varimax		Equamax
Salient loading	n	M	SD	M	SD	M	SD	M	SD
.40	150	22.00	9.60	22.03	9.65	43.36	16.21	43.55	16.19
	300	12.95	2.50	12.95	2.51	17.37	4.12	17.26	3.89
	900	10.38	0.99	10.38	0.99	11.16	1.09	11.15	1.09
.60	150	30.54	8.63	30.54	8.63	36.84	10.94	36.65	10.47
	300	26.20	4.54	26.20	4.54	28.28	5.11	28.26	5.03
	900	24.05	2.10	24.05	2.10	24.63	2.23	24.63	2.23
.80	150	50.04	10.02	50.04	10.02	50.20	10.38	50.00	10.20
	300	47.68	6.38	47.68	6.38	47.70	6.58	47.67	6.56
	900	46.28	3.10	46.28	3.10	46.26	3.22	46.26	3.22
		Oblique solutions
		Three factors				Six factors
		Promax		Oblimin		Promax		Oblimin
Salient loading	n	M	SD	M	SD	M	SD	M	SD
.40	150	26.61	13.28	24.99	12.69	49.29	17.52	47.58	17.26
	300	14.21	4.24	13.86	4.03	21.06	7.20	20.25	6.96
	900	10.60	1.05	10.47	1.04	11.75	1.18	11.22	1.11
.60	150	33.14	9.60	32.46	9.28	45.31	13.30	42.52	12.29
	300	27.17	4.80	26.91	4.71	31.48	6.00	30.16	5.48
	900	24.34	2.31	24.27	2.30	25.57	2.50	25.12	2.40
.80	150	53.78	10.83	53.51	10.66	60.46	12.12	58.96	11.47
	300	49.42	6.84	49.33	6.79	53.01	7.88	52.36	7.51
	900	46.83	3.46	46.82	3.45	48.07	3.67	47.91	3.62

Example

A further illustration of the use of factor score predictor intervals is based on a study with the German Version of the Eysenck Personality Questionnaire Revised (EPQ-R; Eysenck & Eysenck, 1991; Ruch, 1999). Overall, 1,223 German participants (828 females, 395 males; age in years: M = 34.12; SD = 12.71) filled in the EPQ-R together with other personality questionnaires. Written informed consent has been indicated by all participants. Although most of the participants (76.40%) were recruited by means of newspaper advertising, university students (23.60%) were recruited through advertising in university courses. Participants received feedback on their individual personality trait scores.

According to Cattell’s (1966) scree-test the course of the eigenvalues of the unrotated maximum-likelihood factors indicates a three-factor solution (see Table 5). Three factors were retained for Promax (kappa = 4) rotation, also because the EPQ-R has been developed to measure Eysenck’s three-factor model of personality (Eysenck & Eysenck, 1991). The interfactor correlations were small and the overall factor pattern was not perfect but compatible with the intended three-factor model comprising factors for Neuroticism (N), Extraversion (E), and Psychoticism (P). However, many items do not load on a single factor, but have relevant secondary loadings (see Table 5). This is a situation where unit-weighted sum scales will not yield an optimal representation of the data. Therefore, researchers might compute the factor–error score predictor and might be interested in the computation of corresponding reliabilities and in the computation of the factor score predictor interval.

Table 5.

Maximum-Likelihood Three-Factor Solution of EPQ-R Items: Promax Factor Pattern.

Item	N	E	P
p_025	.10	.04	.28
p_030	.09	.08	.15
p_041	.10	.14	.24
p_046	.10	.08	.30
p_048	.04	−.02	.14
p_071	.03	−.01	.29
p_079	−.02	.02	.27
p_084	.06	−.01	.15
p_089	.19	.01	.18
p_102	−.01	.10	.19
p_002	.12	.17	.35
p_006	−.11	−.01	.28
p_009	−.07	−.01	.20
p_011	−.08	−.07	.13
p_013	−.07	−.01	.42
p_017	.01	.02	.40
p_020	−.05	−.15	.22
p_021	−.09	−.08	.49
p_026	−.08	−.17	.16
p_040	.00	−.07	.50
p_057	−.15	−.10	.26
p_061	.00	.00	.29
p_065	.09	−.03	.09
p_070	.08	.01	.34
p_075	.10	.01	.26
p_077	.19	.17	.37
p_086	.00	.00	.41
p_094	.00	−.07	.20
p_097	−.08	−.07	.07
p_099	.00	−.09	.26
p_100	−.01	−.05	.20
p_101	−.25	−.05	.38
e_001	−.05	.26	.02
e_005	.05	.69	−.09
e_010	.08	.72	−.03
e_015	.01	.52	.12
e_019	.04	.40	.03
e_029	.08	.50	.07
e_035	−.01	.52	−.11
e_036	.06	.22	.21
e_039	−.21	.57	−.03
e_044	−.08	.46	−.02
e_051	−.07	.59	−.04
e_054	.00	.40	.00
e_056	.03	.47	−.04
e_060	−.12	.31	−.02
e_064	−.10	.32	.14
e_066	.25	.27	.27
e_074	−.06	.63	−.04
e_088	.05	.55	−.03
e_090	−.01	.21	.27
e_093	.08	.65	−.03
e_024	−.05	.62	.05
e_033	.03	.43	−.01
e_047	−.08	.56	−.06
n_003	.60	.07	.17
n_007	.48	−.02	.08
n_012	.58	.03	−.20
n_016	.52	.11	.01
n_022	.55	−.02	−.10
n_027	.60	−.08	.09
n_031	.59	−.06	−.03
n_034	.50	.09	.01
n_037	.59	−.02	−.26
n_042	.45	.02	−.25
n_045	.55	−.23	.07
n_049	.52	−.02	.08
n_052	.26	.01	−.18
n_058	.28	.01	−.06
n_062	.57	−.03	.05
n_067	.37	−.09	.17
n_069	.38	.00	.00
n_072	.36	−.06	.15
n_076	.49	−.04	−.23
n_080	.51	.00	−.01
n_082	.51	−.15	.14
n_085	.46	−.03	−.02
n_091	.33	.07	.14
n_095	.36	.07	−.05
n_098	.44	.08	.10
Interfactor correlations
N	1.00
E	−.17	1.00
P	−.04	.25	1.00
First 10 eigenvalues of unrotated solution
8.21, 6.01, 3.73, 2.06, 1.87, 1.78, 1.66, 1.55, 1.46, 1.39

Note: EPQ-R = Eysenck Personality Questionnaire Revised; N = Neuroticism; E = Extraversion; P = Psychoticism. Intended loadings were given in boldface. Items were marked with a prefix referring to the intended factor (n, e, p) and the number within the EPQ-R.

The reliabilities of the respective factor–error score predictors (see Equation 12) were .90 for N, .90 for E, and .80 for P. Although the factor loadings were rather small, the large number of variables per factor leads to acceptable reliabilities. The corresponding standard errors of measurement (see Equation 13) were 0.32 for N and E and 0.45 for P. The corresponding upper and lower levels of 90% factor score predictor intervals were calculated by means of multiplying the standard errors of measurement by 1.65 and then adding and subtracting the result from the corresponding factor -error score predictor. As an example, the factor score predictor interval for E and N is presented for two participants of the present study (see Figure 1). The standard deviations of the factor–error score predictors for E and N were 1.06. They were expected to be larger than 1, because according to Equation 11, the factor–error score predictor contains a sum of the true-score variance (which is 1) and the error score variance (which is smaller than 1). As the simulation study focused on the comparison of scores between individuals, the comparison of the E and N scores within individuals was considered in the present example. Although the difference between the E and N scores were nearly half a standard deviation for Participant 15, the factor score predictor intervals overlap (see Figure 1). This indicates that the difference between E and N should not be regarded as relevant for this participant. Only very large differences (considerably greater than 1, as for Participant 16) were interpretable when the factor score predictor interval was considered. It should be noted that the intraindividual differences between E and N depend also on the scores of the other participants of the study. For example, if the other participants had larger scores on N, the relative score of the Participant 16 on N would have been lower so that the difference between N and E might have been smaller for this participant. Moreover, intraindividual differences on personality dimensions might have specific theoretical relevance. For example, the relative magnitude of E and N on the individual level has been related to the concepts of impulsivity and anxiety (e.g., Gray, 1987).

Figure 1.

Example: The 90% factor score predictor interval for N and E for two participants.

Discussion

It has been shown that factor score predictors based on linear combinations of observed variables cannot be simultaneously compatible with the defining equation of the factor model and the fundamental factor theorem. It was therefore proposed to take the variability that is due to the measurement error explicitly into account when individual factor score predictors are to be computed. It was proposed to compute Harman’s (1976) factor score predictor, which is a sum of factor scores and error scores that represents a combined factor–error score predictor. Due to the similarity of the assumptions of the factor model and the assumptions of classical test theory, the additive combination of factor and error in the factor–error score predictor can be regarded as a combination of true score and error score. Accordingly, the formula of classical test theory can be applied to compute reliability, standard error of measurement, and “factor score predictor intervals” for the combined factor–error score predictor. Instead of reporting a single factor–error score predictor for a person, it is proposed to report an individual factor score predictor interval that is based on the individual factor–error score predictor and the standard error of measurement. Thereby, the indeterminacy of factor score predictors is explicitly acknowledged and can be taken into account when individual scores based on factor analysis are used in applied settings. Thus, computing factor score predictor intervals is a consequence but not a solution of the indeterminacy of factor score predictors.

The consequences of computing reliabilities for the factor–error score predictor and factor score predictor intervals were illustrated by means of a simulation study. The main results of the simulation study were that there was a considerable overestimation of reliabilities when sample sizes were small. Sample sizes of about 900 cases were necessary to keep the reliability estimates close to the population estimates. Overestimation of reliabilities in small samples was slightly more pronounced for oblique solutions than for orthogonal solutions. The reason for the overestimation of reliability that occurred in small and medium samples might be that factor models that are based on sample data might be in part based on correlations that are due to sampling error. Thus, this effect might be regarded as an effect of capitalization on chance. However, the magnitude of this effect was considerable and might be explored in further studies. However, it should also be acknowledged that the simulation study was conducted with just one set of models where each manifest variable loads only on one factor. It can therefore not be excluded that the effect of capitalization on chance is smaller or does not occur in models with other loading patterns.

Considerably less than 90% of population factor scores were found within the 90% factor score predictor interval when sample sizes were small and medium. This is probably a direct consequence of the overestimation of reliabilities in small samples, which leads to an underestimation of the sizes of the factor score predictor intervals. On the one hand, these results of the simulation study demonstrate that, in the samples, the population factor scores might be more rarely within the factor score predictor interval than one would expect from the population parameter values. Accordingly, the method presented here should be used with caution when sample sizes and factor loadings are small. On the other hand, this does not mean that factor score predictors should be used without factor score predictor intervals when sample sizes and loadings are small. In contrast, the implication of the results is that it might be reasonable to perform a simulation study with exactly the number of variables, the loadings, and the number of cases corresponding to the empirical data at hand, to estimate the coverage rates and to adjust (enlarge) the size of the factor score predictor interval appropriately so that the intended percentage of population factor scores within the factor score predictor interval is obtained. Thus, the overestimation of reliabilities should be taken into account when sample sizes are small or medium. It should also be concluded that samples of at least 900 cases are necessary to get reasonable coverage rates.

Moreover, the overlap between the factor score predictor intervals reduces the number of interindividual differences that are interpreted. This is a direct consequence of taking the error term of the factor model into account. Even for solutions based on population main loadings of .80, only about 50% of the differences between individuals are interpreted on the basis of a 90% factor score predictor interval. As the factor score predictor interval reflects the effects of the error term in factor analysis on the scores, about 50% of the interindividual differences cannot be interpreted under the conditions of the present simulation study when the assumption that the factor model also contains an error term is taken into account. However, it would be inconsistent to have a term for measurement error in the factor model and to expect that the factor score predictor could be computed as if this measurement error does not occur. The overestimation of the reliabilities that has been found for small and medium sample sizes will, of course, also affect the amount of interindividual differences that are interpreted on the basis of the factor score predictor interval. Therefore, the method presented here should be used for samples of at least about 900 cases or an additional simulation study based on the parameters of the empirical model should be conducted to improve the factor score predictor interval.

Finally, an example was based on a factor analysis of the EPQ-R (Eysenck & Eysenck, 1991). It was shown that even when the main loadings are rather small, the reliabilities can be reasonably high, when the number of items per factor is large. Moreover, it is unlikely that there has been a considerable overestimation of the reliabilities as the sample size in the empirical example was larger than 900 cases. It was, moreover, shown that intraindividual comparisons of scores may turn out to be more realistic when factor score predictor intervals are considered. As a final remark, it is noted that taking the error term or uniqueness into account when the factor loadings are to be estimated is a main characteristic of the factor model. However, this implies that the error term should also be taken into account when factor score predictors are computed. In this sense, the factor score predictor interval is a direct consequence of the properties of the factor model.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

References

Anderson

T. W.

Rubin

(1956). Statistical inference in factor analysis. Proceedings of the Third Berkeley Symposium of Mathematical Statistics and Probability, 5, 111-150.

Bartlett

M. S.

(1937). The statistical conception of mental factors. British Journal of Psychology, 28, 97-104.

Browne

M. W.

(2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36, 111-150.

Cattell

R. B.

(1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245-276.

Eysenck

H. J.

Eysenck

S. B. G.

(1991). Manual of the Eysenck Personality Scales (EPS Adult). London, England: Hodder & Stoughton.

Gray

J. A.

(1987). Perspectives on anxiety and impulsivity: A commentary. Journal of Research in Personality, 21, 493-509.

Grice

J. W.

(2001). Computing and evaluation factor scores. Psychological Methods, 6, 430-450.

Guttman

(1955). The determinacy of factor score matrices with applications for five other problems of common factor theory. British Journal of Statistical Psychology, 8, 65-82.

Harman

H. H.

(1976). Modern factor analysis (3rd ed.). Chicago, IL: University of Chicago Press.

10.

Hendrickson

A. E.

White

P. O.

(1964). Promax: A quick method for rotation to oblique simple structure. British Journal of Statistical Psychology, 17, 65-70.

11.

Jennrich

R. I.

Sampson

P. F.

(1966). Rotation for simple loadings. Psychometrika, 31, 313-323.

12.

Kaiser

H. F.

(1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187-200.

13.

Krijnen

W. P.

(2006). Some results on mean square error for factor score prediction. Psychometrika, 71, 395-409.

14.

Krijnen

W. P.

Wansbeek

T. J.

Ten Berge

J. M. F.

(1996). Best linear predictors for factor scores. Communications in Statistics: Theory and Methods, 25, 3013-3025.

15.

Lord

F. M.

Novick

M. R.

(1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

16.

Lovie

A. D.

(1995). The cold equations: Spearman and Wilson on factor indeterminacy. British Journal of Mathematical and Statistical Psychology, 48, 237-253.

17.

McDonald

R. P.

(1981). Constrained least squares estimators of oblique common factors. Psychometrika, 46, 337-341.

18.

McDonald

R. P.

Burr

E. J.

(1967). A comparison of four methods of constructing factor scores. Psychometrika, 32, 381-401.

19.

Ruch

(1999). Die revidierte Fassung des Eysenck Personality Questionnaire und die Konstruktion des deutschen EPQ-R bzw. EPQ-RK [The revised version of the Eysenck Personality Questionnaire and the construction of the German EPQ-R and EPQ-RK]. Zeitschrift für Differentielle und Diagnostische Psychologie [Journal of Individual Differences], 20, 1-24.

20.

Sass

D. A.

Schmitt

T. A.

(2010). A comparative investigation of rotation criteria within exploratory factor analysis. Multivariate Behavioral Research, 45, 73-103.

21.

Saunders

D. R.

(1962). Trans-varimax. American Psychologist, 17, 395-396.

22.

Schönemann

P. H.

Steiger

J. H.

(1976). Regression component analysis. British Journal of Mathematical and Statistical Psychology, 29, 175-189.

23.

Thurstone

L. L.

(1935). The vectors of mind. Chicago, IL: University of Chicago Press.