Factor Analysis in Counseling Research and Practice

Test or Survey Construction

One vital application of factor analysis concerns the development of psychometrically sound measures (test, survey, and questionnaire). The purpose of factor analysis in this instance is both data reduction and simplification. In other words, the test developer wants to derive a small number of comprehensible factors from a much larger number of variables (e.g., a data set comprised of test items). Superfluous, indistinct, or irrelevant items can be identified for elimination. The following scenario should elucidate how factor analysis is employed during the test construction process.

A group of counseling researchers were interested in measuring the construct of “life satisfaction” in young adolescents, aged 11–15. After perusing existing measures of life satisfaction, only two possible options were located. Because neither instrument was validated with young adolescents, our investigators chose to design their own. Returning to the theoretical and research literature, they found enough solid work addressing life satisfaction issues in their target population to create enough survey items (variables). They generated 40 sample statements (e.g., “My family makes me feel happy”) that respondents would react to using a 5-point Likert-type scale, ranging from 1 (I completely disagree with the statement) to 5 (I completely agree). Since their intent was to measure several dimensions associated with early adolescent life satisfaction, the researchers anticipated that subsets of the 40 variables would cluster or group on several conceptually related dimensions or factors. In other words, once the factor analysis was run, conceptually related items would moderately to strongly intercorrelate (e.g., from .40 to .85), clustering on one or more interpretable dimensions.

Next, they gathered survey data from nearly 500 students attending four different schools. After entering the data into statistical software program (e.g., IBM SPSS) and computing the factor analysis procedure, the fictional output (not shown) revealed three relatively clear patterns in the item groupings, forming three factors composed of 10 items each. One set of 10 conceptually related items were highly intercorrelated creating Factor 1. Because these variables appeared to measure the respondents’ perceptions of their family life, they named this factor “family satisfaction.” Ten other items were moderately to strongly intercorrelated, clustering on Factor 2, and labeled the “satisfaction with peer relations” dimension. A third factor emerged from the results, with 10 items moderately correlating with each other. It was subsequently titled the “optimism–hopefulness” dimension.

The 10 remaining survey items weakly intercorrelated (e.g., rs between .00 and .30) with each other and with the other items comprising the three derived factors. The researchers reviewed the content of these “leftover” items finding that they addressed adolescent qualities unrelated to family satisfaction, peer satisfaction, or optimism–hopefulness. These variables were considered irrelevant and dropped from the 30-item Young Adolescent Life Satisfaction Inventory (YALSI). Finally, using a factor analysis procedure option, factor scores were computed for each of the three dimensions. The YALSI was now ready for its trial run with a new group of youngsters. In brief, factor analysis used in test construction reduced (or condensed) the original set of 40 variables to 30 usable items, which in turn clustered on three relatively distinct dimensions or factors.

Revising Established Tests

For various reasons, existing instruments need to be reworked. Perhaps, the test completion time is far too long for certain client groups (e.g., a group of TBI clients) or certain items are outdated or even biased toward respondents from a particular ethnicity. As a practical example, suppose a rehabilitation counselor desires to assess his clients’ level of satisfaction with their new work environment. The counselor locates a high-quality 150-item questionnaire—“Workplace Fulfillment Questionnaire” (WFQ), one with a wide range of items and scales pertaining to satisfaction with various aspects of the work setting (e.g., supervisor–worker relations, safety issues, and difficulty level of job responsibilities). The counselor wonders if all 150 items have to be administered or could the measure be shortened without compromising reliability and validity. The counselor pays a visit to the agency’s data and records manager looking for some statistical assistance.

The WFQ has the following items, among the many others, which are conceptually linked with the “Supervisor-Work Relations” scale: “Item 1: Workers are able to comfortably voice their opinions to their supervisors;” “Item 29: The boss is very open to employees’ viewpoints;” and, “Item 45: People who work at this company are free to express their concerns.” The data manager conducts a factor analysis on the data collected from a large sample of previously administered inventories, finding that these three variables are highly intercorrelated (ranging from .87 to .89) and strongly “load” on the 15-item “Supervisor-Work Relations” scale. In this situation, one might want to eliminate two of the three highly correlated items, because they seem to measure the same content. If the manager combed through all the results, she might locate other highly correlated items loading strongly on different factors. This process could hypothetically reduce the number of items on the survey by say one third. Of course, permission to administer the revised WFQ would have to be obtained from the hypothetical publisher.

Theory Testing

Another common application of factor analysis involves the empirical testing of theoretical data structures. Here, the central goal is to confirm an existing theory and to identify key variables that represent the theory. A researcher, for example, may be interested in revising a theory of adult personality formation. She can use factor analysis to investigate the underlying correlational patterns shared by personality-related variables in order to test or confirm her theoretical model of personality development.

To conclude, factor analysis has a number of worthwhile applications. Perhaps most importantly, it is a powerful statistical method to determine how best to represent a set of p observed variables with a set of m derived variables so that m < p (Velicer, Eaton, & Fava, 2000). From a large inter-item correlation matrix, the factor analysis process, as explained later, results in a smaller number of interpretable factors or conceptually related dimensions. Technically, a factor (also called a latent dimension or construct) can be described as a “condensed statement of the relationships between a set of variables” (Kline, 1994, p. 5) and it is defined by the magnitude of its factor loadings (i.e., the strength of the correlations of specific variables with a particular factor).

Item Creation

Working from relevant theoretical and research literature, researchers must decide on the construct(s) they want to actually measure. To operationalize this construct and its potential latent dimensions, they generate, after much planning (e.g., determining the number and nature of the variables), a series of interrelated variables, generally more than what is actually required for a good factor analytic solution. Obviously, if the variables do not conceptually “go together,” finding common factors with substantial shared variance is highly unlikely (Kahn, 2006). In devising the variables, the type of data to be analyzed needs to be considered as well; namely, the data collected should be measured on a scale that is suitable for correlational analysis. Although there are ways to factor analyze categorical (e.g., dichotomous “yes or no” items) information, best practice is to start with continuous (ratio or interval) data. Data measured on, for example, a 5- or 7-point Likert-type scale are generally acceptable for factor analysis.

Sample Size Estimation

Prior to collecting data, researchers determine the nature and size of the sample to be studied. In most psychometric investigations, a large sample size is recommended. There are several good reasons for this practice. As the sample size increases, the standard errors of the resulting factor loadings generally decrease. Large samples also tend to provide more stable results and the derived sample loadings more closely estimate population factor loadings. Scholarly consensus is lacking, however, on what sample size is sufficiently large to achieve reliable and stable estimates.

Most researchers determine the minimum sample size required by calculating the ratio of sample size N (total cases) to the number of variables being analyzed, p. Depending upon the expert one reads, the recommended N:p ratio ranges from 3 to 20 (e.g., Cattell, 1978; Comrey & Lee, 1992; Everitt, 1975). Perhaps a middle ground is a ratio of 10 cases per variable. Thus, when 20 variables are under study, the number of cases should be at least 100 and as high as 200. In general, samples sizes greater than 200 are preferred. Comrey and Lee (1992) provided the following sample size guidelines: N = 50—very poor; 100—poor; 200—fair; 300—good; 500—very good; and 1000+ excellent.

It should be noted that while there are recommended minimum sample sizes, studies have shown adequate stability and reproduction of population factor loadings with smaller samples (e.g., MacCallum, Widaman, Zhang, & Hong, 1999). Hogarty, Hines, Kromrey, Ferron, and Mumford (2005) encouraged careful planning in conducting factor analysis, where sample size considerations should also reflect the context of the study and various aspects of the factor solution that are most important to the research (e.g., the communality and overdetermination of factors). With “strong data” a smaller sample size can be adequate. Characteristics of such a robust data set is one that yields high communalities (≥.80), low cross factor loadings (<.32), and moderate to high factor loadings (≥.50). If the researcher is unsure about the quality of the data set, it is better practice to maximize, as much as feasible, the sample size.

Data Collection, Screening, and Checking for Parametric Assumptions

After the data have been secured, they must be closely vetted for missing information, data entry errors, and irregular response patterns. Next, the researcher determines whether the parametric assumptions of factor analysis can be met. Although, similar to other multivariate statistical procedures, factor analysis tends to be robust to modest violations of normality, the solution is enhanced if the variables are normally distributed. Field (2009) recommended a number of procedures to examine the data of normality. For instance, skew and kurtosis indices can be computed for each variable. Preferably, these are less than ±1.0. Variable distributions should be visually inspected using histograms, as well as box, probability–probability (P-P), and quartile–quartile (Q-Q) plots. Basically, a P-P plot (see, e.g., Figure 2) compares the cumulative distribution of sample data to a normal distribution. Similarly a Q-Q plot (see, e.g., Figure 3) assesses quartile values of the sample data to those of a normal distribution. When there is a relatively close match the plot will appear nearly linear.

OPEN IN VIEWER

Figure 2.

Sample P-P plot. P-P = probability–probability.

OPEN IN VIEWER

Figure 3.

Sample Q-Q plot. Q-Q = quartile–quartile.

Moreover, extreme outliers must be identified and, in most cases, removed. Whereas, simple scatter and box plots are used to check for bivariate outliers, multivariate outliers can be detected using a diagnostic tool like the Mahalanobis (D) distance (Field, 2009). The scale-invariant D index is the distance between an observation (e.g., a test score) and the mean of a distribution, taking into account correlations within the data set. Finally, since factor analysis assumes that the variables are linearly related, bivariate scatterplots between variables should be examined.

Creating a Correlation Matrix and Inspecting It for Factorability

Once the data set has been screened and checked for normality, the researcher computes correlations among all the variables. The intercorrelation matrix is then examined for its suitability for factor analysis. Generally, the intercorrelation matrix is factorable if the majority of the correlation coefficients are least low–moderate to strong (r = .20+). If one finds a number of correlations between variables exceeding .85, multicollinearity becomes a concern. Table 3 is a hypothetical 8 by 8 intercorrelation matrix that appears, by visual inspection, to be factorable.

OPEN IN VIEWER

Table 3.

Sample 8 by 8 Intercorrelation Matrix.

	Variables
Vari- ables	1	2	3	4	5	6	7	8
1	1.0
2	.77	1.0
3	.66	.87	1.0
4	.09	.04	.11	1.0
5	.12	.06	.10	.51	1.0
6	.08	.14	.08	.61	.49	1.0
7	.30	.45	.09	.11	.33	.07	1.0
8	.28	.44	.42	.12	.67	.39	.21	1.0

Note. Of the 28 correlations, 15 are >.20, suggesting the matrix is perhaps factorable.

When factor analyzing an intercorrelation matrix comprised of say 50 variables, examination of each correlation is tedious and error-prone. Mercifully, two quantitative methods to assess correlation matrix are available through well-known statistical packages (e.g., IBM SPSS and SAS). One option is Bartlett’s test of sphericity, which examines whether the variables are largely uncorrelated. In the worst case scenario, one finds an identity matrix, where the correlations on the diagonals of the intercorrelation matrix are 1.00 and the rest are 0.00. A significant (p < .05) sphericity χ² suggests that the data set, and thus, the correlation matrix are factorable. Another alternative is to compute the Kaiser–Meyer–Olkin (KMO) to measure the sampling adequacy. The KMO index specifies how small the partial correlations are relative to the original correlations. When the intercorrelation matrix is an identity matrix, the KMO should be .05, the default value in IBM SPSS. Small KMO values indicate that correlations between pairs of variables cannot be explained by other variables. For data to be at least marginally factorable, the KMO should be greater than .60 (Kaiser, 1974). KMO estimates closer to .80 or .90 suggest the intercorrelation matrix is almost ideal for factor analysis (Pett et al., 2003).

Factor Extraction

Once the intercorrelation matrix’s suitability for factor analysis has been established, initial factors (EFA) or components (PCA) are extracted from the matrix. As mentioned previously, one can use an EFA or PCA extraction method. For various technical reasons, statisticians generally prefer EFA over PCA, but with large sample sizes the differences in factor solutions are rarely dramatic (Costello & Osborne, 2005). Without going into too much detail, the extraction process entails the partitioning of the common or shared variance associated with each variable from its unique variance (i.e., variance in each variable that is not shared with any of the other variables) and error variance (i.e., variance not otherwise accounted for in a variable such as random variance) to reveal the underlying factor structure (Brown, 2010). A good factor analytic solution is one where the shared variance represented by the variable’s communality (h ²) is maximized and the unexplained and error variance minimized. Variable communalities are represented as values ranging from 0.0 to 1.0. The closer the h ² is to 1.0, the greater the percentage of variance of a variable (e.g., a test item) is accounted for by a set of common factors or components. In short, a variable’s communality is the proportion of its variance that is explained by each of the extracted factors.

Communalities are derived by squaring each variable’s factor loading across each of the derived factors and summing them. To obtain a percentage, the sum is multiplied by 100. To illustrate the simple math involved, assume survey Item A’s loadings (correlations) on Factors 1, 2, 3, and 4 were .96, −.02, −.08, and −.04, respectively. Clearly, Item A loads highly on Factor 1 and negligibly on the other three factors. The resulting h ² for Item A is calculated as follows: [.96]² + [−.02]² + [−.08]² + [−.04]² = .93. Thus, an exceptionally high 93% (.93 × 100) of the shared variance of Item A is explained by Factors 1 through 4.

It makes sense that the quality of the factor analysis solution improves with increasing communalities. A good factor solution explains most of the variance (50–75%) in the intercorrelation matrix with the possible fewest factors. If most of the communalities are relatively low (<0.5), the variables still possess a substantial amount of unexplained variance. The researcher should therefore consider either extracting more factors to explain the variance or removing the variables with low communalities. High communalities (e.g., >0.8) indicate that the extracted factors explain most of the variance in the variables being analyzed.

Furthermore, a good factor solution means all the derived factors are overdetermined. Such factors possess two major characteristics: (a) each reveals moderate-to-strong loading (e.g., .40 to .80) on at least three variables and (b) the overall factor matrix achieves simple structure. This latter notion implies that the factor solution is most parsimonious, where each variable loads moderately to strongly on one factor and very weakly (near 0.0) on the other factors. A situation where a factor is underdetermined suggests that the underlying factor is difficult to identify and thus, interpret (Comrey & Lee, 1992).

As each item has a communality, each factor has an Eigenvalue (EV or λ). An EV represents the amount of variance that is accounted for by each factor relative to the total variance of the factor matrix and is calculated as the sum of the squared factor loadings for each variable. If the four loadings comprising Factor 1 are −.50, .36, .89, and .92, then the EV equals 2.02 (−.50² + .36² + .89² + .92²). Moreover, EVs are allocated to factors according to the amount of variance explained out of the total variance. The proportion of variance explained by each factor is determined by dividing the factor’s EV by the total number of variables comprising the factor (i.e., EV/number of variables). If Factor 1 from our above example is composed of four variables and the EV is 2.02, it then accounts for about 50% of the total variance in the intercorrelation matrix (2.02 ÷ 4).

The factor analysis process extracts factors one by one until as much variance in the intercorrelation matrix is accounted for by the least number of salient factors. The initial factor always accounts for the most variance. As long as there is additional variance to explain after the first factor is extracted, as is almost always the case, the extraction process continues. When little or no variance is left to be explained in the intercorrelation matrix by the derived factors, the process stops. Each successively extracted factor will have a lower EV, because the amount of variance remaining is gradually diminished. A meaningful extracted factor should always explain more variance than a single variable. Some researchers (e.g., Reise, Waller, & Comrey, 2000) argue that it is preferable to overextract than to underextract factors. Underextraction is more likely to result in factors that contain substantial error. The standard error in factor loadings is influenced by elements other than sample size, including the method of rotation (see below), the number of factors, and the degree to which the factors are correlated (MacCallum & Tucker, 1991). Finally, factors with EVs of at least 1.0 are more stable and thus more likely to be retained as part of the solution (Pett et al., 2003).

Although there is an array of extraction methods that factor analysts can choose from, the discussion here focuses on the two conventional strategies, PCA and principal axis factoring (PAF; also called principal factor analysis). Before the advent of high-speed computing, PCA was the most expedient way of extracting components (or factors). Its central goal is to simplify a large number of items (e.g., 40) to a far smaller number of parsimonious components (e.g., 4). By analyzing both variance unique to each variable and the variance the variables have in common (shared variance), PCA is a process that reproduces the data structure using the smallest number of components. Every pre-extraction item communality must equal 1.0, for all the variance associated with each item is initially represented. When a component is extracted, all the variance associated with it is partialled out, and thus, post-extraction item communalities are far less than 1.0. What remains, the residual, is independent of all extracted components, meaning that each component is treated as if it is independent of all others with intercorrelations of zero. It should be noted that because PCA analyzes all the variance in each variable, some experts contend that PCA is not a true factor analysis procedure (Fabrigar et al., 1999). They contend that the goal of factor analysis is identifying common factors, whereas PCA finds linear combinations that retain as much information as possible.

PAF is another commonplace extraction method. It is best suited for exploring the underlying factors theorized by the researcher (DeCoster, 1998). In other words, the method reveals the latent structure of a set of original variables. The emerging factors from PAF explain the variance common to more than one variable. More specifically, unlike PCA, PAF analyzes only shared variance and leaves out the variables’ unique variance. That is, PAF analyzes only the communality. Figure 4 distinguishes between PCA and PAF in terms of the variance elements (pre-extraction and post-extraction) each method analyzes. Figure 5 presents the initial and extracted communalities for 15 hypothetical self-efficacy survey items using PCA and PAF, respectively. Notice that for the PCA, the initial communalities (pre-extraction) are 1.0 and for PAF, they are relatively lower. The reason for this difference lies in the fact that PCA places 1.0 in intercorrelation matrix as the initial communality. Instead of 1.0, PAF uses the R ² of intercorrelation matrix as the initial estimate of communality. Computationally, the R ² will always be less than 1.0, so the extracted communities for PAF will also be somewhat smaller than those for PCA (see Figure 5). Since the extracted communalities for both PCA and PAF represent the proportion of variance in an item explained by other items, the h ² for each variable with both extraction methods is less than 1.0.

OPEN IN VIEWER

Figure 4.

Variance analyzed by PCA and PAF. PCA = principal component analysis; PAF = principal axis factoring.

OPEN IN VIEWER

Figure 5.

PCA (left) and PAF (right) extraction output for 15 items (IBM SPSS).

Quite often, different extraction methods computed on data sets with large sample sizes produce comparable factor solutions. Simulation studies, however, show PAF to generate a more reliable solution when communalities are low (Kahn, 2006). Furthermore, PAF is robust to violations of normality in the data. In contrast, PCA can lead to misleading results as it tends to yield larger loadings than do other methods. Again, if one’s purpose is to determine latent factors in the data, PAF and related methods (e.g., maximum likelihood [ML]) are preferred. As we mentioned earlier, some researchers use multiple approaches to factor extraction, retention, and rotation, and then compare the results. The most advantageous factor structure is then reported. Doing this is an unsound practice, compromising the primary aims of each method.

Factor Retention

To reiterate, the goal of factor analysis is to explain the maximum amount of variance in the intercorrelation matrix with the fewest factors or components. Choosing the right factor extraction approach is vital, but just as importantly, one must accurately decide on the number of factors to be retained for subsequent rotational analysis. Most factors that could be extracted are not meaningful (i.e., they do not account for much variance), and therefore, should not be retained. The major statistical programs use a default setting called the Kaiser criterion, which keeps only those factors with an EV over 1.0. Researchers need to override the setting, for accepting default position may lead to erroneous decisions. Pett, Lackey, and Sullivan (2003), among many other experts, recommended starting with the Kaiser 1.0 guideline and then examining the scree plot (see, e.g., Figure 6). Basically with this line graph, factor EVs (Y-axis) are plotted against the number of possible factors (X-axis). The point at which the line begins to show a clear bend indicates the actual number of factors that should be retained. In other words, the cutoff for the number of factors to retain is the point on the graph where additional factors fail to add appreciably to the cumulative variance. By inspecting the sample scree plot depicted in Figure 6, an experienced factor analyst would probably retain three factors.

OPEN IN VIEWER

Figure 6.

Sample scree plot (IBM SPSS).

Conducting a parallel analysis (PA) can augment one’s decision-making process (Horn, 1965). Unlike the Kaiser’s criterion and scree plot approaches, PA takes into account sampling error. Essentially with PA, random data sets are generated from those observations that match the observed data in size and number of variables. The mean EVs for the random data are calculated. The factors that are retained are at those points where the EV of the sample data exceeds that of the randomly generated data. As illustrated in Figure 7, three factors would be retained. Accuracy of prediction is improved by replicating the process with multiple sets of random data.

OPEN IN VIEWER

Figure 7.

Illustration of parallel analysis.

Yet another option to assist with determining how many factors to retain is the minimum average partial (MAP) correlation procedure (Velicer, 1976). In the MAP procedure, a new correlation matrix is calculated after each factor is partialled out of the original matrix. If the removal of the factor results in the removal of common variance, the MAP function will decrease. If the removal of the factor results in the removal of unique variance, the MAP function will increase (see Figure 8). The point where the MAP function is at a minimum indicates the number of factors to be retained. Here, the matrix most closely resembles an identity matrix. Examining Figure 8, the MAP procedure suggests that two factors would be retained. In short, adding PA and MAP to the decision-making process improves the likelihood of retaining the correct number of factors (Zwick & Velicer, 1986).

OPEN IN VIEWER

Figure 8.

Illustration of MAP procedure. MAP = minimum average partial.

Factor Rotation

Once the decision on the number of factors to be retained is made, the researcher deploys the statistical software to rotate the initial factors in such a way to maximize simple structure. Before discussing the nuances of the process and the different rotation strategies, further explanation of key concepts is needed. First, factor loadings indicate the relative importance of each variable to each factor. If one squares a factor loading, the amount of variance that the variable explains on a particular factor is calculated. Consequently, a variable with a higher factor loading (e.g., .77) is relatively more important to the factor than a variable with a minimal loading (e.g., .20), because after squaring the loading, the amount variance explained is more substantial (e.g., .77² = .59 or 59% vs. .20² = .04 or 4% of the variance). The variables with the strongest loadings on a particular factor are used to “mark” or “define” a factor. The threshold magnitude to mark a factor varies from study to study, but in most cases, it should be no lower than .30 or approximately 10% of the explained variance. Conservatively speaking, the minimum factor loading should be .35 or even .40 (Comrey & Lee, 1992).

To reiterate, the initial solution indicates that all variables have a tendency to load strongly on the first factor, for it accounts for the maximum amount of the variance. In other words, since the first factor is generally more highly correlated with the variables than the second factor, it is to be expected that Factor 1 will account for the most overall variance. The second factor is orthogonal to the first, as is the second from the third, and so on. With each successive factor extraction (called iterations), less and less variance overall will be explained.

The initial (prerotation) solution is referred to as the unrotated factor structure or a matrix of derived factor loadings. Variables in the matrix are represented as rows and factors as columns. For the most part, the unrotated factor matrix is indecipherable. Loadings on particular factors will vary widely and many of the variables will load on two or more factors (crossload). Table 4 illustrates this situation. To create a more interpretable factor structure, the factor loading matrix most often is rotated. If each row of the loadings matrix is a coordinate of a point in M-dimensional space, then each factor corresponds to a coordinate axis. Factor rotation is equivalent to rotating those axes and computing new loadings in the rotated coordinate system. In an attempt to maximize simple structure, rotation essentially changes the “viewing angle” of the factor space. After rotation, the vectors are rearranged to optimally go through clusters of shared variance. The resulting factor loadings and the factors so produced can be more readily interpreted. After rotating the initial factor matrix (see Table 4), the PAF-rotated factor matrix (see Table 5) shows 12 items clearly loading on Factor 1 and 8 items marking Factor 2. Notice in rotated factor matrix that Items ESA18 and ESA19 on Factor 1 and Items ESA10, ESA5, and ESA7 located on Factor 2 have weak factor loadings (>|.35|). These may be eliminated.

OPEN IN VIEWER

Table 4.

Prerotation Factor Matrix.

Variables	Factor
	1	2	3	4	5
ESA1	−0.22	0.42	0.06	−0.16	0.24
ESA2	0.36	0.22	−0.06	−0.20	0.03
ESA3	0.64	0.07	0.30	−0.11	−0.10
ESA4	0.69	0.11	−0.08	0.15	0.15
ESA5	0.05	0.30	−0.06	−0.07	−0.19
ESA6	0.58	0.11	0.00	0.19	0.07
ESA7	0.09	0.29	0.06	0.08	0.03
ESA8	0.63	0.12	−0.50	−0.22	−0.01
ESA9	0.64	0.15	−0.09	0.09	0.06
ESA10	0.05	0.34	0.06	0.26	−0.20
ESA11	0.38	0.28	0.19	0.11	−0.24
ESA12	−0.34	0.47	−0.22	0.00	−0.12
ESA13	0.57	0.12	−0.12	0.26	0.04
ESA14	0.56	0.06	0.33	−0.12	−0.02
ESA15	−0.21	0.58	0.20	−0.09	0.11
ESA16	0.70	0.12	0.02	0.05	0.09
ESA17	−0.31	0.51	−0.19	−0.09	−0.21
ESA18	−0.39	0.09	0.39	0.35	0.03
ESA19	−0.40	0.20	−0.42	0.35	0.13
ESA20	−0.25	0.53	0.20	−0.12	0.21

Note. Extraction method = principal axis factoring; five factors extracted with Eigenvalue 1.0.

OPEN IN VIEWER

Table 5.

Rotated PAF Factor Matrix (Varimax Rotation).

Variables	Factor
	1	2
ESA16	.72
ESA4	.69
ESA9	.66
ESA3	.62
ESA6	.60
ESA8	.59
ESA13	.58
ESA14	.55
ESA11	.42
ESA2	.39
ESA18	−.34
ESA19	−.33
ESA15		.61
ESA20		.56
ESA17		.55
ESA12		.52
ESA1		.44
ESA10		.30
ESA5		.28
ESA7		.27

Note. PAF = principal axis factoring. Loadings ≤.20 were suppressed.

Researchers can rotate the factor matrix orthogonally or obliquely. By definition, the orthogonal approach rotates the vectors at 90° angles to minimize the amount of factor covariation. Once rotated, the factor loadings from the rotated factor matrix should be interpreted. The magnitude of the factors loadings will change, but the variable communalities will remain the same after rotation. IBM SPSS has multiple options that generate relatively similar factor solutions, including varimax, quartimax, and equamax rotations. The orthogonal procedure may be appropriate when theory and research support uncorrelated factors, namely, when one is measuring distinct concepts. For example, research attempting to validate the Cross-cultural Counseling Inventory, a three-factor orthogonal solution of cross-cultural counseling skill, sociopolitical awareness, and cultural sensitivity was reported (LaFromboise, Coleman, & Hernandez, 1991). Orthogonal rotation, however, should be rarely used in counseling investigations, because most constructs are intercorrelated (e.g., depression and optimism). Ignoring this reality yields a factor solution that is largely synthetic in nature.

Oblique rotation addresses this artificiality but increases the interpretation complexity of the factor solution. With this approach, the rotated vectors are intentionally put at angles less than 90° (oblique angles), allowing for some factor covariation (e.g., r ≤ .30). With IBM SPSS, oblique rotation options are direct oblimin and promax. The parameter delta (Δ) determines the extent of obliqueness of the factors. The default setting is 0.0; however, researchers can override by changing the Δ to a negative value. Doing so lessens the degree of factor intercorrelations, because the factors or components become less oblique. Positive Δ values increase the correlation between factors. Harman (1976) advised researchers to avoid using Δ values greater than .80, for they result in highly correlated factors that are virtually indistinguishable. Obtaining the best solution (i.e., simple structure while limiting the size of the factor intercorrelations) using the oblique approach requires extensive factor analytic skills.

Since the final factor solution (post-oblique rotation) includes the amount of overlap (correlation) between factors, the amount of variance explained by a rotated factor differs from its initial solution. Like orthogonal rotation, oblique rotation does not alter the communalities.

The choice of the rotation method is guided by the researcher’s goal (Rennie, 1997). If the primary objective is a factor structure that best fits the data, oblique rotation is a better choice. Sample measurements substantially impact oblique rotation factor solutions, making the results less likely to be replicated in other studies. Should the research goal center on factor generalizability, orthogonal rotation is preferred. In many research scenarios, the results of oblique and orthogonal rotation tend to be similar. This is particularly the case when the factors are weakly intercorrelated. Rennie suggested that orthogonal rotation be the default method unless the results of oblique rotation are noticeably different. Pedhazur and Schmelkin (1991) claimed that orthogonal rotation produces results that are parsimonious and more replicable. Hence, in their view, this advantage outweighs the criticism that the orthogonal results fail to represent real-world conditions.

Whichever rotation strategy the counseling researcher adopts, the key is to have defensible theoretical and research-based reasons for one’s decision making (Comrey & Lee, 1992). To reiterate, the choices should ensure that the derived rotated factor structure is coherent, achieving simple structure. Each variable must have at least a moderate loading (i.e., at least .30 to .40, depending on the investigation) on a single rotated factor. However, the judgment of whether a variable “belongs” with a factor must be based on the squared value of each particular loading, not simply the loading. The squared loading reflects the amount of variance that the variable contributes to the factor. Additionally, each factor must have at least three moderate-to-strong loadings, as more loadings tend to yield greater reliability (i.e., stronger factor internal consistency). Four to 10 items per factor are typically considered reasonable.

Naming the Factors

The final step is to label each factor or component in such a way as to represent as a whole the conceptual meaning of each variable defining a particular latent dimension. Admittedly, this is a somewhat subjective process, requiring a strong grasp of the theoretical and research literature from with the measure was constructed. To illustrate this process, suppose a new survey based on Lanes, Kuk, and Tamim’s (2011) research related to depression in postpartum mothers was created. Following an EFA, two reliable and valid factors were generated. Fifteen items strongly loaded on Factor 1. A counseling researcher understood these items to be principally addressing a mother’s “ability to cope with life events” after birth. The content of the 12 items comprising Factor 2 involves facets generally associated with “negative clinical implications for maternal-infant attachment.” Perhaps, the researcher would title the first dimension “coping ability” and the second, “negative attachment.”

A Word of Caution

Counselors and researchers must understand which conclusions can and cannot be drawn from factor analytic studies, particularly those using EFA. This procedure provides good information about relationship among variables and factors, but one cannot draw causal inferences from these relationships. CFA, however, allows the researcher to make such inferences. In the next section, CFA is overviewed.

Defining the Model

Based on existing theory and research, the first phase is to articulate the conceptual model and the hypotheses to be confirmed. Since they cannot be observed or directly measured, the factors or dimensions are referred to as latent variables. A counseling researcher, for instance, may be interested in evaluating a recognized model of test anxiety with a new sample. Data are collected on observed and measurable variables which the researcher determines to be proxies for the latent variables. Pulse rate serves as a proxy for the participants’ level of physiological anxiety. Once the latent and observed variables are identified, a structural model is developed that predicts which variables will load on the hypothesized factors (see, e.g., Figure 9). Extending the scenario a bit further, the two ovals in Figure 9 could represent two latent and unobserved dimensions called physiological anxiety and psychological anxiety. The arrows point toward measurable and observed indictors of these dimensions (rectangles). The goal is to determine whether these indicators actually represent the hypothesized constructs. Stated differently, because the latent phenomena are theorized to cause the observed variables, the model shows arrows pointing from the unobserved to the observed variables. The double-headed arrow at the top of the model indicates that the two unobserved variables are correlated to some degree. This suggests that physiological and psychological anxiety dimensions are correlated. Once the CFA is computed, the magnitude of this relationship becomes known.

OPEN IN VIEWER

Figure 9.

Sample hypothesized structure model used in confirmatory factor analysis.

Data Collection

Because the goal is inference, for CFA, samples are generally larger than what would be considered adequate for EFA. The size of the sample speaks to the reliability of the estimates. Whatever sample size is used, the rule of thumb guiding the selection of N should be specified. Data are then collected on the observed variables and the data set must be complete to ensure factorability. Specifically, unless data are missing at random, researchers must address this issue and describe what method they have used to rectify the situation. A couple of ways to deal with missing data is to insert a hypothetical score or to exclude the individual from the sample (Huck, 2012). IBM SPSS provides two options. The first is to drop the cases from CFA. Doing this is problematic, for it may lead to a bias in estimates in cases where data are not missing at random. The second option is to eliminate the variable in instances where data are absent. This strategy obviously changes the covariance matrix to be factored. AMOS and LISREL have a built-in procedure (full information maximum likelihood [FIML]) to estimate the values of the missing data, allowing the researcher to use all participant data. FIML has its limitations as well.

Correlation Matrix

CFA uses a correlation or a covariance matrix developed from the sample data as a beginning point of the analysis. Covariances and correlations between variables allow for the inclusion of a relationship between two variables that is not necessarily causal. Typically structural equation models contain both causal and noncausal relationships. The correlations are restricted to hypothesized relationships. This is a distinction from EFA which assumes that all variables are correlated with all factors. The CFA model parameters are estimated based on these sample values. Often times the residuals matrix is examined, for it represents the difference between the sample correlation matrix and the implied correlation matrix. Residuals analysis is particularly useful when the observed or measured variables are scaled differently.

Fitting the Hypothesized Model to the Data

The SEM framework is typically used in testing models for data fit. This structure is valid under the assumption of multivariate normality. That is, each observed variable should largely follow a normal distribution; all combinations of observed variables must have normal joint distributions; and all bivariate scatterplots are approximately linear with distributions of equal variance. When this assumption of multivariate normality is even moderately violated, there is a potential distortion in the fit statistics as well as increased probability of committing a Type I error. Once the assumptions of CFA are met, the overall analysis can be computed.

As previously mentioned, CFA emphasizes model fit, which is not synonymous with model validity. Specific analyses are designed to evaluate one or the other. In EFA, the GOF test is conducted on individual factors using a holistic approach. GOF looks at a set of relationships between the observed and the latent variables. The model that is being tested is referred to as the default model. When all the variables are linked together and the model fits the data as well as possible, the model is said to be saturated. When no such connections exist and the model represents a poor fit to the data, it is said to be independent. Clearly then, the objective of CFA is to have the hypothesized model reflect a saturated model as closely as possible.

Evaluating the Model

Experts have developed numerous indices to assess fit by looking at both the degree of difference between the model and data and the degree to which the model and data match (see Table 6 for widely accepted sample indices). They can be classified as absolute, incremental, and parsimony measures. Whereas absolute fit indices evaluate how well the hypothesized model fits the data, incremental fit indices compare the model to a null model in which all the observed variables are uncorrelated. Moreover, incremental indices measure the improvement so that greater fit corresponds with greater improvement to the null model. Parsimony fit measures indicate if the specified model is parsimonious. They help determine whether the model can be improved by specifying fewer estimated parameter paths and thus creating a simpler model. When the indices are used in combination, the researcher’s conclusion about best fit tends to be more reliable. Most statistical software report similar indices, but none of them output every alternative.

OPEN IN VIEWER

Table 6.

Sample Well-Researched CFA Indices for Assessing Model Fit.

Index	Type	Goodness-of-Fit Values
Root Mean Square Error Approximation	Parsimony	>.10 = poor; <.10 = adequate
Standardized Root Mean Square Residual	Absolute	<.08 = adequate
Nonnormed Fit Index	Incremental	>.90 = adequate
Comparative Fit Index	Incremental	>.90 = adequate

Note. CFA = confirmatory factor analysis.

Generally, CFA researchers first examine two χ² values (χ² and χ²/df), both conventional measures of overall model fit. These statistics describe the similarity of the observed sample covariance matrix and expected covariance matrix. The null hypothesis is as follows: There is a nonsignificant difference between the two covariance matrices. Since a good CFA result is when the matrices are quite similar, one anticipates a nonsignificant χ²(p > .05), allowing the researcher to retain the null hypothesis. For a variety of reasons (e.g., large sample sizes generate significant χ²s), this GOF index is most often inaccurate, thus other indices must be considered.

Next, the Root Mean Squared Error of Approximation (RMSEA) index should be examined. The RMSEA indicates the amount of unexplained variance or the residual. It gives a suggestion of how well the parameter estimates fit the population covariance matrix. The value of RMSEA is influenced by sample size, the degrees of freedom, and the χ² value. RMSEA values nearer to 0.0 indicate a quality fit. A general guideline for an adequate and good fit is <.10 and <.05, respectively. In short, RMSEA denotes the level to which lack of fit is due to misspecification of the tested model as opposed to being a result of sampling error.

Second, researchers check the Standardized Root Mean Square Residual (SRMR) index, as well as the Nonnormed Fit (NNFI) and Comparative Fit (CFI) indices. The SRMR is an absolute measure of fit that represents the standardized difference between the observed covariance (correlation) and that implied (predicted covariance) by the model. As the sample size increases and the number of parameters increases, the SRMR becomes smaller, and thus, the hypothesized model better fits the data. Even though researchers strive for a “perfect” fit (SRMR = 0.0), a value less than .08 is generally considered a good fit. The NNFI is comparable to R ², the coefficient of determination. A zero value indicates the worst possible model fit and a value close to 1.0 indicates a good fit. The NNFI is a fit index that is less affected by sample size. Finally, the CFI measures the extent to which the hypothesized model differs from the null model. In the latter model, the variances and covariances are zero. CFI values nearing 1.0 (≥.95) are considered sufficient evidence for a good model fit.

The model fit may be inadequate because an observed variable loads heavily on multiple factors. A solution to this may be to eliminate the problematic variables. One consideration when eliminating variables is the size of the main factor loading. The suggested minimum loading is .35 to .40. Furthermore, the variables should have cross-loading of no more than .30. Attention must also be given to the contribution the variable makes to the factor’s interpretability as well as its face validity. Further guidance as to whether a particular variable should be retained or excluded is provided by the number of variables already comprising the factor. Should the researcher decide that the best way to resolve an inadequately fitting model is to eliminate variables, this process must be done one variable at a time with the CFA repeated before the next exclusion. If two variables have comparable factor loadings and reliabilities, the variable with the least skew and kurtosis should be preserved.

Comparing Models

It could also be that the original hypothesized model itself is inadequate. In such a circumstance, a solution would be to return to the data and conduct an EFA to generate a model that can then be later tested. It is advisable to compare models with different latent variables. The researcher can examine the fit statistics for each model as well as conduct a χ² test to compare the models. Separate model fit χ² statistics are computed. The difference between the two is then evaluated for statistical significance. The researcher should also provide evidence showing that a model that has been modified and reanalyzed is statistically superior to the original model with a χ² test. A model that has been modified, a trimmed model, is referred to as a nested or hierarchical model. There must be a theoretical basis for the modification to the model. Changes must not be made simply because an analysis called for an addition or reduction in parameters. By paying attention to the theoretical basis of modification decisions, the likelihood of making a Type I error is reduced (Schreiber, Stage, King, Nora, & Barlow, 2006).