Exploratory Factor Analysis

Abstract

The Problem

Exploratory factor analysis (EFA) serves many useful purposes in human resource development (HRD) research. The most frequent applications of EFA among researchers consists of reducing relatively large sets of variables into more manageable ones, developing and refining a new instrument’s scales, and exploring relations among variables to build theory. Because researchers face a number of decisions when conducting EFA that can involve some subjectivity (e.g., factor extraction method, rotation), poor analytic decisions regarding how the EFA should be conducted (e.g., number of factors to extract) can produce misleading findings to the detriment of these efforts, especially theory building.

The Solution

Steps must be taken to improve the quality of the decision making associated with conducting EFAs if sound theory building and research related to this statistical method is to continue. Higher quality EFAs facilitate higher quality theory building and research.

The Stakeholders

HRD theorists, researchers, and scholar-practitioners are the intended audience of this article. In particular, those interested in refining measures and theory building would benefit most from being exposed to best EFA decision-making practices.

Keywords

exploratory factor analysis theory-building factor extraction rotation

The process of theory generation and development is a cornerstone of social science research like human resource development (HRD) because it serves to guide the systematic examination and understanding of curiosity-inducing phenomena occurring in our daily world. Testing theory generates novel empirical research whose findings can inform additional theory building, empirical testing, and ultimately practice (Cumming, 2014). Indeed, without theory and its continuous development, it seems most unlikely a field could move meaningfully forward (Onwuegbuzie & Daniel, 2003; Reio, 2010). Social science theory generation and building in much of the 20th century was dominated by deductive, quantitative approaches for creating new knowledge at the expense of underemphasizing more inductive, mixed-methods and qualitative approaches, with notable exceptions (e.g., anthropology; Jaccard & Jacoby, 2010). More recently, the theory-building pendulum seems to have swung to mixed-method and qualitative approaches, perhaps unnecessarily undervaluing the possible contributions of quantitative approaches (Haig, 2013; Hesse-Biber & Leavy, 2008). As one part of an entire issue of Advances in Developing Human Resources concerning quantitative theory building, this article attempts to clarify how exploratory factor analysis (EFA) as a quantitative method can and should be used for theory generation and building purposes.

There is a broad array of statistical tools available to social science researchers for conducting research that is theoretically, empirically, and/or practically relevant. EFA is an example of one such tool. Distinguished from confirmatory factor analysis (CFA), the primary purpose of which is to test theory about underlying latent processes at later stages of research, EFA’s purpose is to ascertain the most parsimonious number of interpretable factors required to explain the correlations among the observed variables, with or without underlying theoretical processes in mind (Thompson, 2004). Thus, EFA is a method for identifying the factor structure of a set of multiple indicators or variables without imposing an a priori structure on the factors (Stevens, 1996; Widaman, 2012). EFA is performed at early stages of research to consolidate variables and generate new hypotheses about underlying theoretical processes. In essence, it can be used to inform and generate or develop theory; CFA, however, is used to test or confirm an a priori theory (Stevens, 1996). The authors emphasize that neither EFA nor CFA has more or less scientific merit than the other, as both are equally important for advancing theory and research.

As social scientists, HRD researchers use EFA for evaluating construct validity, testing hypotheses, and refining measures (Conway & Huffcutt, 2003). Although the use of EFA is widespread in organizational research, as with any research or statistical method, there are limitations associated with its use (Cumming, 2014). Researchers must remain attentive to these issues and consider them accordingly in their research designs. We present three of the major considerations here (Fabrigar, Wegener, MacCallum, & Strahan, 1999; Tabachnick & Fidell, 2007). First, EFA lacks an external criterion beyond interpretability against which the researcher can evaluate the solution. This is in contrast to, for example, multivariate analysis of variance where the solution is evaluated by how well it predicts the external criterion of group membership. Nimon and Reio (2011) highlighted the utility of examining the replicability of measurement across groups (factorial invariance) as a possible means to handle the external criterion issue. Second, EFA is frequently used to salvage poorly designed research. Even the “sloppiest” research results can be factor analyzed; thus, it can be associated with something akin to a “fishing expedition” that can lead to the faulty analysis and interpretation of findings. Such research errors can ultimately hamper theory generation and building (Tabachnick & Fidell, 2007). A way to avoid this issue is to resist the temptation to salvage sloppy research because it is bad research practice and instead design and conduct more theoretically and empirically sound research. Finally, and the focus of this article, the quality of the decision processes behind conducting EFAs tends to be uneven at best (Conway & Huffcutt, 2003; Fabrigar et al., 1999; Gorsuch, 1997; Thompson, 2004). Poor decisions about factor extraction methods, number of factors to extract, type of rotation to use, factor score estimation method, appropriate sample size, and handling missing data are examples of issues related to EFA decision practices that can render theory development activities problematic. Researchers’ careful attention to implementing quality decision processes related to EFA would reduce greatly the likelihood of analytic, interpretation, and theory-building errors. Inasmuch as the field of HRD is an emergent one, with a sense of urgency for developing and testing new theory in support of moving the field forward (Reio, 2009, 2010; Torraco, 2005), the aim of this article is to highlight the aforementioned issues and offer solutions through recommending best EFA decision-making practices.

Best Practice in EFA Decision Making

EFA requires the researcher to make a number of decisions that are too often misinformed to the detriment of theory, research, and practice. Appropriate decisions associated with conducting EFAs tend to be poorly supported theoretically and empirically (e.g., violation of multivariate normality assumptions), leading to errors of use to the detriment of quantitative theory building (Conway & Huffcutt, 2003; Fabrigar et al., 1999; Kahn, 2006; Schmitt & Sass, 2011). We present the following best practices in EFA decision making and note how each might unduly influence theory building if used incorrectly: selection of observations (sample size and participant-to-variable ratio), factor extraction method, number of factors to extract, type of rotation, and interpretation and naming of factors.

Selection of Observations: Indicator Selection, Sample Size, and Participant-To-Variable Ratio

When designing a study, quality decision making requires attending to the selection of individuals into the sample. Besides the notion that the sample should be representative of the population in question, researchers largely favor a more heterogeneous sample for generalization purposes to better assure a spread of scores related to the variables and factors they measure (Widaman, 2012). If all the participants attained roughly the same score on a factor, correlations will be lower and the factor may not emerge in the EFA (Tabachnick & Fidell, 2007). An example of this would be where a researcher, as part of a larger study of technical skills in the accounting industry, administered a battery of basic arithmetic tests to a sample of certified professional accountants (CPAs). As the range of individual differences among the CPAs may be rather narrow (there would be a ceiling effect because almost everyone would perform well), the absolute magnitude of correlations among the arithmetic tests may be restricted as compared with a general sample of workers, making the results not especially meaningful. Thus, it is vital that the researcher selects participants who are expected to vary on the observed variables and underlying factors (and avoid possible restriction of range issues as noted above)—hence, the recommendation for heterogeneous samples, as appropriate, in most situations.

Sample size and participant-to-variable ratio are additional decision issues. Research remains unclear about what constitutes an ideal sample size for an analysis (considering that the sample is representative), but larger samples are better than smaller samples because the probability of error is less, population estimates are more accurate, and the findings are more generalizable (Treiblmaier & Filzmoser, 2010). Comrey (1973) suggested sample sizes of 50 to be very poor, 200 as being fair, and those with 300 or more as being good or very good. Stevens (1996) recommended considering both the absolute magnitude of the loadings (pattern and structure coefficients; see Henson & Roberts, 2006) and absolute sample size. Specifically, components or factors would be reliable if there were four or more loadings of .60 or above, regardless of sample size. More recently, participant-to-variable ratios ranging from 5:1 to 10:1 rather than sample size have been touted as being more useful for analysis (Widaman, 2012). Demonstrating that sample size is not the key criterion for conducting EFAs, Dahling, Chau, Mayer, and Gregory (2012) used 179 participants to help refine an initial version of a 21-item pro-social rule-breaking workplace measure to 13 items. The participant-to-variable ratio was 8.5:1, and the analyses yielded three clear, interpretable factors that supported the next two phases of instrument development and validation.

To sum, because small samples tend to yield less reliable correlation coefficients, and few, if any, interpretable factors, it is vital having a sample large enough to reliably estimate the correlations (Henson & Roberts, 2006). Participant-to-variable ratios of 5:1 or greater are strongly recommended (with at least 100 participants), despite evidence though that 3:1 ratios can be useful for EFA work in the presence of strong, reliable correlations and few distinct factors (Tabachnick & Fidell, 2007). With ratios less than 5:1, however, the question of replicability of the factors remains. Lack of replicability can yield misleading results that can distort theory-building efforts.

Factor Extraction Method

There are a number of factor extraction approaches available, most of which can be categorized generally as either a components or common factor approach (Henson, Capraro, & Capraro, 2004. Principal components analysis (PCA; component model) and principal axis factoring (PAF) with estimated communalities (common factor model) tend to be most common high-quality extraction strategies used (Conway & Huffcutt, 2003; Onwuegbuzie & Daniel, 2003). PCA produces components, which represents the linear combinations of variables that retain as much information as possible about the original measured variables. PAFs, however, produce factors that reveal the latent structure of the original measured variables (Mvududu & Sink, 2013). The terms are often used interchangeably for ease of discussion, but the researcher must remain mindful that there are “theoretical and semantical differences” between the two extraction models (Henson et al., 2004, p. 63).

What differs primarily between the two methods is their respective purposes. The aim of a PCA is to reduce large numbers of variables into something more manageable that retains as much as possible of a set of variables’ observed variance, with little attention to interpreting latent constructs. Thus, all the observed variables’ variance is analyzed in a PCA. However, the aim of PAF is to “understand the latent (unobserved) variables that account for relationships among measured variables” (Conway & Huffcutt, 2003, p. 150). PAF uses communality estimates (h²; a measure of shared variance with values between 0 and 1) instead of ones in the correlation matrix’s diagonal (a PCA uses ones that represent both common and unique variance in the observed variables) to eliminate measurement error due to common variance (Henson et al., 2004).

Accordingly, if the researcher wanted to simply reduce the number of variables without interpreting the resulting variables as latent constructs, a PCA is appropriate. If the researcher’s purpose was to understand the latent structure of a set of variables, PAF would be most appropriate. Gorsuch (1990) suggested because PCA and PAF often produce equivalent results, it would make more sense using PAF in most cases because, generally, researchers want to better understand a set of variables’ latent structure. Caution must be exercised with both extraction methods, however, because the results can be heavily biased by non-normally distributed data and outliers (Treiblmaier & Filzmoser, 2010).

There remains quite a bit of scholarly debate as to the relative merit of each approach, particularly because each can produce virtually identical results in terms of interpretation (Thompson, 1992; Velicer & Jackson, 1990). The differences between the two extraction methods tend to decrease with higher score reliabilities and a greater number of factored variables (see Henson & Roberts, 2006; Onwuegbuzie & Daniel, 2003). Because one measure of the stability of a factor-analytic solution is that it materializes regardless of extraction technique, this state of affairs is arguably useful for theory-building purposes. PCAs, for example, are often used by researchers as a preliminary extraction technique, followed by varying the number of factors (guided by theory) and rotational methods (Tabachnick & Fidell, 2007).

The results from research studies using PCA can clearly augment the social science knowledge base. Reio and Ghosh (2009), for instance, in their study of the possible links among workplace incivility, job satisfaction, and physical health used PCAs to generate factor scores for use in their predictive regression models. The researchers also used PCAs to get a preliminary sense of the modified measures’ construct validity, while recommending new research to further validate the scales. The new knowledge gained from such studies can inform the design of future research studies that can be used to generate or build theory (an indirect approach). Still, theory-building efforts are best served by using PAF rather than PCA because it draws heavily on prior theory to guide factor extraction and subsequent interpretation of the latent constructs representing a theory (Kahn, 2006).

It is not hard imagining a situation where an organizational researcher with access to a secondary data set containing a wide range of typical employee data might want to explore the variables most associated with voluntary turnover. The data set might include age, sex, education, professional development workshop participation, annual salary, organizational tenure, job experience, professional experience, hours worked per week, unexcused absences, absences, tardiness, turnover, and so on. If the researcher had no particular theory or research in mind, a PCA would be useful for simple data reduction purposes. The PCA would reduce the larger set of variables into a smaller, more manageable set of components. The resulting components should not be interpreted as latent constructs because they lack a theoretical basis. Alternatively, an EFA or CFA would be best if theory and research suggested the presence of latent constructs (called factors) undergirding the observed variables (e.g., unexcused absences, absences and tardiness might load together to form a factor called “absenteeism” and organizational tenure, job experience, professional experience might form an “experience” factor).

Number of Factors to Extract

Another vital decision for conducting quality EFA is choosing the correct number of factors (components) to retain (Conway & Huffcutt, 2003). There are a number of factor retention rules to assist the researcher in making sense of their data, but each has its limitations (Henson et al., 2004; Onwuegbuzie & Daniel, 2003). The most common rules include the eigenvalue-greater-than-one rule (EV > 1; Kaiser, 1956), scree test (Cattell, 1966), minimum average partial (MAP) correlation (Velicer, 1976), parallel analysis (Horn, 1965), Bartlett’s (1950) chi-square test, and a priori theory (Henson & Roberts, 2006). Extracted factors should explain at least 40% of the total variance in the original variables, although some recommend at least 75% (e.g., Stevens, 1996). Henson and Roberts (2006) challenged Stevens’ rule-of-thumb as being unreasonable for applied psychological research. Factor eigenvalues (eigenvalues are an index of the variance explained by a factor; eigenvalues greater than one are considered interpretable; Gorsuch, 1983) and the percentage of variance explained by each factor should be reported, as well as item communalities related to the rotated factors (Kahn, 2006). Because Bartlett’s chi-square test is heavily influenced by sample size, Henson and Roberts (2006) did not recommend its use in EFA studies as they typically use large samples. Both MAP and parallel analysis have been found to be relatively accurate factor retention rules, with parallel analysis emerging as the most accurate (Fabrigar et al., 1999; Henson et al., 2004). Parallel analysis compares sample data eigenvalues to eigenvalues that would be expected from random data. The factors with eigenvalues greater than the random eigenvalues are retained. Because the procedure remains unavailable in most statistical packages (Thompson & Daniel, 1996, have provided a syntax program, however, for running the analysis in SPSS), the procedure has limited use in social science research (Kahn, 2006). We will focus on the EV > 1 rule, scree test, and a priori theory for the purposes of this discussion as they are by far the most common methods used by social science researchers. See Henson and Roberts (2006) for further exploration of factor retention rules.

Because each of the retention rules has been shown to have limitations, research has shown that using multiple techniques to inform factor retention as being superior to using single decision rules (Fabrigar et al., 1999). The EV > 1 rule is such that eigenvalues greater than 1 can be interpreted as a factor; values less than 1 would not be (Kaiser, 1956). The EV > 1 rule is inconsistent across differing numbers of variables (10-30 is optimal); overall, it tends to extract too many factors, making it extremely problematic because it is the default option in most statistical packages (Henson et al., 2004). Scree test analysis (Cattell, 1966) is a graphical method where the number of eigenvalues is plotted in descending order against the number of factors. The researcher is tasked with finding a visual “elbow” in the plot where there is a distinct transition from large to small eigenvalues; unfortunately, a clear elbow is not always obvious. Consequently, the researcher is forced to make an ambiguous subjective judgment about the number of factors to retain (Ruscio & Roche, 2012). Notwithstanding evidence that scree plots can underestimate or overestimate the number of factors to extract, they tend to be more accurate than when using the EG > 1 rule, despite its subjectivity (Henson et al., 2004). In contrast, Ruscio and Roche (2012) claimed that overfactoring (EV > 1) “leads to fewer errors when factor loadings were estimated . . . [yet] specifying too many factors might lead to the creation of constructs with little theoretical value” (p. 199). A priori theory is also a useful decision rule criterion because we must remain mindful that factor extraction must be theoretically plausible. Thus, the researcher conducting an EFA should triangulate the factor extraction results of the EV > 1 rule and scree test with the theory supporting the research. Errors in factor extraction can have considerable implications for theory building because any insights that might have been gained from the analysis might be masked by the misleading results (Treiblmaier & Filzmoser, 2010).

Nimon, Zigarmi, Houson, Witt, and Diehl’s (2011) research on validating the Work Cognition Inventory illustrates best decision practices for factor extraction. In Study 1, in a series of EFAs (PAFs with promax rotation), not only did they consult the eigenvalue-greater-than-one rule and scree test results, but they also reported the eigenvalues and percentage of variance explained by each factor and subsequently contrasted the results with conceptual or theoretical considerations. This EFA work supported CFAs and further instrument development in the two subsequent studies.

Type of Rotation to Use

When conducting factor-analytic work, most researchers rotate the extracted factors (components) to assist interpretation (Ruscio & Roche, 2012). There are two major factor rotation strategies: (a) orthogonal and (b) oblique. Orthogonal rotations constrain the factors to make them uncorrelated; oblique rotations allow correlated factors, but it is not required (Fabrigar et al., 1999; Treiblmaier & Filzmoser, 2010). Examples of orthogonal rotations include varimax, quartimax, and equamax, with varimax being by far the most common. Oblimin, quartimin, and promax are examples of oblique rotation methods. Because orthogonal solutions are usually the default in most statistical packages, they also seem to be used most frequently by researchers (Henson et al., 2004). However, if the factors are in reality correlated, orthogonal rotations can yield illusory solutions thereby threatening theory building, suggesting instead that oblique rotations are to be preferred (Treiblmaier & Filzmoser, 2010).

For determining the variables related to the respective factors, both a factor pattern matrix and factor structure matrix must be consulted (Tabachnick & Fidell, 2007). A factor pattern matrix presents coefficients (analogous to regression beta weights) that reflect the unique contribution of each variable to each factor. Structure coefficients, however, reflect bivariate correlations between each factored variable and latent factor (Henson & Roberts, 2006). For orthogonal solutions, the pattern and structure matrices are one and the same (report as factor pattern/structure matrix); oblique solutions require reporting and interpreting both when defining factors. Neglecting to interpret each can lead to incorrect inferences about not only a factor’s structure, but also a variable’s importance in light of possible multicollinearity issues (Thompson, 2004). This state of affairs where a factor’s interpretation might be masked by the researcher’s poor decision making again threatens theory building; selection of rotation criteria can have a significant influence on interfactor correlations and the magnitude to which a variable correlates with multiple factors (Schmitt & Sass, 2011). Thus, it unnecessarily introduces error into a study.

Dahling et al. (2012), after conducting a PAF to support their development of the pro-social rule-breaking measure mentioned earlier, used an oblique rotation (direct oblimin). The researchers were clear and precise in justifying why they used an oblique rotation; that is, consistent with best EFA decision-making practice, an oblique rotation was used because they expected that the different types of pro-social rule breaking measured by the instrument would be highly intercorrelated. Similarly, Nimon et al. (2011) used an oblique rotation (promax) in conjunction with the PAFs used in developing the Work Cognition Inventory because research led them to believe that the factors would be correlated.

Interpretation and Naming of Factors

Decisions about minimum threshold values for factor (component) coefficients (frequently ambiguously called loadings; see Henson & Roberts, 2006) are important for interpreting a factor. Typically, coefficients range from .20 to .60 (Treiblmaier & Filzmoser, 2010). Stevens (1996) clarified the issue by explaining how a factor coefficient of .20 could be statistically significant in a large sample, yet would only explain 4% of the variance; thus, becoming an issue of practical significance. With this in mind, values with a magnitude of .32 (explains 10% of variance) would be a minimal acceptable factor coefficient (Treiblmaier & Filzmoser, 2010), with .40 (16% of variance) or more being best for interpreting what the higher coefficients have in common for factor naming purposes (i.e., those ≥ .40; Stevens, 1996). In general, it is best having three or more higher loading coefficients to constitute a meaningful, interpretable factor. Both Dahling et al. (2012) and Nimon et al. (2011) focused on the pattern matrix and removed items that cross-loaded or demonstrated loadings that were less than .33 (Dahling et al., 2012) or .40 (Nimon et al., 2011). The result in both studies was the presentation of clear, interpretable factors that supported scale development because the ambiguous (cross-loadings) or weak loadings (explained less than 10% of variance) were eliminated.

Interpretation also includes considering whether the solution (a) is replicable across groups or time, (b) makes a significant contribution to the research literature, and (c) is sufficiently complex as to be interesting, but not so complex that the solution is rendered uninterpretable (Tabachnick & Fidell, 2007). Nimon and Reio (2011) highlighted the importance of the consistency or replicability of measurement across groups (measurement invariance) to theory building in the field of HRD (applicable to the social sciences in general). Nimon and Reio recommended a number of factorial invariance indices for assessing measurement invariance (i.e., salient similarity index, coefficient of congruence, and the correlation between pattern coefficients). To properly build theory, measurement should be invariant across groups or risk biasing theory, research, and ultimately practice.

Finally, a factor or component must be named or labeled in such a way as to capture “as a whole the conceptual meaning of each variable defining a particular latent dimension” (Mvududu & Sink, 2013, p. 90). Thus, the naming of a factor is based on its conceptual underpinnings. Still, naming a factor remains subjective and as much art as science. The researcher then must draw on the theoretical and research literature supporting the research to make a decision regarding naming the factor. To avoid confusion, the researcher should resist naming a factor the same as one of the scales or items contributing to the factor.

Implications for Theory, Research, and Practice

This article answers Holton’s (2003), Torraco’s (2005) and Reio’s (2009, 2010) calls for more theory-building methods articles (in this case quantitative) and theory building in the field of HRD. The article also heeds Cumming’s (2014) call for more reporting precision in published social science research. We selected EFA because of its extensive use by social science researchers to support such theory building, but in a quantitative sense. Extensive research has demonstrated that EFAs are too frequently plagued by researchers’ poor decision making, creating conditions in peer-reviewed journals where conflicting and confusing information is published that is simply incorrect (Cumming, 2014; Henson & Roberts, 2006; Onwuegbuzie & Daniel, 2003). Theorists need accurate information of course to support future theory building (e.g., through meta-analyses, secondary data analyses; see Newman, Hitchcock, & Newman, 2015; Nimon, 2015) and hypothesis testing (Jaccard & Jacoby, 2010). The purpose of this article was to examine best EFA decision-making practices in social science research like HRD and its links to quantitative theory building. We distinguish five best practices and tie them to concrete examples primarily from the Dahling et al. (2012) and Nimon et al. (2011) studies that used heterogeneous samples to increase generalizability (Widaman, 2012). Recognizing the need for making the implications of these EFA practices clear, we highlight and discuss each in sequence. The five best EFA decision-making practices are as follows: selection of observations, factor extraction method, factor retention, type of rotation, and interpretation.

Selection of observations deals with sample size and participant-to-variable ratios (Widaman, 2012). Extensive research on best EFA practice regarding sample size and participant-to-variable ratios indicates that participant-to-variable ratios should be at least 5:1, but preferably 10:1 or more. Onwuegbuzie and Daniel (2003) decry using sample size only as a decision criterion because this approach does not take into consideration the number of variables; a participant-to-variable ratio less than 5:1 threatens the reliability of variable and factor scores and is thus an internal validity threat. With the 21-item pro-social rule-breaking scale examined in the Dahling et al. (2012) study, for example, the sample size seemed reasonable in that there were 179 participants (Comrey, 1973). The participant-to-variable ratio was 8.5:1. The implication is that we have preliminary evidence the measure warrants further use as a research tool for HRD researchers or as a measure for guiding a training workshop for HRD practitioners.

The factor extraction method used in an EFA is also an important decision for the researcher. PCAs are useful for reducing large sets of variables into smaller, more parsimonious ones. PAFs, alternatively, reduce large sets of variables into smaller ones too, but with theory and research guiding interpretation and understanding of the resulting latent structures. Both the Dahling et al. (2012) pro-social rule-breaking and Nimon et al. (2011) work cognition studies informed the reader about the factor extraction method used, leaving the HRD researcher or practitioner with critical replication information. Hence, the results could be used to build pro-social rule breaking or work cognition theory or guide research and practice.

Factor retention is another significant issue in best EFA decision practices. Researchers have a number of decision tools to aid this process. Acknowledging that MAP and parallel analysis have been shown to be best, yet rarely used, we recommended using the EF > 1 rule, scree test, and a priori theory as decision tools. Both the Dahling et al. (2012) and Nimon et al. (2011) studies reported the decision rule(s) used in detail for making a final determination on the number of factors to retain. For example, each study reported the number of factors extracted, eigenvalues, and factor loadings, and the percentage of variance explained by each factor. This information leaves researchers with some assurance of the validity of the authors’ findings related to the factor analyses and supports HRD theorists, researchers, and practitioners using the findings to guide future theory building, research, and practice.

Most researchers use factor rotation to increase the interpretability of factors once the factor retention decision has been made. We report on two major rotation strategies; that is, orthogonal and oblique and recommended an oblique strategy most often because it allows for overlap among factors that can be more theoretically enriching. Again, consistent with best practice, Dahling et al. (2012) and Nimon et al. (2011) indicated whether factor rotation was used to increase interpretability and, just as importantly, why.

Interpretation of the factors emerging from an EFA is the last important step. Researchers should report cutoff values for interpreting item coefficients within factors (.40 or above is recommended) and avoid naming factors the same as scales or items within the scales. Dahling et al. (2012) and Nimon et al. (2011) expertly presented cutoff values relevant to their research.

The scientific merit of the Dahling et al. (2012) and Nimon et al. (2011) studies is clear and defensible because the authors followed best EFA decision-making practice. The information generated from these studies provided sufficient precision for scholars to independently ascertain the validity of the authors’ claims; therefore, using the information generated by the studies (e.g., to guide future research or to contribute to a meta-analysis) would support pro-social rule breaking and work cognition theory building.

Conclusion

We answer the call for publishing research related to theory building in HRD. Poor EFA decision-making practices can lead to erroneous findings being published in social science journals, including those in HRD. The published erroneous findings then present ambiguous information for theorists, researchers, and practitioners to follow. By adhering to the best EFA decision-making practices presented in this article, HRD researchers would be able to more precisely and accurately report their findings to support new theory building and research, and inform best HRD practice.

Footnotes

Authors’ Note

This article was subjected to a two-tier, blind review process that did not involve any of the contributing authors who are currently members of the editorial board.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Thomas G. Reio Jr. is an associate dean of graduate studies and professor of adult education and human resource development at Florida International University in Miami, Florida. He has been a past editor of Human Resource Development Review and currently is the co-editor of New Horizons in Adult Education and Human Resource Development. His research concerns curiosity and risk-taking motivation, workplace socialization, workplace incivility, and workplace learning. His work has been published in leading journals in education, business, and psychology.

Brad Shuck is an assistant professor of organizational leadership and learning at the University of Louisville in Louisville, Kentucky. His primary areas of research include the application, meaning, and measurement of employee engagement, emerging areas of positive psychology, and leader development. His scholarship has appeared in refereed journals such as Leadership and Organizational Studies, Human Resource Development Review, Human Resource Development Quarterly, Human Resource Development International, Couple and Family Psychology: Research and Practice, Journal of Continuing Higher Education, Journal of Children’s Health Care, and the Journal of Management Development, among others. In addition to his primary faculty appointment, Shuck holds affiliate faculty status in the Department of Educational and Counseling Psychology, Counseling, and College Student Personnel at the University of Louisville.

References

Bartlett

M. S.

(1950). Test of significance in factor analysis. British Journal of Psychology, 3, 77-85.

Cattell

R. B.

(1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245-276.

Comrey

A. L.

(1973). A first course in factor analysis. New York, NY: Academic Press.

Conway

J. M.

Huffcutt

A. I.

(2003). A review and evaluation of exploratory factor analysis practices in organizational research. Organizational Research Methods, 6, 147-168.

Cumming

(2014). The new statistics: Why and how. Psychological Science, 25, 7-29.

Dahling

J. J.

Chau

S. L.

Mayer

D. M.

Gregory

G. B.

(2012). Breaking rules for the right reasons? An investigation of pro-social rule breaking. Journal of Organizational Behavior, 33, 21-42.

Fabrigar

L. R.

Wegener

D. T.

MacCallum

R. C.

Strahan

E. J.

(1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272-299.

Gorsuch

R. L.

(1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Gorsuch

R. L.

(1990). Common factor analysis versus component analysis: Some well and little known facts. Multivariate Behavioral Research, 25, 33-39.

10.

Gorsuch

R. L.

(1997). Exploratory factor analysis: Its role in item analysis. Journal of Personality Assessment, 68, 532-560.

11.

Haig

B. D.

(2013). The philosophy of quantitative methods. In Little

T. D.

(Ed.), The Oxford handbook of quantitative methods (pp. 7-31). New York, NY: Oxford University Press.

12.

Henson

Capraro

R. M.

Capraro

M. M.

(2004). Reporting practice and use of exploratory factor analysis in educational research journals. Research in the Schools, 11, 61-72.

13.

Henson

R. K.

Roberts

J. K.

(2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66, 393-416.

14.

Hesse-Biber

Leavy

(Eds.). (2008). Handbook of emergent methods. New York, NY: Guilford Press.

15.

Holton

E. F.

III . (2003). Where are the theory-building research methods articles? [Editorial]. Human Resource Development Review, 2. 227-228.

16.

Horn

J. L.

(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185.

17.

Jaccard

Jacoby

(2010). Theory construction and model-building skills: A practical guide for social scientists. New York, NY: Guilford Press.

18.

Kahn

J. H.

(2006). Factor analysis in counseling psychology research, training, and practice: Principles, advances, and applications. The Counseling Psychologist, 34, 684-718.

19.

Kaiser

H. F.

(1956). The varimax method of factor analysis (Unpublished doctoral dissertation). University of California, Berkeley.

20.

Mvududu

N. H.

Sink

C. A.

(2013). Factor analysis in counseling research and practice. Counseling Outcome Research & Evaluation, 4, 75-98.

21.

Newman

Hitchcock

J. H.

Newman

(2015). The use of research syntheses and nomological networks to develop HRD theory. Advances in Developing Human Resources, 17, 117-134.

22.

Nimon

(2015). Secondary Data Analyses From Published Descriptive Statistics: Implications for theory, research, and practice. Advances in Developing Human Resources, 17, 26-39.

23.

Nimon

Reio

T. G.

Jr. (2011). Measurement invariance: A foundational principle for quantitative theory building. Human Resource Development Review, 10, 198-214.

24.

Nimon

Zigarmi

Houson

Witt

Diehl

(2011). The Work Cognition Inventory: Initial evidence of construct validity. Human Resource Development Quarterly, 22, 7-35.

25.

Onwuegbuzie

A. J.

Daniel

L. G.

(2003). Typology of analytical and interpretational errors in quantitative and qualitative educational research. Current Issues in Education [Online], 6(2). Retrieved from cie.asu.edu/ojs/index.php/cieatasu/article/downloadSuppFile/439/76/

26.

Reio

T. G.

Jr. (2009). Contributing to the emergent research method conversation [Invited ed.]. Human Resource Development Quarterly, 20, 143-145.

27.

Reio

T. G.

Jr. (2010). The ongoing quest for theory-building research methods articles [Editorial]. Human Resource Development Review, 9, 223-225.

28.

Reio

T. G.

Jr. Ghosh

(2009). Antecedents and outcomes of workplace incivility: Implications for research and practice. Human Resource Development Quarterly, 20, 237-264.

29.

Ruscio

Roche

(2012). Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological Assessment, 24, 282-292.

30.

Schmitt

T. A.

Sass

D. A.

(2011). Rotation criteria and hypothesis testing for exploratory factor analysis: Implications for factor pattern loadings and interfactor correlations. Educational and Psychological Measurement, 71, 95-113.

31.

Stevens

(1996). Applied multivariate statistics for the social sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.

32.

Tabachnick

B. G.

Fidell

L. S.

(2007). Using multivariate statistics (5th ed.). New York, NY: Pearson.

33.

Thompson

(1992). A partial test distribution for cosines among factors across samples. In Thompson

(Ed.), Advances in social science methodology (Vol. 2, pp. 81-97). Greenwich, CT: JAI.

34.

Thompson

(2004). Exploratory and confirmatory factor analysis. Washington, DC: American Psychological Association.

35.

Thompson

Daniel

L. G.

(1996). Factor analytic evidence for the construct validity of scores: An historical overview and some guidelines. Educational and Psychological Measurement, 56, 197-208.

36.

Torraco

R. J.

(2005). Theory development and research methods. In Swanson

R. A.

Holton

E. F.

III (Eds.), Research in organizations: Foundations and methods of inquiry (pp. 351-374). San Francisco, CA: Berrett-Koehler.

37.

Treiblmaier

Filzmoser

(2010). Exploratory factor analysis revisited: How robust methods support the detection of hidden multivariate data structures in IS research. Information & Management, 47, 197-207.

38.

Velicer

W. F.

(1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41, 321-327.

39.

Velicer

W. F.

Jackson

D. N.

(1990). Component analysis versus common factor analysis: Some issues in selecting an appropriate procedure. Multivariate Behavioral Research, 17, 97-114.

40.

Widaman

K. F.

(2012). Exploratory factor analysis and confirmatory factor analysis. In Cooper

(Ed.) APA handbook of research methods in psychology: Data analysis and research publication (Vol. 3, pp. 361-390). Washington, DC: American Psychological Association.