Abstract
This study explored the measurement equivalence of the Career Indecision Profile (CIP) in a Chinese sample with both U.S. and South Korean samples. Past measurement invariance research on the CIP in four international samples (Icelandic young adults, Italian adolescents, French-speaking young adults, and South Korean adolescents) has supported a four-factor structure in the U.S. and in the three Western samples but not in the South Korean sample. Rather, a five-factor structure emerged in South Korea. This study sought to identify whether either the four- or five-factor structure would demonstrate suitable fit for a Chinese adolescent sample. Results indicated that the four-factor structure developed in the United States did not replicate in China, but the five-factor structure found in South Korea showed adequate fit. Additional analysis suggested full metric invariance on all five scales and scalar invariance on four of the five scales. These findings extend the past measurement invariance work with the CIP to suggest two potential ways with which to understand career indecision: a four-factor structure in Western cultures and a five-factor model in Eastern cultures. Future research needs are discussed.
Inquiry into career indecision in the vocational psychology literature has been sizable for some time, with the bulk of this research focusing on understanding the correlates and causes of career indecision. This research has established links between career indecision and both person (e.g., decision-making styles, negative affect, self-esteem) and contextual (e.g., external barriers and conflicts with important others) variables. There have also been several attempts to organize and condense this wealth of information by rationally constructing models to identify overarching sources of career indecision (e.g., Chartrand, Robbins, Morril, & Boggs, 1990; Gati, Krausz, & Osipow, 1996).
More recent data-driven model-building efforts have suggested an alternative way of conceptualizing sources of career indecision. Specifically, this research has factor analytically identified four primary sources of career indecision that subsume a large number of variables previously identified as correlates of indecision (Brown et al., 2012; Brown & Rector, 2008). Brown and colleagues (2012) labeled these four potential latent sources of career indecision as: (a) neuroticism/negative affectivity (NNA), (b) choice/commitment anxiety (CCA), (c) lack of readiness (LR), and (d) interpersonal conflicts (IC).
The first factor, NNA, is defined by a tendency to experience negative affect during the decision-making process as well as placing an emphasis on what may go wrong while making a decision. It is also marked by heavily relying on input from others and prematurely foreclosing on decisions in order to avoid prolonged experiences of negative affect. CCA, the second latent source, is marked by an inability to commit to a decision, feelings of anxiety, approach–approach conflict, and increased needs for self- and occupational information during the decision-making process. The third latent dimension, LR, reflects a lack of preparedness to make a decision. Items loading on this factor reflect low levels of goal directedness, career decision-making self-efficacy, conscientiousness, and rational decision-making style. IC, the fourth and last factor, is defined by the absence of support from, and the presence of conflict with, important others in the decision-making process (Brown et al., 2012).
Hacker, Carr, Abrams, and Brown (2013) developed the Career Indecision Profile-65 (CIP-65) to measure these four latent sources of career indecision. The measure was created using both exploratory factor analytic (EFA) and confirmatory factor analytic (CFA) methods in samples of college students from the United States. CFA results indicated that the four-factor structure demonstrated a good fit to the data.
Subsequent research has tested for the measurement equivalence of the CIP-65 across the following samples: (a) Icelandic young adults, (b) Italian adolescents, (c) and French-speaking (i.e., France and French-speaking Swiss) young adults. Configural invariance (invariance of factor structure) was found between the U.S. samples and all three Western-European samples (Abrams et al., 2013; Carr et al., 2014). In sum, international research using the CIP-65 suggests that the four-factor model of career indecision, when tested etically, may represent salient sources of career indecision in all three Western-European samples studied thus far.
However, Abrams, Lee, Brown, and Carr (2015) found that the four-factor model did not show similar levels of configural invariance, via a CFA, in a South Korean adolescent sample. Instead, a subsequent EFA, using a subset of the South Korean sample, suggested a five-factor structure. Three of the factors from the original four-factor model (NNA, LR, and IC) remained largely unchanged in this new five-factor model. However, CCA split into two factors, with items reflecting a need for self- and occupational information forming a separate fifth factor named need for information (NI). Items reflecting choice anxiety, fear of commitment, and approach–approach conflict continued to load on CCA in the South Korean sample. Thirteen of the CIP-65 items failed to load saliently on any factor and were thus removed to yield a new 52-item measure, the CIP-52. CFA results obtained from a second subsample of South Korean participants supported the five-factor structure of the CIP-52.
When considering the difference in factor structures between the four Western (including the United States) samples and the South Korean sample, it appeared that among South Korean adolescents, a need for self- and occupational information did not emerge as a concern only when experiencing CCA as it seemed to among adolescence and young adults in Western countries (at least those so far studied). Instead, needs for information seem to represent a separate concern among South Korean adolescents that may be related to, but is not solely a characteristic of, CCA.
The Current Study
The current study sought to assess the measurement invariance of the U.S. and South Korean versions of the CIP with a sample of Chinese adolescents to further explore cross-cultural differences (or equivalences) in sources of career indecision as measured by the CIP. Evidence suggests that Chinese and South Korean adolescents are more similar to each other than they are to adolescents from the U.S. and other Western countries, despite recent modernization, technological advances, and Westernization that both countries have experienced (Hwang, 2009; Zhang, Hu, & Pope, 2002). A strong emphasis on collectivism, performance, and value of planning for the future define both cultures (Javidan, Dorfman, Sully deLuque, & House, 2006). As a result of these cultural similarities, we expected that the five-factor model (as measured by the CIP-52) would show better measurement invariance in the Chinese sample than the four-factor model (as measured by the CIP-65).
Method
Participants
Chinese sample description
The sample from China was comprised of 588 secondary students, with more women than men (women: n = 330, 56%; men: n = 258, 44%). The mean age of this sample was 17.6 years (range = 16–20 years, SD = .68). This sample was younger in age than the original U.S. sample but very close in age to the Italian adolescent (Carr et al., 2014) and the South Korean samples (Abrams, Lee, Brown, & Carr, 2015).
U.S. sample description
The U.S. sample used to test for configural invariance was the same sample used in past measurement invariance testing with the CIP-65 (Abrams et al., 2015; Abrams et al., 2013; Carr et al., 2014). The total sample was 488 participants recruited from a variety of undergraduate courses at two Midwestern universities. Participants ranged in age from 18 to 54 (M = 21.83, SD = 6.02) and were largely female (76.6%) and Caucasian (68.4%).
South Korean sample description
The South Korean sample was the same sample used by Abrams et al. (2015) to test for measurement invariance in South Korea. High school students comprised the sample (n = 374), with an average age of 17.97 (SD = .46, range = 17–19). Females represented a majority of the sample (60.2%).
Instrument
Career Indecision Profile-65
Hacker et al. (2013) developed the CIP-65 to tap into the four latent sources of career indecision posited by Brown et al (2012). Responses are given on a 6-point (1 = strongly disagree, 6 = strongly agree) scale, with higher scores reflecting greater indecision. Scores on all four scales of the CIP-65 (NNA, CCA, LR, and IC) have yielded high internal consistency estimates, with Cronbach’s α estimates ranging from .88 for IC to .96 for CCA in a U.S. college student sample (Hacker, Carr, Abrams, & Brown, 2013). Included with the English version of the CIP-65 is a demographic questionnaire to collect information on participants’ age, gender, ethnicity/nationality, level of career decidedness (1 = very undecided, 6 = very decided), and year in school.
Translation Procedure
The following procedures were used in creating both the South Korean and Chinese versions of the CIP-65. First, the original English version of the CIP-65 was translated into Korean and Mandarin and then back translated to English by the multilingual coauthor of the study and a graduate assistant. Second, the back-translated version was returned to the U.S. research team for comments. Third, the teams in South Korea and China made minor revisions to item wording based on the U.S. team’s comments to create final versions of the Korean CIP-65 and Mandarin CIP-65.
Invariance Testing
Test for configural invariance
In order to test for configural invariance, single-group CFAs using LISREL 8.80 were conducted on the U.S., Korean, and Chinese samples independently. Data were analyzed via maximum likelihood estimation using the covariance matrices. Four separate measures of fit, in addition to the χ2 value, were used to evaluate each model’s overall goodness of fit. Absolute fit of the models was determined through root mean square error of approximation (RMSEA) and standardized root mean residual (SRMR). Relative fit of the models was assessed via non-normed fit index (NNFI) and comparative fit index (CFI). Per existing recommendations, models were judged as having acceptable fit with RMSEA and SRMR values below .10, and good fit if these values were below .08 (Hu & Bentler, 1998). For NNFI and CFI, the standard for acceptable fit is .90 and above, and good fit is determined if values exceed .95 (Bentler, 1990).
Using the fit indices noted above, the four- and five-factor models were assessed for their fit in the Chinese sample. Since the four- and five-factor models were composed of a different number of items (65 vs. 52), we could not test for their comparative fit. The better fitting model was then assessed for progressively more stringent levels of measurement invariance—metric and scalar invariance.
Tests for metric invariance
In the second phase of this study, metric invariance was tested on the model (four or five factor) that demonstrated the more appropriate fit in the Chinese sample. The Chinese sample would be compared to the U.S. sample if the four-factor model showed better fit, while the South Korean sample would be the comparison group if the five-factor structure was the better fitting model. Two levels of analysis were used following a procedure similar to the one used by Carr and colleagues (2014). First, an omnibus test of metric invariance was performed that compared a baseline model (where all items’ factor loadings were freed to be estimated in the Chinese and comparison sample) to a constrained model (where all items’ factor loadings were fixed to be equal in both samples). If the results of this omnibus test suggested lack of metric invariance, the next step was to test factor-level invariance by analyzing each factor individually for invariance. In order to conduct these tests, factor loadings were constrained to be equal across samples one factor at a time, while all other loadings were freed to be estimated. To determine the presence of invariance in these analyses, two criteria were used: (a) χ2 difference test between nested models and (b) change in CFI. A significant χ2 difference would suggest a lack of metric invariance (i.e., that the unconstrained, baseline model fits the data better than the model that constrained factor loadings to be equal). Since χ2 tests are highly sensitive to the sample size and may result in inflated experiment-wide error rates when multiple tests are employed, change in CFI was used as an additional metric to assess for invariance. Meade, Johnson, and Brady (2008) suggested that a difference of .002 or less in the CFI can be interpreted as supporting invariance (i.e., that unconstrained, noninvariant model did not provide sufficiently improved fit to reject the constrained, invariant model).
Tests for scalar invariance
If metric invariance was established, tests for scalar invariance were then performed using the same procedure used for the metric invariance testing. Scalar invariance assesses the extent to which intercepts are equal across groups while simultaneously constraining the factor loadings to be equal. First, an omnibus-level test was performed. If that test suggested a lack of scalar invariance, individual factor-level scalar invariance analyses were performed to determine whether there was invariance at the factor level. Last, should the factor-level results suggest a lack of invariance on at least one factor, item-level analysis would be performed, fixing individual item parameters one at a time (Steenkamp & Baugrtner, 1998).
Results
Configural Invariance
Tests of normality for the original four factors of the CIP-65 were acceptable in all three samples (skew < 2.00, kurtosis < 7.00; Fabrigar, Wegener, MacCallum, & Strahan, 1999). Internal consistencies of scores for all factors were acceptable. Cronbach’s α ranged from .79 (IC) to .91 (CCA) in the Chinese sample.
Independent CFAs of the four-factor model on the U.S. and China samples using the CIP-65 revealed a poor fit of the model to the Chinese data. None of the values of the fit indices were acceptable (see Table 1). Thus, the CIP-65 in China did not show configural invariance with the United States. However, when the five-factor model was tested on the Chinese data, with the 13 nonsaliently loading items from the CIP-65 removed from the analysis, all fit indices reflected an adequate fit of the model to the data (see Table 1). Thus, configural invariance of the five-factor model identified in South Korea was supported in China.
Configural Invariance: United States, South Korea, and China.
Note. RMSEA = root mean square error of approximation; SRMR = standardized root mean residual; CFI = comparative fit index; NNFI = non-normed fit index.
Factor loadings were similar in the Chinese sample to those Abrams and colleagues (2015) found in the South Korean sample. Loadings ranged from .29 (NI) to .77 (both CCA and NI). Factors accounted for between 8% (NI) and 74% (IC) of the variance in the items within the Chinese data. Within the South Korean data, factors accounted for 14% (NNA) to 68% (CCA) of variance in the items. The intercorrelations among the five factors ranged from .15 (LR and IC) to .62 (NNA and CCA) in the Chinese sample. In the South Korean sample, intercorrelations between the five factors ranged from .12 (LR and IC) to .36 (CCA and NNA; Abrams et al., 2015).
Metric Invariance
Given that configural invariance of the five-factor model was found between China and South Korea, it was appropriate to proceed to the next level of measurement equivalence testing and test for metric invariance. First, an unconstrained multigroup CFA model using LISREL 8.80 was tested with factor loadings freed to be estimated in both the South Korean and Chinese data sets. For comparison, a constrained model where factor loadings were fixed to be the same in both the Chinese and South Korean samples was tested.
The χ2 difference test for metric invariance between these two nested models yielded a significant result, ▵χ2(47, n = 962) = 135.78, p < .01 (see Table 2). Yet, as also revealed in Table 2, the change in CFI (▵CFI = .001) fell below the .002 threshold. This suggests that there was not a meaningful difference in the fit between the constrained and unconstrained models, which supports metric invariance. In order to further confirm these initial findings, factor-level invariance was tested. This was achieved by constraining loadings for individual factors one at a time and comparing them to the unconstrained model. As with the omnibus- (full model) level metric invariance test (see Table 2), the change in χ2 emerged as significant for four of the five factors (all except for NI; see Table 2). Changes in CFI were negligible or nonexistent, ranging from .000 on four of the five factors to .001 on NNA. Ultimately, these factor-level results when taken in conjunction with the omnibus model test suggest that the CIP-52 has metric invariance across the Chinese and South Korean samples.
Omnibus and Factor-Level Metric Invariance of CIP-52 Five-Factor Model.
Note. CIP = Career Indecision Profile; NNA = neuroticism/negative affectivity; CCA = choice/commitment anxiety; LR = lack of readiness; IC = interpersonal conflicts; NI = need for information.
Scalar Invariance
Given metric invariance was supported, we next tested for scalar invariance by creating an unconstrained model in which factor loadings were constrained to be equal, while item intercept values were freed to be estimated. This unconstrained model was then compared to a constrained model, in which factor loadings and item intercept values were constrained to be equal in the two samples. Results from the omnibus-level tests yielded a statistically significant change in χ2, ▵χ2(52, n = 962) = 921.42, p < .01, as well as a considerable change in CFI (▵CFI = .012).
Changes in χ2 were significant across all five factors when each factor was analyzed separately (see Table 3). However, changes in CFI values suggested that both NI and CCA were scalar invariant, while NNA, LR, and IC may not be scalar invariant.
Omnibus and Factor-Level Scalar Invariance of CIP-52 Five-Factor Model.
Note. CIP = Career Indecision Profile; NNA = neuroticism/negative affectivity; CCA = choice/commitment anxiety; LR = lack of readiness; IC = interpersonal conflicts; NI = need for information.
Item-level analyses were completed for the three factors that did not demonstrate scalar invariance at the factor level (see Table 4). Of the 18 total items on NNA, 11 (61.1%) showed significant χ2 differences, but the largest change in CFI was only .001. Although the results of these item-level analyses were somewhat equivocal, an inspection of the contents and intercept values of the 11 items that showed significant χ2 differences revealed that all but one intercept value was higher in the South Korean than in the Chinese sample. Such a consistent pattern of results suggests that the NNA scale may not be scalar invariant and that South Korean adolescents may receive higher scores than Chinese adolescents on this scale even if they display similar levels of the latent trait (i.e., NNA). The contents of these noninvariant items reflected primarily affective components of NNA (e.g., feelings of being overwhelmed, stressed, and worried by what might go wrong in the future).
Item-Level Scalar Invariance Results and Item Intercepts.
Note. — indicates items on factors found to be invariant during factor level analyses. NNA = neuroticism/negative affectivity; CCA = choice/commitment anxiety; LR = lack of readiness; IC = interpersonal conflicts; NI = need for information.
aItem is not invariant across Chinese and South Korean samples.
Item-level analysis of items on the LR factors showed that eight of the eight (61.5%) LR items had significant changes in χ2, but again no items on this scale showed sufficiently large changes in CFI values (maximum change = .001). However, unlike the results with the NNA scale, there were no clear patterns in the intercept values or contents of items with significant χ2 differences. For example, the South Korean sample had higher intercept values on three items, while the Chinese sample had higher intercept values on five items.
Results for the item-level analysis of IC followed the same pattern as the results on the LR scale: (a) four of the five items (80%) had a significant change in χ2, (b) 1 item yielded a borderline change in CFI (.002), and (c) there were no clear patterns in the intercept values (i.e., higher intercept values were obtained by the South Korean sample on two items, while the opposite was true for the other two items with significant χ2 differences). Taken together, these results largely suggest that all CIP-52 scales may be scalar invariant except the NNA scale, which may yield higher scores in South Korean than in Chinese samples.
Discussion
Congruent with our hypotheses, the results of this study indicate that the four-factor structure of career indecision found in the United States and other Western countries did not replicate in a sample of Chinese young people. Rather, the five-factor structure that emerged in South Korea provided a good fit to the sample from China, which supports the configural invariance of the five-factor CIP-52. Further measurement invariance testing also suggested that the CIP-52 was metric invariant (that factor loadings were all invariant). These results suggest that the CIP-52 can be used in both countries for research purposes (e.g., to compare the correlates and consequences of various source of indecision in the two countries).
Last, the results of the scalar invariance testing, though less definitive, suggested that scalar invariance may also be present, except for the NNA scale. Presence of scalar invariance suggests that individuals in China and South Korea who have the same level of a latent trait (career indecision facets) will obtain equivalent item scores. Lack of scalar invariance, on the other hand, suggests that young people from China and South Korea may not obtain equivalent item scores even with the same level of the latent trait. Our scalar variance results tentatively suggest that scores obtained by South Korean adolescents on the NNA scale may be inflated compared to the Chinese sample. Thus, it appears that the CIP-52 can be employed to make cross-country (in this case South Korea and China) comparisons in mean scores on four of the five CIP-52 scales; however, this application must be held as tentative until the scalar invariance results can be replicated with other samples of South Korean and Chinese adolescents. On the other hand, the results are also clear that the NNA scale should not at this time be used to compare mean scores in the two countries—South Korean adolescents may show higher levels of NNA than Chinese adolescents as a result of measurement artifact (i.e., lack of scalar invariance). No explanation is immediately apparent for why South Korean adolescents would report more stress, worry, and feelings of being overwhelmed about the future than their Chinese counterparts experiencing similar levels of NNA on the CIP-52, but this deserves research attention in the future.
As previously found with the South Korean sample, configural invariance testing revealed that items reflecting needs for self- and occupational information formed a separate factor that was related to, but not subsumed within, CCA. Abrams et al. (2015) suggested that in the United States, young people (and those from other Western countries) may not feel a need for self- and occupational information until they have to make a decision. On the other hand, in South Korea, and now China, a need for information may be present throughout the planning process and not just when making a decision becomes imminent.
There are a variety of explanations for the structural differences found on the CIP between the three Western samples and the two Eastern samples. For one, there has been a high degree of internal change in Chinese (and South Korean) work and cultural contexts. Prior to 1995, the Chinese government saw to the job placement of university students, whereas 30 years later, it is now the responsibility of the individual to decide on and obtain vocational placement (Zhang et al., 2002. The Chinese labor market has experienced a profound evolution in recent decades with the infiltration of advanced technology (Zhang et al., 2002). Many recent college graduates and current students are thus finding themselves navigating an evolving workforce, with little support from past generations who obtained work under markedly different circumstances (Hou, Leung, & Duan, 2009; Zhang et al., 2002). Information needs may, therefore, be especially salient in cultures (e.g., Chinese and South Korean), where independence in career choice making is a recent cultural phenomenon.
An additional explanation may revolve around the high levels of future orientation and performance attainment endorsed in both South Korean and Chinese cultures (Abrams et al., 2015; Gupta, Hanges, & Dorfman, 2002). Performance orientation is the extent to which a culture encourages and rewards individuals and organizations for performance and excellence, while future orientation reflects the degree to which society encourages and rewards future planning among its citizens. Cross-cultural research indicates that both China and South Korea strongly endorse both performance-oriented and future-oriented practices and values (Broadbeck, Chhokar, & House, 2007; Gupta et al., 2002). Abrams et al. (2015) postulated that South Korean adolescents, and in light of this study Chinese adolescents as well, are more focused on the planning process and performance achievement than students in the United States. Thus, adolescents in the two Asian cultures may see gathering information as an integral part of the planning process rather than something that is necessary only when difficulties arise in the choice-making process.
An additional consideration is that these two possible explanations are not necessarily mutually exclusive and that the rapid changes in both South Korean and Chinese labor markets make high levels of future and performance orientations a necessity. Indeed, an inability to gather information from traditional cultural sources (e.g., parents and important others who had been “placed” in their jobs by the government) may strongly motivate Chinese and South Korean adolescents’ needs for information.
This study is not without limitations. One limitation is the age differences between the U.S. sample (young adults) and the Chinese and South Korean (adolescents) samples. Thus, the configural invariance results favoring the five- versus four-factor structure could be due to age rather than cultural differences. However, prior results comparing the U.S. young adult sample to a sample of Italian adolescents (Carr et al., 2014) found that the four-factor structure associated with the CIP-62 showed both configural and metric invariance. Carr et al. (2014) also conducted post hoc EFA on the covariance matrices obtained from all Western samples (including Italian adolescents) and failed to identify a clear fifth factor in any of the samples. These results seem to support a cultural versus age explanation.
Another limitation is the impact that taking an etic (universal) approach to measurement development over an emic (within culture) approach may have had. The CIP was developed in the United States by American researchers and inherently reflects an American conceptualization of career indecision. It is quite possible that a measure of career indecision developed in South Korea and/or China may tap into culturally specific facets of career indecision that the CIP-52 may have missed. Thus, future research would benefit from taking an emic approach by conceptualizing and developing items to assess career indecision in a more culturally relevant fashion.
Ultimately, the findings from this study suggest that the CIP-52 can be used in both South Korea and China to help adolescents in the career exploration process. It can also be used for research in these countries to explore possible antecedents and consequences of each of the five major sources of indecision. The ultimate goals of such research would be not only to gain greater theoretical clarity but also to suggest how each source might be prevented or remediated. At the same time, further study of culturally specific factors that may have an impact on the career indecision experienced by young people in these two countries is necessary to ensure that South Korean and Chinese adolescents receive empirically and culturally sound career services.
Footnotes
Acknowledgments
We thank Blake Baltazar, Plamena Daskalova, Gwendolyn Foehringer, Louis Formica, Ethan Rucker, Alex Tatum, and Kathryn Thomas for reading and commenting on an earlier draft of this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Meghan Roche was supported by Graduate Assistantships provided by the Graduate School and School of Education of Loyola University Chicago.