Development and Initial Validation of a Measure to Assess Career Goal Feedback

Abstract

Career goal feedback provides information about career goal suitability, adequacy of goal progress, and whether changes are needed to reach the goals. Feedback comes from external (e.g., parents, peers) and internal sources (e.g., self-reflection), and plays an important role in the career development of young people. As there is no existing measure that adequately captures this construct, we devised and validated a 24-item inventory for use with young adults. In Study 1, initial items were developed, expert reviewed, and administered to a sample of Chinese university students (N = 1,055; M_AGE = 19 years). We used exploratory factor analysis to test the factor structure and confirmatory factor analysis on a holdout sample to validate a third-order solution (one third-order factor manifested by three second-order factors). In addition, we provided evidence for convergent and incremental validity. In Study 2, we confirmed the factor structure on Australian university students (N = 184; M_AGE = 19 years).

Keywords

career goal feedback career goals career goal management career-related stress career self-efficacy

According to goal-oriented and self-regulatory perspectives, people are active agents who set goals and then consciously and unconsciously regulate their behaviors to reach them (Bandura, 1991; Latham & Locke, 1991). Feedback, which informs people about the adequacy of their goals and the progress they are making toward them, has long been proposed as crucial in influencing this goal management process (Greenhaus, Callanan, & Kaplan, 1995). Although studies have supported the important role of feedback in the career goal pursuit of young people (Creed, Wamelink, & Hu, 2015), no comprehensive measure exists to assess this in young adults. We report two studies where we developed and initially validated an inventory suitable to measure career goal feedback in adolescents and young adults.

The Feedback Construct

Feedback is an important construct in many areas of the social sciences (e.g., education, organizational behavior), and both goal setting and self-regulatory theories (Bandura, 1991; Locke & Latham, 1990) indicate that feedback alerts people to discrepancies between their current state and their desired goals. These goal/performance discrepancies are central to this process as they stimulate self-reflection about one’s situation and elicit adjustments in cognition (e.g., outcome expectations), affect (e.g., satisfaction), behavior (e.g., effort), and/or goals (e.g., lowering goals; Lord, Diefendorff, Schmidt, & Hall, 2010).

Feedback has been construed as both unidimensional (e.g., about the correctness of a response; Kluger & DeNisi, 1996) and multidimensional. In a review of studies in learning settings, Hattie and Timperley (2007) identified three dimensions of effective feedback: feed-up (i.e., information about level of performance or goal to be attained), feed-back (i.e., information regarding performance), and feed-forward (i.e., information about future actions needed). This perception is consistent with propositions from other areas, where, in addition to suggesting feedback should provide information about performance, feedback should also provide directions for the future (Shute, 2008) and advice about how to proceed (Nawaz, Jahanian, & Manzoor, 2012). Hattie and Timperley argued that feedback conveying these three types of information should be more motivating, and, thus, more effective in reducing any gap between current performance and the desired standard.

Most research has focused on feedback from external sources (e.g., from supervisors, peers, parents), although feedback from internal sources has also been recognized (Butler & Winne, 1995). Internal feedback involves self-evaluation, judgment, and self-resolve, and reflects one’s own ideas, feelings, and intuition about the current situation versus the desired state. Internal feedback can be consistent with, complementary to, or conflicting with feedback from external sources (Butler & Winne, 1995).

From a goal-oriented perspective, feedback is directed toward the management, pursuit, and achievement of goals (Latham & Locke, 1991). Thus, goal characteristics need to be taken into account. Most studies in learning and organizational settings have focused on goals that are short-term (i.e., less than a year), specific (i.e., clear and concrete), and set by others (e.g., school or organization; de Kleijn, Mainhard, Meijer, Brekelmans, & Pilot, 2013; Ilies, Judge, & Wagner, 2010), and have evaluated feedback related to outcomes or processes (e.g., improving performance and performance procedures; Hattie & Timperley, 2007) or been aimed at clarifying and achieving existing goals (e.g., explaining how to complete a task; Shute, 2008).

In contrast, career goals for young adults who have not formally entered the full-time workforce are long-lasting (often for years), higher order (i.e., complex and non-specific), mostly self-set, and related to identity development (Perrone, Sedlacek, & Alexander, 2001). Thus, career feedback for young adults is primarily directed at higher order, general career goals (rather than specific actions), long-term self-regulation (e.g., engagement), the integration of the self (e.g., values and talents), and helping them select and achieve suitable goals. Career goal feedback, then, is information received regarding the suitability of a career goal, the progress being made toward achieving it, and improvements or adjustments needed to reach it, which is provided by external (e.g., parents, teachers, and peers) and internal sources (e.g., internal feelings and ideas based on intuition and social comparison), and which is aimed at directing and motivating the young person’s self-regulatory processes so that they make better progress toward their career goals.

Feedback can range from negative (i.e., implying there is a discrepancy between current state and goal) to positive (i.e., indicating the person is meeting or exceeding their goals). However, as negative feedback is the most influential form (Kluger & DeNisi, 1996) and ultimately leads to disproportionately more goal revision than positive or neutral feedback (i.e., goal management adjustment is more pronounced when the feedback is negative, compared with when it is positive; Ilies et al., 2010; Wang & Mukhopadhyay, 2012), we tailored our inventory so that higher scores reflected more negative feedback.

Existing Measures of Career-Related Feedback

Most studies of career-related feedback have used either experimental procedures (e.g., Kerpelman & Pittman, 2001) or single-item measures (e.g., Paa & McWhirter, 2000). One measure, the eight-item Feedback on Career Progress From Significant Others Scale (FCPSO; Creed et al., 2015), measures career feedback directly, but it focuses only on negative external feedback about career progress (e.g., “My friends doubt that I can achieve my ideal career”). Thus, it does not capture other important dimensions of the construct.

Feedback scales from the broader literature (e.g., education, organizational behavior) are also limited by assessing narrow constructs. Many are unidimensional, such as the five-item Feedback subscale of the Job Characteristics Inventory (assessing performance feedback; Sims, Szilagyi, & Keller, 1976). Those that are multidimensional do not assess the three domains of goal suitability, goal progress, and future actions, as recommended by Hattie and Timperley (2007). For example, the 32-item Feedback Scale (Herold & Greller, 1977) taps five domains of negative feedback; however, the focus is on feedback on progress and outcomes. Only one existing scale, the Master Students’ Perceptions of Supervisor Feedback Scale (de Kleijn et al., 2013), was based on Hattie and Timperley’s framework, but the narrow focus of this scale limits its applicability to use with masters’ students. It is a notable gap that there is no multidimensional career feedback scale or suitable scale from other areas that can be adapted. We address this by devising a brief, multidimensional, psychometrically sound measure that can assess feedback on career goals in young adults.

Present Studies

We report two studies. In Study 1, we followed classical test theory approach (Nunnally, 1978), and, based on a review of the literature regarding feedback, generated an initial pool of items that covered the three identified dimensions and two sources. After refining the items, we administered them to a large sample of Chinese students. We then split the sample, and used Sample A for exploratory factor analyses (EFAs) to reduce the number of items and determine the underlying structure, and used Sample B for confirmatory factor analyses (CFAs) and to test validity.

For convergent validity, we employed the FCPSO (Creed et al., 2015), which primarily assesses external feedback, but has the advantage that it is the only existing scale that directly measures career feedback in young adults. We expected scores on the FCPSO to be associated positively with scores from the new feedback scale. Second, previous studies have found negative feedback to be related positively to career distress (Creed et al., 2015) and negatively to self-efficacy (Ilies et al., 2010), which is in line with goal setting theory (Bandura, 1991). We expected the feedback scale to relate to these two constructs in these ways. Third, to support incremental validity, we tested if the new measure could explain additional variance in distress and self-efficacy to that explained by FCPSO, as it captures feedback from multiple dimensions and multiple sources.

In Study 2, we cross-validated the newly devised scale on a sample of English-speaking young adults. The focus was on replicating the multidimensional factor structure using a second sample, which would demonstrate the usefulness of the scale in a population other than Chinese students.

Study 1

Method

Item generation

We generated 61 items to assess three aspects of career goal feedback: goal suitability (i.e., do goals fit the person’s temperament, abilities, and values), progress (i.e., are goals being met by being engaged, hard-working, and achievement focused), and on how to improve (i.e., how goals can be advanced). Most items were worded to tap negative feedback, but we also included positively worded items to reduce response-set bias. As there is evidence that having an equal number of positively and negatively worded items in a scale is not effective in reducing response bias and might actually result in lower internal consistency and artifactual positive and negative factors in factor analyses (Van Sonderen, Sanderman, & Coyne, 2013), we only included a small number of reverse-worded items as per Salazar’s (2015) recommendation. Equal numbers of items were included to cover feedback from external and internal sources.

We conducted focus groups with 20 young people who discussed their own experiences and rated how adequately the items reflected these. As all items had a mean rating >5 on a 6-point scale, none was deleted at this point. The items were then reviewed by eight experts with knowledge of scale and career development (seven doctoral-level psychology academics and one doctoral student researching career development). We retained 42 items that had a mean rating >5, and added five new items to supplement internal feedback on how to improve. These 47 items, plus demographic questions and scales to assess validity, were compiled into one questionnaire and administered to a sample of tertiary students.

Participants

We distributed 1,354 questionnaires to students at four universities and two vocational colleges in Nanjing, China. Of these, 1,267 were returned (response rate = 93.5%), although only 1,055 (67.5% university and 32.5% colleges students) were used, as some participants did not complete the questionnaire, failed attention check questions, or used patterned responses. There were 416 young men (39%) and 636 young women (three did not indicate gender), with an average age of 19.18 years (SD = 1.33 years; 30 did not indicate age). Students were drawn from diverse disciplines, including medicine, psychology, law, design, and engineering, with 59% being first-year students, 22% from second, 13% from third, and 4% from fourth/fifth years.

Students came from both rural and urban provinces and had a variety of college entrance scores, which we standardized to give a measure of academic aptitude (M = 500; SD = 100; range = 150-683). For socioeconomic status (SES), students reported both parents’ educational level (1 = primary school or lower to 5 = master’s or above) and job status (1 = unemployed, casual laborer, job-waiting, non-technical worker, peasant to 5 = professional, higher managerial staff, and private owners), which we summed (M = 10.56; SD = 3.61; range = 4-20; Shi & Shen, 2007).

Using the SPSS Select (50% random) function, the dataset was split randomly into Sample A (n = 531; 42.5% male; M_AGE = 19.19 years, SD = 1.33 years) and Sample B (n = 524; 36.6% male; M_AGE = 19.16 years, SD = 1.28 years). When tested, there were no differences between the two samples on any demographic variable, indicating no bias as a result of splitting.

Materials

The questionnaire contained the 47 feedback items, demographic questions, and the three validity scales. The validity scales were translated into Mandarin using standard translation/back-translation methodology (Jones, Lee, Phillips, Zhang, & Jaceldo, 2001). The first author translated the items into Chinese, and they were back-translated into English by a graduate student. All authors then compared the back-translated versions with the originals and made adjustments where required. The final questionnaire was then piloted with eight university/college students (age range = 19-22 years) to assess language level and readability.

FCPSO

This eight-item scale assesses negative feedback on career progress from significant others using a 6-point Likert-type response format (1 = strongly disagree to 6 = strongly agree; higher scores = more negative feedback). The scale had good internal consistency reliability with Australian university students (α = .90), and, in support of validity, can be represented by a single-factor structure and was related to higher career goal-progress discrepancy and career distress (Creed et al., 2015). Using CFA, we confirmed a single factor; alpha was .79.

Korean College Stress Inventory

The five-item Career Ambiguity subscale assesses stress associated with students’ lack of self-knowledge, confidence, and certainty about their career future. Responses were given on a 5-point Likert-type scale (1 = strongly disagree to 5 = strongly agree; higher scores = more stress). The subscale had good reliability (α = .89 and .92 in two studies) and validity with Korean students (e.g., correlated with lack of self-identity; Choi et al., 2011; Kim et al., 2014). Our CFA confirmed one factor; alpha was .86.

Occupational Self-Efficacy Scale

This eight-item scale, which we modified slightly to make it specific to the current study, assesses students’ belief in their ability to perform successfully in their chosen career. Responses were given on a 5-point Likert-type scale (1 = completely disagree to 5 = completely agree; higher scores = higher self-efficacy). This scale has satisfactory reliability (α = .88) and validity (e.g., correlated with general self-efficacy and self-esteem; Schyns & von Collani, 2002). We confirmed a one-factor structure; alpha was .80.

Procedure

Ethical approval was granted by the authors’ university ethics committee, and written permission to recruit participants was obtained from the administration departments of the universities/colleges. Questionnaires were distributed by the first author and completed and returned by the volunteer students in class (non-participating students were given a substitute activity by their course convener). Completing the survey was taken as expressing informed consent for the students; a prize draw of a CNY500 shopping voucher was offered to participants.

Results

Item reduction and EFA

Using Sample A (n = 531), we assessed the 47 feedback items for skewness and kurtosis, and high inter-item (r ≥.80) and low item-total correlations (r ≤ .30). Four items were deleted due to low item–total correlations. Next, a principal-axis EFA (direct oblimin rotation; Δ = 0) was conducted to assess the underlying structure of the remaining 43 items. The Kaiser-Mayer-Olkin (KMO) measure of sampling adequacy (.90) and Bartlett’s test of sphericity (p < .001) indicated that these items were suitable for EFA. We used various criteria to determine the number of factors, including the scree plot, Velicer’s minimum average partial (MAP) test and parallel analysis, a minimum of three items per factor, minimum factor coefficient of .40 for each item, and factorial meaningfulness. We expected three (i.e., domains of feedback on progress, goal suitability, and how to improve) or six factors (i.e., 3 Domains × 2 Sources).

The first EFA produced nine factors with eigenvalues > 1, which accounted for 58.3% of variance. However, the scree plot, Velicer’s MAP test, and parallel analysis all suggested retaining six factors. We accepted this solution and removed five items that had primary loadings on the last three factors. We also removed four items with low factor loadings (<.40), four cross-loading items (≥.30), and six low-loading items to give each subscale an equal number of items (i.e., four). The final six-factor solution accounted for 64.0% of variance (factor loadings range = .44-.92, item–total correlation range = .32-.64; inter-factor correlation range = .14-.50; see Table 1).

Table 1.

Factor Loadings: Sample A (n = 531).

Item	F1 Internal: How to improve	F2 External: How to improve	F3 External: On progress	F4 External: Goal suitability	F5 Internal: On progress	F6 Internal: Goal suitability
I have not worked out my own strategies to get the career I want	.83	−.04	.15	−.02	−.01	−.02
I feel uncertain about what I have to do to reach my career goals	.72	−.06	−.14	.12	.13	−.01
Even by looking at what others are doing, I cannot see what I need to do to get the career I want	.67	−.06	−.07	.06	.12	.05
I do not have many ideas about how to reach my career goals	.62	−.05	.07	−.05	−.02	.14
I do not get helpful advice from others about how I can reach my career goals	−.10	−.92	−.01	.03	.02	.02
I do not get helpful information from people about how to get into the career I want	.02	−.83	−.03	.01	−.06	.02
I do not get good guidance from the people around me about progressing my career goals	.00	−.71	−.01	−.03	−.01	−.02
I find it hard to get constructive career advice from others	.19	−.53	.04	−.01	.08	−.02
People tell me that I am not working hard enough to get into my chosen career	−.13	−.05	.71	.09	−.04	−.00
People around me tell me that I am not motivated enough to achieve my chosen career	.10	.01	.67	.13	−.10	.10
People tell me that what I have achieved so far is not good enough to get into the career I want	−.04	.03	.61	−.03	.12	−.03
I am told that I am not making good progress toward my career goals	.11	−.02	.45	.14	−.03	.00
I am told that my preferred career choice will not let me display my real talents	−.05	−.01	−.03	.76	.06	.03
People I trust think that my preferred career direction is not consistent with my values	−.01	−.02	.02	.68	.03	.02
People tell me that I have chosen a career direction that does not really suit me	.07	.01	.08	.62	−.03	.01
People who know me well think that my chosen career will not bring me what I really want in life	.02	−.01	.05	.61	−.02	.02
I know that the career progress I have made is less than that of others of my age	−.04	−.14	.08	.03	.66	.09
My experience tells me that I am not progressing my career goals as well as I should	.08	.02	.02	−.08	.59	−.07
When I reflect on it, I don’t think I have made as much progress toward my chosen career as I wanted	.06	.02	−.03	.10	.59	.06
Compared with others of my age, I think I am less on track for the career that I want	−.01	−.11	.09	.03	.57	.01
The career direction I have chosen feels right to me (R)	−.07	−.04	−.01	−.02	−.08	.89
I do not think that my chosen caree r would be good for me	.03	.01	−.03	.16	.12	.59
My career goals do not excite me	.05	.01	−.02	.19	.07	.56
My career plans make a lot of sense to me (R)	.22	−.02	.09	−.11	.06	.44
Cronbach’s α	.86	.84	.74	.78	.75	.77

Note. Main loadings are highlighted in bold.

CFA

Using Sample B (n = 524), we conducted CFA analyses with maximum likelihood estimation (AMOS V22) to validate the factor structure of the 24-item Feedback on Career Goals (FCG) Inventory. Model fit was indicated by the χ² / df ratio (<3:1 desired), the comparative fit index (CFI) and Tucker–Lewis index (TLI; >.90), and the root mean square error approximation (RMSEA) and standardized root mean square residual (SRMR; <.08; Hair, Black, Babin, & Anderson, 2010). The Bayesian information criterion (BIC) was used to discriminate among models that were not nested (a BIC difference >6 indicates a better fitting model; Raftery, 1995). No correlated error terms were included, and there was no post hoc respecification. We tested and compared four competing first-order models: one-factor, two-factor (two correlated factors of internal and external feedback), three-factor (three correlated factors of feedback on goal suitability, progress, and improvement), and six-factor (six factors identified in the EFA). Only the six-factor model had an adequate model fit (see Table 2).

Table 2.

Confirmatory Factor Analyses: Sample B (n = 524).

Models	χ²	df	χ² / df	BIC	TLI	CFI	RMSEA	90% CI	SRMR
First-order
One-factor	2,450.22	252	9.72	2,750.77	.45	.50	.13	[.13, .13]	.11
Two-factor	2,183.80	251	8.70	2,490.609	.52	.56	.12	[.12, .13]	.14
Three-factor	1,557.87	249	6.26	1,877.203	.67	.70	.10	[.10, .11]	.09
Six-factor	510.60	237	2.15	905.070	.93	.94	.05	[.04, .05]	.05
Second-order
One-factor	635.71	246	2.58	973.828	.90	.91	.06	[.05, .06]	.07
Two-factor	670.10	246	2.72	1,008.221	.89	.90	.06	[.05, .06]	.08
Three-factor	543.07	244	2.23	893.710	.92	.93	.05	[.04, .05]	.06
Third-order
One-factor	543.12	245	2.22	887.505	.92	.93	.05	[.04, .05]	.06

Note. A negative variance was produced for one error term in the second-order models with two and three factors, and two were identified in the third-order model. We examined the CIs (generated by AMOS bootstrapping) around the negative variances and found that the upper limits all included positive values, which indicates that the negative error variances likely occurred due to sampling errors (Kolenikov & Bollen, 2012). Following Anderson and Gerbing’s (1988) recommendations, we constrained these residuals to .005. BIC = Bayesian information criterion; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual; RMSEA = root mean square error approximation; CI = confidence interval.

Next, we tested three competing second-order models and compared them with the six-factor first-order model. These were a single second-order factor model (the six first-order factors loaded onto one general factor), a second-order model with two factors (external and internal feedback), and a second-order model with three factors (the three dimensions of feedback). Each second-order model fit the data, but the model with three factors had the best fit. This model also had a lower BIC than the first-order model with six factors (difference value > 6) and was accepted as the preferred model. Correlations among these three second-order factors ranged from .48 to .72 (p < .001); all factor loadings were >.45 (p < .001; see Table 2).

Finally, we tested if this second-order model could be represented by a superordinate factor of career goal feedback. A third-order factor model fitted well (see Table 2). The three second-order factors loaded on the single third-order factor >.67, indicating strong relationships between the second- and third-order factors. Although this model was statistically equivalent to the preferred second-order model, the third-order model is conceptually more parsimonious. These analyses supported conceptualizing career goal feedback as a hierarchical, multidimensional construct. We provided both composite reliabilities (CR; Lucke, 2005; Yang & Green, 2011) and internal consistency reliabilities for the full scale (CR = .92; α = .87), scale of feedback on progress (CR = .80; α = .77), goal suitability (CR = .85; α = .83), and improvement needed (CR = .88; α = .82), and the six first-order subscales (CR range = .71-.86; α range = .70-.84).

Validity

Also using Sample B, we evaluated convergent validity of the 24-item FCG by examining the correlations with the FCPSO Scale. The full FCG and the three second-order factors correlated positively with FCPSO (rs = .24-.58; see Table 3). Of the individual subscales, only external feedback on how to improve did not have a significant association.

Table 3.

Summary Data for Sample B in Chinese (n = 524; Correlations Above Diagonal) and Australian Samples (n = 184; Below Diagonal).

Variables	Chinese		Australian		1	2	3	4	5	6	7	8	9	10	11	12	13
Variables	M	SD	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13
Career Goal Feedback Scale
1. Full scale	77.53	14.51	65.30	19.50	—	.74**	.77**	.78**	.60**	.61**	.58**	.67**	.73**	.70**	.48**	.61**	−.44**
2. Combined Progress subscale	28.20	6.02	24.35	8.16	.87**	—	.38**	.37**	.86**	.30**	.20**	.83**	.34**	.40**	.58**	.41**	−.36**
3. Combined Goal Suitability subscale	22.71	6.34	18.29	6.98	.80**	.55**	—	.39**	.31**	.87**	.28**	.31**	.87**	.37**	.29**	.45**	−.32**
4. Combined How to Improve subscale	26.62	6.62	23.67	7.89	.86**	.63**	.53**	—	.23**	.23**	.82**	.41**	.46**	.81**	.24**	.53**	−.33**
5. External Progress subscale	13.14	3.72	10.73	4.28	.82**	.89**	.57**	.61**	—	.31**	.10*	.44**	.24**	.28**	.52**	.28**	−.26**
6. External Goal Suitability subscale	10.81	3.66	9.12	4.24	.76**	.54**	.92**	.50**	.55**	—	.16**	.20**	.51**	.20**	.29**	.34**	−.22**
7. External How to Improve subscale	12.40	4.09	11.29	3.89	.74**	.53**	.41**	.90**	.52**	.42**	—	.26**	.32**	.33**	.06	.29**	−.19**
8. Internal Progress subscale	15.06	3.37	12.61	4.74	.74**	.91**	.43**	.52**	.62**	.42**	.44**	—	.34**	.41**	.47**	.43**	−.35**
9. Internal Goal Suitability subscale	11.90	3.63	9.17	3.46	.69**	.45**	.88**	.46**	.46**	.63**	.31**	.35**	—	.43**	.22**	.45**	−.34**
10. Internal How to Improve subscale	14.22	4.02	12.38	4.71	.83**	.61**	.55**	.93**	.59**	.49**	.67**	.51**	.51**	—	.33**	.58**	−.36**
Validity Scales
11. Feedback From Significant Others	27.58	5.79	—	—											—	.32**	−.22**
12. Career-Related Stress	15.80	4.80	—	—												—	−.37**
13. Career Self-Efficacy	26.74	4.60	—	—													—

Note. *p < .05, **p < .01.

To test incremental validity, we used two hierarchical regression analyses with FCPSO as baseline predictor, total scores on the three second-order FCG subscales at Step 2, and career stress and career self-efficacy as dependent variables. In both analyses, the three FCG subscales explained additional variance to that explained by FCPSO in stress (β = .24, .13, .36, respectively; ts = 2.94-9.36; ΔF = 78.43, p < .001) and self-efficacy (β = −.17, −.24, −.18, respectively; ts = −4.59-−3.84; ΔF = 32.35, p < .001); an additional 28% of variance was explained in stress and 15% in self-efficacy.

Finally, the correlations between the overall FCG score and age, gender, university year, rural area, college entrance score, and SES were negligible (r range = |.01-.03|), indicating that participants did not respond differently across these different groupings.

Study 2

Method

We used standard translation/back-translation (Jones et al., 2001) to convert the 24-item FCG Inventory into English, and administered it as an online survey to 184 young adults (78% female; M_AGE = 19.44 years, SD = 2.08 years) recruited from one Australian university. These students mostly identified as Australian, although a small number of English-speaking international students participated. Students were recruited from a course website and received course credit and entry to a AUS$50 shopping voucher draw for completing the survey.

Results

We examined the 24 items individually and found none to be skewed (>1) or kurtotic (>3), indicating normally distributed item responses. Correlations ranged from .31 to .67 among the six first-order factors and from .53 to .63 for the three second-order factors. The second-order factors correlated from .80 to .87 with the total feedback score. In addition, the overall FCG score was unrelated to gender (r = −.11, p = .14) and age (r = .04, p = .62; see Table 3).

We used CFA to test a model where the six first-order factors loaded onto the three second-order factors, which, in turn, loaded onto a single superordinate factor of career goal feedback. The model showed an acceptable fit, χ² / df = 1.95, CFI = .93, TLI = .92, RMSEA = .07 [.06, .08, p < .001], SRMR = .06, and all factor loadings were significant (p < .001; range = .60-.96).These results were consistent with the multidimensional factor structure found in the Chinese sample. Composite reliabilities (CR) and internal consistency reliabilities were .95 and .94 for the full scale, .93 and .91 for feedback on progress, .91 and .89 for feedback on goal suitability, and .95 and .93 for feedback on improvement needed, respectively. Ranges for the six first-order subscales were .78 to .91 (CR), and .78 to .90 (α).

Discussion

Feedback about career goals is a crucial influence on career development in young adults (Kerpelman & Pittman, 2001). Based on a synthesis of existing feedback literature in related fields (e.g., Hattie & Timperley, 2007; Shute, 2008), we operationalized career goal feedback as a multidimensional construct regarding information about one’s career goal suitability, goal progress, and improvements or adjustments needed, gained from both external and internal sources. From this, we developed and presented initial evidence of validity for a psychometrically sound, 24-item inventory to assess these three dimensions and two sources of negative career goal feedback in young adults. The EFAs and CFAs with the Chinese sample indicated that the new measure reflected the six intercorrelated factors. We confirmed that these six factors could be represented as three second-order factors (i.e., feedback on goal suitability, progress, and how to improve), which could be reflected in a single, superordinate factor (i.e., feedback could be modeled as multidimensional and hierarchical).

We provided evidence that the FCG Inventory is internally reliable, the association between the FCG Inventory and the FCPSO scale supported convergent validity, and the regression analyses supported incremental validity over the existing FCPSO scale for both cognitive (i.e., self-efficacy) and affective (i.e., stress) outcomes. Previous research has demonstrated the importance of feedback in individual self-regulation (e.g., Creed et al., 2015; Lord et al., 2010), and the present results show that assessing multiple aspects and sources of career feedback adds to the understanding of these self-regulatory processes. We also showed that the inventory was unrelated to demographic variables (e.g., age, gender), indicating no inherent bias based on these characteristics, and demonstrated that the inventory performed similarly with English-speaking young adults, in terms of factor structure and psychometric characteristics, highlighting the usefulness of the scale in a Western sample.

Future Research

First, both samples contained disproportionately more young women than men, and our analyses using the Western sample were restricted to confirming the factor structure and assessing reliability. Thus, future studies should test the applicability of the scale with other populations. Second, although we provided initial evidence for validity, in the Chinese sample, the External Feedback on Improvement subscale was not related to FCPSO, while the Internal Feedback on Improvement subscale was. In addition, the External Improvement subscale was only weakly, though significantly, related to the other two external feedback subscales (on goal suitability and progress) in the Chinese sample, although in the Australian sample, there were moderate to strong associations among all external feedback subscales. These inconsistencies in the results for the Chinese sample suggest that there might be something specific about the way external feedback on improvement is perceived in Chinese young people. Therefore, future studies should continue to evaluate the scale’s validity. We were not able to assess predictive validity in these two cross-sectional studies, and this needs to be assessed in the future. For example, longitudinal studies could assess if higher scores on the FCG Inventory predicted subsequent changes in career self-regulatory processes, such as adjustments to career goals, as theory suggests (Kerpelman & Pittman, 2001).

We anticipate that this new measure will encourage research into the role of feedback in young peoples’ career development. Researchers can use this measure to investigate the effects of career goal feedback. Feedback is hypothesized to influence performance improvement and goal attainment, with these effects mediated by self-regulatory behaviors, such as implementing goal-striving strategies, exerting effort, and adjusting goals (Nicklin & Williams, 2011). Feedback affects cognitive (e.g., discrepancy, self-efficacy, outcome expectations; Creed & Hood, 2015; Lord et al., 2010), affective (e.g., anxiety, distress; Creed et al., 2015), and motivational processes (e.g., intention to exert more or less effort; Brown & McConnell, 2011), which, in turn, motivate goal adjustment (Ilies et al., 2010). These relationships have been proposed in several theories and supported by empirical studies (e.g., goal setting theory, Latham & Locke, 1991). However, little is known about the strength of the relationship between each of the three dimensions of the career goal feedback construct and these outcomes; nor do we know how these relationships are affected by personal (e.g., goal orientation) and contextual (e.g., collectivism vs. individualism) factors. This research is critical to a better understanding of the complex, underlying mechanisms of career feedback perception and utilization in young people.

Practical Implications

This brief, multidimensional measure can be used in career counseling and educational programs with adolescents and young adults. It could be used as a screening tool to identify students who might benefit from help in setting suitable career goals, in adjusting their goals, and in identifying adaptive strategies to better progress them. For example, for young people who report receiving negative feedback on their goal suitability, providing information to help them further explore their values and interests, their current goals, and other career possibilities would be helpful in goal setting. For those who perceive negative feedback on their goal progress and future improvement, learning how to utilize this feedback in self-appraisal and behavioral regulation would be beneficial. These strategies, in turn, could facilitate more effective management of, and appropriate responses to, unfavorable feedback, thereby reducing negative emotional and behavioral outcomes and stimulating more career exploration. This is likely to lead to more active and adaptive career management, and consequently help adolescents better prepare for their future career and life.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Anderson

J. C.

Gerbing

D. W.

(1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411-416.

Bandura

(1991). Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes, 50, 248-287.

Brown

C. M.

McConnell

A. R.

(2011). Discrepancy-based and anticipated emotions in behavioral self-regulation. Emotion, 11, 1091-1095.

Butler

D. L.

Winne

P. H.

(1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65, 245-281.

Choi

B. Y.

Park

Nam

S. K.

Lee

Cho

Lee

S. M.

(2011). The development and initial psychometric evaluation of the Korean Career Stress Inventory for college students. Career Development Quarterly, 59, 559-572.

Creed

P. A.

Hood

(2015). The development and initial validation of a scale to assess career goal discrepancies. Journal of Career Assessment, 23, 308-317.

Creed

P. A.

Wamelink

(2015). Antecedents and consequences to perceived career goal-progress discrepancies. Journal of Vocational Behavior, 87, 43-53.

de Kleijn

R. A.

Mainhard

M. T.

Meijer

P. C.

Brekelmans

Pilot

(2013). Master’s thesis projects: Student perceptions of supervisor feedback. Assessment and Evaluation in Higher Education, 38, 1012-1026.

Greenhaus

J. H.

Callanan

G. A.

Kaplan

(1995). The role of goal setting in career management. International Journal of Career Management, 7, 3-12.

10.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis. Upper Saddle River, NJ: Prentice Hall.

11.

Hattie

Timperley

(2007). The power of feedback. Review of Educational Research, 77, 81-112.

12.

Herold

D. M.

Greller

M. M.

(1977). Feedback the definition of a construct. Academy of Management Journal, 20, 142-147.

13.

Ilies

Judge

T. A.

Wagner

D. T.

(2010). The influence of cognitive and affective reactions to feedback on subsequent goals. European Psychologist, 15, 121-131.

14.

Jones

P. S.

Lee

J. W.

Phillips

L. R.

Zhang

X. E.

Jaceldo

K. B.

(2001). An adaptation of Brislin’s translation model for cross-cultural research. Nursing Research, 50, 300-304.

15.

Kerpelman

J. L.

Pittman

J. F.

(2001). The instability of possible selves: Identity processes within late adolescents’ close peer relationships. Journal of Adolescence, 24, 491-512.

16.

Kim

Jung

S. H.

Jang

S. H.

Lee

Rhee

Cho

S. H.

Lee

S. M.

(2014). Construction and initial validation of the Planned Happenstance Career Inventory. Career Development Quarterly, 62, 239-253.

17.

Kluger

A. N.

DeNisi

(1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-284.

18.

Kolenikov

Bollen

K. A.

(2012). Testing negative error variances: Is a Heywood case a symptom of misspecification? Sociological Methods & Research, 41, 124-167.

19.

Latham

G. P.

Locke

E. A.

(1991). Self-regulation through goal setting. Organizational Behavior and Human Decision Processes, 50, 212-247.

20.

Locke

E. A.

Latham

G. P.

(1990). Work motivation and satisfaction: Light at the end of the tunnel. Psychological Sicence, 1, 240-246.

21.

Lord

R. G.

Diefendorff

J. M.

Schmidt

A. M.

Hall

R. J.

(2010). Self-regulation at work. Annual Review of Psychology, 61, 543-568.

22.

Lucke

J. F.

(2005). The α and the ω of congeneric test theory: An extension of reliability and internal consistency to heterogeneous tests. Applied Psychological Measurement, 29, 65-81.

23.

Nawaz

Jahanian

Manzoor

S. W.

(2012). Critical elements of constructive performance feedback: An integrated model. European Journal of Business and Management, 4, 76-84.

24.

Nicklin

J. M.

Williams

K. J.

(2011). Self-regulation of goals and performance: Effects of discrepancy feedback, regulatory focus, and self-efficacy. Psychology, 2, 187-200.

25.

Nunnally

J. C.

(1978). Psychometric theory. New York, NY: McGraw-Hill.

26.

Paa

H. K.

McWhirter

E. H.

(2000). Perceived influences on high school students’ current career expectations. Career Development Quarterly, 49, 29-44.

27.

Perrone

K. M.

Sedlacek

W. E.

Alexander

C. M.

(2001). Gender and ethnic differences in career goal attainment. Career Development Quarterly, 50, 168-178.

28.

Raftery

A. E.

(1995). Bayesian model selection in social research. Sociological Methodology, 25, 111-163.

29.

Salazar

M. S.

(2015). The dilemma of combining positive and negative items in scales. Psicothema, 27, 192-200.

30.

Schyns

von Collani

(2002). A new occupational self-efficacy scale and its relation to personality constructs and organizational variables. European Journal of Work and Organizational Psychology, 11, 219-241.

31.

Shi

B.-G.

Shen

J.-L.

(2007). The relationships among family SES, intelligence, intrinsic motivation and creativity. Xin Li Fa Zhan Yu Jiao Yu, 23, 30-34.

32.

Shute

V. J.

(2008). Focus on formative feedback. Review of Educational Research, 78, 153-189.

33.

Sims

H. P.

Szilagyi

A. D.

Keller

R. T.

(1976). The measurement of job characteristics. Academy of Management Journal, 19, 195-212.

34.

Van Sonderen

Sanderman

Coyne

J. C.

(2013). Ineffectiveness of reverse wording of questionnaire items: Let’s learn from cows in the rain. PLoS ONE, 8, e68967.

35.

Wang

Mukhopadhyay

(2012). The dynamics of goal revision: A cybernetic multiperiod Test-Operate-Test-Adjust-Loop (TOTAL) model of self-regulation. Journal of Consumer Research, 38, 815-832.

36.

Yang

Green

S. B.

(2011). Coefficient alpha: A reliability coefficient for the 21st century? Journal of Psychoeducational Assessment, 29, 377-392.