Development and Validation of the Career Counseling Outcome Questionnaire in Two Clinical Settings

Abstract

This study reports the development and initial validation of the Career Counseling Outcome Questionnaire (CCOQ) in two individual career counseling settings using a college sample (n = 1,140) and a community sample (n = 161). Exploratory and confirmatory factor analyses revealed five correlated factors in both samples: (a) knowledge of the career decision-making process (4 items), (b) knowledge of the self (3 items), (c) knowledge of career information (3 items), (d) anxiety towards career decision-making (3 items), and (e) career undecidedness (2 items). The CCOQ scale scores changed in the expected theoretical direction during the career counseling interventions and did not change when clients were not receiving counseling. Except for anxiety, all CCOQ subscales predicted satisfaction with the career decision twelve months after counseling. The CCOQ total score predicted satisfaction with the decision twelve months after counseling, over and beyond a widely used instrument assessing sources of career decision difficulties (i.e. the Career Decision Difficulties Questionnaire; CDDQ). Career counselors could use the CCOQ to monitor the effectiveness of their interventions in complement to diagnostic measures such as the CDDQ.

Keywords

career counseling outcome effectiveness career decision-making

Introduction

One of the primary goals of individual career counseling is to help individuals make satisfying career decisions (Levin et al., 2022). Meta-analyses (Brown & Ryan Krane, 2000; Oliver & Spokane, 1988; Ryan Krane, 1999; Spokane & Oliver, 1983; Whiston et al., 1998, 2017) conducted in the past decades demonstrate that individual career counseling can lead to positive pre-post change in career decision outcomes with moderate to large effect sizes (ds ranging between 0.41 and 1.08). Although previous research supports the effectiveness of individual career counseling, there is important variability in clients’ responses to this type of intervention (Covali et al., 2011; Milot-Lapointe et al., 2016, 2019; Multon et al., 2001, 2007). Milot-Lapointe and Le Corff (2023) found that while an important proportion of clients (66%) experienced an optimal change in their career decision difficulties during individual career counseling, 21% experienced a suboptimal change and 13% did not experience any change. Clients who did not experience any change during counseling had significantly higher decision difficulties, were less satisfied with their career decision, career situation and counseling, and had lower life satisfaction twelve months after counseling compared to other clients (Milot-Lapointe & Le Corff, 2024a).

Because of the longitudinal consequences associated to variability in career counseling effectiveness, Milot-Lapointe and Le Corff (2024a) highlighted the importance of early detection of non optimal changes during individual career counseling. They suggested that by monitoring client progress using objective assessments, counselors could identify clients at risk of not making significant progress and might prevent them from experiencing a non optimal change. Despite the importance of clinical assessment of outcomes of career counseling (Milot-Lapointe et al., 2019; Whiston, 2001), career counselors who work in natural settings are often left without concrete feedback regarding the impact of the intervention they delivered (Rochat, 2019). To remedy this situation, several researchers argued (Milot-Lapointe et al., 2019; Rottinghaus et al., 2017; Toggweiler & Künzli, 2020; Whiston & Rahardja, 2008) that there is a strong need for sound career-specific outcome measures that career counselors could use to monitor the effectiveness of their interventions in natural settings. Indeed, as it has been extensively demonstrated in meta-analyses in the field of personal counseling (e.g., Shimokawa et al., 2010), clinical use of outcome measures helps counselors identify clients who are at risk of leaving counseling without clinical gains and improve counseling effectiveness. In addition, given the increasing importance of accountability (e.g., funders, administrators, parents) to counseling outcomes, the use of this type of scale could be helpful in providing evidence of the effectiveness of individual career counseling (Rottinghaus et al., 2017; Whiston, 2001).

In view of the preceding, this study aimed to develop and validate a career counseling outcome measure that career counselors could use to monitor the effectiveness of their interventions in natural settings.

Why Developing a Career Counseling Outcome Measure?

Over the last decades, several measures of career decision difficulties, most notably the Career Decision Difficulties Questionnaire (CDDQ; Gati et al., 1996) and the Career Thoughts Inventory (Sampson et al., 1998), have been developed for diagnostic purposes, with the aim of helping counselors assess the causes of their clients’ career indecision (Xu & Bhang, 2019). Although diagnostic measures have strong utility in clinical practice, their purpose is qualitatively different from the purpose of measures that aim to monitor the outcomes of counseling and therefore their use as an outcome measure might not be optimal (Vermeersch et al., 2000, 2004). Indeed, while high levels of reliability and validity are emphasized in the construction of diagnostic measures, questionnaires that are intended to monitor client change during counseling also need to emphasize sensitivity to change as a central and primary property before it can be confidently used to track the effects of counseling interventions on clients (Toussaint et al., 2020; Vermeersch et al., 2000, 2004).

Sensitivity to change refers to the degree to which an instrument accurately reflects changes that occur or should occur following participation in a counseling intervention (Hill & Lambert, 2004; Vermeersch et al., 2004). To demonstrate optimal sensitivity to change, scales of an outcome measure should: (1) significantly change in the expected theoretical direction during a counseling intervention (Tryon, 1991; Vermeesch et al., 2000), and (2) not change when clients are not exposed to a counseling intervention (Tryon, 1991) or show greater change in treated than in untreated clients (Vermeersch et al., 2000). A number of factors are required to foster the sensitivity to change of an outcome measure. First, outcome measures used in clinical settings should be consistent with the intended target of the career counseling intervention (Verbruggen et al., 2013; Whiston, 2001). Second, they should assess constructs that are susceptible to change (Versmeesh et al., 2004). Third, they should assess constructs not subject to floor or ceiling effects (Vermeesh et al., 2004).

Many researchers and clinicians (e.g. Masdonati et al., 2009, 2014; Massoudi et al., 2008; Milot-Lapointe et al., 2018, 2020, 2022; Rochat, 2019) have adopted the CDDQ to assess decision outcomes of career counseling because of the strong theoretical and empirical support it received as a measure of career decision difficulties (e.g. Gati et al., 1996, 2000). However, previous findings in the specific context of individual career counseling (Masdonati et al., 2009, 2014; Massoudi et al., 2008; Milot-Lapointe et al., 2022) suggest that the CDDQ’s sensitivity to change may not be optimal. Indeed, the afore mentioned studies suggest that some of the constructs assessed by the CDDQ (i.e. motivation, indecisiveness, dysfunctional career beliefs, and external conflicts) are relatively stable in the short-term and thus, potentially not a feasible target of intervention for natural individual career counseling (which is often two or three sessions; Elad-Strenger & Littman-Ovadia, 2012; Milot-Lapointe et al., 2018).

In light of these findings, although the CDDQ is strongly supported as a diagnostic measure of career decision difficulties and is frequently employed in outcome research, it may not fully reflect the magnitude of career counselors’ impact in natural settings. Additionally, although the CDDQ contains a relatively low number of items (32 items), clinical outcome assessment requires multiple assessments (ideally at each session) and should thus contain as few items as possible. For these reasons, there is a need to develop a very short career counseling outcome measure that could complement diagnostic instruments such as the CDDQ in career counselors’ day-to-day practice. To our knowledge, no such measure exists in the English or French language.

Theoretical Background of the New Scale

We propose that an outcome measure should assess content or indicators that should change and can realistically change positively during career counseling to help clients make a satisfying career decision. To identify these indicators, we examined the content involved in each phase of two empirically supported intervention models of career decision-making that provide individuals with procedures for making satisfying career decisions. These models are the Prescreening, In-depth, and Choice model of career decision (PIC; Gati & Asher, 2001) and the Cognitive Information Processing model (CIP; Sampson et al., 1999). These models were selected because their effects on client satisfaction with career decisions have been supported in one-year follow-up (Milot-Lapointe & Le Corff, 2024a, 2024b) and six-year follow-up (Gati et al., 2006) studies.

Based on these frameworks, the career counseling process requires that clients, among other things, increase their decision-making readiness, learn the steps of the decision-making process, learn how to collect reliable information about themselves and options, and learn how to analyse and use collected information as a basis for choosing the alternative that best suits them (Gati & Kulcsár, 2021; Milot-Lapointe & Le Corff, 2024a). Our examination of the content involved in these models led us to propose five content dimensions that should and can realistically change positively during short-term individual career counseling to help clients make a satisfying career decision. These dimensions are: (1) Client’s anxiety toward career decision-making, Client’s knowledge on (2) the process, (3) the self and (4) career information, and (5) Client’s undecidedness. In the following sections, we explain why these dimensions were retained and present our conceptual definitions of these dimensions.

Client’s Anxiety Toward Career Decision-Making

The career decision-making process requires that counselors increase client’s readiness. Client’s anxiety toward career decision-making is an indicator of client’s readiness to make an optimal career decision in the scale developed in this study. Based on the CIP, it is defined as dysfunctional anxious thoughts about the career decision-making process, the consequences of the career decision, and one’s ability to make a satisfying career decision that can « prevent [an] individual from thinking in a systematic and organized manner about the problem and making a rational decision » (Kleiman et al., 2004, p. 314). While in the PIC model increasing client’s readiness involves intervening on difficulties that arose prior to entering the career decision-making process (i.e. lack of motivation, indecisiveness and dysfunctional beliefs), increasing readiness in the CIP model involves intervening on dysfunctional anxious thinking that arise during the decision-making process. This conception appears more suitable for short-term career counseling since dysfunctional anxious thoughts stem from conscious or preconscious levels while core dysfunctional beliefs (Clark & Beck, 2011), indecisiveness and lack motivation (Levin et al., 2022) may require long-term interventions.

Client’s Knowledge

To help clients make satisfying career decisions, the PIC model and the CIP approach recommend that career counselors increase client’s knowledge. In line with their conception, we conceptualized client’s knowledge as the level of knowledge they possess about (a) the process of career decision-making (i.e. steps involved in the process and ways of obtaining the information required at each step), (b) self (i.e. interests, values, personality, abilities, and career-related preferences), and (c) career information (i.e. characteristics of educational programs and jobs, labor market needs). Our conception of client’s knowledge is similar to the content of the lack of information category in the CDDQ (Gati et al., 1996). The differences between both are twofold. First, the scale developed in this study aims to measure the increase in client’s knowledge during the counseling process while the lack of information scale of the CDDQ measures the reduction of lack of information during the process (when it is used as an outcome measure). In a clinical context where feedback on client progress is needed, we believe that an outcome measure providing the client with feedback about the extent to which their knowledge (e.g. about the self) has increased align more with the goals of counseling. Second, the client’s knowledge subscales assess whether clients possess sufficient knowledge to make a good decision about their career, while the lack of information subscale measures the extent to which clients are undecided because they lack information. Thereby, the CDDQ lack of information subscale indicates to what extent client’s career indecision is caused by lack of information whereas our client’s knowledge subscale is an indicator of whether clients have sufficient knowledge to make a good career decision. Since the scale developed in this study is meant to be used as an indicator of whether or not a client is on track to make a satisfying career decision, this focus on the quality of the decision appeared more suitable.

Client’s Career Undecidedness

Reduction of undecidedness is a change that clients ideally expect before leaving career counseling. According to the PIC model, the implementation of the career decision may be delayed, impeded, or avoided if the client is uncertain whether they are making the right decision or not for them (Gati & Asher, 2001). Assessment of undecidedness is typically absent from diagnostic measures of career decision difficulties since their focus is on the causes of the state of undecidedness. In the scale developed in this study, undecidedness represents the client’s subjective uncertainty regarding what is the best possible decision for them.

Aims of the Current Study

The current study aimed to provide career counselors with a very short career counseling outcome measure that they could use to track client progress during counseling. This measure could assist them in identifying clients at risk of leaving counseling without meaningful clinical gains or with difficulties that could lead them to make a career decision that is not satisfying in the months following the process. To ensure the clinical utility and the psychometric qualities of the scale developed, this study aimed to (1) assess the factor structure of the scale developed; (2) assess its sensitivity to change between pre and post counseling and compare it with the CDDQ’s sensitivity to change; (3) assess its predictive validity and its (4) incremental validity in predicting satisfaction with career decision over the CDDQ’s overall level of difficulties.

Method

Development of the Career Counseling Outcome Questionnaire

Item Construction

The authors first generated a set of 24 items in French (six for client’s anxiety toward decision-making, twelve for client’s knowledge, and six for undecidedness) based on the definitions of the five content dimensions. After two review processes with five licensed career counselors (four with a master’s degree and one with a PhD), 17 items (three for client’s anxiety toward decision-making, eleven for client’s knowledge, and three for undecidedness) were retained based on how appropriately they fitted with our conceptualization of career counseling outcomes. Items were written using simple and straightforward language, and phrased so that all clients can provide appropriate responses and to minimize social desirability (Simms & Watson, 2007).

Revision of Items Based on Empirical Data

A sample of 151 French-Canadian clients from the community aged between 14 and 56 years completed the 17-item version of the questionnaire at the beginning of their first career counseling session and at the end of their last session at a university career counseling services center. Mean, standard deviation and pre-post change for each item were analyzed. Two items were removed from the questionnaire because of a lack of change sensitivity. The final version of the questionnaire comprised 15 items, each measured on a seven-point Likert scale ranging from ‘Strongly Disagree’ to ‘Strongly Agree. Items are listed in Table 2.

Participants

Two different individual career counseling samples participated in the validation of the final 15-item version of the questionnaire developed in this study: a college sample and a community sample.

College Sample

This sample included 1 140 college students (69% women, 30% men; 1% identified as trans, non-binary, or to another gender) aged between 17 and 46 years (M = 18.83; SD = 3.00) who received an average of 3.73 sessions (SD = 1.78) of naturally occurring individual career counseling at the career services center of their college (n = 22). Most clients were born in Canada (87%) and were White (80%), while 6% designated themselves Black, 3% Asian, 2% Arab, 2% Hispanic, and 6% preferred not to answer. Career counseling sessions were delivered by 61 experienced career counselors with a master’s degree in career counseling. Clients reported that making a good career decision was their main career counseling goal.

Community Sample

This sample consisted of 161 French-Canadian individuals from the community (105 women, 56 men; no client identified as trans, non-binary, or to another gender) aged between 16 and 59 years (M = 28.61; SD = 10.44) who received an average of 5.78 career counseling sessions at a university career counseling services center. Almost all clients were born in Canada (98%). Among the 161 clients, 76% were workers, 3% were unemployed, and 21% were high school, college, undergraduate or graduate students. Employed and unemployed clients reported that they were deliberating about making a career change (e.g., moving from a field to another). Student clients reported that they were deliberating about choosing an educational program to pursue in the next year. Career counseling sessions were delivered by 108 undergraduate or graduate students in career counseling trained to an integrative cognitive career counseling intervention based on the CIP and the PIC (see Milot-Lapointe & Le Corff, 2024a, 2024b).

Other Measures

Career Decision-Making Difficulties Questionnaire (CDDQ)

The French version (Rossier et al., 2022) of the CDDQ (Gati et al., 1996) was used to ensure that the scale developed in this study brings an additional value to the measurement of career counseling effectiveness. The CDDQ is a 32-item Likert-type questionnaire that assesses various difficulties individuals may encounter when making career decisions. The CDDQ has been found to have high internal consistency (alpha coefficient of 0.92 for the total scale) and construct validity in a French sample (Rossier et al., 2022).

Satisfaction With Career Decision (SWCD)

To assess the incremental validity of the scale developed in this study, clients’ satisfaction with their career decision was assessed twelve months after counseling using a single question rated on a 5-point Likert-type scale ranging from very unsatisfied to very satisfied (Milot-Lapointe & Le Corff, 2024a, 2024b).

Procedure

The protocol of this study has been approved by the ethical committee of the authors’ institution and conducted in accordance with APA ethical standards. Clients from the community sample completed the career counseling outcome questionnaire (CCOQ) developed in this study and the CDDQ approximately one month before their first career counseling session, immediately before their first session, after their last counseling session, and six months after counseling. They also answered the satisfaction with career decision item twelve months after counseling. Clients from the college sample completed the CCOQ and the CDDQ before their first career counseling session, after their last counseling session, and six months after counseling. They also completed the satisfaction with decision item twelve months after counseling. Clients received a financial compensation of 40$ for completing the pretest and the posttest measures, and another compensation of 40$ for completing the questionnaires at follow-ups.

Results

Factor Structure (Objective 1)

Although a five-factor structure was posited when developing the CCOQ, a two-step procedure was conducted to empirically establish its structure and ascertain the relevance of each item. In the first step, exploratory factor analyses (EFA) were computed to determine the factor structure that best fitted the data. In the second step, the structure obtained in the first step was tested using confirmatory factor analysis (CFA) in two independent samples. In order to do so, sample 1 (college students) was randomly divided into two samples, one exploratory sample (on which EFA were conducted) and one confirmatory sample (on which CFA were conducted), each including 570 participants. CFA were also conducted on sample 2 (community), separately from the confirmatory college sample.

Exploratory Factor Analyses

To obtain an estimate of the number of factors to extract, an EFA with principal axis factoring and oblimin rotation (to take into account the theoretically expected relations between scales/factors) was conducted, a Horn parallel analysis (Horn, 1965) was computed, and Catell’s scree plot for eigenvalues was examined using SPSS 29. The exploratory college sample included only two missing data, and pairwise deletion was used in the EFA. Item scores did not deviate significantly from normality, with skewness and kurtosis ranging between +/− 2.0. Bartlett’s test of sphericity (χ² = 4639.80; p < .001) and the Keiser-Meyer-Lokin measure (KMO = 0.89) indicated that the items were sampled adequately for factor analysis and that factors were likely to be reliable (Tabachnick & Fidell, 2019). Horn parallel analysis results suggested the presence of five factors, Catell’s scree plot included two “elbows” suggesting the presence of either two or five factors, while four factors had an eigenvalue greater than one.

Subsequently, a series of EFA with the robust maximum likelihood extraction method and Geomin rotation (.5) were conducted using Mplus 8.1 to test models including one to six factors. As shown in Table 1, all three comparative fit indices used (Akaike Information Criterion, Bayesian Information Criterion, and sample-size adjusted Bayesian Information Criterion) indicated that the five-factor model had the best fit to the data. Relative (Comparative Fit Index, Tucker-Lewis Fit Index) and absolute (Root Mean Square Error of Approximation) fit indices also indicated that the five-factor model best fitted the data. Examination of standardized factor loadings for this model (see Table 2) shows that each item had a strong primary loading on its expected factor (ranging from .50 to .84) and that cross loadings were low (the highest was .33 and all others were below .30). These five factors correspond to the scales that the authors intended when developing the CCOQ, as explained in the Introduction. Consequently, factor 1 was labeled “Knowledge of the career decision-making process”, factor 2 “Self-knowledge”, factor 3 “Knowledge of career information”, factor 4 “Anxiety towards career decision-making”, and factor 5 “Career undecidedness”.

Table 1.

Goodness-of-fit indices for the exploratory factor analyses.

Models	Scaling factor	χ ²	df	AIC	BIC	sBIC	CFI	TLI	RMSEA
1-factor	1.1717	1506.09	90	29,024	29,219	29,076	.628	.566	.166
2-factor	1.1323	807.19	76	28,201	28,457	28,270	.808	.734	.130
3-factor	1.0282	591.97	63	27,921	28,234	28,006	.861	.768	.121
4-factor	1.0546	317.08	51	27,671	28,036	27,770	.930	.856	.096
5-factor	1.1148	42.44	40	27,406	27,819	27,517	.999	.998	.010
6-factor	0.6456	38.08	30	27,403	27,860	27,526	.998	.993	.022

Note. N = 570; χ² = chi-square; df = degrees of freedom; AIC = Akaike Information Criterion; BIC = Bayesian Information Criterion; sBIC = sample-size adjusted BIC; CFI = Comparative Fit Index; TLI = Tucker Lewis Index; RMSEA = Root Mean Square Error of Approximation.

Table 2.

Geomin-rotated factor loadings for the exploratory five-factor model.

	F1	F2	F3	F4	F5
1. I know enough about the important steps needed to make a good decision about my career.	.70*	.08*	.09*	−.01	−.07*
2. I know how to obtain additional information about myself (for example, about my interests, my abilities or my personality)	.64*	.20*	.05	−.03	−.08*
3. I know how to obtain accurate and up-to-date information about the educational programs or the occupations that may interest me.	.67*	−.04	.16*	.00	.01
4. I know how to combine the information I have about myself with the information I have about careers in order to make a good decision about my career	.50*	.28*	.12*	−.08*	−.09*
5. I know myself well enough (interests, values, personality, and skills) to make a good decision about my career	.11*	.69*	.07*	−.06*	−.08*
6. I know what are the most important aspects of my personality to consider to make a good decision about my career	.07*	.76*	.04	−.03	−.05
7. My job expectations are clear enough to me to make a good decision about my career	.03	.57*	.33*	−.01	−.04
8. I know enough about the characteristics of existing occupations and educational programs to make a good decision about my career	.05	.13*	.77*	−.02	−.08*
9. I know enough about the occupations and educational programs that interest me to make a good career decision about my career	.10*	.11*	.69*	−.01	−.11*
10. I know enough about the needs of the job market to make a good decision about my career	.16*	.01	.64*	−.05	−.04
11. I feel anxious thinking about the decision I have to make for my career	−.02	−.01	.03	.79*	.09*
12. I am worried about the long-term consequences of my career decision	.02	.02	−.06*	.83*	.05
13. I have doubts about my ability to make a career decision that will be satisfying for me in the long run	−.05	−.17*	.00	.58*	.25*
14. I feel uncertain about what decision I should make about my career	.01	.02	−.04	.09*	.84*
15. I’m struggling to make a career decision that will be optimal for me	−.03	−.04	−.03	.04	.80*

Note. N = 570; *p < .05; Loading greater equal than .40 are in bold; F1 = Knowledge of the career decision-making process; F2 = Self-knowledge; F3 = Knowledge of career information; F4 = Anxiety towards career decision-making; F5 = Career undecidedness.

Confirmatory Factor Analyses

CFAs were calculated on the confirmatory college sample and on the community sample to confirm the five-factor structure obtained with the EFA. Thus, a first-order model with five correlated factors was tested. In addition, since it is intended that a total score for the CCOQ can be used as an overall index of whether a client is on track to make a good career decision, a second-order CFA with five first-order factors and one second-order factor was also tested. Moreover, for the sake of completeness, a bifactor model with one general and five specific factors was tested as a supplemental analysis.

The college sample included only 6 missing data (that is, six participants had one unanswered item) which were accommodated via the full information maximum likelihood estimation. There were no missing data in the confirmatory community sample. As shown in Table 3, the five-factor model had an excellent fit to the data according to usual standards (Hu & Bentler, 1995; Kline, 2016; Marsh et al., 2005) in both the confirmatory college sample (CFI = .955; TLI = .942; RMSEA = .058) and in the community sample (CFI = .974; TLI = .966; RMSEA = .048). Standardized loadings, shown in Table 4, were high for all items, varying from .65 to .89 in the confirmatory college sample and from .74 to .93 in the community sample. Correlations between factors 1, 2 and 3 (the “knowledge” factors) were high in the college sample (.64–.73) and moderate in the community sample (.44–.45), correlations between factor 4 (anxiety) and 5 (undecidedness) were also high (college = .67; community = .58), but correlations between factors 1–3 and 4–5 were moderate to low (college = −.25 to −.47; community = −.17 to −.33). McDonald’s Omega coefficients (McDonald, 1999) were high, ranging from .82 to .86 in the confirmatory college sample and from.85 to .90 in the community sample, indicating that the factors had high internal consistencies.

Table 3.

Goodness-of-fit indices for the confirmatory factor analyses.

Models	Scaling factor	χ ²	df	CFI	TLI	RMSEA
5-factor
College sample	1.1704	234.96	80	.955	.942	.058
Community sample	0.9931	110.19	80	.974	.966	.048
Second-order
College sample	1.1718	384.00	85	.914	.894	.079
College sample - respecified	1.1650	253.63	84	.951	.939	.060
Community sample	1.0039	149.83	85	.945	.931	.069
Bifactor
College sample	1.1454	334.35	81	.927	.906	.074
College sample - respecified	1.1430	206.77	80	.964	.952	.053
Community sample	1.1858	142.69	82	.948	.934	.068

Note. N (confirmatory college) = 570; N (community) = 161; χ² = chi-square; df = degrees of freedom; CFI = Comparative Fit Index; TLI = Tucker Lewis Index; RMSEA = Root Mean Square Error of Approximation.

Table 4.

Standardized loadings, factor correlations, and McDonald’s Omega for the five-factor model in confirmatory factor analysis.

	Confirmatory college sample (n = 570)					Community sample (n = 161)
	Standardized factor loadings
	F1	F2	F3	F4	F5	F1	F2	F3	F4	F5
1. I know enough about the important steps needed to make a good decision about my career.	.78					.86
2. I know how to obtain additional information about myself (for example, about my interests, my skills or my personality)	.77					.76
3. I know how to obtain accurate and up-to-date information about the educational programs or the occupations that may interest me.	.65					.74
4. I know enough about how to combine the information I have about myself and occupations to make a good decision about my career	.78					.76
5. I know myself well enough (interests, values, personality, and skills) to make a good decision about my career		.82					.89
6. I know what are the most important aspects of my personality to consider to make a good decision about my career		.81					.82
7. My job expectations are clear enough to me to make a good decision about my career		.74					.74
8. I know enough about the characteristics of existing occupations and educational programs to make a good decision about my career			.84					.90
9. I know enough about the occupations and educational programs that interest me to make a good career decision about my career			.83					.91
10. I know enough about the needs of the job market to make a good decision about my career			.68					.78
11. I feel anxious thinking about the decision I have to make for my career				.80					.87
12. I am worried about the long-term consequences of my career decision				.89					.86
13. I have doubts about my ability to make a career decision that will be satisfying for me in the long run				.78					.75
14. I feel uncertain about what decision I should make about my career					.84					.93
15. I’m struggling to make a career decision that will be optimal for me					.85					.79
	Factor correlations
		F1	F2	F3	F4	F5	F1	F2	F3	F4	F5
F1
F2	.65*					.44*
F3	.73*	.64*				.45*	.45*
F4	−.25*	−.30*	−.28*			−.17*	−.18	−.14
F5	−.31*	−.47*	−.37*	.67*		−.18	−.33*	−.20*	.58*
	McDonald’s omega coefficient
		F1	F2	F3	F4	F5	F1	F2	F3	F4	F5
Omega	.84	.84	.82	.86	.83	.86	.86	.90	.86	.85

Note. *p < .05; Loadings greater equal than .40 are in bold; F1 = Knowledge of the career decision-making process; F2 = Self-knowledge; F3 = Knowledge of career information; F4 = Anxiety towards career decision-making; F5 = Career undecidedness.

Meanwhile, the second-order model had an adequate fit to the data in both the confirmatory college (CFI = .914; TLI = .894; RMSEA = .079) and community (CFI = .945; TLI = .931; RMSEA = .069) samples (Kline, 2016; Marsh et al., 2005), but fit indices were lower than for the five-factor model, especially in the college sample (see Table 3). However, examination of the modification indices (MI) revealed that allowing residuals terms for factors 4 and 5 to correlate in the college sample would increase model fit (MI = 112.81). Indeed, the respecified second-order model had a better fit to the data (CFI = .951; TLI = .939; RMSEA = .060), which was similar to that of the five-factor model. As shown in Table 5, standardized loadings for items were highly similar to those observed in the five-factor model. In both sample, factors 1, 2, and 3 (pertaining to knowledge) were strongly and positively associated with the second-order factor (with λ varying from .78 to .85 in the confirmatory college sample and from .61 to .70 in the community sample) while factors 4 (anxiety) and 5 (undecidedness) had moderate negative association with the second-order factor (λ = −.33 and −.46 in the college sample and λ = −.37 and −.47 in the community sample).

Table 5.

Standardized loadings for the second-order model in confirmatory factor analysis.

	Confirmatory college sample					Community sample
	Standardized factor loadings
	F1	F2	F3	F4	F5	F1	F2	F3	F4	F5
Second-order factor	.83	.78	.85	−.33	−.46	.62	.70	.61	−.37	−.47
1. I know enough about the important steps needed to make a good decision about my career.	.78					.86
2. I know how to obtain additional information about myself (for example, about my interests, my abilities or my personality)	.77					.77
3. I know how to obtain accurate and up-to-date information about the educational programs or the occupations that may interest me.	.65					.74
4. I know how to combine the information I have about myself with the information I have about careers in order to make a good decision about my career	.78					.76
5. I know myself well enough (interests, values, personality, and skills) to make a good decision about my career		.82					.89
6. I know what are the most important aspects of my personality to consider to make a good decision about my career		.81					.82
7. My job expectations are clear enough to me to make a good decision about my career		.74					.74
8. I know enough about the characteristics of existing occupations and educational programs to make a good decision about my career			.84					.89
9. I know enough about the occupations and educational programs that interest me to make a good decision about my career			.83					.92
10. I know enough about the needs of the job market to make a good decision about my career			.68					.78
11. I feel anxious thinking about the decision I have to make for my career				.80					.87
12. I am worried about the long-term consequences of my career decision				.89					.86
13. I have doubts about my ability to make a career decision that will be satisfying for me in the long run				.78					.74
14. I feel uncertain about what decision I should make about my career					.82					.91
15. I’m struggling to make a career decision that will be optimal for me					.86					.81

Regarding the bifactor analysis, model fit was excellent in the confirmatory college sample (CFI = .964; TLI = .952; RMSEA = .053), again after allowing residuals terms for factors 4 and 5 to correlate (MI = 120.92). Model fit for the bifactor model was slightly better than for the second-order and five-factor models. Model fit for the bifactor model was good in the community sample (CFI = .948; TLI = .934; RMSEA = .068) and was similar to that of the second-order model but lower than that of the five-factor model. Factor loadings for the bifactor model in both samples are provided in the online supplemental material (Table 6).

Sensitivity to Change (Objective 2)

Of the 1 301 clients who participated in this study, 951 completed the CCOQ and the CDDQ at posttest and 862 at 6-month follow-up (FU-6). Effect sizes (Cohen’s d; Cohen, 1988) with 95% confidence intervals were calculated to assess whether CCOQ scales: (1) changed in the expected theoretical direction during the career counseling interventions (first criterion of sensitivity to change), (2) did not change when clients were not exposed to a counseling intervention (second criterion), and (3) showed greater change in clients when compared to the CDDQ scales.

Regarding the first criterion of sensitivity, pre-post changes (between the first and last sessions) observed in the college and in the community samples were in the theoretically expected direction (undecidedness and anxiety decreased while client’s knowledge increased). Effect sizes were very large (ds ranging between −1.07 and 2.11) for all scales in the community sample. In the college sample, larges effects sizes were observed on four of the five scales (ds ranging between −.71 and 1.23) while a moderate effect size was observed for the Anxiety subscale (d = −.51).

Concerning the second criterion, changes observed during the periods when the community sample did not receive counseling (i.e. during the month preceding counseling and the 6 months following counseling) were all clinically nonsignificant (ds ranging between −.14 and .18) and did not overlap with pre-post changes. Similarly, changes observed during the period when the college sample did not receive counseling (i.e. during the six months following counseling) were also all clinically nonsignificant (ds ranging between −.11 and .12) and did not overlap with pre-post changes.

When looking at changes on the CDDQ, pre-post effect sizes were moderate or large for process (−.86 and −1.44), self (−.61 and −1.05), options (−.71 and −1.23), ways (−.65 and −.74), unreliable information (−.39 and −.41) and the CDDQ total (−.85 and −1.39) scales in the college and the community samples, respectively. Effect sizes were null or small for the lack of motivation (.05 and −.05), indecisiveness (.03 and .01), dysfunctional beliefs (−.03 and −.21), internal conflicts (−.32 and −.34), and external conflicts (.01 and .01) scales. Finally, the upper end of the changes for the CDDQ total scale did not overlap with the lower end of the changes observed for the CCOQ total scale in both samples indicating that overall, the CCOQ was significantly more sensitive to change. All effect sizes are provided in the online supplemental material (Tables 7 and 8).

Predictive (Objective 3) and Incremental Validity (Objective 4)

Hierarchical multiple regressions controlling for sample type (with college and community samples coded as zero and 1, respectively) were conducted to examine the predictive validity of the five CCOQ subscales (objective 3) and the incremental validity of the CCOQ total score (objective 4). Regarding predictive validity, Model 1, which included only sample type as predictor, was statistically significant (F = 17.09; p = .001), and explained 2% of the variance in satisfaction with decision twelve months after counseling. Model 2 indicates that adding the five CCOQ subscales scores in a second step was also statistically significant (F = 49.23; p = .001), contributing an additional 24% to the total variance explained by the model. Except for the anxiety subscale (p = .08), all subscales significantly predicted satisfaction with career decision. The regression coefficients are provided in the online supplemental material (Table 9).

Regarding incremental validity, Model 1 including sample type and CDDQ total score as predictors was statistically significant (F = 77.90; p = .001) and explained 16% of the variance in satisfaction with decision. Model 2 shows that adding the CCOQ total score in a second step was also statistically significant (F = 120.14; p = .001), increasing the total variance explained by the model by an additional 12%, thereby confirming its incremental validity. The regression coefficients for incremental validity analyses are provided in the online supplemental material (Table 10).

Additional Analyses

Further analyses were conducted in order to increase the clinical utility of the CCOQ for career counselors. A reliable change index (RCI) was computed using Jacobson and Truax (1991) procedure to allow counselors determine whether the change exhibited by a client on the CCOQ total scale is reliable. The RCI value that has been computed is 10, meaning that a client’s total score must change by at least 10 points on the CCOQ to be considered a reliable change (i.e. a change of 10 points has a 95% probability of indicating a “true” change, when accounting for measurement error from both assessments).

Additionally, to help counselors distinguish during the counseling process between clients who appear on track to make a satisfying career decision and those who are not, potential cutoff scores on the CCOQ total scale at posttest were derived using ROC curves. A dichotomous variable (satisfied or not satisfied) using client’s satisfaction with career decision scores twelve months after counseling was used in the analysis. The ROC analysis provides an index (area under the curve; AUC) of the overall accuracy of a scale. An AUC of .5 indicates that a measure is unable to discriminate between clients on track and those who are not on track, and an AUC of 1.0 is obtained when a scale allows perfect discrimination between clients on track and those who are not. The obtained AUC is 0.84 with a 95% confidence interval from 0.81 to 0.87, meaning that the developed scale has a very good ability to discriminate between clients on track to make a satisfying career decision and those who are not. Sensitivity and specificity values are provided in the online supplemental material (Table 11). For instance, a cut-off score of 72 on the CCOQ total score yields sensitivity and specificity values of .79 and .81, respectively. This means that using 72 as the cut-off score would capture 79% of clients who are likely to make a satisfying career decision twelve months after counseling and exclude 81% of clients who are at risk not to make a satisfying decision. Using higher cut-off scores would increase the questionnaire ability to detect clients who are not likely to make a satisfying career decision but would diminish the scale ability to capture clients who are likely to make a satisfying career decision. Conversely, using lower cut-off scores would increase the questionnaire ability to capture clients who are likely to make a satisfying career decision, but would diminish its ability to identify clients who are at risk not to make a satisfying decision.

Discussion

The current study aimed to develop a short career counseling outcome measure that career counselors could use to monitor the effectiveness of their interventions in natural settings.

Factor Structure

As theoretically expected, EFAs and CFAs showed that a five-factor structure corresponding to the intended scales best fitted the data in both the college and the community samples. All items had a strong primary loading on their expected factor with low cross loadings, which provide support for the content validity of the CCOQ. Omega coefficients were also high, thus supporting the internal consistency of factors. Furthermore, analyses showed that the five first-order factors could be explained by a second-order factor, thus showing that the five CCOQ subscales also measure a unitary construct and supporting the use of the total CCOQ score. Our results also suggest that the CCOQ structure could adequately be modelized using a bifactor model.

Sensitivity to Change

The CCOQ total score met both criteria for change sensitivity: it changed in the expected theoretical direction during the career counseling interventions and did not change when clients were not receiving counseling. The large pre-post effect sizes obtained for the CCOQ total score in the present study suggest that the CCOQ is highly sensitive to changes that occur during career counseling in clients and appears to accurately reflect changes that occur or should occur following participation in a career counseling intervention.

The CCOQ sensitivity to change was also supported at the subscale level. Three CCOQ subscales – Knowledge about (a) the process of decision-making, (b) the self, and (c) career information – were particularly sensitive to change. These scales share similarities with four CDDQ subscales – Lack of information about (a) the process, (b) the self, (c) career options and (d) way of obtaining additional information – which were also highly sensitive to change. However, changes were significantly greater for the CCOQ knowledge subscales in both samples when compared to their related CDDQ lack of information subscales, except for changes in process subscales and career options subscales that did slightly overlap in the community sample and the college sample, respectively.

The difference between the CCOQ and the CDDQ sensitivity to change can also be explained by the inclusion of the Undecidedness and the Anxiety subscales in the CCOQ, which were more sensitive to change than six subscales among the CDDQ. This suggests that undecidedness and anxiety toward decision-making were a main target of individual career counseling interventions. Another reason for this difference stems in the lack of change sensitivity for four CDDQ subscales (lack of motivation, indecisiveness, dysfunctional beliefs, and external conflicts). The absence of changes in these subscales converge with previous research (Masdonati et al., 2009; Milot-Lapointe et al., 2022), suggesting that their use might be more relevant for diagnostic purpose than for outcomes measurement. Indeed, as observed in previous studies, external conflicts and lack of motivation difficulties were low and thus susceptible to floor effect, which might explain why they did not change during career counseling. Moreover, since indecisiveness is conceptualized as a trait highly correlated with neuroticism (Brown & Rector, 2008), it may not be a feasible target of intervention in career counseling. Similarly, core dysfunctional career beliefs may not be a feasible target of intervention when individual career counseling is short-term (2–3 interventions) but might be in longer career counseling interventions given the small pre-post effect size (d = −.21) obtained in the community sample where interventions lasted between 4 and 8 sessions.

Predictive and Incremental Validity

While four of the five CCOQ subscales predicted satisfaction with career decision twelve months after counseling, the nonsignificant effect of anxiety might be explained by a lack of statistical power (p = .08). Given that anxiety is conceptualized as a lack of readiness in the CCOQ, it is also possible that anxiety had an indirect effect in predicting satisfaction with the decision. Although the CDDQ total score at the end of counseling was a very good predictor of satisfaction with career decision twelve months after counseling, the CCOQ total score explained additional variance. It also demonstrated a very good capacity to discriminate between clients on track to make a satisfying career decision and those who are not. Since it is difficult for career counselors to predict the long-term impact of their interventions with their clients, it is important for an outcome measure used during the counseling process to be a good predictor of positive long-term outcomes. The CCOQ ability to predict satisfaction with career decision twelve months after counseling might be explained by the fact that its theoretical background is based on two decision-making models whose long-term effects have been empirically supported – the PIC and the CIP. Based on these frameworks, our results might suggest that the CCOQ total score at the end of career counseling is an indicator of the extent to which clients are able to deal with the uncertainty of the decision-making process (anxiety subscale), possess sufficient knowledge of the process and the content dimensions of career decisions (knowledge subscale), and are confident that they are making the right decision for them (undecidedness subscale).

Contributions for Research and Practice

The CCOQ may complement existing measures by enabling researchers to evaluate the effectiveness of career counseling at both individual and group levels. Its sensitivity to change could help researchers provide robust evidence of the effectiveness of individual career counseling. Career counselors could use the CCOQ to monitor the effectiveness of their interventions in complement to diagnostic measures such as the CDDQ. They might use the CCOQ subscales to determine certain goals of career counseling sessions. For example, a given counseling session might aim to increase a client’s self-knowledge if their score is low on this subscale. Counselors might use the CCOQ total score to evaluate if their clients are making meaningful progress using the RCI, and adjust their interventions if clients are not progressing as expected. A change larger than 10 on the CCOQ total scale would indicate that the client has made meaningful progress during the process. Counselors might also use cutoff scores as indicators of whether or not clients are on track to make a satisfying career decision. Counselors might increase the duration of counseling if a given client does not seem ready to commit in a decision regarding their CCOQ total score.

Limitations and Suggestions for Future Research

The results of this study should be interpreted with its limitations in mind. Given that there was a minority of adolescents in our total sample, future research should assess psychometric properties of the CCOQ on clients in the early stages of their career development. Another limitation of this study is that while we were able to demonstrate the stability of the CCOQ during the periods preceding and following counseling for the community sample, we were not able to evaluate its stability in the period preceding counseling for the college sample. Although it is unlikely that college students would have experienced large effect sizes during the month preceding an appointment with a career counselor (like it was the case during counseling), future research would benefit from including this time assessment to estimate if changes occur during this period. Additionally, while follow-up assessments conducted twelve months after counseling provide valuable insights regarding client satisfaction with career decision, it is important to examine the predictive validity of the CCOQ over a more extended period (e.g., several years; Milot-Lapointe & Le Corff, 2024b), as career dissatisfaction with career decisions may emerge later, after transitioning into a new program or job. Finally, to ensure clinical utility of the CCOQ in natural settings, future research should assess if effects of career counseling on the CCOQ scales are larger when counselors are provided with feedback regarding their client progress on this scale compared to when they do not benefit from this feedback. To achieve this, researchers could employ longitudinal designs, such as pre-post designs or time-series analyses.

Supplemental Material

Supplemental Material - Development and Validation of the Career Counseling Outcome Questionnaire in Two Clinical Settings

Supplemental Material for Development and Validation of the Career Counseling Outcome Questionnaire in Two Clinical Settings by Francis Milot-Lapointe and Yann Le Corff in Journal of Career Assessment

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Social Sciences and Humanities Research Council of Canada.

ORCID iDs

Francis Milot-Lapointe

Yann Le Corff

Supplemental Material

Supplemental material for this article is available online.

References

Brown

Ryan Krane

N. E.

(2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In Brown

Lent

R. W.

(Eds.), Handbook of counseling psychology (3rd ed., pp. 740–766). Wiley.

Brown

S. D.

Rector

C. C.

(2008). Conceptualizing and diagnosing problems in vocational decision making. In Brown

S. D.

Lent

R. W.

(Eds.), Handbook of counseling psychology (4th ed., pp. 392–407). John Wiley & Sons, Inc.

Clark

D. A.

Beck

A. T.

(2011). Cognitive therapy of anxiety disorders: Science and practice. Guilford Press.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

Covali

Bernaud

J.-L.

Di Fabio

(2011). Processus de changement et variabilité individuelle au cours des consultations d’orientation. Swiss Journal of Psychology, 70(3), 175–183. https://doi.org/10.1024/1421-0185/a000053

Elad-Strenger

Littman-Ovadia

(2012). The contribution of the counselor-client working alliance to career exploration. Journal of Career Assessment, 20(2), 140–153. https://doi.org/10.1177/1069072711420850

Gati

Asher

(2001). Prescreening, in-depth exploration, and choice: From decision theory to career counseling practice. The Career Development Quarterly, 50(2), 140–157. https://doi.org/10.1002/j.2161-0045.2001.tb00979.x

Gati

Gadassi

Shemesh

(2006). The predictive validity of a computer-assisted career decision-making system: A six-year follow-up. Journal of Vocational Behavior, 68(2), 205–219. https://doi.org/10.1016/j.jvb.2005.08.002

Gati

Krausz

Osipow

S. H.

(1996). A taxonomy of difficulties in career decision making. Journal of Counseling Psychology, 43(4), 510–526. https://doi.org/10.1037/0022-0167.43.4.510

10.

Gati

Kulcsár

(2021). Making better career decisions: From challenges to opportunities. Journal of Vocational Behavior, 126, Article 103545. https://doi.org/10.1016/j.jvb.2021.103545

11.

Gati

Osipow

S. H.

Krausz

Saka

(2000). Validity of the career decision-making difficulties questionnaire: Counselee versus career counselor perceptions. Journal of Vocational Behavior, 56(1), 99–113. https://doi.org/10.1006/jvbe.1999.1710

12.

Hill

C. E.

Lambert

M. J.

(2004). Methodological issues in studying psychotherapy processes and outcomes. In Lambert

M. J.

(Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (pp. 87–135). John Wiley & Sons.

13.

Horn

J. L.

(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185. https://doi.org/10.1007/BF02289447

14.

L. T.

Bentler

P. M.

(1995). Evaluating model fit. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues and application (pp. 77–99). Sage.

15.

Jacobson

N. S.

Truax

(1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. https://doi.org/10.1037/0022-006X.59.1.12

16.

Kleiman

Gati

Peterson

Sampson

Reardon

Lenz

(2004). Dysfunctional thinking and difficulties in career decision making. Journal of Career Assessment, 12(3), 312–331. https://doi.org/10.1177/1069072704266673

17.

Kline

R. B.

(2016). Principles and practice of structural equation modeling (4th ed.). Guilford Press.

18.

Levin

Lipshits-Braziler

Gati

(2022). The identification and validation of five types of career indecision: A latent profile analysis of career decision-making difficulties. Journal of Counseling Psychology, 69(4), 452–462. https://doi.org/10.1037/cou0000603

19.

Marsh

H. W.

Hau

K.-T.

Grayson

(2005). Goodness of fit evaluation. In Maydeu-Olivares

McArdle

(Eds.), Contemporary psychometrics (pp. 275–340). Lawrence Erlbaum Associates Publishers.

20.

Masdonati

Massoudi

Rossier

(2009). Effectiveness of career counseling and the impact of the working alliance. Journal of Career Development, 36(2), 183–203. https://doi.org/10.1177/0894845309340798

21.

Masdonati

Perdrix

Massoudi

Rossier

(2014). Working alliance as a moderator and a mediator of career counseling effectiveness. Journal of Career Assessment, 22(1), 3–17. https://doi.org/10.1177/1069072713487489

22.

Massoudi

Masdonati

Clot-Siegrist

Franz

Rossier

(2008). Évaluation des effets du counseling d'orientation: influence de l'alliance de travail et des caractéristiques individuelles. Pratiques Psychologiques, 14(2), 117–136. https://doi.org/10.1016/j.prps.2007.11.010

23.

McDonald

R. P.

(1999). Test theory: A unified treatment. Lawrence Erlbaum Associates Publishers. https://doi.org/10.1207/S15327841Mpee0402_7

24.

Milot-Lapointe

Le Corff

(2023). Trajectories of change in career decision difficulties during a manualized individual career counseling intervention: The influence of counselor adherence, working alliance and client personality traits. Journal of Career Assessment, 31(3), 607–628. https://doi.org/10.1177/10690727221141983

25.

Milot-Lapointe

Le Corff

(2024a). Predicting outcomes of a manualized individual career counseling intervention over a one-year follow-up from trajectories of change in career decision difficulties. Journal of Career Assessment, 32(4), 745–759. https://doi.org/10.1177/10690727241232438

26.

Milot-Lapointe

Le Corff

(2024b). Longitudinal effects of two individual career counseling interventions on clients’ career decision-making during the covid-19 pandemic over a 1-year follow-up. Journal of Employment Counseling, 61(1), 46–58. https://doi.org/10.1002/joec.12215

27.

Milot-Lapointe

Le Corff

Savard

(2019). A study of clinical change in individual career counseling. The Career Development Quarterly, 67(4), 357–364. https://doi.org/10.1002/cdq.12204

28.

Milot-Lapointe

Le Corff

Savard

(2022). Processes and outcomes of individual career counselling for different types of clients: A latent profile analysis replication and extension. British Journal of Guidance and Counselling, 50(5), 677–686. https://doi.org/10.1080/03069885.2021.1959518

29.

Milot-Lapointe

Savard

Le Corff

(2016). Effet du counseling de carrière sur la santé mentale: utilisation d’une méthode du changement clinique. Revue Canadienne de Counseling et de Psychothérapie, 50(3), 278–289.

30.

Milot-Lapointe

Savard

Le Corff

(2018). Intervention components and working alliance as predictors of individual career counseling effect on career decision-making difficulties. Journal of Vocational Behavior, 107, 15–24. https://doi.org/10.1016/j.jvb.2018.03.001

31.

Milot-Lapointe

Savard

Le Corff

(2020). Effect of individual career counseling on psychological distress: Impact of career intervention components, working alliance, and career indecision. International Journal for Educational and Vocational Guidance, 20(2), 243–262. https://doi.org/10.1007/s10775-019-09402-6

32.

Multon

K. D.

Heppner

M. J.

Gysbers

N. C.

Zook

Ellis-Kalton

C. A.

(2001). Client psychological distress: An important factor in career counseling. The Career Development Quarterly, 49(4), 324–335. https://doi.org/10.1002/j.2161-0045.2001.tb00960.x

33.

Multon

K. D.

Wood

Heppner

M. J.

Gysbers

N. C.

(2007). A cluster-analytic investigation of subtypes of adult career counseling clients: Toward a taxonomy of career problems. Journal of Career Assessment, 15(1), 66–86. https://doi.org/10.1177/1069072706294508

34.

Oliver

L. W.

Spokane

A. R.

(1988). Career-intervention outcome: What contributes to client gain? Journal of Counseling Psychology, 35(4), 447–462. https://doi.org/10.1037/0022-0167.35.4.447

35.

Rochat

(2019). The career decision-making difficulties questionnaire: A case for item-level interpretation. The Career Development Quarterly, 67(3), 205–219. https://doi.org/10.1002/cdq.12191

36.

Rossier

Rochat

Sovet

Bernaud

J.-L.

(2022). Validation of a French version of the career decision-making difficulties questionnaire: Relationships with self-esteem and self-efficacy. Journal of Career Development, 49(4), 906–921. https://doi.org/10.1177/08948453211009975

37.

Rottinghaus

P. J.

Eshelman

Gore

J. S.

Keller

K. J.

Schneider

Harris

K. L.

(2017). Measuring change in career counseling: Validation of the career futures inventory-revised. International Journal for Educational and Vocational Guidance, 17(1), 61–75. https://doi.org/10.1007/s10775-016-9329-7

38.

Ryan Krane

N. E.

(1999). Career counseling and career choice goal attainment: A meta-analytically derived model for career counseling practice. Doctoral dissertation, ProQuest Dissertation and Theses. Loyola University, Chicago. (Publication No 9930585).

39.

Sampson

J. P.

Jr. Peterson

G. W.

Lenz

J. G.

Reardon

R. C.

Saunders

D. E.

(1998). The design and use of a measure of dysfunctional career thoughts among adults, college students, and high school students: The Career Thoughts Inventory. Journal of Career Assessment, 6(2), 115–134. https://doi.org/10.1177/106907279800600201

40.

Sampson

J. P.

Lenz

J. G.

Reardon

R. C.

Peterson

G. W.

(1999). A cognitive information processing approach to employment problem solving and decision making. The Career Development Quarterly, 48(1), 3–18. https://doi.org/10.1002/j.2161-0045.1999.tb00271.x

41.

Shimokawa

Lambert

M. J.

Smart

D. W.

(2010). Enhancing treatment outcome of patients at risk of treatment failure: meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298–311. https://doi.org/10.1037/a0019247

42.

Simms

L. J.

Watson

(2007). The construct validation approach to personality scale construction. In Robins

R. W.

Fraley

R. C.

Krueger

R. F.

(Eds.), Handbook of research methods in personality psychology (pp. 240–256). Guilford.

43.

Spokane

A. R.

Oliver

L. W.

(1983). The outcomes of vocational intervention. In Walsh

W. B.

Osipow

S. H.

(Eds.), Handbook of vocational psychology (pp. 99–116). Erlbaum.

44.

Tabachnick

B. G.

Fidell

L. S.

(2019). Using multivariate statistics (7th ed.). Pearson.

45.

Toggweiler

Künzli

(2020). The change-inventory for career counseling - an instrument for measuring counseling-correlated changes. Frontiers in Psychology, 11, 1916. https://doi.org/10.3389/fpsyg.2020.01916

46.

Toussaint

Hüsing

Gumz

Wingenfeld

Härter

Schramm

Löwe

(2020). Sensitivity to change and minimal clinically important difference of the 7-item Generalized Anxiety Disorder Questionnaire (GAD-7). Journal of Affective Disorders, 265, 395–401. https://doi.org/10.1016/j.jad.2020.01.032

47.

Tryon

W. W.

(1991). Activity measurement in psychology and medicine. Plenum Press. https://doi.org/10.1007/978-1-4757-9003-0

48.

Verbruggen

Chambers

C. D.

Logan

G. D.

(2013). Fictitious inhibitory differences: How skewness and slowing distort the estimation of stopping latencies. Psychological Science, 24(3), 352–362. https://doi.org/10.1177/0956797612457390

49.

Vermeersch

D. A.

Lambert

M. J.

Burlingame

G. M.

(2000). Outcome questionnaire: Item sensitivity to change. Journal of Personality Assessment, 74(2), 242–261. https://doi.org/10.1207/S15327752JPA7402_6

50.

Vermeersch

D. A.

Whipple

J. L.

Lambert

M. J.

Hawkins

E. J.

Burchfield

C. M.

Okiishi

J. C.

(2004). Outcome questionnaire: Is it sensitive to changes in counseling center clients? Journal of Counseling Psychology, 51(1), 38–49. https://doi.org/10.1037/0022-0167.51.1.38

51.

Whiston

S. C.

(2001). Selecting career outcome assessments: An organizational scheme. Journal of Career Assessment, 9(3), 215–228. https://doi.org/10.1177/106907270100900301

52.

Whiston

S. C.

Goodrich Mitts

Wright

(2017). Effectiveness of career choice interventions: A meta-analytic replication and extension. Journal of Vocational Behavior, 100, 175–184. https://doi.org/10.1016/j.jvb.2017.03.010

53.

Whiston

S. C.

Rahardja

(2008). Vocational counseling process and outcome. In Brown

S. D.

Lent

R. W.

(Eds.), Handbook of counseling psychology (4th ed., pp. 444–461). Wiley.

54.

Whiston

S. C.

Sexton

T. L.

Lasoff

D. L.

(1998). Career-intervention outcome: A replication and extension of Oliver and Spokane (1988). Journal of Counseling Psychology, 45(2), 150–165. https://doi.org/10.1037/0022-0167.45.2.150

55.

Bhang

C. H.

(2019). The structure and measurement of career indecision: A critical review. The Career Development Quarterly, 67(1), 2–20. https://doi.org/10.1002/cdq.12159

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.65 MB