Comparison of the Effectiveness of Two Assessment Feedback Models in Reducing Career Indecision

Abstract

We examined the relative effectiveness of information giving (IG) and therapeutic assessment (TA) models of career assessment feedback in reducing career indecision. Clients initiating career counseling at a large Midwestern University completed measures of career choice anxiety (CCA), vocational identity, and career decision-making self-efficacy prior to and following a two-session intervention. We also administered measures of treatment integrity and session helpfulness immediately following treatment. Clients were randomly assigned to an intervention based upon either an IG or TA model of assessment feedback. Results indicated that TA participants’ vocational identity scores improved significantly more than those of IG participants; a medium-to-large effect size was identified. TA and IG participants’ CCA and decision-making self-efficacy scores significantly increased but not differentially following treatment. Participants of both groups rated their feedback sessions as “greatly helpful.” We discuss implications for career assessment as well as the limitations of the study.

Keywords

therapeutic assessment career indecision vocational identity career intervention assessment feedback

According to recent U.S. Department of Education statistics, more than 18 million students attended degree granting postsecondary institutions in 2009 (Aud et al., 2011). It is well established that many students experience college major and career indecision (Gordon, 2007). Indecision can be reflected by self-reports and changes of declared majors. Regarding the latter, Gordon (2007) reported that as many as 75% of college students change their undergraduate major at least once. Further, approximately 7% of college-bound students who took the SAT in 2009–2010 reported that they were undecided about their college major (College Entrance Examination Board, 2010). The American College Testing (ACT) reported that an even larger number—14% of college-bound students—reported indecision about their college major (ACT, 2009). Research has demonstrated that indecision has negative effects on college student persistence, academic performance, and timely degree completion (Allen & Robbins, 2010; Tracey & Robbins, 2006).

Because college major and career indecision are commonly experienced and have negative consequences, higher education institutions have invested significantly in student support services. The U.S. Department of Education reported that 2- and 4-year postsecondary institutions invest approximately 4.7% of their budgets (Aud et al., 2011) in student services, including career counseling. The U.S. higher education system has placed a high priority on efficient college major decision making and expeditious degree completion and career entry. Therefore, those who provide support services to higher education students have much to gain by developing and implementing more effective career counseling interventions.

Career Counseling Outcome Research

Outcome research has demonstrated that career interventions are at least moderately effective. Four significant meta-analyses have reported average overall effect sizes ranging from .34 to .82 (Oliver & Spokane, 1988; Ryan, 1999; Whiston, Brecheisen, & Stephens, 2003; Whiston, Sexton, & Lasoff, 1998). These effect sizes indicate that the average career counseling participant experiences more positive outcomes than 63–79% of control group participants.

Three implications can be drawn from these meta-analyses. First, the weight of evidence indicates the necessity for counselor involvement in career counseling. Both Whiston, Sexton, and Lasoff (1998) and Whiston, Brecheisen, and Stephens (2003) found that counselor-free intervention is the least effective modality. Second, individual counseling and structured workshop/classes appear to be the most effective career counseling modalities (Oliver & Spokane, 1988; Whiston et al., 1998, 2003). Third, Ryan (1999) identified five critical ingredients of effective career interventions: individualized interpretations and feedback, providing world-of-work information, modeling effective career decision-making processes, engaging clients in written exercises, and choice-related support. Accordingly, these treatment components should be the focus of career counseling outcome studies.

Although the aforementioned meta-analyses have produced valuable information that informs practice, there has been a general decline in the volume of career counseling outcome research (Whiston et al., 2003). More recently, Scheel et al. (2011) identified a decrease in career counseling research studies in the top primary counseling psychology publications over the past 30 years. This decrease can be juxtaposed against the multiple calls of researchers for more critical examination of career counseling interventions (e.g., Bernes, Bardick, & Orr, 2007; Heppner & Heppner, 2003; Whiston, 2011). Whiston et al. (2003) specifically highlighted the lack of research comparing career assessment feedback methods. The lack of comparative research regarding assessment feedback is particularly problematic because of the central role of interest assessment in career counseling (Hansen, 2005) and the increase in Internet-mediated career assessment tools available directly to students and counselors (e.g., Tracey, 2010). Three points should be considered. First, interest assessment will remain an essential component of career counseling. Second, counselor-mediated interventions provide superior outcomes. Third, the costs of counseling personnel are increasing in a higher education budget with limited funding. Therefore, it is important for researchers to continue exploring the most effective methods of counselor involvement in career assessment feedback.

Therapeutic Assessment

Claiborn, Goodyear, and Horner (2001) reviewed research on assessment feedback methods and provided the following recommendations. They suggested that assessment feedback is more effective when it (a) involves a collaborative therapeutic relationship, (b) is conducted in a structured environment, (c) involves discussion of feedback comprehension, and (d) follows goal or question setting. Therapeutic assessment (TA; Finn & Tonsager, 2002) is a promising approach to assessment feedback that incorporates these components.

Finn and Tonsager (2002) defined TA as “a semi-structured form of collaborative assessment that uses psychological testing as the centerpiece of brief therapy” (p. 10). The TA process is based on three defining principles. The first principle, self-verification, was derived from Kohut’s (1977) theory of self-psychology that stipulates that many problems of the self, particularly anxiety, result from failed integration of self-view with the view mirrored by one’s environment. A primary counseling focus is to enable clients to attain an accurate and cohesive view of self (Patton, Connor, & Scott, 1982). In TA, self-verification occurs by providing clients with feedback that closely mirrors their self-views. Subsequently, new information is provided to facilitate the formation of a more cohesive and encompassing sense of self.

The second primary principle of TA is self-enhancement, which is derived from social cognitive and self-psychology theories (Finn & Tonsager, 1997). According to Fiske and Taylor (2008), people are motivated to be positively viewed by themselves and others. Combining self-enhancement with the principle of self-verification, clients in TA are given information initially determined to closely match their self-view and then presented with new and potentially discrepant information, which results in an enhanced sense of self for clients. To further self-enhancement, TA incorporates the humanistic principles of respect, empathy, and positive regard of client qualities. With a more cohesive sense of self, clients are able to consider methods for achieving assessment-directed goals.

The third principle of TA is self-efficacy/self-discovery, which is rooted in Bandura’s social cognitive theory. Bandura (1993) referenced the basic human need for mastery and control over one’s environment. He postulated that self-efficacy is developed through personal mastery, vicarious experience, social persuasion, or analysis of physiological state. In gaining mastery and environmental control, people gain increased efficacy to perform a particular set of actions. In TA, clients are actively engaged to form personal goals for assessment and develop personally relevant questions for exploration. Self-efficacy/self-discovery is fostered as clients collaborate in discovering assessment information relevant to their questions and goals (Finn & Tonsager, 1997). The TA process serves to verify a client’s experience, positively reframe and enhance negative self-evaluations, and facilitate client self-discovery.

TA shares some common features with the traditional method of assessment feedback, which we refer to as the information giving (IG) model. The central purpose of the IG approach is for the assessor to provide clinically relevant information to clients to guide decision making and action planning. The IG approach, which has been labeled test-and-tell or delivered assessment, frequently has been employed to provide psychological and career assessment feedback (e.g., Cutler, 2004; Finn & Tonsager, 1997; Goodyear, 1990; Hanson, Claiborn, & Kerr, 1997).

The IG approach has been characterized as “interpreted to the client at a particular fixed point (e.g., during the second interview) in the counseling process; often this is in a ‘data dump’ fashion and with the counselor maintaining responsibility for interpretations and inferences” (Goodyear, 1990, p. 252). Cutler (2004) described the IG approach as consisting of data gathering, interpretation, and client feedback steps. The primary distinction between the two approaches is that IG is test and therapist centered and TA is client and relationship centered. By shifting the focus from the test to the client, TA feedback honors the reality that “how we interpret an event greatly influences how we respond to it” (Finn, 1999, p. 1) The TA model represents small but significant modifications to the IG model (Finn & Tonsager, 1997). These minor adjustments retain validity and objectivity while enhancing the impact of assessment feedback for the client.

The Present Study

We conducted this study to compare TA and IG methods of career assessment feedback. In conducting this study, we incorporated four research design recommendations from previous investigators, including (a) direct counselor involvement in career assessment feedback, (b) intentionality of study outcome measurement selection, (c) inclusion of treatment integrity checks, and (d) use of real career counseling clients. We hypothesized that TA would yield significantly greater reduction of career indecision than the traditional IG approach. We had three general expectations regarding the incremental benefits of TA over the IG approach reflecting the TA tenets of self-verification, self-efficacy, and self-discovery. First, TA should be more powerful than IG in strengthening vocational identity. Second, TA should yield higher career decision-making efficacy (CDSE). Third, TA should decrease CCA.

We had five hypotheses regarding the relative positive effects of TA in comparison to IG immediately following assessment feedback:

TA condition participants will demonstrate a significantly greater increase in CDSE.

TA participants will report a significantly greater increase in vocational identity.

TA participants will indicate a significantly greater decrease in CCA.

TA participants will rate their assessment feedback session as significantly more helpful.

TA and IG participants together will demonstrate significantly higher CDSE and vocational identity and lower CCA following completion of the respective assessment processes.

Method

Participants

Clients

Twenty-three participants, including 18 men and 5 women, completed the study and outcome measures in full. Participants were actual clients seeking career counseling services at a large Midwestern university counseling center. An additional 22 participants began but did not complete the study due to scheduling problems, choosing not to complete the career assessment or failing to return for their feedback sessions. Sixteen participants reported ethnicity, with 14 identifying as Caucasian/White, 1 as African, and 1 as Asian American. The age range, reported by 20 of the 23 participants, was 17–27 with a mean age of 20.54. Of the 17 students who reported their academic status, 6 were first-year, 6 were second-year, 2 were third-year, and 1 was a fourth-year student. The sample also included one graduate student and one alumni/community member. Note that not all participants reported demographic information.

Counselors

Seven members of the counseling center staff were recruited and trained as counselors for this study. Due to scheduling difficulties, only four counselors had participants who completed the study in full. These four counselors were either currently completing their American Psychological Association (APA)-accredited doctoral internship in psychology or had completed their doctoral degree in counseling or clinical psychology. All four counselors had familiarity and experience conducting career assessment and feedback prior to the study. Each counselor completed a training session covering study protocols. Following protocol training, all counselors conducted assessment sessions for both the TA and IG treatment conditions. Table 1 shows relevant counselor demographics of the four counselors participating in the study.

Table 1.

Counselor Demographics.

Counselor number	Age	Ethnicity	Sex	Degree and number of postgraduate years	Number of participants
1	68	European American	Male	EdD, counseling psychology, 38	18
2	31	European American	Female	PsyD, clinical psychology, 1	1
3	32	European American	Male	PhD, counseling psychology, 1.5	3
4	38	European American	Male	Predoctoral intern, counseling psychology	1

Measures

Career Factors Inventory (CFI; Chartrand & Robbins, 1997)

The CFI has a total of 21 items with a 5-point Likert-type response scale; the CFI is intended to measure career needs and potential barriers to decision making. There are four CFI scales: Need for Career Information, Need for Self-Knowledge, CCA, and Generalized Indecisiveness (Chartrand, Robbins, Morrill, & Boggs, 1990). Chartrand et al. (1990) reported respective CFI scale score internal reliability coefficients of .86, .79, .73, and .83 with a total scale reliability of .87. These results indicate sufficient to good internal consistency. The authors also were able to use CFI scores to distinguish between students high and low in academic major choice decidedness. Chartrand, Robbins, Morrill, and Boggs (1990) and Lewis and Savickas (1995) demonstrated that CFI scale scores are significantly correlated with other previously validated, theoretically similar instruments. The CCA scale was designed to measure anxiety experienced in relation to the career decision-making process. Research indicates that the CCA scale is related to similar constructs such as trait anxiety. We used the CCA scale in this study in consideration of the TA tenet of self-verification because anxiety around the decision-making process should be lower for clients with greater self-verification. We calculated pretest and posttest internal reliabilities of .890 and .802, respectively, for the CCA scores of the study sample.

Career Decision-Making Self-Efficacy Scale–Short Form (CDSE-SF; Betz & Taylor, 2001)

Taylor and Betz (1983) developed the CDSE to apply Bandura’s (1977) self-efficacy theory to career decision making, with the specific aim of measuring self-efficacy related to Crites’s (1978) five career choice competencies. Consequently, CDSE reflects confidence in one’s ability to complete five tasks theoretically related to career decision making, including: self-appraisal, occupational information, goal selection, planning, and problem solving (Nilsson, Schmidt, & Meek, 2002). The CDSE-SF has a total of 25 items and the same five scales as the original. Each item is scored on a 10-point Likert-type scale ranging from 1 (no confidence at all) to 10 (complete confidence). In a review of 14 studies employing the CDSE-SF, Nilsson, Schmidt, and Meek (2002) reported scale score reliabilities to range from .71 to .82, .78 to .82, .83, .77 to .83, and .69 to .75, respectively. The mean total reliability coefficient was .94. Nilsson et al. found no statistical differences in the reliability properties of the CDSE and CDSE-SF.

Concurrent validity has been found between CDSE-SF scores and other career indecision measures. Betz, Klein, and Taylor (1996) found positive associations between the total CDSE-SF scores and positive career decision-making outcomes such as higher decision-making certainty and vocational identity and lower career indecision. We used the CDSE-SF total score in consideration of the TA tenets of self-efficacy/self-discovery and self-enhancement. We calculated pretest and posttest internal reliabilities of .902 and .913, respectively, for the CDSE-SF scores of the study sample.

My Vocational Situation (MVS; Holland, Daiger, & Power, 1980)

The MVS is a 26-item measure intended to assess vocational identity, need for career information, and career decision-making barriers. It consists of three separate scales including the 18-item Vocational Identity scale (VIS), 4-item Occupational Information, and 4-item Barriers scales. An additional free response question, which precedes the three primary scales, asks the respondent to “List all the occupations you are considering right now.” Using scores from a college student sample, Holland, Gottfredson, and Power (1980) reported internal consistency reliability estimates for the three MVS scales of .89, .79, and .45, respectively, for men; the estimates were .88, .77, and .65, respectively, for women. The VIS items are negatively scored and stated as true–false responses with higher scores indicating a clearer vocational identity (Holland, Johnston, & Asama, 1993). Example items include I don’t know what my major strengths are and I am uncertain about which occupation I would enjoy.

Holland et al. (1993) summarized the MVS research and concluded that VIS scores are positively correlated with measures of career development, decision making, ego identity, conscientiousness, choice confidence, hope, and self-esteem. They indicated that VIS scale scores were negatively correlated with measures of undecided beliefs, neuroticism, dependent decision-making style, CCA, indecisiveness, and need for career information and self-knowledge. We used the VIS in consideration of the TA tenet of self-verification. Note that the VIS was not negatively scored because the purpose of this study was to detect pre/post change. Thus, lower VIS scores in our results reflect clearer vocational identity. We calculated pretest and posttest internal reliabilities of .838 and .702, respectively, for the VIS scores of the study sample.

Counselor Rating Form–Short Form (CRF-SF; Corrigan & Schmidt, 1983)

The CRF-SF, a 12-item version of the full CRF, is used to assess client perceptions of counselor attractiveness, expertness, and trustworthiness. Each scale consists of positive adjective items rated on a 7-point scale from not very to very. Higher scores reflect more positive perceptions of the counselor. Epperson and Pecnik (1985) reported internal consistency coefficients of .88, .86, and .88, respectively, with an average value of .87.

Several researchers have reported confirmation of a three-factor structure for the CRF-SF (Corrigan & Schmidt, 1983; Epperson & Pecnik 1985; Tracey, Glidden, & Kokotovic, 1988). Tracey, Glidden, and Kokotovic (1988) also reported a global construct of overall counselor positivity perception. Corrigan and Schmidt (1983) reported high intercorrelations between the three scales’ scores ranging from .381 to .900. Further evidence of validity was provided by Kokotovic and Tracey (1987), who found that client CRF-SF scores on the trustworthiness and expertness scales were significantly related to client dropout and continuation behavior. In consideration of the high correlations among the three theoretical constructs and evidence of a global counselor positivity measure, we used the total counselor rating and three separate CRF-SF scales in this study. Using the current study data, we calculated internal reliabilities of .957 for the total scale scores, .945 for expertness, .926 for attractiveness, and .884 for the trustworthiness scale.

Assessment

We used the Strong Interest Inventory (SII; Donnay, Morris, Schaubhut, & Thompson, 2005) to provide clients with feedback. The SII is a career interest assessment tool widely used in both counseling practice and research; it has undergone several revisions. The current SII consists of 291 items that yield four sets of subscale scores. All scale scores are presented as standard scores. The General Occupational Themes (GOT) measure preferences within each of the Holland six interest types (i.e., realistic, investigative, artistic, social, enterprising, and conventional). The Basic Interest Scales (BIS) measure interest in 30 types of activities organized by primary interest type (e.g., performing arts and culinary arts within the artistic theme; counseling and helping and teaching and education with the social theme, sales and management within the enterprising theme). The Occupational Scales (OS) show the respondent’s similarity to groups of satisfied workers of their sex in 244 different occupations. Occupations are organized within each of the six interest types according to the occupation’s primary interest theme. Realistic occupations include electrician, forester, and radiologic technologist. Investigative occupations include dietician, pharmacist, and mathematician. The five dichotomous Personal Style Scales (PSS) reflect preferences for work style (working alone vs. with people), learning environment (practical vs. academic), leadership style (level of comfort), risk taking, and team orientation (alone vs. group). The GOT, BIS, and OS items are rated on a 5-point scale ranging from strongly like to strongly dislike.

The following psychometric data are primarily based on the 2004 revision of the SII. It is significant to note that correlations between scores on the 1994 and 2004 versions of the SII have been reported to range from .90 to .98 for the GOTs and from .80 to .98 between parallel BISs. The internal consistency coefficients for the Realistic, Investigative, Artistic, Social, Enterprising, and Conventional (RIASEC) GOT scale scores were all .91 or greater (Donnay et al., 2005). The BIS scores had a mean internal consistency coefficient of .87. The median test–retest reliability for the OS scores was .86. The internal consistency coefficients for the PSS scores were in the range of .82 to .87. A significant amount of research has demonstrated the concurrent, construct, and discriminant validity of the SII scales (Donnay et al., 2005; Gasser, Larson, & Borgen, 2007).

Treatment Checks and Perception

We used treatment validity check and session helpfulness items similar to those developed by Hanson and Claiborn (2006). The session helpfulness question was modified to a 9-point scale following Elliott’s (1985) rating method. To assess treatment perception each participant responded to the following questions with 5-point response scales after the assessment feedback session: “To what extent was the testing process personalized by … ” (a) “having you develop questions, prior to taking the test, about yourself you wanted answered?” and (b) “being involved actively in the interpretation of the test results?” They then responded to the helpfulness question with a 9-point scale “How helpful or hindering was your test feedback session?” Higher scores indicated greater perceived session involvement and helpfulness.

Procedure

Counselor Training

The recruited counselors were trained in two 2-hr sessions consisting of didactic and role-play learning. The first author conducted the training sessions. The counselors were asked to review the SII manual and the TA and IG protocols developed for the study. Each counselor was asked to memorize both protocols and practice mock sessions, one of which was taped, using each protocol until judged competent by the trainer. Competence was determined based on counselor’s successful adherence to a protocol checklist. At the request of the counselors, we developed a quick reference sheet for the protocols for in-session use. As an additional check, two independent undergraduate students rated the taped mock session of each counselor with the CRF-SF. The CRF-SF scores were compared using an analysis of variance (ANOVA) to determine if the counselors were perceived similarly in terms of their expertness, attractiveness, and trustworthiness.

Test Interpretation and Feedback

Our test interpretation and feedback procedures were similar to those reported by Cutler (2004) in her comparison of TA and IG approaches. Of note, Cutler’s feedback procedures and protocol were adapted from assessment feedback protocols by Finn (1996), Hanson (2001), and Pawlowski (2002). The IG feedback protocol used in this study additionally incorporated the recommended interpretation format detailed in the 1994 SII manual (Harmon, Hansen, Borgen, & Hammer, 1994). The TA protocol was modified to include elements consistent with the principles of self-verification, self-enhancement, and self-efficacy/self-discovery.

Participation was invited by the counseling center reception staff as clients checked in for their intake sessions. Those clients who agreed to participate completed the CFI, MVS, and CDSE-SF pretest measures prior to their intake session with a counselor. Participants then completed their intake session according to the assigned feedback protocol. Each participating counselor alternated between TA and IG protocols for each successive participant. The participants then were scheduled to complete the SII following their intake sessions.

In the first intervention session, participants met with their assigned counselor prior to taking the SII for a 45-min session that was used to either (a) gather information only about the participant’s career concerns (IG condition) or (b) gather information and establish collaborative questions and goals (TA condition). For the second and final intervention session, participants received SII feedback according to their assigned treatment condition. At the conclusion of the second session, participants completed the posttest measures of the CFI, MVS, and CDSE-SF and the treatment check and reaction questions. Career counseling continued at the discretion of the counselor and client following study completion.

Results

Data Screening and Preanalysis Results

All outcome variables were screened for normality and outliers following procedures outlined by Tabachnik and Fidell (2001). To evaluate any potential covariate effects, difference scores were calculated between participants’ pre- and posttreatment scores on the three dependent measures. Correlations were then evaluated between the difference scores and participant age, sex, and academic status. No correlations were significant, indicating that difference scores were not significantly related to demographic characteristics. We also calculated Pearson correlations between participant difference and pretest scores and all three were statistically significant. Three separate ANOVAs were calculated with treatment group as the independent variable and pretest scores as the respective dependent variables. None of the ANOVAs were significant, indicating that treatment group did not affect the dependent measures prior to treatment. In consideration of the significant correlation between participant difference and pretest scores, we decided to use one-way analysis of covariances (ANCOVAs) with pretest scores as a covariate, posttest scores as the dependent variable, and treatment group as the independent variable for Hypotheses 1, 2, and 3.

A total of 45 participants initially completed the pretest measures. Of those 45 participants, 22 failed to complete the SII or return for a feedback session. Twelve participants who did not complete the study either reported some demographics or completed a portion of the pretest measures. Of the 12, 7 were women and 5 were men. There were five first-year, two second-year, two third-year, and two fourth-year students. Their average age was 23.18 with a range of 18–42. Nine reported Caucasian/White as their ethnicity. We conducted three ANOVAs using completion status (completed vs. failure to complete) as the independent variable and pretest scores as the dependent variable. There were no significant effects for completion status. We compared other characteristics of the completed and dropout groups and could not identify any characteristics distinguishing the two groups. Ultimately, 23 participants completed both sessions and the pre- and post measures with one participant failing to complete the pretest CFI and VIS measures and one omitting the posttest CFI measure.

Counselor Check

Two undergraduate students blind to the hypotheses viewed 15-min segments of each counselor’s taped mock session and then completed the CRF-SF for each counselor. We conducted four ANOVAs with counselor as the independent variable and CRF-SF total, expertness, attractiveness, and trustworthiness scores as the dependent variables. The analyses did not reflect a significant counselor effects for CRF-SF total, expertness, attractiveness, or trustworthiness. Therefore, we concluded that the four counselors were perceived as comparably expert, trustworthy, and attractive by the participants. Table 2 below displays relevant F statistics and p values for each CRF-SF dependent variable.

Table 2.

Counselor CRF-SF Scores Analysis of Variance (ANOVA) Results.

CRF-SF dependent variable	F statistic (N = 4)	p	η²	Observed power
Total score	.028	.993	.020	.052
Expertness	.500	.783	.036	.054
Attractiveness	.254	.856	.160	.071
Trustworthiness	.048	.984	.034	.054

Note. CRF-SF = Counselor Rating Form–Short Form.

*Indicates statistical significance at the .05 level.

Treatment Check

To assess treatment integrity, we examined participants’ perceptions of their feedback sessions as measured by their responses to the treatment check questions. We conducted two ANOVAs with treatment as the independent variable and perceived feedback personalization and session involvement as the dependent variables. The ANOVA results are shown in Table 3. There was a treatment effect for session personalization; TA participants perceived a higher level of personalization than IG participants. The session involvement scores of the TA and IG groups did not significantly differ. However, the session involvement scores of TA participants were an average of .45 higher than those of IG participants, with a large effect size of .148. Together these results suggest that the two treatments were perceived as intended.

Table 3.

Treatment Integrity Analysis of Variance (ANOVA) Results.

Session personalization	M	SD	F statistic	p	η²	Observed power
Developing questions	(N = 12) IG = 2.83	.577	5.456	.029*	.206	.606
Developing questions	(N = 11) TA = 3.36	.505
Session involvement	(N = 12) IG = 3.25	.622	3.479	.077	.148	.427
Session involvement	(N = 10) TA =3.70	.483

Note. IG = information giving; SD = standard deviation; TA = therapeutic assessment.

*Indicates statistical significance at the .05 level.

Analysis of Hypotheses

Hypothesis 1 was that TA participants would demonstrate a significantly greater increase in CDSE than IG participants immediately following assessment feedback. We conducted a one-way ANCOVA using treatment group as the independent variable, CDSE-SF pretreatment scale score as the covariate, and CDSE-SF posttreatment score as the dependent variable. All η² values were calculated following recommendations by Levine and Hullett (2002). There was not a significant treatment group effect, F(1, 21) = .778, p = .388. The results are presented in Table 4. The CDSE scores of TA and IG participants did not differ.

Table 4.

Dependent Measures Analysis of Covariance (ANCOVA) Results.

Measure	IG M (SD)		TA M (SD)		F statistic	p	η²	Observed power
Measure	Pre	Post	Pre	Post	F statistic	p	η²	Observed power
CDSE (N = 12, 11)	87.75 (10.55)	95.17 (12.33)	85.45 (12.09)	98.14 (9.59)	.778	.388	.032	.134
VIS (N = 11, 11)	10.82, (3.34)	8.73, (3.23)	11.09, (4.55)	6.45, (3.56)	5.32*	.032	.124	.591
CCA (N = 10, 11)	19.90, (2.92)	18.00, (3.27)	18.55, (6.77)	16.00, (3.52)	1.43	.247	.049	.205
Helpfulness (N = 11, 11)	N/A	7.92, (1.06)	N/A	7.77, (.75)	.138	.714	.007	.065

Note. CCA = Career Choice Anxiety scale; CDSE = Career Decision-Making Self-Efficacy scale; IG = information giving; SD = standard deviation; TA = therapeutic assessment; VIS = Vocational Identity scale.

*Indicates statistical significance at the .05 level.

The second hypothesis was that TA participants would report a significantly greater increase in vocational identity than IG participants following feedback. We conducted a one-way ANCOVA using treatment group as the independent variable, VIS pretreatment as the covariate, and VIS posttreatment as the dependent variable. The results are presented in Table 4. There was a significant treatment group effect, F(1, 20) = 5.32, p = .032. The VIS scores of TA participants were significantly lower than those of IG participants. Note that a decrease in VIS indicates a more positive vocational identity. This outcome provides support for Hypothesis 2.

The third hypothesis was that TA participants would have significantly lower CCA than IG participants following assessment feedback. We conducted a one-way ANCOVA using treatment group as the independent variable, CCA pretreatment as the covariate, and CCA posttreatment as the dependent variable. The results are presented in Table 4. There was not a significant treatment group effect for CCA posttreatment scores, F(1, 19) = 1.43, p = .247. The TA treatment did not differentially affect CCA.

The fourth hypothesis was that TA participants would rate their assessment feedback session as significantly more helpful than IG participants immediately following assessment feedback. We conducted an ANOVA using treatment group as the independent variable and perceived helpfulness as the dependent variable. The results are presented in Table 4. Treatment groups did not affect participant ratings of feedback helpfulness, F(1, 21) = .138, p = .714. It is noteworthy that the average helpfulness scores for both groups were just less than the “Greatly Helpful” scale descriptor, which suggests a possible ceiling effect.

The fifth and final hypothesis was that TA and IG participants together would demonstrate significantly higher CDSE and vocational identity and lower CCA following assessment feedback sessions. Three repeated measures ANOVAs were conducted using pretest and posttest scores as the dependent variable and time as the independent variable. Results are displayed in Table 5. The hypothesis was fully supported. Participants’ scores of CDSE, vocational identity, and CCA did significantly improve.

Table 5.

Dependent Measures Time Analysis of Variance (ANOVA) Results.

Measure	M (SD)		F statistic	p	η²	Observed power
Measure	Pre	Post	F statistic	p	η²	Observed power
CDSE (N = 12,11)	86.65 (11.11)	96.59 (10.96)	14.96*	.001	.405	.959
VIS (N = 11,11)	10.95 (3.90)	7.59 (3.51)	25.74*	.000	.551	.998
CCA (N = 10,11)	19.19 (5.22)	16.95 (3.47)	5.98*	.024	.230	.643

Note. CCA = Career Choice Anxiety scale; CDSE = Career Decision-Making Self-Efficacy scale; SD = standard deviation; VIS = Vocational Identity scale.

*Indicates statistical significance at the .05 level.

Discussion

The purpose of this study was to explore the comparative effectiveness of two approaches to career interest assessment feedback. We examined the effectiveness of TA, an alternative method of career assessment feedback, and IG, a traditional method, in increasing CDSE and vocational identity and reducing CCA. All participants were actual clients seeking career counseling at a university counseling center. The results indicated that TA was more effective than IG in increasing vocational identity. In addition, the assessment feedback process (i.e., TA and IG together) was effective in increasing CDSE and vocational identity and at reducing CCA. Overall, participants in both the TA and IG conditions rated their assessment feedback experience as “greatly helpful.”

Discussion of Hypotheses

The first hypothesis was that TA participants would demonstrate a significantly greater increase in CDSE than IG participants immediately following assessment feedback. The results did not support this hypothesis as CDSE scores significantly increased for both groups with a large effect size of .405. Because self-efficacy scores improved in both treatment groups, there was not sufficient evidence in support of the TA principle of self-efficacy/self-discovery. It is important to note that the self-efficacy increase for TA participants was greater than that of IG participants by .48 of a standard deviation (SD) for this measure. However, this was not a statistically significant difference.

There are three potential reasons for the absence of a significant treatment difference for decision-making self-efficacy. First, both treatments were effective; the posttreatment scores were near the ceiling of the CDSE measure. There may have been little room for treatments to significantly differ. Second, the limited number of participants in each treatment group did not yield statistical power (.134) to detect actual difference between the groups. Third, it is possible that both treatments were equally effective in increasing CDSE. Cutler (2004) proposed that if conclusions from psychotherapy outcome research are extended to career counseling outcome research, we should not expect treatment modality to be a critical factor influencing change. In this vein, the results regarding Hypothesis 1 reflect outcomes of minimal-to-no differences between active treatments common in psychotherapy outcome research (e.g., Luborsky et al., 1999).

Hypothesis 2 was that TA participants would report a significantly greater increase in vocational identity than IG participants following assessment feedback. The results supported this hypothesis. The vocational identity scores significantly improved for both treatment groups and to a significantly greater extent for those in the TA condition. A medium to large effect size of .124 was found for group difference, accounting for 12.4% of the variance between treatment groups. On average, TA participants’ vocational identity scores decreased by 2.55 (SD = 3.514) more than those of IG participants. A large effect size of .551 was found for treatment, accounting for 55% of the variance. This finding suggests that the self-verification principle of TA may enhance sense of vocational identity, even following a relatively brief treatment.

Hypothesis 3, that TA would yield a significantly greater decrease in CCA than IG, was not supported. A small to medium effect size of .049 was calculated for the group difference in CCA, accounting for 4.9 % of the variance in group posttest scores. On average, TA participants’ CCA scores decreased by 3.6 more (SD = 3.47 for posttest) than those of IG participants. A large effect size (ES) of .230 was found for the overall treatment effect, accounting for 23% of the variance. The failure to find a significant change in CCA does not substantiate the immediate effect of TA nor its principles.

The lack of a statistically significant finding may be due to three reasons. We first must consider the small sample size and corresponding low statistical power (.205). Second, a brief TA process may not adequately address CCA. In consideration of the smaller relative ES, it is possible that self-verification creates more immediate changes in identity and only affects anxiety in the longer term. A third possibility is that our TA protocol did not emphasize participant anxiety; counselors were not instructed to attend to client anxiety. A TA protocol that specifically invites the sharing of affect related to career choice dilemmas might lead to a greater reduction in anxiety.

The fourth hypothesis was that TA participants would rate their assessment feedback session as significantly more helpful than IG participants immediately following assessment feedback. Hypothesis 4 was not supported. Participants in both groups perceived treatments as helpful with average overall scores of 7.92 and 7.77, which fall just below the maximum value of 9 on the helpfulness rating scale. There are three potential reasons for the absence of a differential treatment effect for helpfulness. First, the use of a single item may not have produced a measure with sufficient spread or dimensionality to adequately measure helpfulness. Second, the small sample size again led to a minimal power value of .065. Third, both treatments may have been genuinely and equally helpful.

Hypothesis 5 was that TA and IG participants together would demonstrate significantly higher CDSE and vocational identity and lower CCA immediately following their assessment feedback session. There was full support for Hypothesis 5 with large treatment effect sizes of .405, .551, and .230 for CDSE, vocational identity, and CCA, respectively. These positive findings indicate that, as a whole, TA and IG assessment protocols were effective. In addition, the findings suggest that the tenets of TA (e.g., self-verification, self-enhancement, and self-efficacy/self-discovery) at least in part lead to the theoretically predicted changes.

Implications

These results both support and contrast with findings from previous career counseling and TA research. With respect to the career counseling literature, the current study provides further evidence for the overall efficacy of career counseling and specifically for the effectiveness of individualized career assessment (cf. Brown & Ryan Krane, 2000; Whiston et al., 1998, 2003). Although not a direct focus of the current study, the results provide further evidence for the effectiveness of career assessment feedback with the SII in increasing decision-making self-efficacy (cf. Luzzo & Day, 1999; Uffelman, Subich, Diegelman, Wagner, & Bardash, 2004). Our findings were also comparable to those of Cutler (2004), who found no significant differences between the outcomes of TA and IG assessment feedback. The current study also adds to career counseling intervention research in demonstrating the comparable effectiveness of TA as an alternative method of assessment feedback. These results represent a response to the call of Whiston et al. (2003) for more comparisons of different methods (e.g., individual, group, computer, stylistic differences) of career assessment feedback.

The results also were consistent with those of several studies in the TA literature. Specifically, the findings are similar to other studies documenting positive outcomes from TA feedback ranging from small to large effect sizes (e.g., Ackerman, Hilsenroth, Baity, & Blagys, 2000; Hanson et al., 1997; Hilsenroth, Peters, & Ackerman, 2004; Houser, 2005; Ingram, 2000; Newman & Greenway, 1997; Peters, 2000; Tharinger et al., 2009). Further, the results are directly similar to Hanson, Claiborn, and Kerr (1997), who also found no difference between delivered and interactive career counseling feedback on ratings of session helpfulness.

These results also have two important implications in light of the significant increase in Internet-based career assessments that has occurred over the last decade. First, Gati and Asulin-Peretz (2011) identified the need for integrating expert knowledge into Internet-based career assessment tools. Our results suggest that the TA principle of self-discovery can be integrated into these online systems. For example, the step of asking respondents to develop personally relevant exploration questions can be integrated into the design of on online career assessment tool. Asking questions can be designed to precede information generation. Elaboration of the insight sought from the tool and the reasons for the relevance of this additional information could enhance self-discovery in counselor-free online assessments and interventions. Second, the TA principles of self-verification and self-enhancement can be important for counselors in working with the increasing number of clients who initiate counseling after already having used online assessments (Gati & Asulin-Peretz, 2011). These clients may not have been able to integrate or synthesize the information generated from these Internet-based tools. In these circumstances, counselors can begin by focusing on assessment feedback that closely mirrors the self-views of clients (self-verification) and then proceed to consider potentially discrepant information. The growth of automated, open access career assessment and intervention systems produce a potentially overwhelming amount of information. The TA principles provide guidance for counselors who work with clients to make meaning of this information.

Finally, this study adds to the TA literature in three ways. It is the first study that has demonstrated the positive effects of TA assessment feedback on career counseling outcomes. Although Hanson et al. (1997) also found positive outcomes from an interactive career counseling assessment feedback intervention, they examined outcomes only related to session and counselor quality and did not examine career counseling outcomes. Second, the results add further support to the findings of Poston and Hanson (2010) that psychological assessment, particularly involving collaboration, is effective. Finally, in consideration of the identified significant difference and medium to large effect size between TA and IG participants’ vocational identity scores, the results of the study provide partial evidence in support of the TA principle of self-verification.

Taken together, these findings suggest that TA is effective in a variety of contexts. The results also suggest that TA is more effective than IG in increasing vocational identity. It remains unclear if TA leads to additional treatment outcomes such as decreased anxiety and resistance and increased goal achievement and motivation. It also is important to consider the limitations of the study.

Limitations

There were several limitations in the present study that can be attributed to the implementation of the study at a university counseling center. The first pertains to the sampling method. Because actual clients seeking career counseling were solicited for participation, only a limited number of potential participants were available. There was a 51% attrition rate, which occurred as a result of scheduling problems, inappropriate fit with study purpose, counselor illness, failure to complete the assessment or outcome measures, and failure to return for a second session. The small sample size reduced the experimental power of all of our analyses and our ability to detect treatment effects. As a consequence, the sample size placed a significant limitation on the interpretability of our results.

It also is important to consider the strengths of the selected sampling method. Using real clients strengthened the external validity of the study—at a cost to internal validity (Gelso & Fretz, 2001). However, we determined this enhancement to be of greater importance given the direct clinical applicability of the study. Heppner and Heppner (2003) noted a concern with the questionable generalizability created by the preponderance of recruited client samples and unnatural settings used in career counseling outcome research. External validity in the current study was improved through use of a sample of individuals who sought counseling on their own and indicated a genuine need for career counseling. The sampling method also helped achieve excellent population and ecological validity. As a result, the findings can be directly generalized to university counseling center client populations and settings.

A second potential limiting factor was that several counselors performed the two study interventions. We attempted to control for counselor differences by comparing perceptions of the counselors’ expertness, attractiveness, and trustworthiness; we also used a protocol checklist during counselor training to monitor treatment integrity. We standardized training in consideration of variability in the counselors’ experience and availability for scheduling and training. Despite all these efforts, the results still may have been affected by counselor differences and protocol adherence.

It is important to note that the counselors were also assessed for protocol preference or treatment allegiance following training and prior to the start of the study. All counselors except counselor Number 1 stated a preference for protocol A (i.e., the TA protocol). Counselor Number 1, who stated a preference for the IG protocol, also saw the majority of participants. It is difficult to determine if and how these preferences may have affected the results. We were not able to statistically evaluate counselor effect due to the small sample size. However the significant influencing effect of treatment allegiance has been well documented (Luborsky et al., 1999). Because counselor Number 1 saw the majority of participants, the results may have been positively skewed toward the IG treatment. This effect presents a significant limitation to the interpretation of the results particularly considering the significant positive outcomes identified from the TA treatment.

A third limiting factor was the lack of longer term follow-up measurement. Participants were allowed to continue with career counseling following study participation; there was no way to maintain comparability between the participants following study completion. Any follow-up measurement results could not have been attributed to differences in study treatments. In consequence, any longer term effects were not detected and the results are limited to the immediate effects of intervention. Whiston (2001) emphasized the importance of determining both the immediate and long-term effects of career interventions. Implementation of follow-up measures in future research will provide better evaluation of treatment effects over time.

In sum, there were three potential limitations pertaining to the design, sampling, and measurement selection of the current study. These factors may have affected the interpretability and generalizability of results and the ability to detect treatment effects, particularly the small sample size. However the study design, sampling, and measurement selection also provided several strengths to the significance of the study results. Although highly challenging, the choice to conduct the study as part of the regular functioning of a university counseling center with unsolicited career counseling clients afforded adequate internal and strong external validity. We believe intentional measurement selection (i.e., theoretically derived and population relevant) also provided a greater ability to detect treatment effects.

Future Directions

In consideration of the previous research, theoretical nature, methodology, and results of the current study, we make the following suggestions for the direction of future research in three primary areas. The first suggestion involves similar research aimed at exploring methods of career assessment interpretation. Multiple researchers in career counseling have identified subtypes of career undecided individuals that present with distinct sets of career decision-making concerns (e.g., Chartrand et al., 1994; Jones & Chenery, 1980; Kelly & Pulver, 2003; Lucas & Epperson, 1988). In order to fully recognize the individual presentation of career decision-making needs, researchers have encouraged the development of differential treatment approaches (e.g., Lent, 2001; Whiston & Oliver, 2005). Kivlighan, Hageseth, Tipton, and McGovern (1981) conducted the only study we identified in which the effectiveness of differential assessment feedback approaches by personality type was directly examined. They found that treatments that matched personality type (i.e., either task or people oriented) were more effective. We echo the sentiment that future researchers continue these steps and in particular examine the utility of TA and IG approaches with clients with distinct clusters of career concerns.

The second suggestion for future researchers is to implement three methodological enhancements related to the limitations of the current study. First researchers should consider implementing studies of middle and high school students, community college students, or adults in career transition. More specialized groups could also be targeted, including first-generation or international college students. Second, longer term outcomes can be measured in a more controlled setting, such as a departmental training clinic. Third, researchers can add process measures (e.g., specific measurement of aspects of the assessment feedback process, expanded session helpfulness ratings, session evaluations). Although the addition of process measures will increase the research time commitment and potentially reduce the overall number of participants, it also can provide greater understanding of why positive changes do or do not occur.

In closing, we encourage researchers to continue critical examination of the career assessment feedback process. Career assessment continues to be practiced by 56% of APA members who identify counseling as their specialty (Watkins, Campbell, & Nieberding, 1994). Holland’s (1997) person environment fit theory remains a dominant theoretical approach to career counseling and assessment. Career assessment is the central mechanism in practical application of Holland’s theory and, according to Hansen (2005), continues to be the construct most prevalently assessed in practice. Meta-analytic research reflects the effectiveness of career counseling assessment. Further, Brown and Ryan Krane (2000) specifically identified assessment as a critical ingredient of effective career counseling. In consideration of these factors, it is imperative that researchers continue to explore the process and impact of, and further develop improved methodological approaches to career counseling assessment.

Footnotes

Authors’ Note

This article is based on a doctoral dissertation conducted by the first author under the supervision of the second author.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by a small summer dissertation grant, Purdue Research Foundation Summer Dissertation Grant, Purdue University, West Lafayette, IN, June 2007 to July 2007.

References

Ackerman

S. J.

Hilsenroth

M. J.

Baity

M. R.

Blagys

M. D.

(2000). Interaction of therapeutic process and alliance during psychological assessment. Journal of Personality Assessment, 75, 82–109.

ACT. (2009). ACT high school profile: Section IV, career and educational aspirations. Retrieved August 17, 2011, from http://www.act.org/news/data/09/data.html

Allen

Robbins

S. B.

(2010). Effects of interest-major congruence, motivation, and academic performance on timely degree attainment. Journal of Counseling Psychology, 57, 23–35.

Aud

Hussar

Kena

Bianco

Frohlich

Kemp

Tahan

(2011). The Condition of Education 2011 (NCES 2011-033). U.S. Department of education, National center for education statistics. Washington, DC: U.S. Government Printing Office.

Bandura

(1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215.

Bandura

(1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28, 117–148.

Bernes

K. B.

Bardick

A. D.

Orr

D. T.

(2007). Career guidance and counseling efficacy studies: An international research agenda. International Journal of Vocational Guidance, 7, 81–96.

Betz

N. E.

Klein

K. L.

Taylor

K. L.

(1996). Evaluation of a short form of the decision-making self-efficacy scale. Journal of Career Assessment, 4, 47–57.

Betz

N. E.

Taylor

K. L.

(2001). Manual for the career decision self-efficacy scale and CDSE – Short form. Columbus: Department of Psychology, Ohio State University.

10.

Brown

S. D.

Ryan Krane

N. E.

(2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In Brown

S. D.

Lent

R. W.

(Eds.), Handbook of counseling psychology (3rd ed., pp. 740–766). New York, NY: John Wiley.

11.

Chartrand

J. M.

Martin

W. F.

Robbins

S. B.

McAuliffe

G. J.

Pickering

J. W.

Calliotte

J. A.

(1994). Testing a level versus an interactional view of career indecision. Journal of Career Assessment, 2, 55–69.

12.

Chartrand

J. M.

Robbins

S. B.

(1997). Career factors inventory: Applications and technical guide. Palo Alto, CA: Consulting Psychologists Press.

13.

Chartrand

J. M.

Robbins

S. B.

Morrill

W. H.

Boggs

(1990). Development and validation of the career factors inventory. Journal of Counseling Psychology, 37, 491–501.

14.

Claiborn

C. D.

Goodyear

R. K.

Horner

P. A.

(2001). Feedback. Psychotherapy, 38, 401–405.

15.

College Entrance Examination Board. (September, 2010). College-bound seniors: Total group profile [National] report, selected years, 1995-96 through 2009-10. Retrieved August 17, 2011, from http://professionals.collegeboard.com/data-reports-research/sat/cb-seniors-2010

16.

Corrigan

J. D.

Schmidt

L. D.

(1983). Development and validation of revisions in the counselor rating form. Journal of Counseling Psychology, 30, 64–75.

17.

Crites

J. O.

(1978). Career maturity inventory. Monterey, CA: McGraw Hill.

18.

Cutler

S. C.

(2004). Efficacy of therapeutic assessment in counseling for career indecision (Doctoral dissertation, University of Nebraska, 2004). Dissertation Abstracts International, 65, 3275.

19.

Donnay

D. A.

Morris

M. L.

Schaubhut

N. A.

Thompson

R. C.

(2005). Strong interest inventory manual: Research, development, and strategies for interpretation. Mountain View, CA: CPP.

20.

Elliott

(1985). Helpful and nonhelpful events in brief counseling interviews: An empirical taxonomy. Journal of Counseling Psychology, 32, 307–322.

21.

Epperson

D. L.

Pecnik

J. A.

(1985). Counselor rating form—Short version: Further validation and comparison to the long form. Journal of Counseling Psychology, 32, 143–146.

22.

Finn

S. E.

(1996). Manual for using the MMPI-2 as a therapeutic intervention. Minneapolis: University of Minnesota Press.

23.

Finn

S. E.

March, 1999). Giving feedback to clients about “defensive” test protocols: Guidelines from therapeutic assessment. Paper presented at the midwinter meeting the Society for Personality Assessment, New Orleans, LA.

24.

Finn

S. E.

Tonsager

M. E.

(1997). Information-gathering and therapeutic models of assessment: Complementary paradigms. Psychological Assessment, 9, 374–385.

25.

Finn

S. E.

Tonsager

M. E.

(2002). How therapeutic assessment became humanistic. The Humanistic Psychologist, 30, 10–22.

26.

Fiske

S. T.

Taylor

S. E.

(2008). Social cognition: From brains to culture. Boston, MA: McGraw-Hill Higher Education.

27.

Gasser

C. E.

Larson

L. M.

Borgen

F. H.

(2007). Concurrent validity of the 2005 strong interest inventory: An examination of gender and major field of study. Journal of Career Assessment, 15, 23–43.

28.

Gati

Asulin-Peretz

(2011). Internet-based self-help career assessments and interventions: Challenges and implications for evidence-based career counseling. Journal of Career Assessment, 19, 259–273.

29.

Gelso

Fretz

(2001). Counseling psychology (2nd ed.). Fort Worth, TX: Harcourt College.

30.

Goodyear

R. K.

(1990). Research on the effects of test interpretation: A review. The Counseling Psychologist, 18, 240–257.

31.

Gordon

V. N.

(2007). The undecided college student: An academic a career advising challenge (3rd ed.). Springfield, IL: Charles C. Thomas.

32.

Hansen

J. C.

(2005). Assessment of interests. In Brown

S. D.

Lent

R. W.

(Eds.), Career development and counseling: Putting theory and research to work (pp. 281–304). Hoboken, NJ: John Wiley.

33.

Hanson

W. E.

(2001). Minnesota multiphasic personality inventory-2 (MMPI-2) test interpretation feedback studies counselor training manual (Unpublished document). University of Nebraska-Lincoln, Lincoln, NE.

34.

Hanson

W. E.

Claiborn

C. D.

(2006). Effects of test interpretation style and favorability in the counseling process. Journal of Counseling & Development, 84, 349–357.

35.

Hanson

W. E.

Claiborn

C. D.

Kerr

(1997). Differential effects of two test-interpretation styles in counseling: A field study. Journal of Counseling Psychology, 44, 400–405.

36.

Harmon

L. W.

Hansen

J. C.

Borgen

F. H.

Hammer

A. L.

(1994). Strong interest inventory: Applications and technical guide. Palo Alto, CA: Consulting Psychologists Press.

37.

Heppner

M. J.

Heppner

P. P.

(2003). Identifying process variables in career counseling: A research agenda. Journal of Vocational Behavior, 62, 429–452.

38.

Hilsenroth

M. J.

Peters

E. J.

Ackerman

S. J.

(2004). The development of therapeutic alliance during psychological assessment: Patient and therapist perspectives across treatment. Journal of Personality Assessment, 83, 332–344.

39.

Holland

J. L.

(1997). Making vocational choices (3rd ed.). Odessa, FL: Psychological Assessment Resources.

40.

Holland

J. L.

Daiger

D. C.

Power

P. G.

(1980). My vocational situation: Description of an experimental diagnostic form for the selection of vocational assistance. Palo Alto, CA: Consulting Psychologists Press.

41.

Holland

J. L.

Gottfredson

D. C.

Power

P. G.

(1980). Some diagnostic scales for research in decision making and personality: Identity, information, and barriers. Journal of Personality and Social Psychology, 39, 1191–1200.

42.

Holland

J. L.

Johnston

J. A.

Asama

N. F.

(1993). The vocational identity scale: A diagnostic and treatment tool. Journal of Career Assessment, 1, 1–12.

43.

Houser

S. A.

(2005). The use of therapeutic assessment in treatment planning and outcomes measurement of clients in a community mental health center (Ph.D. dissertation). Texas Woman’s University, TX. Retrieved March 27, 2009, from Dissertations & Theses: Full Text (Publication No. AAT 3195980).

44.

Ingram

S. M.

(2000). The effects of psychological assessment on psychological distress, self-esteem, and self-knowledge of cognitive abilities (Ph.D. dissertation). Georgia State University, GA. Retrieved August 27, 2009, from Dissertations & Theses: Full Text. (Publication No. AAT 3008151).

45.

Jones

Chenery

(1980). Multiple subtypes among vocationally undecided college students: A model and assessment instrument. Journal of Counseling Psychology, 27, 469–477.

46.

Kelly

K. R.

Pulver

C. A.

(2003). Refining measurement of career indecision types: A validity study. Journal of Counseling & Development, 81, 445–454.

47.

Kivlighan

D. M

Hageseth

J. A.

Tipton

R. M.

McGovern

T. V.

(1981). Effects of matching treatment approaches and personality types in group vocational counseling. Journal of Counseling Psychology, 28, 315–320.

48.

Kohut

(1977). The restoration of the self. New York, NY: International Universities Press.

49.

Kokotovic

A. M.

Tracey

T. J.

(1987). Premature termination in a university counseling center. Journal of Counseling Psychology, 34, 80–82.

50.

Lent

R. W.

(2001). Vocational psychology and career counseling: Inventing the future. Journal of Vocational Behavior, 59, 213–225.

51.

Levine

T. R.

Hullett

C. R.

(2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28, 612–625.

52.

Lewis

D. M.

Savickas

M. L.

(1995). Validity of the career factors inventory. Journal Career Assessment, 3, 44–56.

53.

Luborsky

Diguer

Seligman

D. A.

Rosenthal

Krause

E. D.

Johnson

… Schweizer

(1999). The researcher’s own therapy allegiances: A “wild card” in comparisons of treatment efficacy. Clinical Psychology: Science and Practice, 6, 95–106.

54.

Lucas

M. S.

Epperson

D. L.

(1988). Personality types in vocationally undecided students. Journal of College Student Development, 29, 460–466.

55.

Luzzo

D. A.

Day

M. A.

(1999). Effects of strong interest inventory feedback on career decision-making self-efficacy and social cognitive career beliefs. Journal of Career Assessment, 7, 1–17.

56.

Newman

Greenway

(1997). Therapeutic effects of providing MMPI-2 test feedback to clients at a university counseling service: A collaborative approach. Psychological Assessment, 9, 122–131.

57.

Nilsson

J. E.

Schmidt

C. K.

Meek

W. D.

(2002). Reliability generalization: An examination of the career decision-making self-efficacy scale. Educational and Psychological Measurement, 62, 647–658.

58.

Oliver

L. W.

Spokane

A. R.

(1988). Career-intervention outcome: What contributes to client gain? Journal of Counseling Psychology, 35, 447–462.

59.

Patton

M. J.

Connor

G. E.

Scott

K. J.

(1982). Kohut’s psychology of the self: Theory and measures of counseling outcome. Journal of Counseling Psychology, 29, 268–282.

60.

Pawlowski

C. A.

(2002). Therapeutic effects of MMPI-2 feedback favorability on client subjective well-being: A process and outcome study (Doctoral dissertation, University of Nebraska, 2002). Dissertation Abstracts International, 63, 4382.

61.

Peters

J. M.

(2000). The effect of therapeutic assessment on women with eating disorders (Ph.D. dissertation). The University of Texas at Austin, TX. Retrieved August 27, 2009, from Dissertations & Theses: Full Text. (Publication No. AAT 9983317).

62.

Poston

J. M.

Hanson

W. E.

(2010). Meta-analysis of psychological assessment as a therapeutic intervention. Psychological Assessment, 22, 203–212.

63.

Ryan

N. E.

(1999). Career counseling and career choice goal attainment: A meta-analytically derived model for career counseling practice (Ph.D. dissertation). Loyola University of Chicago, IL. Retrieved August 27, 2009, from Dissertations & Theses: Full Text. (Publication No. AAT 9930585).

64.

Scheel

M. J.

Berman

Friedlander

M. L.

Conoley

C. W.

Duan

Whiston

S. C.

(2011). Whatever happened to counseling in counseling psychology? The Counseling Psychologist, 39, 673–602.

65.

Tabachnik

B. G.

Fidell

L. S.

(2001). Using multivariate statistics (4th ed.). Boston, MA: Allyn & Bacon.

66.

Taylor

K. M.

Betz

N. E.

(1983). Applications of self-efficacy theory to the understanding and treatment of career indecision. Journal of Vocational Behavior, 22, 63–81.

67.

Tharinger

D. J.

Finn

S. E.

Gentry

Hamilton

Fowler

Matson

Krumbholtz

Walkowiak

(2009). Therapeutic assessment with children: A pilot study of treatment acceptability and outcome. Journal of Personality Assessment, 91, 238–244.

68.

Tracey

T. J. G.

(2010). Interest assessment using new technology. Journal of Career Assessment, 18, 336–344.

69.

Tracey

T. J. G.

Glidden

C. E.

Kokotovic

A. M.

(1988). Factor structure of the counselor rating form-short. Journal of Counseling Psychology, 35, 330–335.

70.

Tracey

T. J. G.

Robbins

S. B.

(2006). The interest-major congruence and college success relation: A longitudinal study. Journal of Vocational Behavior, 69, 64–89.

71.

Uffelman

R. A.

Subich

L. M.

Diegelman

N. M.

Wagner

K. S.

Barhas

R. J.

(2004). Effect of mode of interest assessment on clients’ career decision making self-efficacy. Journal of Career Assessment, 12, 366–380.

72.

Watkins

C. E.

Campbell

V. L.

Nieberding

(1994). The practice of vocational assessment by counseling psychologists. The Counseling Psychologist, 22, 115–128.

73.

Whiston

S. C.

(2001). Selecting career outcome assessments: An organizational scheme. Journal of Career Assessment, 9, 215–228.

74.

Whiston

S. C.

(2011). Vocational counseling and interventions: An exploration of future “big” questions. Journal of Career Assessment, 19, 287–295.

75.

Whiston

S. C.

Brecheisen

B. K.

Stephens

(2003). Does treatment modality affect career counseling effectiveness? Journal of Vocational Behavior, 62, 390–410.

76.

Whiston

S. C.

Oliver

L. W.

(2005). Career counseling process and outcome. In Walsh

W. B.

Savickas

M. L.

(Eds.), Handbook of vocational psychology (pp. 155–194). Mahwah, NJ: Lawrence Erlbaum.

77.

Whiston

S. C.

Sexton

T. L.

Lasoff

D. L.

(1998). Career-intervention outcome: A replication and extension of Oliver and Spokane (1998). Journal of Counseling Psychology, 45, 150–165.