Progress and Prospects in the Evaluation of Career Assistance

Abstract

Reviews of research on the evaluation of career assistance from 1970 to 2014 (N = 23) are examined for common conclusions and recommendations. Conclusions are tempered based upon the small numbers of studies included. Recommendations for strengthening research using integrated treatment chains and incorporating epidemiologically based statistics (i.e., risk ratio, odds ratio, and population attributable fraction) are described. A shift in prevailing research paradigm from an emphasis on the individual to an emphasis on the community would supplement evaluations based upon individual outcomes and make a more compelling case to policy makers and funders who could support future evaluative research and provision of career assistance.

Keywords

career counseling meta-analysis epidemiology

The study of how people aspire, select, and adjust to work and careers is a complex cross-disciplinary enterprise involving psychology, sociology, economics, education, and cultural studies (Brown & Ryan Krane, 2000; Crites, 1969; Osipow & Fitzgerald, 1996; Spokane, 2013; Walsh, 2003). The sweeping nature of the psychology of careers, therefore, necessitates some parsing into subareas when evaluating a body of empirical studies for contributions over time. This article concentrates on the design, delivery, and evaluation of a broad range of strategies for assisting individuals, groups, and populations engaged in the choice of a career (as opposed to implementation of a chosen career). Our invitation from the editor was to write a retrospective “review of reviews” of research conducted over the past several decades in the subject area called career intervention or as we are calling the topic, “career assistance.” Our intent in this retrospective is to (a) collect reviews of research on career assistance from 1970 to present, (b) use aggregate conclusions and recommendations from these reviews to suggest ways to advance future research and practice in career assistance, (c) utilize these understandings from prior reviews to redirect future efforts so as to increase research resources and support, and (d) better integrate the practice of career assistance with findings from its empirical and scientific base. We begin by explicating our definition of career assistance and how it is evaluated.

Delimiting and Evaluating Career Assistance

In a seminal review of counseling for the first Handbook of Psychotherapy and Behavior Change, Myers (1971, 1986) reasoned that counseling was, by definition, education and vocational intervention, not that counseling included educational and vocational intervention. A similar view was espoused by Tyler (1992). Myers and Tyler implied that life decisions were invariably both career and personal in nature and the therapeutic discussion of such issues was counseling. However, increasing specialization in journal content has led to a bifurcation of career counseling and assistance and mental health oriented or “personal counseling.” This separation has had benefits as well as unfortunate consequences. One benefit is the ability to focus on specific career-related behaviors some of which can be strengthened in alternative ways without the need for direct service. A negative consequence is that only a small number of empirical studies in counseling include or emphasize career issues (60–70 controlled evaluations). Further, the separation of definitions of career and personal counseling raises questions about whether the underlying mechanisms responsible for client gains in career and personal counseling are the same or different—and, thus, leads to studies of the therapeutic elements or combination of elements are responsible for gains when providing career assistance as opposed to “counseling.” It seems likely to us that the underlying mechanisms responsible for gain rest in the therapeutic relationship and process as moderated by the client’s cultural and socioeconomic context. The specific content of an intervention may vary depending on the client’s goals but may not be the most critical determining factor in positive outcomes (Wampold, 2009).

Myers (1971, 1986) raised a second definitional issue—when studying career assistance any classification ought to be on the basis of whether the investigation was a study attempting to manipulate variables for understanding or whether, in contrast, the study was based upon the optimal provision and evaluation of services. The implication of Myers’ view is that we should focus both on the underlying mechanisms responsible for improved decision making and role functioning (analytic and process studies), as well as the efficacy of any attempts to assist individuals in structured ways (outcome studies). Thus, the present review focuses on what the laboratory findings reveal about how we might deliver services, and the very critical problem of treatment fidelity and implementation as we attempt to do so.

Definitions employed in studies

Several of the reviews we included utilized Holland, Magoon, and Spokane’s (1981) inclusive definition of multiple forms of vocational assistance, including vocational counseling, self-help devices, career courses, occupational information, interest inventories, and vocational card sorts. Others utilized Spokane and Oliver’s (1983) definition of career intervention as “any treatment or effort intended to enhance an individual’s career development or to enable the person to make better career decisions” (p. 100). Although a case can be made for a delimited definition emphasizing dyadic, or group therapeutic interventions, we utilize the broader view of any activity or effort, either self-guided or externally applied, to enhance the quality of career decisions. This broad definition is consistent with trends in providing preventative and educative strategies in field-based settings with population-based samples—particularly interventive efforts that address the needs of vulnerable social groups and for which dyadic or group interventions are impractical, given limited professional resources and lack of access to services. It remains to be seen whether the underlying mechanisms responsible for gain in dyadic and preventative intervention are the same or even similar.

We note that the meta-analytic reviews we include generally begin by utilizing one of the broad definitions of career assistance. However, because such reviews must have available data such as Ns, means, standard deviations (SDs), treatment control comparisons, and so on, these inclusion criteria limit, de facto, the scope and nature to more traditional strategies that typically include needed data. In cases with complete data, the resulting operational definition of career assistance is more often limited to face-to-face individual, group, workshop or class interventions, or at least to interventions with necessary data available (Oliver & Spokane, 1982; Whiston & Oliver, 2005). Thus, care must be taken to delimit conclusions and recommendations and to state the generalizability of findings in light of the nature of studies included and the settings in which they are conducted. As additional studies accumulate, separate reviews of preventative and developmental interventions may be more justifiable, as was done by Baker and Popowicz (1983), or of dyadic interventions by themselves as suggested by Blustein and Spengler (1995). Previous reviews have used broad definitions to encompass a large number of career assistance studies, but limit studies included in the review due to quantitative and experimental inclusion criteria. At present, however, the bodies of literature in both the dyadic definition of career counseling and the more comprehensive definition of career intervention or assistance are small and argue for broad inclusion criteria, carefully circumscribed conclusions, and additional studies that compare direct and indirect assistance in actual settings and under realistic conditions.

Advances in Career Intervention Outcome and Process Research Over Time

There have been multiple advances in our understanding of career assistance in its various forms (e.g., career counseling, interest inventory interpretations, and instruction in decision making). Nonetheless, we caution that these advances must be viewed within the limits of the rigor and practical relevance of included studies. Before we examine reviews from 1970 on, we describe five phases in research on career assistance in which a particular empirical focus resulted in a significant advance that can reasonably be generalized beyond the study findings. We note, however, that studies that approach a clinical trial with respect to methodological rigor have not yet been conducted in the area of career assistance.

In an early research phase, a small group of less than 10 studies were reported in the 1940s and 1950s that evaluated traditional approaches with college students. The initial studies, often carefully designed and utilizing random assignment, contributed new understandings in the field of career assistance despite limitations in sampling, outcomes, and strategic focus. A good example of these early studies was Hoyt (1955) who compared individual counseling to group counseling and a no-treatment control in male college students. Both individual and group clients reported about ½ SD greater self-reported certainty and satisfaction than controls, and group counseling was considered to be the more economical choice. These early studies confirmed that career counseling could be carefully studied with positive outcomes.

A second phase of empirical studies followed in the 1960s and 1970s incorporating a more sophisticated array of treatments, variable design rigor, and better validated outcome measures. Generally, samples were still college student volunteers, but not necessarily randomly assigned. An example of these later studies (Babcock & Kaufman, 1976) compared individual counseling to a seven-session career course using self and career knowledge and planfulness as outcomes. Among the gains from these second phase studies were a significant increase in the number of studies; rigorous literary reviews of, and recommendations for, the increasing body of research (Myers, 1971); the evaluation of manualized treatments (Bergland & Lundquist, 1975; Knickerbocker & Davidshofer, 1978); the use of computer-assisted interventions (Cochran, Hoffman, Strand, & Warren, 1977; Maola & Kane, 1976); and a more sophisticated array of outcome measures involving multiple domains such as information seeking and exploration (Samaan & Parker, 1973). Overall, Phase II studies’ outcome gains appeared to be greater than in Phase I studies, occasionally exceeding one SD of gain. These Phase II studies also demonstrated that career intervention more broadly defined and with a more diverse set of outcome measures could also be evaluated with positive outcomes.

A third research phase during the 1980s saw the first meta-analyses; more diverse sampling, particularly of minority and underserved populations; an ever broader array of interventive strategies and outcome measures; the initial use of diagnostic information as a potential moderator of treatment effectiveness (Fretz, 1981; Rounds & Tinsley, 1984; Robbins & Tucker, 1986); the suggestion of common ingredients in successful career interventions (Holland, Magoon, & Spokane, 1981); and the worry that treatments differing in nature and strategy did not always show differential outcomes. The third phase of studies also saw additional growth in research rigor.

The 1990s were characterized by increasing attention to the process of career assistance and its relation to outcomes as demonstrated in Kirschner, Hoffman, and Hill (1994) who studied counselor behavior and client responses using a case study, discovery-oriented paradigm. There were calls for more research on best and worst sessions; an emphasis on the role of the working alliance in career counseling (Meara & Patton, 1994); and increasingly sophisticated designs, outcome measures, and analytic strategies combining process and outcome in career intervention (Heppner & Hendricks, 1995; Heppner, Multon, Gysbers, Ellis, & Zook, 1998; Marko & Savickas, 1998; Swanson, 1995). There were also more sophisticated replications of meta-analytic studies (Whiston, Sexton, & Lasoff, 1998) that further clarified the potentially additive nature of common ingredients and the differential effect of a counselor’s presence. Additionally, participant samples in theory studies were increasingly diverse [global] (Lent, Brown, Nota, & Soresi, 2003) and suggested that the interventions could be adapted for diverse populations with salutary effects. Finally, there was a growing emphasis on federally driven needs for more research in science, technology, engineering, and math (STEM) careers and School to Work Transition (U.S. General Accounting Office, 1991). These later studies emphasized the complexity of the relationship between process and outcome in the provision of career assistance and suggested more modest outcomes than previously found.

A fifth and ongoing phase of research on career assistance during the 2000s and beyond has continued the call for process and outcome studies (Heppner & Heppner, 2003), further refined our understanding of the therapeutic ingredients associated with positive outcomes (Brown & Ryan Krane, 2000), and struggled to untangle the complex problem of which treatment might be effective for which particular client (Whiston, Brecheisen, & Stephens, 2003). This fifth and recent phase of research is characterized by a very diverse set of interventive strategies, including social media and the Internet (Whiston & Oliver, 2005), narrative approaches (Hartung, 2013; Savickas, 2005), and a new generation of outcome measures focusing on barriers and impediments to choice (Swanson, Daniels, & Tokar, 1996). The fifth and current phase of career intervention research has continued on the trend of process-outcome research, while exploring new strategies.

Slow steady progress

Roger Myers (1971, 1986), who generally took the “long view” on the field of career development, noted the lumbering pace of researchers compared with the fleet imaginative changes by practitioners. Myers (1971) argued that counseling rarely advanced “behind a well-established body of scientific evidence” moving instead from practice backward, a conclusion that we still find to be true. This back and forth interchange between practice and research, while effective to a certain extent, rarely follows a coherent path or direction. Competing theoretical perspectives, resource constraints, and scant opportunities for collaboration between a small number of researchers and active practitioners limit the sharing of knowledge and experience. Thus, sustained research-to-practice or practice-to-research exchanges have not tended to produce “gold standard” studies or rigorous clinical trials that might exert greater influence on policy and practice in the provision of career assistance to a broad and diverse general population.

Although there have been several advances during the five phases of research, significant improvements in the sophistication of studies evaluating the efficacy of career assistance should be a professional priority. The questions we pose should be considerably more complex, and address common limitations that while individually may not seem serious but are collectively problematic. With effort, the limitations in the design and analysis of career interventions can be corrected over a series of future studies. Principal among these problems is that current conclusions about the efficacy of career intervention and assistance are drawn largely from studies conducted within the frame of 1–3 sessions, a few weeks of sessions, or even a few months, but almost never longer than a semester of interventive attempts. This short intervention and evaluation frame is in contrast to a substantial body of longitudinal studies that describe, but do not intervene, in career development across years or even decades (Gribbons & Lohnes, 1982). Interventive and longitudinal studies have not converged. Longitudinal studies have generally not incorporated treatments preferring, understandably, to study development as it naturally occurs. As our scope shifts to a more population-based preventative model, we should be able to incorporate serial, integrative interventions over extended periods of time to examine cumulative gains. Such studies will require substantial resources, use of novel methodologies, and colleagues who are willing to invest time and effort over long periods. The conclusions and recommendations from previous reviews point the way toward needed future studies at the level of rigor and relevance appropriate to current contexts.

Twenty-Three Reviews of Career Assistance

There appear to be two general categories of reviews of research on career assistance. The first category is a general review of empirical studies (either meta-analytic or literary) evaluating the effects of career assistance across populations and outcomes (a convergent review). The second category of reviews concentrates on a particular subgroup or decision point (e.g., college and career readiness; Lapan, Turner, & Pierce, 2012), with an emphasis on unique challenges faced by a subgroup or at a particular choice point. It is worth noting that these review categories do not focus on a strategy—rather on a population or developmental moment of challenge. There are not yet a sufficient number of studies within a specific strategy to review the underlying mechanisms of career strategies independently or to examine the collaborative process of career assistance closely. However, there is sufficient overlap in existing studies to draw conclusions about the necessity of future studies on how to intervene with individuals making career decisions. We focused on choice rather than implementation in this review and consolidate findings and recommendations regarding the design and evaluation of career assistance to populations not characterized by significant pathology. The next section of this review considers the research reviews that discuss findings across the five phases described previously. We note that reviews have either been general reviews of career assistance or focused on a certain population/choice point, but no review has been made on a specific career intervention due to a limitation in the number of studies

Review selection and inclusion

We found 23 substantive reviews of career assistance research. In the present review process, we excluded annual reviews from the Journal of Vocational Behavior, Career Development Quarterly, and Journal of Career Assessment that covered only a single year. We did not include reviews found in textbooks, though some of the insights found in textbook sources are appropriate for discussion (e.g., Blustein, 2006; Herr, Cramer, & Niles, 2004; Swanson & Fouad, 2009). Annual Reviews of Psychology that reviewed specific career interventions were included as they typically covered 2 years. Reviews prior to 1970 were judged to be more general in scope (general counseling) and limited in sampling as to be of minimal value now, though a close review of the studies included in these early reviews suggests that while less sophisticated they were “experimental” in nature and useful in understanding how traditional clients respond to interventions (Myers, 1971). Twenty-three reviews of career assistance research were included after meeting inclusion criteria.

We did not conduct an exhaustive review covering all studies. Rather, we conducted a review of reviews, referring to individual studies where they informed the conclusions or recommendations in a review. Some reviewers have noted that the career assistance literature overall is not large and roughly consist of between 60 and 70 studies with only a few comparing individual career counseling to a suitable control condition (Whiston, 2002). Thus, overly specific recommendations for practice seem premature at this point. It is possible that another exhaustive or meta-analytic review would be indicated in the future with the accumulation of a significant number of additional rigorous process/outcome studies. However, we believe that what is needed now is careful attention to practical considerations in the design, cost analysis, prototyping, and adaptation of rigorous strategies of career assistance to efficaciously address the needs of vulnerable subgroups that face potent constraints while making career decisions.

Table 1 is a summary of included reviews of research on career assistance from 1970 to 2014. The reviews in this general category overlap to a considerable extent since new studies are sometimes combined with or considered in the context of previous studies. Additionally, more than one review has been written by the same author(s) resulting in some duplication in conclusions and recommendations (see Table 1 for summary of reviews in chronological order).

Table 1.
Conclusions and Recommendations From 23 Reviews of Career Intervention and Assistance From 1970 to 2014.

Authors/date Review Method ES Conclusions Recommendations for Research and/or Practice

Myers (1971) Lit — Array of procedures of tentatively established worth under controlled conditions, including using prior self-estimates before test results, pretreatments before counseling, reinforcement of specific constructive behaviors, and structured treatment content Utilize client attribute differences—i.e., developmental status; don’t group all treatments together to assess effects; customize outcome indicators, vary treatment content; treatment and control groups with equal motivation

Super and Hall (1978) Lit — Much more research needs to be done Utilize correlational and longitudinal studies to inform “experimental” interventions. Utilize behavioral measures of exploratory behavior and decision making; intervene by developmental level; specific training for the “hard to employ,” compare different treatments

Krumboltz, Becker-Haven, and Burnett (1979) Lit — Students with different decision types may need different treatments; decision-making skills can be increased by administering inventories, presenting as “experts,” retaining external control with clients low on internal locus of control, using computer-based systems, using modeling and guided practice, teaching anxiety management and problem-solving skills, provide role models, provide occupational and sex-role myth-destroying information, reinforce “congruent” choices, and give accurate self-knowledge feedback; no clear relationship between counseling and career maturity, and between counseling and success or satisfaction Use external or objective measures rather than exclusively self-report. Improve research designs to minimize confounding

Holland, Magoon, and Spokane (1981) Lit — Number and variety of treatments have increased; beneficial effects are due to the following common elements: (a) exposure to occupational information; (b) cognitive rehearsal of aspirations; (c) acquisition of cognitive structure for organizing information about self, occupations, and their relations; and (d) social support or reinforcement from counselors or workshop members. Positive outcomes hold for a wide range of evaluations, populations, and designs Rigorous evaluations of all forms of vocational intervention; client goals linked to treatments; comprehensive knowledge of client treatment interactions and related outcomes; develop more potent treatments; ordering effects of integrated treatment chains across time. Study interventions to overcome barriers and biases rather than documenting origins and perpetuation. Isolate critical occupational information and study differential presentation and influence; careful planning and organization of comprehensive treatments

Fretz (1981) Lit — Combination treatments more effective and lack of differences among treatments notable Vary treatment parameters; incorporate attribute-treatment interactions; conduct cost–benefit analysis; employ multiple treatments, treatment levels, and outcomes

Spokane and Oliver (1983) Meta .85/52/6,701 unweighted Treated clients better off than 80% of controls; group/class clients better off than 87% of controls but ES was affected by high outliers in class interventions; individual clients better off than 81% of controls. Interventions may improve symptom relief more often than role functioning. Differences in opinion exist in how to further strengthen treatments—either ATIs or stronger treatments Utilize real clients not volunteers and random assignment; use multiple outcome criteria; use no-treatment control and check whether received assistance elsewhere; assess client parameters and characteristics and also social and economic contexts; improve reporting and check for errors. Make raw data available

Baker and Popowicz (1983) Meta .50/18/- unweighted Average effects of controlled career education interventions are “medium.” Interventions are effective enough to warrant continued use New treatments with similar content should have positive effects

Rounds and Tinsley (1984) Lit — Career intervention is a form of psychotherapy designed to affect vocational feelings, attitudes, cognitions, and behaviors; range of potential interventions unlimited; reviews suggest that career interventions do have modest effects, but we do not understand how they do this Examine behavior change process and theory underlying interventions—do not assume uniformity of behavior change underlying different interventions: Utilize dismantling or constructing strategy to isolate effective treatment parameters; utilize diagnostic categories for treatment assignment: specify target behaviors derived from vocational problems. Use wait list or “best available” control

Myers (1986) Lit — A variety of interventions increase information-seeking, client motivation, locus of control, and people/task orientation influence response; decision readiness can be increased with counseling; over aspirers do better in group interventions while under aspirers do better in individual interventions Prior interventions were weak and short term; shift to programmatic approaches utilizing training interventions; incorporate measures of decision making and role functioning

Oliver and Spokane (1988) Meta .48/58/7,311 weighted by sample size Career counseling has generally positive effects; individual counseling produced more client gain per hour (or session) than any other intervention mode. Intensity of treatment was significant contributor to outcome magnitude. Clear differences in effectiveness among intervention modes Use diagnostic indicators to tailor treatment; use real clients; use blocking variables with self-esteem sociability and goal instability; follow up with dropouts

Blustein and Spengler (1995) Lit — Career counseling and psychotherapy closely related; ES for personal adjustment outcomes of career intervention = .58. There is counselor bias against career counseling Study integrative interventions that address cross-domain roles; there is overlap between career interventions and psychotherapy in the change/gain mechanisms; include career outcomes in psychotherapy research; counselors should be trained to work across domains and career attitudes should be addressed early in supervision

Swanson (1995) Lit — Some counseling psychologists hold negative views of career counseling and may communicate them to clients; although there is difficulty in defining career counseling, the outcome data suggest that it is effective as measured by commonly used outcomes (e.g., maturity, identity, and realism); group and individual career counseling may be effective for different clients; career counseling process is still largely unexplored Examine aptitude by treatment interactions; compare equivalent treatments; refocus research on the human psychological interaction in career counseling; what interventions are not effective and for whom? Study what happens during career counseling; study process and outcome in tandem

Whiston, Sexton, and Lasoff (1998) .38/47/4,660 weighted by sample size Career interventions have moderate effects overall. Individual career counseling was found to be the most effective per hour of treatment, but were most expensive; whereas computer interventions were the most cost effective; effects of individual counseling occur in 2–3 sessions; type of treatment and intensity were not related to ES Utilize standard procedures for diagnosing clients; examine treatment integrity and employ manualized treatments; include more diverse populations; report data thoroughly

Brown and Ryan Krane (2000) Meta .34/62/7,725 Clients are 1/3 of a standard deviation better off than controls. Effectiveness dropped after 4–5 sessions. Combinations of ingredients associated with higher ES than any one alone. Adding 0, 1, 2, or 3 ingredients yielded ES of .22, .45, .61, .99, respectively. Five critical treatment parameters related to increased ES: Workbooks and written exercises emphasizing goals, future plans; individualized inventory interpretations and feedback; world of work information; modeling exposure; support building In training emphasize the importance of work roles and effectiveness evidence. Test effects of five critical ingredients on intervention outcome using dismantling. Incorporate client goals, needs, and ability as parameters. Investigate prematurely foreclosed options and attribute-treatment interactions using client variables such as barriers, identity development, race, sex, and SES and, particularly problem diagnostics

Brown et al. (2003) Meta — Random additive combinations of 14 noncritical ingredients did not result in linear increase in ES, for example, computer-guided assistance, cognitive restructuring, vocational exploration, outside reading, performance accomplishments Ensure that comparison conditions are comparable; adequate power for small treatment effects; help clients develop written goals for their future, add individualized counselor input; provide, in-session gathering/processing of occupational information; promote information outside of sessions; provide opportunities to compare options in writing; individual consultations for problematic assessment results; use successful models

Whiston, Brecheisen, and Stephens (2003) Meta -/57/4,732 Significant increase in counselor-free interventions; individual counseling more effective than computer intervention without counselor; computer intervention with counselor more effective than computer intervention alone; participant age not a moderator of ES; length of workshop not associated with ES Report data thoroughly, particularly participant ethnicity; need increase in studies of test interpretation and class interventions; vary parameters of similar treatment modalities and compare; need standard outcome battery

Miller and Brown (2005) Lit — Career interventions are modestly effective with ES around .34 and client variables that might lead to differential treatment not identified in studies Incorporate ability assessments in interventions; broaden theoretical bases underlying interventions to include more than trait theory; involve a counselor in computer-based interventions; tailor interventions based upon client goals and presenting concerns; attend to conditions underlying client choice difficulty; use multiculturally informed interventions

Whiston and Oliver (2005) Lit — Overall, ES between .30 and .60 or moderately effective for career counseling; individual counseling highly effective; counselor involvement important to gains; gains appear to be maintained up to 1 year; haphazard structure activities poorly evaluated; scant and inconsistent evidence of client and counselor differences in career counseling process or outcome; working alliance potent for some clients but not others Develop a standardized outcome/process battery; calculate cost-effectiveness; examine dose-response; develop a system of diagnosis for differential treatment; increase research on Internet and web-based career intervention; increase cross intervention and attribute/mode comparisons; find effective ways to incorporate occupational information

Brown and McPartland (2005) Lit — Career interventions for choice difficulty are generally effective (1/3 SD of gain); interventions can be more effective develop goals and options write and talk about them in session and engage in information search outside Replications needed; use most appropriate control condition; test inferences from more than trait theory as basis for efficacy; test what works for whom; improve treatments that don’t involve therapeutic intensity; utilize decisional status and type and/or client goals to vary treatment parameters; compare client characteristics (e.g., race gender, SES, and sexual orientation)

Whiston and Rahardja (2008) Lit — Significant variation in ES for various outcomes; larger effects for intermediate process outcomes such as maturity or identity, and smaller effects for distal outcomes. Input factors have minimal effects on outcomes—for example, client or counselor factors; optimum effects occur in 4–5 sessions; counselor-free interventions are ineffective; incorporate five critical factors Two critical ingredients, building support and modeling should be routinely integrated in interventions; counselors should be actively engaged in interventions to improve insight and exploration; incorporate a theoretical base; study what works for which client—including cultural difference; use dismantling and constructive research designs; emphasize costs and benefits, including larger cost implications such as major change or tuition costs

Whiston (2011) Lit — Employ role induction, working alliance, and five ingredients Treatment protocols should be adapted to address SES and marginalized subgroup barriers; better diagnostic systems

Whiston and James (2013) Lit — Career choice counseling moderately effective; long-term benefits exist; can be implemented in 4–5 sessions; client factors have inconsistent effects; use diagnostic measures and a standardized outcome battery; focus research on those in poverty and marginalized groups Incorporate five critical ingredients in combination, address client expectations, and involve a counselor

Whiston and Rose (2013) Lit — Emerging adults need career counseling; career counseling is generally effective; interventions should incorporate five critical ingredients; working alliance appears to enhance effects Develop treatment protocols to address barriers faced by low SES and racial minority persons, and women seeking STEM careers; study counselor behaviors when interpreting inventories

Note. SES = socioeconomic status; ES = effect sizes; STEM = science, technology, engineering, and math. “Lit” indicates a literary review and “Meta” indicates a meta-analytic review.

Studies of choice implementation have been reviewed by others separately (Jome & Phillips, 2005) or as a subsection of a more general review (Brown & McPartland, 2005). Evidence in this very crucial interventional domain is accruing more slowly but speaks even more directly to the potent effects of social constraints and barriers on career attainment, and the need to design specific interventions to overcome powerful contributors to career outcome disparities (Holland et al., 1981). Although choice implementation has been reviewed in the literature and requires additional attention, more focus will be placed on the facilitation of career choice in the current review.

Conclusions From Reviews of Research on the Effectiveness of Career Assistance

Each of the 23 reviews that we studied articulated some form of conclusion regarding the effectiveness of career assistance. Utilizing the entries from Table 1, we state five conclusions that we believe to be reasonable and were articulated in multiple reviews, polls, or studies, particularly where they highlighted a different aspect, strength, or limitation regarding the effects of career assistance. We also delineate five specific recommendations for improvements in methodology to address critical issues in the design and conduct of evaluations of career assistance.

Representative national polls consistently underscore an increasing desire for career assistance. In a recent National Career Development Association (NCDA) Harris Poll, more than 8 of the 10 respondents actually seeking assistance found that career assistance was helpful across developmental levels (NCDA, 2011). The need was particularly acute among African Americans, Hispanic Americans, and individuals with less than a high school diploma—one third of Hispanic and African American respondents reported the need for career assistance. Professional effort should be directed to vulnerable population subgroups in need of assistance and to individuals facing challenging internal and external sociocultural barriers and constraints that limit exploration and range of choice (Byars-Winston, Kamtamneni, & Mobley, 2012). Similarly, Balfanz and Letgers (2004) found that high schools differ dramatically in their ability to move students forward across grade levels or “promotion power.” Balfanz and Letgers (2004) concluded that half of the African American students and 40% of Latino students are in schools where graduation is not a norm—compared with only 11% of White students. We know where educational and vocational problems and failures exist. We need to know much more about how to effectively provide career assistance in vulnerable settings.

Meta-analytic reviews of controlled career intervention studies suggest that treated participants report posttest scores that indicate that they are better off than 60%, to 67% of control participants following intervention. These percentages are conservative estimates but are consistent with sample size weighted effect sizes (ES) of .30. Interventions by a counselor that included multiple treatment parameters found larger ES differences between treatment and control participants approaching ES = .50 (Brown & McPartland, 2005; Whiston & Oliver, 2005; Whiston & Rahardja, 2008). These outcomes have been termed tentative, modest, or generally effective in 9 of the 23 reviews (Brown & McPartland, 2005; Fretz, 1981; Holland et al., 1981; Krumboltz, Becker-Haven, & Burnett, 1979; Miller & Brown, 2005; Myers, 1971, 1986; Oliver & Spokane, 1988; Rounds & Tinsley, 1984; Spokane & Oliver, 1983; Swanson, 1995; Whiston & Rose, 2013), and moderate or medium in magnitude in 5 of the 23 reviews (Baker & Popowicz, 1983; Whiston et al., 1998; Whiston & James, 2013; Whiston & Oliver, 2005). Reviewers noted that intervention effects seem to drop off after anywhere from 2 to 5 sessions (Whiston et al., 1998; Whiston & James, 2013; Whiston & Rahardja, 2008), though we note that most attempts at assistance are brief and nonsequential.

Five reviewers noted that somewhere between 5 and 7 key ingredients or combinations of treatments and content were more effective (Brown et al., 2003; Brown & Ryan Krane, 2000; Fretz, 1981; Holland et al., 1981; Whiston et al., 2003). Several of the reviews also noted, however, the absence of differences among treatments or in attributes of the client or length of intervention in the magnitude of outcome. Although most reviewers noted beneficial effects across outcomes, consistent differences in outcomes appear to be in symptom relief and self-report measures such as decisional status, identity or information seeking, but less so in more stringent or distal role functioning measures.¹

Most meta-analytic reviews reflect the absence of controls or inadequately reported data in included studies (usually only 60–70 controlled investigations). Further, the number of studies that can be included is disappointingly small. Given the lack of sophistication and control in study design (e.g., equating time, effort, and expectation for gain expended in control condition with treatment condition), few compelling causal links have been demonstrated between specific therapeutic element or set of elements and outcomes other than client self-report or independent of demand characteristics.

Controlled studies of career intervention that are included in meta-analytic reviews are generally short term and do not include multiple-chained interventions over longer time periods (more than one semester of a class, or 4–5 sessions, or as a follow-up [1–3 years]). Studies are disproportionately one-time interventions with no additional intervening treatments and often one-shot instances of relatively weak treatments and only rarely conducted under field conditions.

The various forms, techniques, strategies, paradigms, and models of career assistance are neither clearly defined nor fully manualized, thus, increasing the difficulty in estimating their efficacy and in making analytic comparisons among various strategies. Outcome measures are largely self-report in nature and do not reflect the full range of possible and observable outcomes (Brickell, 1976; Oliver, 1979; Oliver & Spokane, 1982). The lack of manualized and clearly defined strategies and techniques makes cross comparisons unreliable.

Recommendations for Strengthening the Efficacy of Career Assistance

Each of the reviews suggested ways to improve the rigor and methodology in research studies evaluating the efficacy of career assistance. Among those frequently mentioned were the use of more realistic and more comparable control conditions, the use of a standard battery of diagnostic and outcome measures, including more use of behavioral and cost measures and external barriers and impediments. Several recommendations emphasized more design rigor, including dismantling and analytic designs, attribute-treatment designs—that is, what works for whom—hybrid designs using process and outcome measures, replications, cost–benefit analyses, and treatment parameter variations. Other frequent recommendations were more manualized treatment protocols, incorporation of client goals, and more use of “tailored” treatment protocols. Some improvement has been observed in methodology, but none of the methodologies in any of the studies included in the reviews approaches the rigor level of a clinical trial. Craighead and Dunlop (2014) define an adequate form of treatment for a clinical trial—and adequacy in their view means state-of-the-art treatments or combinations of treatments designed based upon analysis of the findings from previous studies.

Evaluating Career Assistance Where it Occurs

We may be approaching empirically supported consensus treatments, but they are not widely shared yet, nor well understood, and a set of studies utilizing consensus treatments must be a priority for future efforts. Increasing our emphasis on how career assistance is designed and delivered in routine practice may yield significant gains. We should not, as Hagermoser Sanetti and Kratochwill (2014) have argued, assume that the assistance strategies we believe are in use are actually being delivered as intended. Our overall recommendation is that in view of the substantial need for services and the limits on available resources, a significant change in interventive paradigms is needed in addition to a refining of the traditional process outcome paradigm.

One potential avenue for strengthening future career assistance is the careful study of the degree to which interventions that we believe to be effective under optimal circumstances (traditional clients in controlled settings) can actually be implemented consistently on a larger scale in schools, communities, and workplaces. Called variously treatment integrity, procedural fidelity, or implementation integrity (Hagermoser Sanetti & Kratochwill, 2014), these terms are the foundation of an emerging science of interventional implementation. Treatment integrity addresses what is needed with respect to adapting, adjusting, and revising empirically valid and manualized treatments to ensure their effectiveness as interventions in field settings. Integrity checks and detailed records of any variations or adaptations done to adjust a treatment in a field setting are the hallmark of treatment integrity research. Sharply increased attention to intervention methods (e.g., drift checks, integrity checks, adverse events, etc.) would strengthen our research considerably.

Another likely route to strengthening the outcomes of career assistance will be through serial, integrated treatment instances or chains, graded in intensity and focus. Outcomes should emphasize behavioral and attitudinal targets at multiple decision points. A chained intervention paradigm is consistent with Tiedeman’s notion (Tiedeman & O’Hara, 1963) that there are key decision points wherein individuals must confront constraints and barriers to choice in strategic ways (e.g., choice of math and science courses in middle school and choice of first job in adolescence). A treatment chain paradigm suggests not only that interventions might need to occur at several of these points but also that cumulative data of a theoretic and behavioral nature should be gathered and stored in order to make rational decisions in collaboration with the client about the course and goals of intervention (target, structure, support, intensity, etc.). The study of isolated one point-in-time or limited short-term contacts in which outcomes are linked only to the specific intervention are of limited value at this time in our field. Such a view is consistent with Osipow’s (1983) reasoning that significant vocational choices are preceded and followed by several series of collateral decisions and choices (imagine highly dense branching systems) made in an iterative manner over extended periods—some small and seemingly trivial—others larger and more significant. Each of these serial choices is made in a context involving multiple influences and each decision shapes both the next choice to be made, and later decisions and outcomes (Myers, 1971), and may therefore be amenable to intervention. Future studies should be directed toward treatment intensity-related investigations of the degree and extent of interventional effort needed to result in desired career outcomes. To illustrate, Rayman, Bernard, Holland, and Barnett (1983) showed how clients who did not respond to traditional treatments showed impressive gains when intervention efforts were intensified.

The response to intervention (RTI) paradigm utilized by educational and school psychology colleagues (Elliot, 2008) may contribute to our understanding about how to adapt and adjust assistance in response to the characteristics of the client and the extent of operative constraint. RTI monitors progress and intensifies or adapts an intervention until a desired outcome is achieved. This recommendation is similar to the concept of “prototyping” in product development. Before a product is released for sale to the public, multiple prototypes or models are developed and both beta tested and marketed to the populations of intended use. Several prototypes may be tested and hundreds of decisions required before a final product is introduced (Krishnan & Ulrich, 2001). Substantial resources are required for this level of product development, yet we may benefit from rapid prototyping of interventions—for example, 2.0, 2.1, 2.2, until we clearly understand what treatments work and what fails in the face of external constraints and the specifics of how to adjust interventions that fail until they work—particularly in natural field settings.

Although Perry, Dauwalder, and Bonnett (2009) expressed concern about the cost of conducting large-scale, particularly school-based interventions, there is evidence that although comprehensive interventions may have more costs, these interventions are typically more cost effective than individual or group interventions by a factor of 2 (i.e., 10.87 vs. 20.69), when amortized over the number of students reached and hours of professional effort expended (Oliver & Spokane, 1988). Further, such programs, are consistent with Federal and State educational agendas and, therefore, more likely to receive funding (Turner & Lapan, 2013). This appears to happen regularly in efforts to encourage “STEM” careers (Hartung, 2010) and in “Transition From School to Work” (Blustein et al., 2002), priorities for the United States and other industrialized nations, where we are relevant, essential, and supportable. When we engage in such research related to national priorities, we may not be engaging in controlled studies, but we can make progress, albeit in a roundabout manner, in the science of career intervention. Although these projects are more similar to program evaluation efforts than traditional process and outcome studies, they yield valuable information about service delivery in field settings. Further, as Krumboltz, Becker-Haven, and Burnett (1979) suggested, progress need not come from a single study—some pieces or nuggets of understanding as Krumboltz called them can be collected and applied if we read carefully. Extracting wisdom from these more diverse projects, stimulating colleagues to attempt further efforts, enhancing the efficacy of career interventions, and formulating science-based models from these efforts are our biggest challenges. There is much to be done in understanding how to most effectively deliver STEM and transition services and how these focused efforts address the needs of vulnerable subgroups (Juntenen, Ali, & Pietrantonio, 2013; U.S. General Accounting Office, 1991). The extent to which assistance in these areas corresponds with essential implementation paradigms that would advance both foundational theory and intervention practice is what we should study. Although the present review does not specifically attempt to evaluate STEM and transition studies, the issues raised in conducting field-based interventions are evident throughout the reviews we examined.

Why We Don’t Find Attribute Treatment Interactions

It is a challenge to explain why participant differences (Dawis, 1992) have not been found to moderate outcomes from assistance interventions. One possibility is that individual differences are less compelling than external constraints and familial resources as moderators. We may need to adjust levels of assistance until they exceed or overcome external constraints—perhaps in an iterative chain of interventions—while at the same time working to minimize institutional constraints acting on vulnerable groups. In reading these 23 reviews of research, we could not locate a study or a review directly comparing intervention strategies or assistance models for individuals with substantially different economic and social resources and support (Lopez, Concepcion, Kopelowicz, & Vega, 2012). We believe that one reason that demographic and diagnostic moderator variables have not been shown to be related to treatment efficacy in career assistance is that the most potent moderators have not been systematically investigated. It is our contention that sociocultural and economic disparity is one most likely moderator of treatment efficacy, as are age, and a weighted combination of depressive and anxiety status. Thus, younger, more vulnerable clients necessarily need more potent and well-structured assistance. This effect was hinted at in Azrin’s studies of the job club (Azrin, Flores, & Kaplan, 1975), in which powerful gains in reducing length of unemployment, amount of benefits taken, and time to a job, and so on, were consistently found with a vulnerable and underresourced populations. In addition to design problems, the study was cast in a behavioral frame, thus, why or how questions were addressed. Similarly, Santisteban et al. (2003) added to a consistent finding that structured brief approaches are generally successful with Hispanic adolescents with drug problems, a highly vulnerable group. Commonly employed metrics of socioeconomic status (SES) and race presently do not adequately tap disparity and should be supplanted by more sophisticated measures of sociocultural disparity in career studies. To illustrate, Jaeger (2012) concluded that resources in the extended family compensate for lacking resources in low-SES families, which in turn promote children’s educational success. Thus, a simple measure of SES may mask crucial differences in available social and economic support. Likewise, simple categories of race or ethnicity absent identification and acculturation data are necessarily oversimplified. Further, treatment parameters should be varied systematically in studies using composite indices of social, cultural, and economic disparity that accurately assess the client’s access to resources as moderator and mediator variables. It is possible, as well, that some as yet undiscovered set of moderators (e.g., neurobehavioral differences) would demonstrate significant attribute-treatment interactions (Craighead & Dunlop, 2014; Gazzaniga, 2013). Systemic differences and the constraints they impose, rather than individual differences, may moderate specific assistance strategies.

Communicating Findings Effectively to Policy Makers and Funding Entities

Career assistance outcomes to date are considered to be modest by most reviewers, yet we agree with Brown and colleagues (2003) that modest ES can translate to important gains when extrapolated to the general population. We speculate that more potent interventions when carefully structured, prototyped, and adapted on the basis of social disparity and lack of social and economic resources will be more than modestly effective (see, e.g., Azrin et al., 1975). In the interim, however, the nature of the statistics employed in career intervention studies may make translation of research findings and communication with relevant funding sources more difficult and beneficial consequences less obvious. In sum, current research statistics are modest, yet hopeful, but statistical analyses as they are currently utilized make outcomes less obvious and obtaining funding resources difficult.

Lapan, Turner, and Pierce (2012) argue persuasively that the United States needs a standing policy that implements career development research for college and career readiness. How can we best inform policy? Although Sampson, McClain, Musch, and Reardon (2011) make a strong case for the accessibility to career assistance as a social justice concern that cannot be ignored, we believe that any case will be more convincing when supported by data to underscore social justice concerns. Policy makers may be influenced by significant results from outcome studies or intervention programs, but not readily grasp the implications of findings that are reported in terms of means and SDs, p values, ES, variance accounted for, and the like. These significance estimates do not clearly state potential consequences or separate risk to the individual from risk to the population (Scott, Mason, & Chapman, 1999). Epidemiologists have been utilizing outcome indices in health research in ways that communicate both the benefits of intervention and the risks on inaction (Brown et al., 2009; Mason, Scott, Chapman, & Tu, 2000; Scott et al., 1999; Szapocznik et al., 2006). Rather than calculating regression or correlation coefficients, or variance accounted for, three statistics commonly used by epidemiologists and which might be usefully applied to career intervention are (a) the risk ratio, (b) the odds ratio, and (c) the population attributable risk. The risk ratio is “the relative increase or decrease in the probability of a given outcome for an individual when one rather than another condition is true” (Scott et al., 1999, p. 1265). Risk ratios are calculated by dividing the risk associated with one condition—for example, socially vulnerable versus non-vulnerable (social and economic disparity) divided by the risk associated with the other. A ratio of one (equal risk) indicates no comparative risk. If a risk ratio is not possible to calculate, an alternative epidemiological statistic is an odds ratio, “the odds are the probability that an event, typically an undesired or negative outcome, will occur, divided by the probability that it will not occur” (Scott et al., 1999, p. 1266). The odds ratio is after the fact calculation of the increased odds of a deleterious outcome for individuals predisposed by disparity. Odds ratios are calculated by sampling individuals based upon the occurrence of a negative outcome and the history of an individual with a positive outcome is compared to an individual with a negative outcome. Reporting research results as a risk ratio, odds ratio, or population attributable risk may better inform policy makers about the importance of career assistance.

Although the risk ratio and the odds ratio are similar in that they are measures of the relative risk to the individual (probability of an outcome vs. the odds of an outcome), the population attributable fraction or community-level risk is an estimate of the danger to the larger community and, conversely, the benefit of intervention. A population attributable fraction makes a compelling case when the effects are amortized over greater numbers and particularly when the social and economic consequences—that is, the cost of a negative outcome at the macro level—are both estimable and graphic, a population attributable fraction may be better, as it provides data on the potential risk to larger numbers of people.

The issue addressed in these calculations is that a problem that looms large for an individual may be less consequential on a macro level than a problem that is somewhat less severe for an individual but is widespread in the community and, therefore, has more negative impact overall. An example of this problem might be the effect on an individual’s eventual career attainment who is laid off from work and develops a serious mood disorder. Mental health problems are likely and serious (large) for the individual involved and may compromise attainment in the long term, but the instance of such a case may be relatively rare, when compared to the effects of high school dropout on level of career attainment—a commonly occurring issue which among African American adolescents approaches 50% (Balfanz & Letgers, 2004). Although arguably smaller in magnitude for an individual, dropout is much more widespread and therefore potentially more consequential to the community as a whole as compared to mood disorders at work.

A hypothetical example of the population attributable fraction in career is illustrated in Table 2 (adapted from Scott et al., 1999). If we knew, for illustrative purposes,² that comparing, ex post facto, prolonged job layoff instances among 100 fourteen-year-old boys from single-parent homes and 100 fourteen-year-old boys from two-parent homes, then we can calculate the population attributable fraction as 31.26% (see Table 2 for calculations). Thus, 31.26% of the cases of job loss in the population are related to the adolescent coming from a single-parent family. If the probability of job loss for youth born in single-parent homes were the same as the probability for youth in dual parent homes, the number of cases of job loss in the community would be reduced by 31.26%. It is then possible to calculate the social and economic costs and benefits of addressing or ignoring the needs of adolescent males from single-parent families—in dollar terms for unemployment benefits, lost tax dollars, and so on.

Table 2.
Hypothetical Illustration of Population Attributable Fraction for 200 Fourteen-Year-Old Males From Single-Parent or Two-Parent Families (After Scott et al., 1999).

Laid Off (N = 100) Not Laid Off (N = 100)

Single-parent home 42 (a) 21 (b)

Two-parent home 58 (c) 79 (d)

$\frac{a / b}{c / d} = \frac{42 / 21}{58 / 79} = 2.72$ = odds ratio, $\frac{P e (O R - 1)}{P e (O R - 1) + 1} \times 100 %$ = population attributable fraction, $\frac{.21 (2.72 - 1)}{.21 (2.72 - 1) + 1} \times 100 % = 26.58 (31.26) %$

The Future of Career Assistance

We believe that future prospects for career intervention research and practice depend, not only upon increasing the number, rigor, and analytic capability of the next phase of research investigations but also upon our ability to communicate the importance of career interventions as well as the human and economic costs of failure to intervene. A broadened research paradigm and change in direction in intervention strategy is warranted both as a logical conclusion from previous reviews and as a necessary step to sustain vigor in this core area of counseling practice and research. Altering our research focus from the individual outcome and process to the relative individual and community risk for a problem, cost of nonintervention and benefits of early intervention could substantially improve our ability to communicate our relevance to policy makers and funders.

In sum, we have reviewed research reviews of controlled studies of career assistance and consolidated conclusions and recommendations from these reviews. We have suggested alternative paradigms for constructing and evaluating career assistance while more carefully designed analytical studies accumulate to justify more specific findings. In addition, we believe that a shift in interventive paradigms to the community level would be productive and lead to increased public interest and funding (Bumbarger, 2014).

Authors/date	Review Method	ES	Conclusions	Recommendations for Research and/or Practice
Myers (1971)	Lit	—	Array of procedures of tentatively established worth under controlled conditions, including using prior self-estimates before test results, pretreatments before counseling, reinforcement of specific constructive behaviors, and structured treatment content	Utilize client attribute differences—i.e., developmental status; don’t group all treatments together to assess effects; customize outcome indicators, vary treatment content; treatment and control groups with equal motivation
Super and Hall (1978)	Lit	—	Much more research needs to be done	Utilize correlational and longitudinal studies to inform “experimental” interventions. Utilize behavioral measures of exploratory behavior and decision making; intervene by developmental level; specific training for the “hard to employ,” compare different treatments
Krumboltz, Becker-Haven, and Burnett (1979)	Lit	—	Students with different decision types may need different treatments; decision-making skills can be increased by administering inventories, presenting as “experts,” retaining external control with clients low on internal locus of control, using computer-based systems, using modeling and guided practice, teaching anxiety management and problem-solving skills, provide role models, provide occupational and sex-role myth-destroying information, reinforce “congruent” choices, and give accurate self-knowledge feedback; no clear relationship between counseling and career maturity, and between counseling and success or satisfaction	Use external or objective measures rather than exclusively self-report. Improve research designs to minimize confounding
Holland, Magoon, and Spokane (1981)	Lit	—	Number and variety of treatments have increased; beneficial effects are due to the following common elements: (a) exposure to occupational information; (b) cognitive rehearsal of aspirations; (c) acquisition of cognitive structure for organizing information about self, occupations, and their relations; and (d) social support or reinforcement from counselors or workshop members. Positive outcomes hold for a wide range of evaluations, populations, and designs	Rigorous evaluations of all forms of vocational intervention; client goals linked to treatments; comprehensive knowledge of client treatment interactions and related outcomes; develop more potent treatments; ordering effects of integrated treatment chains across time. Study interventions to overcome barriers and biases rather than documenting origins and perpetuation. Isolate critical occupational information and study differential presentation and influence; careful planning and organization of comprehensive treatments
Fretz (1981)	Lit	—	Combination treatments more effective and lack of differences among treatments notable	Vary treatment parameters; incorporate attribute-treatment interactions; conduct cost–benefit analysis; employ multiple treatments, treatment levels, and outcomes
Spokane and Oliver (1983)	Meta	.85/52/6,701 unweighted	Treated clients better off than 80% of controls; group/class clients better off than 87% of controls but ES was affected by high outliers in class interventions; individual clients better off than 81% of controls. Interventions may improve symptom relief more often than role functioning. Differences in opinion exist in how to further strengthen treatments—either ATIs or stronger treatments	Utilize real clients not volunteers and random assignment; use multiple outcome criteria; use no-treatment control and check whether received assistance elsewhere; assess client parameters and characteristics and also social and economic contexts; improve reporting and check for errors. Make raw data available
Baker and Popowicz (1983)	Meta	.50/18/- unweighted	Average effects of controlled career education interventions are “medium.” Interventions are effective enough to warrant continued use	New treatments with similar content should have positive effects
Rounds and Tinsley (1984)	Lit	—	Career intervention is a form of psychotherapy designed to affect vocational feelings, attitudes, cognitions, and behaviors; range of potential interventions unlimited; reviews suggest that career interventions do have modest effects, but we do not understand how they do this	Examine behavior change process and theory underlying interventions—do not assume uniformity of behavior change underlying different interventions: Utilize dismantling or constructing strategy to isolate effective treatment parameters; utilize diagnostic categories for treatment assignment: specify target behaviors derived from vocational problems. Use wait list or “best available” control
Myers (1986)	Lit	—	A variety of interventions increase information-seeking, client motivation, locus of control, and people/task orientation influence response; decision readiness can be increased with counseling; over aspirers do better in group interventions while under aspirers do better in individual interventions	Prior interventions were weak and short term; shift to programmatic approaches utilizing training interventions; incorporate measures of decision making and role functioning
Oliver and Spokane (1988)	Meta	.48/58/7,311 weighted by sample size	Career counseling has generally positive effects; individual counseling produced more client gain per hour (or session) than any other intervention mode. Intensity of treatment was significant contributor to outcome magnitude. Clear differences in effectiveness among intervention modes	Use diagnostic indicators to tailor treatment; use real clients; use blocking variables with self-esteem sociability and goal instability; follow up with dropouts
Blustein and Spengler (1995)	Lit	—	Career counseling and psychotherapy closely related; ES for personal adjustment outcomes of career intervention = .58. There is counselor bias against career counseling	Study integrative interventions that address cross-domain roles; there is overlap between career interventions and psychotherapy in the change/gain mechanisms; include career outcomes in psychotherapy research; counselors should be trained to work across domains and career attitudes should be addressed early in supervision
Swanson (1995)	Lit	—	Some counseling psychologists hold negative views of career counseling and may communicate them to clients; although there is difficulty in defining career counseling, the outcome data suggest that it is effective as measured by commonly used outcomes (e.g., maturity, identity, and realism); group and individual career counseling may be effective for different clients; career counseling process is still largely unexplored	Examine aptitude by treatment interactions; compare equivalent treatments; refocus research on the human psychological interaction in career counseling; what interventions are not effective and for whom? Study what happens during career counseling; study process and outcome in tandem
Whiston, Sexton, and Lasoff (1998)		.38/47/4,660 weighted by sample size	Career interventions have moderate effects overall. Individual career counseling was found to be the most effective per hour of treatment, but were most expensive; whereas computer interventions were the most cost effective; effects of individual counseling occur in 2–3 sessions; type of treatment and intensity were not related to ES	Utilize standard procedures for diagnosing clients; examine treatment integrity and employ manualized treatments; include more diverse populations; report data thoroughly
Brown and Ryan Krane (2000)	Meta	.34/62/7,725	Clients are 1/3 of a standard deviation better off than controls. Effectiveness dropped after 4–5 sessions. Combinations of ingredients associated with higher ES than any one alone. Adding 0, 1, 2, or 3 ingredients yielded ES of .22, .45, .61, .99, respectively. Five critical treatment parameters related to increased ES: Workbooks and written exercises emphasizing goals, future plans; individualized inventory interpretations and feedback; world of work information; modeling exposure; support building	In training emphasize the importance of work roles and effectiveness evidence. Test effects of five critical ingredients on intervention outcome using dismantling. Incorporate client goals, needs, and ability as parameters. Investigate prematurely foreclosed options and attribute-treatment interactions using client variables such as barriers, identity development, race, sex, and SES and, particularly problem diagnostics
Brown et al. (2003)	Meta	—	Random additive combinations of 14 noncritical ingredients did not result in linear increase in ES, for example, computer-guided assistance, cognitive restructuring, vocational exploration, outside reading, performance accomplishments	Ensure that comparison conditions are comparable; adequate power for small treatment effects; help clients develop written goals for their future, add individualized counselor input; provide, in-session gathering/processing of occupational information; promote information outside of sessions; provide opportunities to compare options in writing; individual consultations for problematic assessment results; use successful models
Whiston, Brecheisen, and Stephens (2003)	Meta	-/57/4,732	Significant increase in counselor-free interventions; individual counseling more effective than computer intervention without counselor; computer intervention with counselor more effective than computer intervention alone; participant age not a moderator of ES; length of workshop not associated with ES	Report data thoroughly, particularly participant ethnicity; need increase in studies of test interpretation and class interventions; vary parameters of similar treatment modalities and compare; need standard outcome battery
Miller and Brown (2005)	Lit	—	Career interventions are modestly effective with ES around .34 and client variables that might lead to differential treatment not identified in studies	Incorporate ability assessments in interventions; broaden theoretical bases underlying interventions to include more than trait theory; involve a counselor in computer-based interventions; tailor interventions based upon client goals and presenting concerns; attend to conditions underlying client choice difficulty; use multiculturally informed interventions
Whiston and Oliver (2005)	Lit	—	Overall, ES between .30 and .60 or moderately effective for career counseling; individual counseling highly effective; counselor involvement important to gains; gains appear to be maintained up to 1 year; haphazard structure activities poorly evaluated; scant and inconsistent evidence of client and counselor differences in career counseling process or outcome; working alliance potent for some clients but not others	Develop a standardized outcome/process battery; calculate cost-effectiveness; examine dose-response; develop a system of diagnosis for differential treatment; increase research on Internet and web-based career intervention; increase cross intervention and attribute/mode comparisons; find effective ways to incorporate occupational information
Brown and McPartland (2005)	Lit	—	Career interventions for choice difficulty are generally effective (1/3 SD of gain); interventions can be more effective develop goals and options write and talk about them in session and engage in information search outside	Replications needed; use most appropriate control condition; test inferences from more than trait theory as basis for efficacy; test what works for whom; improve treatments that don’t involve therapeutic intensity; utilize decisional status and type and/or client goals to vary treatment parameters; compare client characteristics (e.g., race gender, SES, and sexual orientation)
Whiston and Rahardja (2008)	Lit	—	Significant variation in ES for various outcomes; larger effects for intermediate process outcomes such as maturity or identity, and smaller effects for distal outcomes. Input factors have minimal effects on outcomes—for example, client or counselor factors; optimum effects occur in 4–5 sessions; counselor-free interventions are ineffective; incorporate five critical factors	Two critical ingredients, building support and modeling should be routinely integrated in interventions; counselors should be actively engaged in interventions to improve insight and exploration; incorporate a theoretical base; study what works for which client—including cultural difference; use dismantling and constructive research designs; emphasize costs and benefits, including larger cost implications such as major change or tuition costs
Whiston (2011)	Lit	—	Employ role induction, working alliance, and five ingredients	Treatment protocols should be adapted to address SES and marginalized subgroup barriers; better diagnostic systems
Whiston and James (2013)	Lit	—	Career choice counseling moderately effective; long-term benefits exist; can be implemented in 4–5 sessions; client factors have inconsistent effects; use diagnostic measures and a standardized outcome battery; focus research on those in poverty and marginalized groups	Incorporate five critical ingredients in combination, address client expectations, and involve a counselor
Whiston and Rose (2013)	Lit	—	Emerging adults need career counseling; career counseling is generally effective; interventions should incorporate five critical ingredients; working alliance appears to enhance effects	Develop treatment protocols to address barriers faced by low SES and racial minority persons, and women seeking STEM careers; study counselor behaviors when interpreting inventories

	Laid Off (N = 100)	Not Laid Off (N = 100)
Single-parent home	42 (a)	21 (b)
Two-parent home	58 (c)	79 (d)
$\frac{a / b}{c / d} = \frac{42 / 21}{58 / 79} = 2.72$ = odds ratio, $\frac{P e (O R - 1)}{P e (O R - 1) + 1} \times 100 %$ = population attributable fraction, $\frac{.21 (2.72 - 1)}{.21 (2.72 - 1) + 1} \times 100 % = 26.58 (31.26) %$

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Azrin

N. H.

Flores

Kaplan

S. J.

(1975). Job-finding club: A group-assisted program for obtaining employment. Behaviour Research and Therapy, 13, 17–27. doi:10.1016/0005-7967(75)90048-0

Babcock

R. J.

Kaufman

M. A.

(1976). Effectiveness of a career course. Vocational Guidance Quarterly, 24, 261–266. doi:10.1002/j.2164-585X.1976.tb00932.x

Baker

S. B.

Popowicz

C. L.

(1983). Meta-analysis as a strategy for evaluating effects of career education interventions. Vocational Guidance Quarterly, 31, 178–186.

Balfanz

Letgers

(2004). Locating the dropout crisis: Which high schools produce the nation’s dropouts? Where are they located? Who attends them? Baltimore, MD: Center for Research on the Education of Students Placed At Risk, Publications Department, Johns Hopkins University.

Bergland

B. W.

Lundquist

G. W.

(1975). The Vocational Exploration Group and minority youth: An experimental outcome study. Journal of Vocational Behavior, 7, 289–296. doi:10.1016/0001-8791(75)90070-6

Blustein

D. L.

(2006). The psychology of working: A new perspective for career development, counseling and public policy. Mahwah, NJ: Lawrence Erlbaum.

Blustein

D. L.

Chaves

A. P.

Diemer

M. A.

Gallagher

L. A.

Marshall

K. G.

Sirin

Bhati

K. S.

(2002). Voices of the forgotten half: The role of social class in the school-to-work transition. Journal of Counseling Psychology, 49, 311–323. doi:10.1037/0022-0167.49.3.311

Blustein

D. L.

Spengler

P. M.

(1995). Personal adjustment: Career counseling and psychotherapy. In Walsh

W. B.

Osipow

S. H.

(Eds.), Handbook of vocational psychology (2nd ed., pp. 295–329). Hillsdale, NJ: Erlbaum.

Brickell

(1976). Needed: Instruments as good as our eyes. Journal of Career Development, 2, 56–66. doi:10.1177/089484537600200305

10.

Brown

S. C.

Mason

C. A.

Lombard

J. L.

Martinez

Plater-Zyberk

Spokane

A. R.

Szapocznik

(2009). The relationship of built environment to perceived social support and psychological distress in Hispanic elders: The role of “Eyes on the street.” Journal of Gerontology: Social Sciences, 64B, 234. doi:10.1093/geronb/gbn011

11.

Brown

S. D.

McPartland

E. B.

(2005). Career interventions: Current status and future directions. In Walsh

W. B.

Savickas

M. L.

(Eds.), Handbook of vocational psychology (pp. 195–226). Mahwah, NJ: Lawrence Erlbaum.

12.

Brown

S. D.

Ryan Krane

N. E.

(2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In Brown

S. D.

Lent

R. W.

(Eds.), Handbook of counseling psychology (3rd ed., p. 740). Hoboken, NJ: John Wiley.

13.

Brown

S. D.

Ryan Krane

N. E.

Brecheisen

Castelino

Budisin

Miller

Edens

(2003). Critical ingredients of career choice interventions: More analyses and new hypotheses. Journal of Vocational Behavior, 62, 411–428. doi:10.1016/S0001-8791(02)00052-0

14.

Bumbarger

(2014). Understanding and promoting treatment integrity in prevention. In Hagermosr Sanetti

L. M.

Ktatochwill

T. R.

(Eds.), Treatment integrity: A foundation for evidence-based practice in applied psychology (pp. 35–54). Washington, DC: American Psychological Association.

15.

Byars-Winston

Kamtamneni

Mobley

(2012). The changing economic order: implications of economic disparities for counseling psychologists. In Fouad

N. A.

(Ed.), APA handbook of counseling psychology. Vol. 2: Practice, interventions, and applications (pp. 255–283). Washington, DC: American Psychological Association.

16.

Cochran

D. J.

Hoffman

S. D.

Strand

K. H.

Warren

P. M.

(1977). Effects of client/computer interaction on career decision making processes. Journal of Counseling Psychology, 24, 308–312. doi:10.1037/0022-0167.24.4.308

17.

Craighead

W. E.

Dunlop

B. W.

(2014). Combination psychotherapy and antidepressant medication treatment for depression: For whom, when, and how. Annual Review of Psychology, 65, 267–300. doi:10.1146/annurev.psych.121208.131653

18.

Crites

J. O.

(1969). Vocational psychology: The study of vocational behavior and development. New York, NY: McGraw Hill.

19.

Dawis

R. V.

(1992). The individual differences tradition in counseling psychology. Journal of Counseling Psychology, 39, 1–7. doi:10.1037/0022-0167.39.1.7

20.

Elliot

(2008). Response to intervention: What and why? The School Administrator. September 10–18, 2008.

21.

Fretz

B. R.

(1981). Evaluating the effectiveness of career interventions [Monograph]. Journal of Counseling Psychology, 28, 77–90. doi:10.1037/0022-0167.28.1.77

22.

Gazzaniga

M. S.

(2013). Shifting gears: Seeking new approaches for mind/brain mechanisms. Annual Review of Psychology, 64, 1. doi:10.1145/annurev-psych-113011-143817

23.

Gribbons

W. D.

Lohnes

P. R.

(1982). Careers in theory and experience: A twenty-year longitudinal study. Albany, NY: SUNY Press.

24.

Hagermoser Sanetti

L. M.

Kratochwill

T. R.

(Eds.). (2014). Treatment integrity: A foundation for evidence-based practice in applied psychology. Washington, DC: American Psychological Association.

25.

Hartung

P. J.

(2010). Practice and research in career counseling and development—2009. The Career Development Quarterly, 59, 98–142.

26.

Hartung

P. J.

(2013). Career as story: Making the narrative turn. In Bruce Walsh

Savickas

M. L.

Hartung

P. L.

(Eds.), Handbook of vocational psychology: Theory, research, and practice (4th ed., pp. 33–52). New York, NY: Routledge.

27.

Heppner

M. J.

Hendricks

(1995). A process and outcome study examining career indecision and indecisiveness. Journal of Counseling & Development, 73, 426–437. doi:10.1002/j.1556-6676.1995.tb01776.x

28.

Heppner

M. J.

Heppner

P. P.

(2003). Identifying process variables in career counseling: A research agenda. Journal of Vocational Behavior, 62, 429–452. doi:10.1016/S0001-9791(02)00053-2

29.

Heppner

M. J.

Multon

K. D.

Gysbers

N. C.

Ellis

C. A.

Zook

C. E.

(1998). The relationship of trainee self-efficacy to the process and outcome of career counseling. Journal of Counseling Psychology, 45, 393–402. doi:10.1037/0022-0167.45.4.393

30.

Herr

E. L.

Cramer

Niles

(2004). Career guidance and counseling through the lifespan: Systematic approaches (6th ed.). Boston, MA: Pearson.

31.

Holland

J. L.

Magoon

T. M.

Spokane

A. R.

(1981). Counseling psychology: Career interventions, research, and theory. Annual Review of Psychology, 32, 279–305. doi:10.1146/annurev.ps.32.020181.001431

32.

Hoyt

D. P.

(1955). An evaluation of group and individual programs in vocational guidance. Journal of Applied Psychology, 39, 26–30. doi:10.1037/h0042099

33.

Jaeger

M. M.

(2012). The extended family and children’s educational success. American Sociological Review, 77, 903–922. doi:10.1177/0003122412464040

34.

Jome

L. M.

Phillips

S. D.

(2005). Interventions to aid job finding and choice implementation. In Brown

S. D.

Lent

R. W.

(Eds.), Career development and counseling (2nd ed., p. 595). Hoboken, NJ: Wiley.

35.

Juntenen

C. L.

Ali

S. R.

Pietrantonio

(2013). Social class, poverty and career development. In Brown

S. D.

Lent

R. W.

(Eds.), Career development and counseling: Putting theory and research to work (2nd ed., pp. 245–274). Hoboken, NJ: Wiley.

36.

Kirschner

Hoffman

M. A.

Hill

C. E.

(1994). Case study of the process and outcome of career counseling. Journal of Counseling Psychology, 41, 216. doi:10.1037/0022-0167.41.2.216

37.

Knickerbocker

Davidshofer

(1978). Attitudinal outcomes of the Life Planning Workshop. Journal of Counseling Psychology, 25, 103–109. doi:10.1037/0022-0167.25.2.103

38.

Krishnan

Ulrich

(2001). Product development decisions: A review of the literature. Management Science, 47, 1–21. doi:10.1287/mnsc.47.1.1.10668

39.

Krumboltz

J. D.

Becker-Haven

J. F.

Burnett

K. F.

(1979). Counseling psychology. Annual Review of Psychology, 30, 555–602. doi:10.1146/annurev.ps.30.020179.003011

40.

Lapan

R. T.

Turner

Pierce

M. E.

(2012). College and career readiness: Policy and research to support effective counseling in schools. In Fouad

N. A.

Carter

J. A.

Subich

(Eds.), APA handbook of counseling psychology, vol. 2: Practice, interventions, and applications (pp. 57–73). Washington, DC: American Psychological Association.

41.

Lent

R. W.

Brown

S. D.

Nota

Soresi

(2003). Testing social cognitive interest and choice hypotheses across Holland types in Italian high school students. Journal of Vocational Behavior, 62, 101–118. doi:10.1016/S0001-8791(02)00057-X

42.

Lopez

S. R.

Concepcion

Kopelowicz

Vega

W. A.

(2012). From documenting to eliminating disparities in mental health care for Latinos. American Psychologist, 67, 511–523. doi:10.1037/a0029737

43.

Maola

Kane

(1976). Comparison of computer-based versus counselor-based occupational information systems with disadvantaged vocational students. Journal of Counseling Psychology, 23, 163–165. doi:10.1037/0022-0167.23.2.163

44.

Marko

K. W.

Savickas

M. L.

(1998). Effectiveness of a career time perspective intervention. Journal of Vocational Behavior, 52, 106–119. doi:10.1006/jvbe.1996.1566

45.

Mason

Scott

K. G.

Chapman

D. A.

(2000). A review of some individual- and community-level effect size indices for the study of risk factors for child and adolescent development. Educational and Psychological Measurement, 60, 3385–3410. doi:10.1177/00131640021970619

46.

Meara

N. M.

Patton

M. J.

(1994) Contributions of the working alliance in the practice of career counseling. The Career Development Quarterly, 43, 161–177. doi:10.1002/j.2161-0045.1994.tb00855

47.

Miller

M. J.

Brown

S. D.

(2005). Counseling for career choice: Implications for improving interventions and working with diverse populations. In Brown

S. D.

Lent

R. W.

(Eds.), Career development and counseling: Putting theory and research to work (pp. 441–464). Hoboken, NJ: Wiley.

48.

Myers

R. A.

(1971). Research on educational and vocational counseling. In Bergin

A. E.

Garfield

S. L.

(Eds.), Handbook of psychotherapy and behavior change: An empirical analysis (pp. 863–891). New York, NY: Wiley.

49.

Myers

R. A.

(1986). Research on educational and vocational counseling. In Bergin

A. E.

Garfield

S. L.

(Eds.), Handbook of psychotherapy and behavior change: An empirical analysis (3rd ed., pp. 715–738). New York, NY: Wiley.

50.

NCDA. (2011). National survey of working America. Broken Arrow, OK: Author.

51.

Oliver

L. W.

(1979). Outcome measurement in career counseling research. Journal of Counseling Psychology, 26, 217–226. doi:10.1037/0022-0167.26.3.217

52.

Oliver

L. W.

Spokane

A. R.

(1982). Sufficiency in the reporting of research results: Some guidelines (Tech. Rep. No. 599). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

53.

Oliver

L. W.

Spokane

A. R.

(1988). Career-intervention outcome: What contributes to client gain? Journal of Counseling Psychology, 35, 447. doi:10.1037/0022-0167.35.4.447

54.

Osipow

S. H.

(1983). Theories of career development. Englewood Cliffs, NJ: Prentice Hall.

55.

Osipow

S. H.

Fitzgerald

L. F.

(1996). Theories of career development (4th ed.). Boston, MA: Allyn & Bacon.

56.

Perry

J. C.

Dauwalder

J. P.

Bonnett

H. R.

(2009). Verifying the efficacy of vocational guidance programs: Procedures, problems, and potential directions. The Career Development Quarterly, 57, 348–357. doi:10.1002/j.2161-0045.2009.tb00121.x

57.

Rayman

J. R.

Bernard

C. B.

Holland

J. L.

Barnett

D. C.

(1983). The effects of a career course on undecided college students. Journal of Vocational Behavior, 23, 346–355. doi:10.1016/0001-8791(83)90047-7

58.

Robbins

S. B.

Tucker

K. R.

(1986). Relation of goal instability to self-directed and interactional career counseling workshops. Journal of Counseling Psychology, 33, 418–424. doi:10.1037/0022-0167.33.4.418

59.

Rounds

J. B.

Jr Tinsley

H. E. A.

(1984). Diagnosis and treatment of vocational problems. In Brown

S. D.

Lent

R. W.

(Eds.), Handbook of counseling psychology (pp. 137–177). New York, NY: Wiley.

60.

Samaan

M. K.

Parker

C. A.

(1973). Effects of behavioral (reinforcement) and advice-giving counseling on information-seeking behavior. Journal of Counseling Psychology, 20, 193–201. doi:10.1037/h0034575

61.

Sampson

J. P.

Jr McClain

Musch

Reardon

R. C.

(2011). The supply and demand for career interventions as a social justice issue. Plenary Presentation: 10th Biennial Conference of the Society for Vocational Psychology: Forging Career Policy for the Greater Good, Boston, MA.

62.

Santisteban

D. A.

Coatsworth

J. D.

Perez-Vidal

Kurtines

W. M.

Schwartz

S. J.

LaPerriere

Szapocznik

(2003). Efficacy of brief strategic family therapy in modifying Hispanic adolescent behavior problems and substance use. Journal of Family Psychology, 17, 121–133. doi:10.1037/0893-3200.17.1.121

63.

Savickas

(2005). The theory and practice of career construction. In Brown

S. D.

Lent

R. W.

(Eds.), Career development and counseling: Putting theory and research to work (pp. 42–70). Hoboken, NJ: Wiley.

64.

Scott

K. G.

Mason

C. A.

Chapman

D. A.

(1999). The use of epidemiological methodology as a means of influencing public policy. Child Development, 70, 1263. doi:10.1111/1467-8624.00091

65.

Spokane

A. R.

(2013). Understanding career aspiration, choice and adaptation: How do individuals decide? Occasional paper , SHRM Foundation, Arlington, VA.

66.

Spokane

A. R.

Oliver

L. W.

(1983). The outcomes of vocational intervention. In Walsh

W. B.

Osipow

S. H.

(Eds.), Handbook of vocational psychology (Vol. II, pp. 99–136) Hillsdale, NJ: Erlbaum.

67.

Super

D. E.

Hall

D. T.

(1978). Career development: Exploration and planning. Annual Review of Psychology 29, 333–372. doi:10.1146/annurev.ps.29.020178.002001

68.

Swanson

J. L.

(1995). The process and outcome of career counseling. In Walsh

W. B.

Osipow

S. H.

(Eds.), Handbook of vocational psychology: Theory, research, and practice (2nd ed., pp. 217–259). Hillsdale, NJ: Lawrence Erlbaum.

69.

Swanson

J. L.

Daniels

K. K.

Tokar

D. M.

(1996). Assessing perceptions of career-related barriers: The career barriers inventory. Journal of Career Assessment, 4, 219–244. doi:10.1177/106907279600400207

70.

Swanson

J. L.

Fouad

N. A.

(2009). Career theory and practice: Learning through case studies (2nd ed.). Thousand Oaks, CA: Sage.

71.

Szapocznik

Lombard

Martinez

Mason

C. A.

Gorman-Smith

Plater-Zyberk

… Spokane

A. R.

(2006). The impact of the built environment on children’s school conduct grades: The role of diversity of use in a Hispanic neighborhood. American Journal of Community Psychology, 38, 299–310. doi:10.1007/s10464-006-9084-x

72.

Tiedeman

D. V.

O’Hara

R. P.

(1963). Career development: Choice and adjustment. New York, NY: College Entrance Examination Board.

73.

Turner

S. L.

Lapan

R. T.

(2013). Promotion of career awareness, development, and school success in children and adolescents. In Brown

S. D.

Lent

R. W.

(Eds.), Career development and counseling (2nd ed., p. 539). Hoboken, NJ: John Wiley.

74.

Tyler

(1992). Counseling psychology why? Counseling psychology: Why? Professional Psychology: Research and Practice, 23, 342–344. doi:10.1037/0735-7028.23.5.342

75.

U.S. General Accounting Office. (1991). Transition from school to work: Linking education and worksite training (Rep. No. GAO/HRD-91-105). Report to Congressional Requestors. Washington, DC: Author.

76.

Walsh

W. B.

(2003). Diversity, flexibility, and career interventions. Journal of Vocational Behavior, 62, 459. doi:10.1016/S0001-8791(02)00059-3

77.

Wampold

B. E.

(2009). The great psychotherapy debate: Models, methods, and findings. New York, NY: Routledge.

78.

Whiston

S. C.

(2002). Application of the principles: Career counseling and interventions. The Counseling Psychologist, 30, 218–237. doi:10.1177/0011000002302002

79.

Whiston

S. C.

(2011). Vocational counseling and interventions: An exploration of future “big”questions. Journal of Career Assessment, 19, 287–295. doi: 10.1177/1069072710395535

80.

Whiston

S. C.

Brecheisen

B. K.

Stephens

(2003). Does treatment modality affect career counseling effectiveness? Journal of Vocational Behavior, 62, 390. doi:10.1016/S0001-8791(02)00050-7

81.

Whiston

S. C.

James

B. N.

(2013). Promotion of career choices. In Brown

S. D.

Lent

R. W.

(Eds.), Career development and counseling (2nd ed., p. 565). Hoboken, NJ: John Wiley.

82.

Whiston

S. C.

Oliver

L. W.

(2005). Vocational counseling process and outcome. In Walsh

W. B.

Savickas

M. L.

(Eds.), Handbook of vocational psychology (pp. 155–194). Mahwah, NJ: Lawrence Erlbaum.

83.

Whiston

S. C.

Rahardja

(2008). In Brown

S. D.

Lent

R. W.

(Eds.), Handbook of counseling psychology (3rd ed., pp. 444–461). New York, NY: Wiley.

84.

Whiston

S. C.

Rose

C. S.

(2013). Career counseling with emerging adults. In Walsh

W. B.

Savickas

M. L.

Hartung

P. J.

(Eds.), Handbook of vocational psychology: Theory research and practice. (4th ed., pp. 249–272). New York, NY: Routledge.

85.

Whiston

S. C.

Sexton

T. L.

Lasoff

D. L.

(1998). Career-intervention outcome: A replication and extension of Oliver and Spokane. Journal of Counseling Psychology, 45, 150–194. doi:10.1037/0022-0167.45.2.150