Psychometric Properties of a Career Exploratory Outcome Expectations Measure

Abstract

Social cognitive career theory and research are advanced by increasing attention to career outcome expectations and by applying this theory earlier in the life span. This article offers the career exploratory outcome expectations construct as a means of applying the more general construct of career outcome expectations during the childhood period and introduces the psychometric characteristics of the Career Exploratory Outcome Expectations Scale (CEOES). Employing data from 446 fifth graders and item response theory (IRT), the CEOES presents a one-dimensional structure with a four-category Likert-type response scale. Favorable results of person- and item-separation reliability were found and the scale appears to perform equally well for both genders. The CEOES also demonstrated concurrent validity through positive associations with established self-efficacy measures. The CEOES constitutes a useful measure to study aspects of career outcome expectations in childhood, and psychometric findings affirm its use in the career assessment literature.

Keywords

career exploratory outcome expectations outcome expectations career exploration childhood career development item response theory

Introduction

Efforts have increased over the past decade to theorize and investigate social cognitive mechanisms in career development, with an emphasis on the adolescent and young adult periods. Social cognitive career theory and constructs have presented a major contribution to understanding if and how individuals’ approach or avoid vocational behavioral domains and academic and career interests, choices, performance, and persistence (Betz, 2000; Lent, 2013). Still, the attention paid to outcome expectations and to childhood has been relatively scarce (Cupani, Minzi, Pérez, & Pautassi, 2010; Fouad & Guillen, 2006; Lent & Brown, 2013). The drive to study social cognitive mechanisms in earlier ages is consistent with the identified centrality of childhood career development and with current calls to expand the social cognitive career theory across the life span (Lent & Brown, 2013). This article focuses on the career outcome expectations of children and a new means to assess it.

Outcome expectations consist of a person’s generalized beliefs about the likelihood of achieving certain outcomes by engaging in a course of action (Lent, 2013). Outcome expectations play regulatory functions, as individuals are more likely to approach and behave in ways that will contribute to valued outcomes (Lent & Brown, 2006, 2013). When applied to careers, outcome expectations have been mostly focused on career decision making (e.g., Betz & Voyten, 1997; Fouad, Smith, & Enochs, 1997). However, the study of career decision-making outcome expectations in childhood may not be developmentally appropriate. Although adolescents and adults are expected to make career decisions, this is not the case for children. Instead, children are expected to fantasize about life roles and to increase their self-knowledge and environment knowledge to translate their imagination into concrete terms. Career exploration has been identified as a central process advancing the children’s development of career cognition and learning (Patton & Porfeli, 2007). Given that career decision making is promoted by career outcome expectations in adolescence and adulthood, career exploration may be an essential process within a social cognitive career theory of children. Moving backward in the developmental course from the construct of career decision-making outcome expectations leads us to the construct of career exploratory outcome expectations.

Career Exploratory Outcome Expectations: Construct and Relations

Children’s career exploration mostly occurs at the home, school, and community settings (Super, 1980), offering experiences contributing to outcome expectations (Lent, Hackett, & Brown, 2004). Career exploratory outcome expectations can be defined as the perceived likelihood of achieving career progress outcomes by performing an exploratory behavior. A child may knowingly or unknowingly engage in career exploration to gain information and test hypotheses about himself or herself in present and future life roles, the working world, and the community (Berlyne, 1960; Jordaan, 1963; Patton & Porfeli, 2007; Taveira & Moreno, 2003).

As cognitive development sustains career exploration and social cognitive mechanisms (Lent & Brown, 2013; Patton & Porfeli, 2007), hypothetical-deductive reasoning is assumed to facilitate the establishment and partially explain the emergence of children’s career exploratory outcome expectations. Middle school children employ hypothetical-deductive reasoning by making inferences on the basis of abstract information. These skills enable children to fantasize, test hypotheses about themselves and the working world, and more intentionally engage in career exploration (e.g., Gottfredson, 1981; Patton & Porfeli, 2007).

As career outcome expectations and career exploration have been related to each other and to self-efficacy expectations (e.g., Betz & Voyten, 1997; Lent, 2013; Rogers, Creed, & Glendon, 2008; Taveira & Moreno, 2003), it is expected that career exploratory outcome expectations is reciprocally related to self-efficacy expectations. Although children engage in career exploration, they observe others’ behaviors and test internal (e.g., psychophysiological reactions) and external feedback (e.g., verbal encouragement) within context, which promotes the development of outcome and self-efficacy expectations (Lent et al., 2004). Outcome and self-efficacy expectations regulate behaviors and sustain children’s intentions to explore careers and approach or avoid related career and academic tasks (Porfeli, Lee, & Weigold, 2012; Porfeli, Wang, & Hartung, 2008; Taveira & Moreno, 2003). Children are more likely to approach tasks in which they feel confident and expect to accomplish desired outcomes, developing related likes and goals. On the other hand, children are prone to avoid tasks in which they expect to fail and get adverse consequences, developing dislikes and intentions to not repeat them (Bandura, 1986; Lent et al., 2004). Thus, career exploratory outcome and self-efficacy expectations can sustain children’s progress in intentional and in-depth exploratory behaviors, strengthen an emerging sense of self, and develop preferences and occupational alternatives (Berlyne, 1960; Lent et al., 2004; Patton & Porfeli, 2007).

As a social cognitive mechanism, career exploratory outcome expectations can be revised according to the children’s mastery and performance experiences (Lent & Brown, 2013). Leisure, extracurricular, and school settings provide children opportunities for such revising experiences (Zaff, Moore, Papillo, & Williams, 2003). Such changes may bolster individuals’ approach to or avoidance of future leisure and extracurricular experiences as well as impact academic engagement and achievement (Knifsend & Graham, 2012). Academic achievement and self-regulation also yield internal and external mastery and performance experiences that both influence and are influenced by social cognitive mechanisms (Cupani et al., 2010; Shell, Colvin, & Bruning, 1995). As children’s career development is embedded in leisure, extracurricular, and academic experiences, it is expected that career exploratory outcome expectations will correlate with self-efficacy expectations in those activities. These relations are also assumed to play a motivational role in the development of academic and occupational aspirations, exploratory intentions, interests, and choices (Bandura, Barbaranelli, Caprara, & Pastorelli, 2001; Cupani et al., 2010; Ferry, Fouad, & Smith, 2000; Lent, 2013).

Assessment

The assessment of career exploratory outcome expectations can be informed by the historical assessment of the two key constructs that bear on it, namely, career outcome expectations and career exploration. Career outcome expectations have been most commonly assessed with two measures—the Career Decision-Making Outcome Expectancies Scale (CDMOES; Betz & Voyten, 1997) and the Middle School Self-Efficacy Scale (MSSE; Fouad et al., 1997)—employing Likert-type scales (1 = strongly disagree to 5 = strongly agree). The CDMOES was designed to assess college students and the MSSE, as its title suggests, was developed for middle school students. The CDMOES includes two outcome expectations subscales, which are career outcome expectations and education outcome expectations. The MSSE includes two outcome expectations subscales, namely, career decision-making outcome expectancies and math/science outcome expectancies. Although these measures focus on career decision making and academics, their items implicitly assess exploratory behaviors, which supports relying on career exploration in a new measure of outcome expectations.

The CDMOES and the MSSE have been modified in subsequent studies to yield one-dimensional scales, improved psychometric characteristics, and evidence of concurrent validity. Focusing on the CDMOES outcome expectations subscales, Rogers, Creed, and Glendon (2008) reduced the number of response categories for high school students. Based on such a modification, the measure yielded a single outcome expectations dimension and it exhibited positive correlations with career decision-making self-efficacy and outcome expectations. As for the MSSE math/science outcome expectancies, the subscale was designed to be one dimensional. After expanding the number of response categories, the subscale exhibited improved reliability (Ferry et al., 2000). Cupani and collaborators (2010) also reduced the number of items for this subscale MSSE. With these modifications, reliability was improved, a total score of math outcome expectations was used and it correlated with logic-mathematics self-efficacy expectations. Based on previous research, a new measure of career exploratory outcome expectations might present a one-dimensional structure, acceptable psychometrics, and be correlated with self-efficacy constructs.

As for career exploration, the subscales Internal and External Search Instrumentality of the Career Exploration Survey (CES; Stumpf, Colarelli, & Hartman, 1983; adapt. by Taveira, 2000) are particularly informative for career exploratory outcome expectations. These subscales assess adolescents’ and adults’ expected consequences of exploratory behaviors with a 5-point Likert-type scale from 1 = very low to 5 = very high probability. Measures such as the Career Exploration Scale (Tracey, Lent, Brown, Soresi, & Nota, 2006) and the Occupational Exploration Scale (Noack, Kracke, Gniewosz, & Dietrich, 2010) have assessed middle school students’ career exploration and these measures yield a one-factor structure and favorable reliability employing a four- and five-category response scales.

The reviewed measures can inform the format, response scale, dimensionality, and validity of a measure of career exploratory outcome expectations. These measures illustrate the possibility of using a self-report format in middle childhood. The use of four- or five-categories Likert-type scales is also consistent with recommendations derived from the assessment of younger participants’ social cognitive mechanisms (Lent & Brown, 2006) and can be considered in a measure of career exploratory outcome expectations. A one-factor structure of career exploratory outcome expectations seems also consistent with other measures of related constructs such as the CDMOES and MSSE (e.g., Cupani et al., 2010; Ferry et al., 2000; Rogers et al., 2008) and with existent measures of middle school students’ career exploration (e.g., Noack et al., 2010; Tracey et al., 2006). The concurrent validity of a measure of career exploratory outcome expectations might be addressed using self-efficacy expectations scales.

Although evidence of invariance of the reviewed measures across genders has not been reported, differences across genders in career exploration (e.g., Noack et al., 2010) and mixed results in career outcome expectations (e.g., Betz & Voyten, 1997; Ferry et al., 2000; Fouad et al., 1997) have been found. Given the limited evidence, the invariance of a measure of career exploratory outcome expectations across genders should also be examined.

Purpose

This study presents the Career Exploratory Outcome Expectations Scale (CEOES) as a potential new measure to advance the social cognitive career research devoted to childhood. The CEOES was designed to assess career exploratory outcome expectations in middle childhood. Increases in systematic career exploration and cognitive development during this period (Gottfredson, 1981; Patton & Porfeli, 2007) might foster the development of career exploratory outcome expectations. This construct should be studied during this childhood period in an attempt to ascertain when and how it emerges and serves as an antecedent to career development during the adolescent period. This study was conducted in Portugal to address European-Portuguese calls to promote career development throughout the school years in preparation for a challenging and changing world of work (Leão, 2006).

The psychometric properties of the CEOES were examined with an item response theory (IRT) model rather than classical test theory (CTT) procedures. IRT is a psychometric research paradigm that includes a set of models that estimate latent traits based on item performance, describing the probability of the response to an item as a function of the examinee’s level on the latent trait and the item characteristics (Hambleton & Jones, 1993). IRT favors the estimation of group- and test-independent parameters, whereby the items’ parameters are independent of the sample recruited and the person scores are not dependent on the measure’s characteristics, as well as constitutes a promising venue to respond to an increasing diverse research and client population requiring brief and precise career assessments (Bond & Fox, 2007; Chartrand & Walsh, 2001; Hambleton & Jones, 1993; Wetzel, Hell, & Pässler, 2012). IRT was used in this article, because its benefits outweigh those offered by CTT procedures while estimating parameters that balance the level of difficulty of each item with the individual’s level on the assessed construct.

The IRT Rasch Rating Scale model (RSM; Andrich, 1978) was selected due to its parsimonious nature (Bond & Fox, 2007) and to its appropriateness for the one-dimensional structure expected for the CEOES. The RSM was also selected due to its previous use in psychological constructs (Reise & Moore, 2012) and capacity to include polytomous-scaled items, such as the Likert-type scales used in reviewed measures (e.g., Betz & Voyten, 1997; Fouad et al., 1997; Lent & Brown, 2006; Stumpf et al., 1983) and those used in the CEOES. Given the ongoing uncertainty in the literature about the appropriate number of response categories for psychological constructs such as career outcome expectations, RSM offers an empirically driven method to discern the appropriate number of response categories (Bond & Fox, 2007), which is also an advantage of IRT over CTT methods.

The concurrent validity of the CEOES was assessed through ascertaining if and how it is associated with established scales of self-efficacy expectations for (a) leisure and extracurricular activities, (b) academic success, and (c) self-regulatory learning. These analyses relied on theory and research suggesting relations of career outcome expectations with career exploration and with self-efficacy expectations (e.g., Betz & Voyten, 1997; Cupani et al., 2010; Lent, 2013; Taveira & Moreno, 2003; Zaff et al., 2003).

Based on the literature, the following measurement hypotheses were considered:

Hypothesis 1: The CEOES will demonstrate a one-dimensional structure.

Hypothesis 2: A five-category response scale will demonstrate the most favorable psychometric characteristics.

Hypothesis 3: The CEOES will demonstrate acceptable reliability.

Hypothesis 4: Evidence will support the applicability of the CEOES for both girls and boys.

Hypothesis 5: The CEOES will demonstrate positive associations with self-efficacy scales.

Method

Participants

After the Portuguese General Direction of Education authorized this study, participants were recruited from a non-probabilistic intentional sampling method. Four Portuguese middle schools collaborated in this study. The consents from the schools’ principals, psychologists, and teachers were obtained. The schools’ principals and psychologists selected the classroom groups to participate in the study. All students from the classroom groups (N = 530) were invited to participate. Based on written consent forms from the caregivers, 84.15% of the students were authorized and also assented to participate in the study. The sample included 446 fifth graders, 212 (47.5%) girls and 234 (52.5%) boys, aged 9 to 13 years (M = 10.23, SD = .49). At the time of this study, children were attending fifth grade.¹ Children derived from low (26.5%), medium-low (53.6%), and medium-high (19.5%) social economic status families.² Most of the children were Portuguese native, followed by 0.4% of German-, Brazilian-, and Chinese-native participants.

Measures

Career exploratory outcome expectations scale (CEOES)

Development of CEOES was based on the childhood career development literature and the reviewed measures of career outcome expectations and career exploration. Fifteen items were included in a self-report measure to assess one’s perceived likelihood of achieving certain outcomes by performing exploratory behaviors. The items were cast in childhood contexts (e.g., school, friends, and home; Super, 1980) and focused on expectations to gain information and test hypotheses about oneself in life roles (Items 2, 3, 5, and 11), the working and occupational world (Items 1, 6, 7, 9, and 15), and the community (Items 4, 8, 10, 12, 13, and 14).

A 5-point Likert-type scale ranging from 1 = very low probability to 5 = very high probability was used. This response scale was consistent with the items’ content, the literature recommendations (Lent & Brown, 2006), and the reviewed measures (e.g., Fouad et al., 1997; Stumpf et al., 1983). The CES Internal and External Search Instrumentality scales (Stumpf et al., 1983; adapt. Taveira, 2000) were of particular interest because the items assess outcome expectations, the response scale is cast in probability-based terms (i.e., a likelihood), and the scale has presented favorable psychometric results with Portuguese students from middle school to college. The possibility of using a probability-based response scale in fifth grade was also consistent with the Portuguese curriculum, according to which children are expected to understand the notions of random situations and probability (Ponte et al., 2007). Written instructions explaining the items’ formulation and the response scale were included.

Multidimensional scales of perceived self-efficacy (MSPSE)

To examine the concurrent validity of the CEOES, the scales of self-efficacy expectations for leisure and extracurricular activities (8 items; e.g., “Is it easy for you to learn sports”), academic success (9 items; e.g., “Is it easy for you to learn math”), and self-regulated learning (11 items; e.g., “Is it easy for you to focus on academic subjects”) from the Portuguese MSPSE version were used (Bandura, 1990; adapt. Teixeira, 2009; Teixeira & Carmo, 2004). The Portuguese version of the MSPSE has been used with Portuguese students from middle school years to college and exhibited acceptable psychometric properties, including evidence of validity based on demonstrated relationships with academic and career variables (e.g., Teixeira, 2009; Teixeira & Carmo, 2004). The items were answered with a 5-point Likert-type scale ranging from 1 = not easy at all to 5 = very easy. A total score for each scale was computed by averaging the items’ scores, with higher scores indicating favorable self-efficacy expectations. In this sample, a Cronbach’s α of .82 was obtained for self-efficacy expectations for leisure and extracurricular learning, .67 for self-efficacy for academic success, and .91 for self-efficacy for self-regulated learning, thus presenting minimally acceptable to very strong results (DeVellis, 1991).

Procedures

The CEOES and the MSPSE scales were administered on paper by a doctoral student, during 3 weeks spanning February to March 2013. Confidentiality was guaranteed. Children completed the instruments in classroom and took an average of 15 min to complete the CEOES and 20 min to complete the MSPSE scales. A report presenting the results was delivered to the schools’ principals 5 weeks after data collection.

Data Analysis

The Rasch measurement software Winsteps, version 3.72.0, for Windows (Linacre, 2011a) was used. The RSM describes a probabilistic relationship between the item location parameter (b_i) and the person’s latent trait (θ _n ), as provided by the formula (Andrich, 1978):

log (P_{n i k} / P_{n i (k - 1)}) = θ_{n} - b_{i} - τ_{k},

P_nik is the probability of the person n chose the category k of the item i, P_ni_{(k − 1)} is the probability of the person n chose the category k − 1 of the item i, θ _n is the level of person n in the latent trait, b_i is the location parameter of the item i, and τ _k is the threshold in category k − 1 relative to category k. Data calibration using RSM provides estimates for the person (level on latent trait) and the item location parameters, both depicted in a trait measurement continuum. Thresholds (τ), or step calibrations, are derived for each adjacent response category in the response scale. Thresholds are determined at the point in which choosing one response category over another is equal. For k response categories, k − 1 thresholds are considered.

The assumptions of unidimensionality and local independence of the items were evaluated with residuals analyses. Although calibrating data in RSM, there are observations that present differences between the expected and the observed pattern of results. These differences constitute residuals that can be analyzed to find additional common variance. To check the assumption of unidimensionality, a principal component analysis (PCA) of the residuals was run. This analysis identifies clusters of residuals, also referred to as secondary dimensions, which share a large amount of variance and can be considered a possible separate dimension relative to the target construct (Linacre, 2011b). Although traditional factor analyses rely on raw scores and rotated factor solutions, the PCA of the residuals is conducted after excluding the target construct and relies on unrotated solutions (Dimitrov, 2012). The interpretation of the results derived from common factor analyses differs from the PCA of the residuals. Although common factor analyses explore or confirm a construct factor structure, the PCA of the residuals detects secondary dimensions. When a PCA of the residuals presents Eigenvalues lower than 2.0 for the secondary dimensions, the shared variance of the residuals can be assumed as random and support for the assumption of unidimensionality is obtained (Linacre, 2011b). To check the local independence of the items, correlations of the items’ linearized residuals were run. Coefficients lower than .70 suggest the local independence of the items and assume that the performance in an item depends of the person’s latent trait and is not influenced by his or her responses to other items (Bond & Fox, 2007; Linacre, 2011b).

RSM was employed to estimate the person and item location parameters, determine the appropriate number of response scale categories, and evaluate the model fit and scale reliability. The diagnostic features for the response category evaluation were (a) a regular distribution of the category frequencies; (b) a minimum of 10 observations per category; (c) a monotonic increase of the average measures, considering that on average individuals presenting high level on the latent trait will endorse high response categories; (d) the inexistence of a response category with an outfit higher than 2.00; and (e) a monotonic increase in the step calibrations (Bond & Fox, 2007; Linacre, 2002a). The last criterion implies that the steps are ordered to indicate that each category is the most probable to be observed at certain intervals of the continuum (Linacre, 1999). Category probability curves can also be used in the response category evaluation to visualize whether each response category presents a peak in a single section of the continuum (Wetzel et al., 2012).

To evaluate model fit, infit and outfit mean square (MNSQ) statistics for person and item parameters were calculated (Bond & Fox, 2007; de Ayala, 2009). Infit and outfit are reported as mean squares, which are derived from the division of χ² statistics by the degrees of freedom. MNSQ presents a ratio scale form with a mathematical expectation of 1 and a range of 0 to +∞ (Bond & Fox, 2007). A good fit is assumed when the model and the observed response patterns do not significantly differ. MNSQ infit and outfit statistics should range from 0.05 to 1.5, being 1.00 indicative of perfect fit (Linacre, 2002b). Misfit derives from aberrant responses, such as individuals who present a low level on the latent trait endorsing a high response category or vice versa. Infit and outfit values higher than 2.00 indicate misfit (Bond & Fox, 2007; Linacre, 2002b). Point-measure correlations were computed to address the agreement of the score in each item to the overall latent trait (Aryadoust, Goh, & Kim, 2012).

The person-separation reliability (PSR) and the item-separation reliability (ISR) coefficients were calculated to examine the reliability. Although PSR illustrates the probability of reproducing the person location parameter order if a parallel set of items were applied, ISR indicates the probability of reproducing the items’ location parameter order if the items were applied to a similar sample (Bond & Fox, 2007). Both PSR and ISR values range from 0 to 1, with a minimum value of .70 being acceptable.

The existence of uniform differential item functioning (DIF) was examined to assess the validity of the CEOES across gender groups. Items exhibit DIF when the probability of selecting a response category does not exclusively depend of the item’s location and the person’s latent trait, but is rather influenced by other individuals’ characteristics, suggesting a group bias in the item’s results (Bond & Fox, 2007; de Ayala, 2009).

To assess the concurrent validity of the CEOES, correlation coefficients were computed between this construct and the self-efficacy expectations scales. If the assumptions of normality of sampling distribution were not fulfilled, both parametric (i.e., Pearson correlation coefficient) and nonparametric tests (i.e., Spearman correlation coefficient) would be computed. When the conclusions provided by both sets of tests consistently led to retaining or rejecting the null hypothesis, parametric tests results are presented; otherwise, nonparametric tests results are reported (Fife-Schaw, 2006).

Results

RSM Assumptions

The assumption of unidimensionality was fulfilled. The PCA of the residuals revealed an Eigenvalue of 9.9 for the main dimension accounting for 39.7% of the variance. A secondary dimension of the item residuals presented an Eigenvalue of 1.5 and explained only 6% of the variance (approximately six times less of the variance explained by the main dimension). The local independence of the items was also verified. Correlations of the items’ linearized residuals ranged from 0 to .20.

Response Scale

The five-category scale presented a minimum of 10 observations per category, a monotonic increase of the average measures, and no category with an outfit of 2.00. Still, it presented a skewed distribution of frequencies across the response categories: While Category 1 included 70 (1%) observations, Category 5 included 3,257 (49%) observations. A monotonic increase of the step calibrations was also not observed, as the first two thresholds were disordered. Category 2 was never the most probable one in the continuum.

These results suggested that collapsing categories could improve the efficiency of the response scale. Bond and Fox (2007) recommended the following two guidelines to select the categories to collapse: The categories to collapse must make sense together in content and the uniformity of the frequencies distribution should be improved or maintained. Based on these guidelines, a three- and a four-category scale were tested. The category diagnostic statistics and the CEOES reliability of the three-, four-, and five-category scales are presented in Table 1. The probability curves for these scales are depicted in Figure 1.

Figure 1.

Category probability curves.

Table 1.

Category Statistics and Reliability.

Response Scale	Observed		Average Measure	MNSQ		Step Calibration	PSR (ISR)
Response Scale	Count	%	Average Measure	Infit	Outfit	Step Calibration	PSR (ISR)
Five-category scale							.80 (.97)
1. Very low	70	1	−2.42	1.35	1.86	—
2. Low	125	2	−1.25	1.22	1.32	−0.58
3. Moderate	998	15	−0.27	0.92	0.94	−1.57
4. High	2,236	33	1.15	0.87	0.88	0.36
5. Very high	3,257	49	3.04	1.00	0.99	1.79
Four-category scale							.81 (.97)
1. Very low or low	195	3	−2.79	1.33	1.49	—
2. Moderate	998	15	−0.85	0.93	0.97	−1.56
3. High	2,236	33	0.86	0.89	0.88	0.04
4. Very high	3,257	49	2.77	0.98	0.99	1.52
Three-category scale							.54 (.94)
1. Very low or low	195	3	−2.00	1.16	1.22	—
2. Moderate	998	15	0.00	0.94	0.88	−0.77
3. Very high or high	5,493	82	2.00	0.97	0.98	0.77

Note. MNSQ = infit and outfit mean square; PSR = person-separation reliability; ISR = item-separation reliability.

To test the four-category scale, the categories “very low” and “low probability” were collapsed, as they both referred to a low perceived likelihood of achieving a certain outcome from an exploratory behavior and presented the two lowest observed frequencies. Collapsing these categories led to a more regular distribution of the observations and to the ordering of steps calibration. The steps also increased monotonically and each of the four categories became the most probable one across the continuum. Average measures still increased monotonically and fit statistics were lower than 2.00. Outfit statistics improved when compared to the five-category scale, that is, the highest outfit value for the five-category scale was 1.86, whereas for the four-category scale it was 1.49. The PSR also improved.

To test the three-category scale, the categories “very high” and “high probability” were also collapsed, as their content considered a high perceived likelihood of achieving a certain outcome from an exploratory behavior. The results suggested that the steps increased monotonically, each of the three categories was the most probable one across the continuum, the fit statistics were lower than 2.00, and the outfit statistics improved. However, the three-category scale presented an extremely skewed distribution of the observations, with most observations lying in the Category 3 “very high or high probability.” The ISR decreased and the PSR also severely decreased from .81 to .54.

The four-category scale presented better results than the five-category scale in the ordering of steps calibration, the monotonically increase of steps, and the outfit statistics. It also presented a better distribution of the observations and a better PSR value than the five- or three-category scales. The remaining analyses were performed with the four-category scale, as this was empirically the most efficient one.

Fit Statistics

No item presented an infit and/or outfit value higher than 1.5 or lower than 0.50, that is, infit ranged from .83 to 1.45 and outfit ranged from .83 to 1.43 (see Table 2). Infit and outfit statistics presented mean values of 1.01, close to the value referenced for perfect fit. Point-measure correlations ranged from .45 to .54.

Table 2.

CEOES Item Statistics.

Items	b	SE	MNSQ		r _pm	Mean	SD
Items	b	SE	Infit	Outfit	r _pm	Mean	SD
1. Asking questions about the world of work at home, school and my friends will help me learn more about careers	0.49	.07	1.02	1.05	.60	3.09	.85
2. Observing how people balance their paid work with family responsibilities will help me learn how to balance my work with other activities in the future	0.83	.07	0.89	0.92	.65	2.93	.88
3. Knowing about my parents’ work will help me to choose my future career	0.05	.08	1.45	1.43	.51	3.27	.91
4. Doing my schoolwork will make my parents and teachers feel happy	−1.22	.10	1.24	1.04	.45	3.67	.65
5. Talking with my friends about the future will help me make better plans	0.61	.07	1.13	1.12	.60	3.04	.91
6. Searching the Internet will help me find important information about occupations	−0.10	.08	1.08	1.12	.53	3.33	.78
7. Knowing other people’s opinions about the world of work will help me develop my own ideas about work	−0.16	.08	0.86	0.85	.61	3.35	.76
8. Talking with my friends will help me better communicate my own ideas to other people	0.42	.07	0.83	0.89	.65	3.12	.83
9. Doing things in my free time will help me identify fields in which I am good at	−0.23	.08	0.91	1.01	.57	3.38	.75
10. Getting to know the people that work in my school might help me in the future should I need someone’s help	−0.36	.08	0.90	0.95	.55	3.43	.70
11. Watching men and women performing different activities will help me learn what to do when I grow up	0.14	.07	1.03	1.01	.58	3.24	.82
12. Participating in school activities will help me learn how to contribute to community	0.05	.08	0.83	0.83	.64	3.27	.82
13. Hanging out with my friends will help me better relate to different people	−0.27	.08	1.03	1.12	.54	3.39	.75
14. Participating in out-of-school activities will help me learn more about the community	0.15	.07	0.91	0.86	.62	3.23	.81
15. Knowing the types of places where my family members work will help me learn more about their specific jobs	−0.41	.08	1.02	0.94	.57	3.44	.78

Note. A translation of the items from Portuguese to English resulting from a discussion between Portuguese- and English-native speakers is presented. b = item location parameter; SE = standard error of the estimates; r_pm = point-measure correlation; SD = standard deviation; CEOES = Career Exploratory Outcome Expectations Scale.

As for person-fit statistics, 30 (6.7%) children presented infit and/or outfit values higher than 2.00. Eleven (2.5%) children presented infit and/or outfit values from 1.5 to 2.00.

Figure 2 depicts the item-person joint representation in the continuum. Individuals are depicted on the left-hand side and items on the right-hand side of the vertical axis. Children easily endorsed high response categories (i.e., “very high” or “high probability”) in Item 4. This was not the case for Item 2, in which the probability of choosing high response categories was higher for children with high career exploratory outcome expectations. Overall, it was easy for the sample to endorse high response categories in the 15 items. The mean person location was 1.79 (range: −2.29 to 5.52) and the mean item location was 0.00 (range: −1.22 to .83)

Figure 2.

Career Exploratory Outcome Expectations Scale person-item map. Note. “#” depicts two persons and “.” depicts one person.

DIF

Multiple t-tests were run to investigate differences in the item parameters for gender. As DIF procedures can increase the probability of Type I error, the Bonferroni correction was applied (Smith, 2004). For the 15 t-tests, a significance value of .003 was considered. Evidence of uniform DIF was not found between the two groups for any item, thus suggesting the measure’s invariance across girls and boys (see Table 3).

Table 3.

Item Location Parameters and DIF Analysis for Gender.

Item	Location Parameter (SE)		t (df)	p
Item	Girls	Boys	t (df)	p
1	0.61 (0.10)	0.38 (0.10)	1.62 (416)	0.106
2	0.83 (0.10)	0.83 (0.10)	0.00 (415)	1.00
3	0.10 (0.11)	0.01 (0.10)	0.64 (416)	0.524
4	−1.06 (0.13)	−1.38 (0.14)	1.65 (417)	0.099
5	0.61 (0.10)	0.61 (0.10)	0.00 (414)	1.00
6	−0.10 (0.11)	−0.10 (0.11)	0.00 (416)	1.00
7	−0.24 (0.11)	−0.08 (0.11)	−1.03 (415)	0.303
8	0.30 (0.11)	0.53 (0.10)	−1.65 (415)	0.099
9	−0.18 (0.11)	−0.29 (0.11)	0.72 (416)	0.474
10	−0.40 (0.12)	−0.33 (0.11)	−0.47 (415)	0.641
11	0.03 (0.11)	0.23 (0.10)	−1.32 (415)	0.187
12	0.18 (0.11)	−0.08 (0.11)	1.74 (416)	0.083
13	−0.40 (0.12)	−0.15 (0.11)	−1.59 (414)	0.113
14	0.15 (0.11)	0.15 (0.10)	0.00 (415)	1.00
15	−0.41 (0.12)	−0.41 (0.11)	0.00 (414)	1.00

Note. SE = standard error; df = degrees of freedom; DIF = differential item functioning.

Concurrent Validity

As the assumption of normality of sampling distribution was not fulfilled, both parametric and nonparametric correlation coefficients were computed. The correlations between the CEOES scores and the self-efficacy expectations for leisure and extracurricular activities, r = .40, p < .001; academic success, r = .38, p < .001; and self-regulated learning, r = .51, p < .001 were all positive and in the expected direction, statistically significant, and moderate to strong in magnitude.

Discussion

This study offered the career exploratory outcome expectations construct to assess social cognitive mechanisms in children’s career development and introduced the CEOES as a means of assessing this new construct. The study focused on the psychometric characteristics of the CEOES, employing IRT procedures and correlational analyses to test the structure, response scale, reliability, and validity of the measure.

The results indicated the fulfillment of the assumption of unidimensionality, thus supporting Hypothesis 1. This result is consistent with the one-factor structure obtained in studies of career outcome expectations (Cupani et al., 2010; Rogers et al., 2008) and in measures of middle school students’ career exploration (Noack et al., 2010; Tracey et al., 2006).

The IRT diagnostics of the response scale were particularly useful in this study, as the number of response categories continues to be questioned in the literature devoted to the current separate assessment of outcome expectations and career exploration. A five-category response scale was expected to yield favorable results due to its previous use in measures of career outcome expectations (e.g., Betz & Voyten, 1997; Fouad et al., 1997) and of career exploration (e.g., Stumpf et al., 1983; Tracey et al., 2006). Still, Hypothesis 2 was not empirically supported, as the results suggested the efficiency of a four-category scale and demonstrated that the reliability of the CEOES was reduced when the number of categories was reduced to three. As measures of outcome expectations used a five-category scale from six grade forward (e.g., Fouad et al., 1997) and this study suggested the efficiency of a four-category scale in fifth grade, further research might use IRT to investigate whether five- or four-category scales are most efficient throughout middle school grades. Moreover, the lack of evidence supporting the CEOES initial five-category response scale suggests that further studies focused on new measures may consider the potential of IRT models to test different possibilities for response scales, so that an empirically efficient one is established.

As a high agreement between each item and the overall latent trait was found, the items were all aligned with the career exploratory outcome expectations construct. The CEOES items were guided by the career exploration literature to ensure that the breadth of possible career exploration outcomes was included (e.g., Berlyne, 1960; Jordaan, 1963; Patton & Porfeli, 2007; Taveira & Moreno, 2003). The team identified key outcomes (i.e., gain information and test hypothesis about oneself) and considered the length of the survey to limit the impact of fatigue and boredom on students. As a consequence, other specific outcomes were not assessed (e.g., likelihood of getting approval from friends, family, or the community as a consequence of exploring a given career-related activity). Future instrument development could involve expanding the CEOES to address additional outcome expectations.

A ceiling effect was found, suggesting that a fraction of students endorsed the highest Likert-type choice for most of the items. Children presenting low levels of career exploratory outcome expectations were measured with a lower error of measurement than children presenting high levels of it, suggesting that high scores may be partly due to less careful consideration of the respondents. This study lacked a social desirability measure, which could have also been useful to control for a potential social desirability bias underlying the CEOES’ ceiling effect. Further studies using the CEOES and a social desirability measure are needed to control for this possibility. The threat to social desirability might also interact with childhood being a period of initial development of hypothetical-deductive reasoning. The combinations of these effects might at once prompt a more desirable response and one that lacks an understanding of conceptual differences between items in the scale. In the absence of hypothetical-deductive reasoning, students might be inclined to respond similarly across all items if they fail to understand their nuance and in the positive direction if they detect a socially desirable pattern. Such a process would explain why, for example, students would tend to score most items as “very high” or “high probability” rather than “very low” or “low probability.” Further research might assess the hypothetical-deductive reasoning and employ a broader age range of students to test whether variably in age and cognitive development is associated with the types of psychometrics reported here.

The results bearing on the ceiling effect can also suggest that the CEOES may be a more precise measure of low outcome expectations rather than the full spectrum. This conclusion has been suggested in previous studies employing IRT RSM in psychological constructs that often present ceiling or roof effects (Reise & Moore, 2012). The CEOES may be best used to identify children presenting low levels of career exploratory outcome expectations and more generally students who could benefit from career interventions. The CEOES can be used to promote children’s intentional career exploration and positive outcome expectations through career-oriented experiences. The CEOES can not only identify children who might benefit the most from these interventions but also serve the evaluation of the effectiveness of these interventions, for example, in pretest and posttest. Thus, the CEOES can serve a vital role in career development assessment and practice, covering social cognitive mechanisms earlier in life (Lent & Brown, 2013) and being consistent with European-Portuguese calls for early career interventions (Leão, 2006).

Favorable results of PSR and ISR were also obtained, thus supporting Hypothesis 3. These results suggest a highly probability that the order of the location parameters is replicable. Although differences for gender have been found in career exploration (e.g., Noack et al., 2010) and mixed findings have been found in the career outcome expectations literature (e.g., Betz & Voyten, 1997; Ferry et al., 2000), the DIF results suggest that the CEOES is equally applicable across both genders, thereby supporting Hypothesis 4.

The correlations between the CEOES and the self-efficacy scales supported Hypothesis 5, providing evidence of the CEOES concurrent validity. Moreover, the coefficient values were not so strong (e.g., .8 or higher) to infer that the construct is duplicating other related constructs. These results also suggest that the leisure, extracurricular, and academic activities may sustain the development and revision of children’s social cognitive mechanisms (Shell et al., 1995; Zaff et al., 2003), which in turn may impact the success and the approach or avoidance of career and academic activities (Knifsend & Graham, 2012; Porfeli et al., 2008, 2012). Further studies employing longitudinal data may examine the predictive validity of the CEOES, considering its predictive role in one’s approach or avoidance of tasks, career aspirations, interests, and intentions (Bandura et al., 2001; Cupani et al., 2010; Ferry et al., 2000; Lent & Brown, 2013). Other research may also explore the discriminant validity of the CEOES to ensure that it is sufficiently different from other established constructs.

Conclusion

This study presented two innovative contributions. First, the career exploratory outcome expectations construct was suggested to advance social cognitive career theory and research during the childhood period. As the CEOES was designed to assess this construct and it presented favorable psychometric properties, the measure holds promise to stimulate attention to career exploratory outcome expectations and to social cognitive mechanisms earlier in the career life span and specifically the childhood period (Cupani et al., 2010; Fouad & Guillen, 2006; Lent & Brown, 2013). Second, IRT was shown to be a useful means of identifying the optimal number of item response categories for the CEOES, the relative fit of the measurement model based on the children’s level of career exploratory outcome expectations, and precise reliability estimates. These examinations would not have been possible if CTT methods were used. The use of IRT in this study might, therefore, stimulate its further application in the career assessment literature (Chartrand & Walsh, 2001; Wetzel et al., 2012).

Footnotes

Acknowledgments

We acknowledge Dr. Robert W. Lent (University of Maryland, USA) for his availability to assist us in this work and to share relevant comments for its improvement.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Portuguese Foundation for Science and Technology through a Doctoral grant (SFRH/BD/84162/2012), supported by POPH/FSE and European Union.

Notes

References

Andrich

D. A.

(1978). A rating formulation for ordered response categories. Psychometrika, 43, 561–573. doi:10.1007/BF02293814

Aryadoust

Goh

C. C. M.

Kim

L. O.

(2012). An investigation of differential item functioning in the MELAB listening test. Language Assessment Quarterly, 8, 361–385. doi:10.1080/15434303.2011.628632

Bandura

(1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

Bandura

(1990). Multidimensional scales of perceived academic efficacy. Stanford, CA: Stanford University.

Bandura

Barbaranelli

Caprara

G. V.

Pastorelli

(2001). Self-efficacy beliefs as shapers of children’s aspirations and career trajectories. Child Development, 72, 187–206. doi:10.1111/1467-8624.00273

Berlyne

D. E.

(1960). Conflict, arousal, and curiosity. New York, NY: McGraw-Hill.

Betz

N. E.

(2000). Self-efficacy theory as a basis for career assessment. Journal of Career Assessment, 8, 205–222. doi:10.1177/106907270000800301

Betz

N. E.

Voyten

(1997). Efficacy and outcome expectations influence career exploration and decidedness. The Career Development Quarterly, 46, 179–189. doi:10.1002/j.2161-0045.1997.tb01004.x

Bond

T. G.

Fox

C. M.

(2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.

10.

Cantante

(2010). A magreza da classe média em Portugal [The thiness of the medium class in Portugal]. Lisboa, Portugal: Diretório das Desigualdades.

11.

Chartrand

J. M.

Walsh

W. B.

(2001). Career assessment: Changes and trends. In Leong

F. T.

Barak

(Eds.), Contemporary models in vocational psychology (pp. 231–256). Mahwah, NJ: Erlbaum.

12.

Cupani

Minzi

M. C.

Pérez

Pautassi

R. M.

(2010). An assessment of a social cognitive model of academic performance in mathematics in Argentinean middle school students. Learning and Individual Differences, 20, 659–663. doi:10.1016/j.lindif.2010.03.006

13.

DeVellis

(1991). Scale development: Theory and applications. Newbury Park, CA: Sage.

14.

Dimitrov

(2012). Statistical methods for validation of assessment scale data in counseling and related fields. Alexandria, VA: American Counseling Association.

15.

de Ayala

R. J.

(2009). The theory and practice of item response theory. New York, NY: The Guilford Press.

16.

Ferry

T. R.

Fouad

N. A.

Smith

P. L.

(2000). The role of family context in a social cognitive model for career-related choice behavior: A math and science perspective. Journal of Vocational Behavior, 57, 348–364. doi:10.1006/jvbe.1999.1743

17.

Fife-Schaw

(2006). Levels of measurement. In Breakwell

G. M.

Hammond

Fife-Schaw

Smith

J. A.

(Eds.), Research methods in psychology (pp. 50–63). London, England: Sage.

18.

Fouad

N. A.

Guillen

(2006). Outcome expectations: Looking to the past and potential future. Journal of Career Assessment, 14, 130–142. doi:10.1177/1069072705281370

19.

Fouad

N. A.

Smith

P. L.

Enochs

(1997). Reliability and validity evidence for the Middle School Self-Efficacy Scale. Measurement and Evaluation in Counseling and Development, 30, 17–31.

20.

Gottfredson

L. S.

(1981). Circumscription and compromise: A developmental theory of occupational aspirations. Journal of Counseling Psychology, 28, 545–579. doi:10.1037/0022-0167.28.6.545

21.

Hambleton

R. K.

Jones

R. W.

(1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12, 38–47. doi:10.1111/j.1745-3992.1993.tb00543.x

22.

Instituto Nacional de Estatística. (2011). Classificação portuguesa das profissões 2010 [Portuguese classification of occupations 2010]. Lisboa, Portugal: Instituto Nacional de Estatística.

23.

Jordaan

J. P.

(1963). Exploratory behavior: The formation of self and occupational concepts. In Super

Starishevsky

Matlin

Jordaan

J. P.

(Eds.), Career development; Self-concept theory (pp. 42–78). Princeton, NJ: College Entrance Examination Board.

24.

Knifsend

C. A.

Graham

(2012). Too much of a good thing? How breadth of extracurricular participation related to school-related affect and academic outcomes during adolescence. Journal of Youth and Adolescence, 41, 379–389. doi:10.1007/s10964-011-9737-4

25.

Leão

(2006). A promoção do desenvolvimento vocacional em context escolar: O(s) tempo(s) e o(s) modo(s) [Promoting career development at the school setting: Time(s) and form(s)]. Revista Portuguesa de Investigação Educacional, 5, 63–78.

26.

Lent

R. W.

(2013). Social cognitive career theory. In Brown

S. D.

Lent

R. W.

(Eds.), Career development and counseling: Putting theory and research to work (pp. 115–146). New York, NY: Wiley.

27.

Lent

R. W.

Brown

S. D.

(2006). On conceptualizing and assessing social cognitive constructs in career research: a measurement guide. Journal of Career Assessment, 14, 12–35. doi:10.1177/1069072705281364

28.

Lent

R. W.

Brown

S. D.

(2013). Social cognitive model of career self-management: Toward a unifying view of adaptive career behavior across the life span. Journal of Counseling Psychology, 60, 557–568. doi:10.1037/a0033446

29.

Lent

R. W.

Hackett

Brown

S. D.

(2004). Una perspectiva social cognitiva de la transición entre la escuela y el trabajo [A social cognitive perspective on school-to-work transition]. Evaluar, 4, 1–22.

30.

Linacre

J. M.

(1999). Category disordering (disordered categories) vs. threshold disordering (disordered thresholds). Rasch Measurement Transactions, 13, 675.

31.

Linacre

J. M.

(2002a). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85–106.

32.

Linacre

J. M.

(2002b). What do Infit and Outfit, Mean-square and Standardized mean? Rasch measurement transactions, 16, 878.

33.

Linacre

J. M.

(2011a). Winsteps rasch measurement (Version 3.72.0) [Computer software]. Chicago, IL: Winsteps.

34.

Linacre

J. M.

(2011b). A user’s guide to WINSTEPS and MINISTEP: Rasch-model computer programs. Program manual 3.72.0. Chicago, IL: Winsteps.

35.

Noack

Kracke

Gniewosz

Dietrich

(2010). Parental and school effects on students’ occupational exploration: A longitudinal and multilevel analysis. Journal of Vocational Behavior, 77, 50–57. doi:10.1016/j.jvb.2010.02.006

36.

Patton

Porfeli

E. J.

(2007). Career exploration for children and adolescents. In Shorikov

V. B.

Patton

(Eds.), Career development in childhood and adolescence (pp. 47–69). Rotterdam, the Netherlands: Sense Publishers.

37.

Pereira

M. C.

Reis

(2014). Grade retention during basic education in Portugal: Determinants and impact on student achievement. Economic Bulletin and Financial Stability Report Articles, 61–83.

38.

Ponte

J. P.

Serrazina

Guimarães

H. M.

Breda

Guimarães

Sousa

… Oliveira

P. A.

(2007). Programa de matemática do ensino básico [Mathematics program in basic education]. Lisboa, Portugal: Ministério da Educação.

39.

Porfeli

E. J.

Lee

Weigold

I. K.

(2012). A multidimensional measure of work valences. Journal of Vocational Behavior, 80, 340–350. doi:10.1016/j.jvb.2011.09.004

40.

Porfeli

E. J.

Wang

Hartung

P. J.

(2008). Family transmission of work affectivity and experiences to children. Journal of Vocational Behavior, 73, 278–286. doi:10.1016/j.jvb.2008.06.001

41.

Reise

S. P.

Moore

T. M.

(2012). An introduction to item response theory models and their application in the assessment of noncognitive traits. In Cooper

Camic

Long

Panter

Rindskopf

Kenneth

(Eds.), APA handbook of research methods in psychology: Foundations, planning, measures, and psychometrics (pp. 699–721). Washington, DC: American Psychological Association.

42.

Rogers

M. E.

Creed

P. A.

Glendon

A. I.

(2008). The role of personality in adolescent career planning and exploration: A social cognitive perspective. Journal of Vocational Behavior, 73, 132–142. doi:10.1016/j.jvb.2008.02.002

43.

Shell

D. F.

Colvin

Bruning

R. H.

(1995). Self-efficacy, attribution, and outcome expectancy mechanisms in reading and writing achievement: Grade-level and achievement-level differences. Journal of Educational Psychology, 87, 386–298. doi:10.1037/0022-0663.87.3.386

44.

Smith

R. M.

(2004). Detecting item bias with the Rasch model. In Smith

J. E. V.

Smith

R. M.

(Eds.), Introduction to rasch measurement (pp. 391–459). Maple Grove, MN: JAM Press.

45.

Stumpf

C. A.

Colarelli

S. M.

Hartman

(1983). Development of the Career Exploration Survey (CES). Journal of Vocational Behavior, 22, 191–226. doi:10.1016/0001-8791(83)90028-3

46.

Super

D. E.

(1980). A life-span, life-space approach to career development. Journal of Vocational Behavior, 16, 282–298. doi: 10.1016/0001-8791(80)90056-1

47.

Taveira

M. C.

(2000). Exploração e desenvolvimento vocacional de jovens [Exploration and career development in adolescence]. Universidade do Minho, Braga: Instituto de Educação e Psicologia.

48.

Taveira

M. C.

Moreno

M. L.

(2003). Guidance theory and practice: The status of career exploration. British Journal of Guidance and Counseling, 69, 248–261. doi:10.1080/0306988031000102360

49.

Teixeira

M. O.

(2009). Uma medida de autoeficácia percebida em contextos sociais e académicos [A measure of perceived self-efficacy in social and academic contexts]. Psychologica, 51, 47–55.

50.

Teixeira

M. O.

Carmo

A. M.

(2004). Estudos com a versão Portuguesa da Escala Multidimensional de Autoeficácia Percebida de Bandura (MSPSE) [Studies with the Portuguese version of Bandura’s Multidimensional Scales of Perceived Self-Efficacy]. In Machado

Almeida

Gonçalves

Carvalho

(Eds.), Avaliação psicológica: Formas e contextos (pp. 198–203). Braga, Portugal: Psiquilíbrios.

51.

Tracey

T. J.

Lent

R. W.

Brown

S. D.

Soresi

Nota

(2006). Adherence to RIASEC structure in relation to career exploration and parenting style: Longitudinal and idiothetic considerations. Journal of Vocational Behavior, 69, 248–261. doi:10.1016/j.jvb.2006.02.001

52.

Wetzel

Hell

Pässler

(2012). Comparison of different test construction strategies in the development of a gender fair interest inventory using verbs. Journal of Career Assessment, 21, 88–104. doi:10.1177/1069072711417166

53.

Zaff

J. F.

Moore

K. A.

Papillo

A. R.

Williams

(2003). Implications of extracurricular activity participation during adolescence on positive outcomes. Journal of Adolescent Research, 18, 599–630. doi:10.1177/0743558403254779