Precision in Career Motivation Assessment

Abstract

The Subjective Science Attitude Change Measures (SSACM; Stake & Mares, 2001) represent a collection of useful self-report tools for assessing change in high school students’ science attitudes as a function of a given motivational intervention. Despite the survey’s utility, little work has been done to examine this tool among other samples (i.e., college students) or to test the psychometric properties and overall construct validity of SSACM scores. Participants (N = 1,368) consisted of undergraduate students enrolled in biology, chemistry, and physics laboratory classes. Analysis of the SSACM’s factor structure using exploratory structural equation modeling indicated support for a bifactor structure consisting of one general science motivation factor and three specific factors labeled intrinsic science interest, science career identity, and science self-efficacy. This model outperformed alternative bifactor and specific two- and three-factor models. Results largely yielded evidence of concurrent validity, as three of the four scale scores were significant positive predictors of relevant outcomes over and above the contribution of gender, parental occupation type, and mastery motivation. Implications for science career counseling and assessment are discussed.

Keywords

career motivation science self-efficacy science interest career identity

The skills that come from science, technology, engineering, and mathematics (STEM) domains are “the skills today’s employers are looking for to fill jobs right now and in the future,” as articulated by President Barack Obama in his 2013 State of the Union address. Jobs in STEM fields are clearly in high demand, and labor market forecasts suggest this will continue well into the future (Bureau of Labor Statistics, 2012). The ever-present need for qualified workers in these fields has given rise to a proliferation of intervention programs designed to facilitate student interest in science. Many of these intervention programs target K–12 students (e.g., Bischoff, Castendyk, Gallagher, Schaumloffel, & Labroo, 2008; Harackiewicz, Rozek, Hullerman, & Hyde, 2012), but college students benefit from such programs as well (Hulleman, Godes, Hendricks, & Harackiewicz, 2010). College students switch out of STEM majors at a much greater rate than they change into them (Seymour & Hewitt, 1997), and of undeclared students, only 16% ultimately choose to major in an STEM field (National Center for Education Statistics, 2009). Thus, even “catching” student interest in secondary school does not always “hold” into the college years (Hidi, 2000). Of course, the most traditional and ubiquitous of intervention opportunities, within the college classroom, remain one of the most effective places for facilitating change in science attitudes. College STEM educators are therefore working hard to capture the interest of their students, and, consequently, strong measurement tools are needed to evaluate whether these efforts are working.

Subjective Science Attitude Change Measures

Educators are typically concerned with evaluating the scientific and/or technical skills of their students. This can be done by evaluating the product of his or her work (e.g., speed or precision of an engineered machine or judges’ ratings of a project) or simply evaluating their level of knowledge in a given content area. However, a student’s development of skill does not guarantee that he or she will perceive themselves as interested or even competent in science. Subjective ratings of these outcomes are needed to supplement objective measures. The subjective science attitude change measures (SSACM; Stake & Mares, 2001) represent such a collection of rating scales used to assess the impact of science intervention programs.

The SSACM were originally developed for use with high school students and their parents for the purpose of measuring the impact of science program interventions. The SSACM subscales include (a) increased science motivation (ISM), (b) increased science confidence (ISC), (c) increased science knowledge (ISK), and (e) new social niche (NSN; see Table 1). ISK taps instructional effects and NSN measures perceptions of social cohesion, but neither was included in the present study because the former reflects material that is more pertinent to educational researchers, whereas NSN refers to social situations in high school settings. There are two unique features of the SSACM that distinguish them from other science motivation measures. First, they measure changes in science motivation at a proximal level of specificity because their items are designed to tap intervention-specific effects rather than global attitude change. Most researchers use global measures of science attitudes and make inferences about causal change using pretest–posttest designs; unfortunately, positive gains in student science attitudes are difficult to sustain, as scores often return to pretest levels when measured at follow-up (e.g., Stake & Mares, 2005). Second, the SSACM contain a parent version that provides researchers and educators with more objective data regarding student attitude change. The SSACM are therefore ideal for conducting multitrait, multimethod studies that can in turn offer greater insight into the construct validity of SSACM scores. Along these lines, Stake and Mares (2001) obtained evidence of significant positive correlations between parental and student ratings of ISM (r = .52) and ISC (r = .43). Their results also revealed significant positive correlations between SSACM scores on both ISM and ISC and pretest–posttest difference scores on global measures of ISM and ISC.

Table 1

Subjective Science Attitude Change Measures–Student Version.

Increased science motivation. My experiences in the program…

1. Made science seem more interesting to me.

2. Stimulated my enthusiasm for science.

3. Increased my interest in a science career.

4. Made science seem more fun.

5. Clarified for me what I want to do in a science career.

6. Made me feel more sure that I want a career in science.

Increased science confidence. My experiences in the program…

1. Made me feel more relaxed about learning science.

2. Increased my confidence in my ability to do science.

3. Increased my confidence that I can succeed in science as a career.

4. Made me feel more self-assured as a student of science.

5. Made the idea of a science career for me seem more possible.

6. Increased my confidence that I could handle science courses at the college level.

Increased science knowledge.My experiences in the program…

1. Enhanced my knowledge of science.

2. Allowed me to learn science techniques I didn't know before.

3. Gave me “hands-on” experience that will help me in the future with science projects and activities.

4. Gave me a better understanding of what science is all about.

5. Gave me a better grasp of how to write a research paper.

6. Helped me learn problem-solving skills needed in science.

New social niche. My experiences in the program…

1. Helped me see that many other students like science, just as I do.

2. Made me realize that some other people my age really respect and appreciate other students who are good at science.

3. Gave me an opportunity to meet students from other schools who were more enjoyable to be with than many of the students at my high school.

4. Gave me an opportunity to make friends with people who are a lot like me.

5. Made me realize it is okay to be smart and like science, even if some students in my high school don't think so.

Note. From “Science enrichment programs for gifted high school girls and boys: Predictors of program impact on science confidence and motivation”, by J. E. Stake and K. E. Mares, 2001, Journal of Research in Science Teaching, 38, pp. 1065-1088. Reprinted with permission from John Wiley.

Our review of the extant literature resulted in no other research on the SSACM other than the original Stake and Mares (2001) study. This is unfortunate because the SSACM hold considerable promise for use with college student populations. This is particularly true of college freshmen who may be undecided as to whether to declare science as their major area of study. In a similar vein, older students who may be struggling with a weakening commitment to their science major might benefit from the motivational “boost” that an intervention can provide. SSACM scores can thus be used as indices of such adaptive motivational effects or as predictors of student intentions to change from science to nonscience majors. In short, administering the SSACM in the context of a college science course could yield useful information for students, educators, and counselors alike as important career-related decisions such as these are being considered.

The ISM and ISC scales each purport to have one latent factor underlying their respective items but this is speculative because Stake and Mares (2001) did not report any factor structures. At face value, our reading of the items suggests there may be three distinct factors underlying the 12 items. The ISM scale contains items that appear to measure the instantiation of science interest in that they reflect students’ phenomenological experiences of interest (e.g., “Made science seem more interesting to me” and “Stimulated my enthusiasm for science”). Although individuals often feel excitement or pleasure when they are interested in something, positive affect does not define interest alone. Interest development also involves collating information about the environment and comparing these features against schemas of related experiences (Berlyne, 1960; Durik & Harackiewicz, 2007). Prominent among these features is novelty, as tasks, objects, and experiences that are incongruent with established schemas are thought to stimulate emotional arousal and curiosity (Silvia, 2005). Scherer (2001) referred to this type of appraisal as a novelty check. It is important to highlight the importance of novelty because educational theory and research (e.g., Brophy, 1999, 2008) consistently suggest that teaching material in ways that are novel and creative increases student task engagement. The experience of interest thus involves both cognitive and affective components, directs students’ attention during task engagement, and even facilitates cognitive resource replenishment needed for sustaining motivated task engagement (Sansone & Smith, 2000; Sansone & Thoman, 2005; Silvia, 2008; Thoman, Smith, & Silvia, 2011). Given that various ISM items seem to tap into aspects of the experience of interest, we refer to the factor that is defined by these items as intrinsic science interest (ISI) in the present study.

Several items clearly map onto ISC (e.g., “Increased my confidence in my ability to do science”), which we would subsume under the hypothesized factor science self-efficacy (SSE),¹ but there are also items from both of the original scales that reflect a mixture of science interest and SSE. Some of these items explicitly measure interest development, whereas others clearly measure SSE, but the element that unifies the ISM and ISC scales appears to be an orienting of the individual to distal outcomes. Items that measure increased career focus (e.g., “Clarified for me what I want to do in a science career”) and goal attainability (e.g., “Made the idea of a science career for me seem more possible”) represent an orientation toward the future, which is thought to be a key determinant of vocational identity (Diemer & Blustein, 2007; Marko & Savickas, 1998; Super, 1980). Short- and long-term (e.g., career) goals are brought into view because skill development leads to the generation of both efficacy percepts and expectancies of positive future outcomes (Lent, Brown, & Hackett, 1994). The synergistic effects of real-time performance feedback, greater confidence in the moment (Nauta & Kahn, 2007), and adaptive expectancies should foster identity development by bridging one’s sense of current self with one’s sense of future self. It is this cognitive consistency in attitudes—from perceptions of present self as a science student to projections of future self as potential scientist—that are thought to contribute to the formation of a firm science identity. We would thus label the latent variable that gives rise to these scores as science career identity (SCI). There are a number of items on the ISM and ISC scales that appear to cohere around this idea, and we explore this possibility in the present study.

Present Study

The primary purpose of the present study was to evaluate the factor structure of the ISM and ISC scales of the SSACM using a college sample of students enrolled in science classes. SSE, interest, and identity are all distinct constructs because they are regulated by slightly different psychological systems (e.g., affective vs. cognitive) and from different temporal orientations. A good science intervention program should address all three by triggering interest and competence perceptions, and initiating the process of integrating interest and self-efficacy into one’s self-concept. We believe that science motivation in its entirety cannot exist independent of these motivational influences. We tested this proposition by conducting a bifactor analysis, whereby ISM and ISC items were allowed to load simultaneously on a general motivation factor and specific latent constructs. Bifactor models are similar to higher order models in that they possess nested structures and are, in fact, mathematically equivalent when proportionality constraints are imposed (Brunner, Nagy, & Wilhelm, 2012). However, bifactor models are thought to be preferable when specific factors are the focus of attention, and the researcher wishes to link both general and specific factors to external variables (Chen, West, & Sousa, 2006). Such is the case in the present study, as we were interested in the extent to which ISI, SSE, and SCI explain variation in relevant outcomes over and above a general factor.

Thus, a second purpose of this study was to evaluate the concurrent validity of the specific factor scores. Four outcomes were chosen for these analyses: (a) absolute task mastery (ATM) motivation for research; (b) intrinsic science motivation; (c) intention to conduct undergraduate research; and (d) science identity. If college students are highly motivated to understand and develop skill in science, then it is reasonable to suspect that they would also be motivated to develop their skills in formal research endeavors. ATM motivation is thought to be a logical outcome of science motivation because involvement in formal research is a necessary step for students to take if they are to pursue scientific careers. Global science motivation was thus hypothesized to be a positive predictor of ATM motivation. Similarly, interest is considered a psychological state resulting from an initial or recurring engagement with an object or activity (Renninger, 2000). The potency of the attention and affect generated by engagement with the task determines whether interest is internalized into the self as a need for competence (science competence in the present case), which is a key component of self-determination theory (SDT; Deci & Ryan, 1985; Ryan & Deci, 2000). ISI was therefore hypothesized to be a significant positive predictor of intrinsic science motivation. Finally, SCI and SSE were hypothesized to be significant positive predictors of general forms of science identity and science confidence, respectively.

The specific factor scores were presumed to demonstrate these concurrent relations with outcomes while statistically controlling for potentially confounding predictors. Because there is a significant association between parental involvement and their children’s math/science career goals (Byars-Winston & Fouad, 2008), we modeled parental occupational type (STEM vs. non-STEM) as a predictor of science motivation. We considered the possibility that variation in the validity analysis outcomes could be accounted for by a general tendency to approach the science lab class with an aim of mastering course content; therefore, mastery motivation was estimated as a statistical control variable. Gender was also included as a covariate, given that women tend to be subject to motivational deficits, particularly lower self-efficacy (e.g., Deemer, Thoman, Chase, & Smith, in press; Rice, Lopez, & Richardson, 2013), owing in large part to environmental factors.

Method

Exploratory Structural Equation Modeling

Researchers often favor the use of confirmatory factor analysis (CFA) in validating psychological tests and measurement instruments because of its stringent requirement of specifying unobserved variables to be modeled as well as patterns of zero and nonzero factor loadings. Factor analytic models often do not conform to such strict expectations because scores of theoretical interest to researchers may share more than one underlying source of variation. In fact, some models estimated by researchers have such complex structures that they cannot be meaningfully interpreted without the aid of factor rotation and item cross loading (Holden & Fekken, 1994; McCrae, Zonderman, Costa, Bond, & Paunonen, 1996). Contributing to this problem is the fact that CFA models sometimes fail to meet the strict fit criteria established by methodology experts (Marsh, Hau, & Grayson, 2005), which compels many investigators to engage in the less than desirable practice of making post hoc modifications to improve the fit of their models (Mueller, 1997).

In contrast, psychological self-report measures often demonstrate acceptable psychometric properties when evaluated using exploratory factor analysis (EFA), but many researchers are reluctant to use this approach because it lacks the sophistication and rigor of CFA. Exploratory structural equation modeling (ESEM; Asparouhov & Muthén, 2009) represents an advancement in factor analysis in that it combines both EFA and CFA in a way that captures the strengths of each approach. ESEM utilizes the strength of traditional EFA by permitting factor rotation through free estimation of all item loadings; this increases the clarity of the factor structure and renders models easier to interpret. The advantage of the CFA aspect of ESEM is that it takes into account the effects of measurement error while also providing the model fit indices and population parameter estimates that are so commonly reported in the psychological literature (Marsh, Liem, Martin, Morin, & Nagengast, 2011). Because the SSACM scales under investigation in the current study are yet to be adequately explored and are hypothesized to have a complex (bifactor) structure, ESEM represents an appropriate choice of methodology.

Participants

A total of 1,368 students participated in the study (53.6% women). Participants’ mean age was 20.70 years (standard deviation [SD] = 3.12) and ranged from 18 to 48. Reported ethnic identities were as follows: (a) White/Anglo American (61.6%), (b) Asian/Asian American (17.3%), (c) Latino (8.4%), (d) African/African American (6.6%), (e) multiracial (3.1%), (f) other (1.7%), Native American (0.8%), and (g) Arabic/Arab American (0.6%). Just over one fourth (26.8%) of the sample reported majoring in something other than STEM, while 0.9% had not yet declared a major. The remainder of the sample reporting majoring in an STEM field, with most identifying biology (31.7%), followed by engineering (28.4%), chemistry/biochemistry (8.8%), computer science (1.4%), mathematics (1.2%), and physics (0.8%).

Measures

SSACM

As noted previously, the SSACM consist of four scales that measure changes in students’ science attitudes and knowledge in psychological and educational domains ranging from affective to social. Responses are scored on a Likert-type scale ranging from 1 (not at all) to 7 (a great deal). Because the substantive focus of the present study was on perceptions of competence and motivation, only the ISM and ISC scales were used. The ISM scale measures the cognitive–affective dimension of science interest by tapping immediate affective reactions to interventions (e.g., “Stimulated my enthusiasm for science”) as well as their influence in shaping appraisals of career goal intentions (e.g., “Clarified for me what I want to do in a science career”). The ISC scale similarly measures proximal and distal effects on students’ perceptions of competence in science as well as feelings of physical well-being associated with knowledge acquisition (i.e., “Made me feel more relaxed about learning science”). The ISK scale measures science-related cognition and behavior with foci ranging from the acquisition of general problem-solving ability to specific scientific skills. The NSN scale evaluates the social dimension of science learning through its assessment of students’ feelings of belonging and cohesion that come as a result of science participation.

ATM Motivation

Mastery motivation for conducting research was measured using the 6-item ATM subscale of the Achievement Goals for Research Scale (AGRS; Deemer, Carter, & Lobrano, 2010). Grounded in achievement goal theory (Dweck & Leggett, 1988; Elliot & McGregor, 2001; Nicholls, 1984), the AGRS is a 25-item instrument that measures the approach- and avoidance-based reasons for engaging in scientific research. The AGRS is comprised of six subscales: (a) ATM, (b) incremental task mastery, (c) self-demonstration of competence, (d) mastery avoidance, (e) performance approach, and (f) performance avoidance. Responses are scored on a Likert-type scale ranging from (1) strongly disagree to (5) strongly agree. ATM scores have been shown to be internally consistent (α = .90) and demonstrate convergent validity through significant positive correlations with drive motivation and sensitivity to rewarding stimuli (Deemer et al., 2010).

Intrinsic Science Motivation

Intrinsic motivation for science was measured using the 5-item intrinsically motivated science learning (IMSL) subscale of the Science Motivation Questionnaire (SMQ; Glynn & Koballa, 2006). The SMQ is a 30-item self-report instrument that covers motivational content using the following subscales: (a) intrinsically motivated science learning, (b) extrinsically motivated science learning, (c) relevance of learning science, (d) responsibility for learning science, (e) confidence learning science, and (f) anxiety about science assessment. Participants rate their response to the statement “When I am in a college science course . . .” on a Likert-type scale ranging from 1 (never) to 5 (always). An example item includes “The science I learn is more important to me than the grade I receive.” Coefficient α for IMSL was .84 in the present study.

Mastery Motivation

The mastery approach goal subscale of the Achievement Goal Questionnaire–Revised (AGQ-R; Elliot & Murayama, 2008) was used to measure mastery motivation. The AGQ-R consists of 12 items that measure four achievement goal constructs (3 items each): (a) mastery approach; (b) mastery avoidance; (c) performance approach; and (d) performance avoidance. Mastery goals refer to desires to either develop skill and understanding or avoid being unable to do so. Performance goals are normatively based aims that orient individuals toward either outperforming others or avoidance of performing poorly relative to others. Participants respond on a Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). An example item includes “I am striving to understand the content of this course as thoroughly as possible.” Elliot and Murayama (2008) obtained evidence of the reliability of the mastery approach items (α = .84); Cronbach’s α was .86 in the present study.

Confidence Learning Science

The 5-item confidence learning science (CLS) subscale of the SMQ (Glynn & Koballa, 2006) was used to measure science confidence as a predicted outcome of the specific SSE factor in the present study. CLS is different from SSE in that it measures science confidence not perceived to be attributed to an instructional intervention yet still retains sufficient specificity by measuring science confidence at the level of the classroom. In other words, CLS does not measure global science confidence. An example item includes “I am confident I will do well on the science labs and projects.” Responses are scored on a Likert-type scale ranging from 1 (never) to 5 (always). The reliability of the CLS scale has been supported in past research (α = .89; Taasoobshirazi & Glynn, 2009). The CLS scale possessed good internal consistency in the present study (α = .89). The predictive validity of the CLS scale has been demonstrated through a significant positive association with problem-solving ability in science (Taasoobshirazi & Glynn, 2009).

Science Identity

Doosje, Ellemers, and Spears’ (1995) group identification items were adapted for the present study to measure identification with science. Doosje et al. developed the 4-item measure to tap the cognitive and affective dimensions of academic identity. An example of an original item includes “I see myself as a psychology student.” The adapted items were as follows: (a) “I see myself as a science student”; (b) “I am pleased to be a science student”; (c) “I feel strong ties with other science students”; and (d) “I identify with other science students.” One additional item was developed to provide more extensive coverage of the science identity construct—“I feel that being a science student is an important reflection of who I am.” These 5 items were found to possess sufficient internal consistency (α = .85). Participants respond on a Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree). Doosje et al. (1995) obtained evidence of the original items’ discriminant validity by showing that they differentially predict positive perceptions of in-group members relative to out-group members.

Background Variables

Other variables to be controlled for in the concurrent validity analysis, other than mastery motivation, included gender and the potential influence of parents’ occupation type on participants’ science motivation. Gender was dummy coded such that men were assigned a code of 0, and women were assigned a code of 1. Parent occupation type was also dummy-coded (non-STEM = 0, STEM = 1). STEM occupations were operationally defined as those in which STEM was either a vocational application (e.g., engineer) or instructional focus (e.g., high school math teacher, science professor); all other occupations (e.g., law, business, healthcare, etc.) were defined as non-STEM.

Procedure

Data were collected from participants at three U.S. universities across four academic terms from January 2011 to November 2012. All data were collected using an Internet-based survey. Laboratory classes were operationalized as the intervention mechanisms for fostering change in student science attitudes, given that laboratories afford opportunities to engage in science tasks, as opposed to lecture settings, where learning is more of a passive activity. Course rosters from chemistry, biology, and physics classes were obtained from the registrar’s office of each university and e-mail messages requesting student participation were subsequently distributed at the midpoint of each academic term. Upon submitting their responses, participants were thanked for their involvement and directed to a webpage that contained a debriefing statement describing the purpose of the study. Participants received a US$10 electronic gift card as compensation for their efforts.

Results

Exploratory Bifactor Analysis

A model with all indicators specified to load on a single first-order factor would represent a logical rival hypothesis to a bifactor model, given that bifactor models are partly defined by general factors, but one-factor models cannot be rotated in EFA. We submitted such a model to CFA using a randomly drawn subset (n = 823) of the overall sample. Although the standardized factor loadings were quite strong, ranging from 0.73 to 0.92, the fit of the model was poor, χ²(54, N = 823) = 1,512.35, p = .00, CFI = 0.88, TLI = 0.86, RMSEA = 0.18 (90% CI: [0.17, 0.19]), SRMR = .04. Having ruled out this model as a viable alternative, we proceeded to perform an EFA using an ESEM approach (Asparouhov & Muthén, 2009). The hypothesized bifactor model was fitted and compared to a number of alternative models: (a) a bifactor model with one general and two specific motivation and confidence factors; (b) a two-factor model reflecting the original SSACM structure; and (c) a three-factor model that included separate SSE, ISI, and SCI factors. A one-factor model could not be tested, given that multiple factors are needed for rotation. Orthogonal rotation methods were used, given that general and specific factors in the bifactor framework are assumed to be uncorrelated (Chen et al., 2006). Geomin rotation was used for the specific-factor models and bigeomin rotation was used for the bifactor models. Fit indices used to assess model fit included the (a) model chi-square test, (b) comparative fit index (CFI), (c) Tucker-Lewis index (TLI), (d) standardized mean square error of approximation (RMSEA), and (d) standardized root mean square residual (SRMR).

Fit statistics are reported in Table 2. Fit index values for the two-factor model were in the acceptable range with the exception of RMSEA, the value for which exceeded the 0.10 threshold for marginal fit (Browne & Cudeck, 1993). The three-factor and alternative bifactor models offered a better fit to the data as indicated by CFI, TLI, and SRMR values, but RMSEA values were still in the marginal range. The alternative bifactor model was statistically equivalent to the three-factor model in terms of the fit statistics but yielded a much different pattern of factor loadings due to the bigeomin rotation (see Table 3). The hypothesized bifactor model produced a more acceptable RMSEA value of 0.06, and standardized loadings for the general factor were slightly higher (mean λ = .865) as compared to general factor loadings in the alternative bifactor model (mean λ = .860). SSACM items were not originally intended to be reverse scored, thus rendering negative factor loadings difficult to interpret. For this reason, only positive factor loadings in the hypothesized model were given consideration to comprise a potential subscale. Specific factor loadings were all fairly low, but this is to be expected as specific factors account for only score variance not accounted for by general factors (Brunner et al., 2012). The following items were therefore identified as producing the highest loadings: (a) Items 1, 2, 3, and 7 for ISI; (b) Items 5, 9, 10, and 11 for SCI; and (c) Items 8 and 12 for SSE. The general factor scale was labeled simply as SSACM Motivation (SSACMM) to refer to a global science motive collectively defined by interest, identity, and self-efficacy. Factor scores were saved from the bifactor analysis and used as predictor variables in the concurrent validity analysis.

Table 2

Summary of ESEM-EFA Model Fit Statistics.

Model	χ²	df	CFI	TLI	RMSEA	90% CI for RMSEA	SRMR
Two factor	594.29	43	.96	.93	.12	[.12, .13]	.02
Three factor	271.23	33	.98	.96	.09	[.08, .10]	.01
Alternative bifactor	271.23	33	.98	.96	.09	[.08, .10]	.01
Hypothesized bifactor	85.61	24	.99	.99	.06	[.04, .07]	.01

Note. CFI = comparative fit index; CI = confidence interval; EFA = exploratory factor analysis (EFA); ESEM = exploratory structural equation modeling; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; TLI = Tucker-Lewis index.

Table 3

Standardized Factor Loadings for ESEM EFA Model.

	Two-Factor Model		Three-Factor Model			Alternative Bifactor Model			Hypothesized Bifactor Model
Item	MOT	CONF	ISI	SCI	SSE	GF	MOT	CONF	MOT	ISI	SCI	SSE
SSACM1	.87	−.00	1.01	.00	−.15	.83	.38	−.02	.84	.33	−.04	−.00
SSACM2	.96	−.10	.77	−.02	.18	.86	.19	−.13	.86	.15	−.14	.01
SSACM3	.86	.08	.86	.12	−.01	.90	.28	−.01	.91	.24	−.04	−.08
SSACM4	.98	−.08	.54	.01	.46	.91	.01	−.20	.91	−.04	−.22	−.01
SSACM5	.52	.42	.45	.48	.02	.87	.12	.17	.89	.07	.14	−.12
SSACM6	.65	.30	.27	.40	.35	.92	−.06	.03	.92	−.12	.00	−.06
SSACM7	.80	.08	.76	.13	.00	.84	.24	.00	.84	.27	.01	.21
SSACM8	.77	.16	.41	.26	.34	.92	.00	−.03	.92	−.02	−.04	.32
SSACM9	−.01	.83	.00	.88	−.10	.71	.00	.42	.71	−.00	.41	.02
SSACM10	.19	.77	.02	.85	.09	.87	−.06	.34	.87	−.08	.33	−.00
SSACM11	.01	.95	−.01	1.03	−.10	.83	−.01	.49	.83	−.02	.47	−.02
SSACM12	.51	.36	−.00	.49	.48	.85	−.19	.04	.83	−.19	.06	.11

Note. SSACM = subjective science attitude change measure; MOT = motivation; CONF = confidence; ISI = intrinsic science interest; SCI = science career identity; SSE = science self-efficacy; GF = general factor.

Hierarchical Regression Analyses

To evaluate the concurrent validity of the SSACM scores, we conducted a series of hierarchical regression analyses using a subsample of 458 participants. Means, SDs, and correlations for the analysis variables are reported in Table 4. Null correlations among the factor scores (rs ranged from –.06 to .06) reflect the orthogonality of the factors. Gender, parent occupation type, and mastery motivation were included as covariates on Step 1. Only individual factor scores were entered on the second step of the hierarchical regressions in order to maintain consistency with the orthogonal structure of the bifactor model (see Table 5). Regressing ATM motivation on Step 1 of the first analysis resulted in 20% of the variance accounted for by the predictors, F(4, 435) = 27.52, p < .001, while SSACMM explained an additional 12.1% of the variance, ΔF(1, 434) = 77.69, p < .001. Mothers in STEM occupations (β = .10, p = .03) and mastery approach goals (β = .44, p < .001) were significant positive predictors of ATM motivation on Step 1, but the former coefficient was reduced to nonsignificance (β = .06, p = .15) when SSACMM was added to the regression equation (β = .38, p < .001). The Step 1 model of the second analysis was significant, F(4, 453) = 45.07, p < .001, with the collection of predictors accounting for 28.5% of the variance in intrinsic science motivation. Only mastery motivation was found to be a significant predictor of intrinsic science motivation (β = .53, p < .001), whereas mothers in STEM (β = .07, p = .10) and gender (β = −.06, p = .13) were marginally nonsignificant. ISI accounted for a unique 0.8% of the variance in intrinsic science motivation on Step 2, ΔF(1, 452) = 5.10, p = .02 (β = .09).

Table 4

Zero-Order Correlations and Descriptive Statistics for the Regression Analysis Variables.

Variable	1	2	3	4	5	6	7	8	9
1. ATMM	—
2. SCI	.21***	—
3. ISI	.06	−.02	—
4. ISM	.65***	.15**	.12**	—
5. MAP	.44***	.07	.03	.51***	—
6. CLS	.49***	.02	−.07	.68***	.46***	—
7. SI	.55***	.27***	−.01	.67***	.39***	.54***	—
8. SSACMM	.50***	.02	.03	.60***	.40***	.55***	.53***	—
9. SSE	.02	−.06	−.06	.01	−.04	.03	.00	.05	—
10. M	3.79	.00	.00	3.86	4.14	3.88	5.23	.00	.00
11. SD	.76	.88	.83	.73	.76	.71	1.14	.99	.81

Note. ATMM = absolute task mastery motivation; SCI = science career identity; ISI = intrinsic science interest; ISM = intrinsic science motivation; MAP = mastery approach goal; CLS = confidence learning science; SI = science identity; SSACMM = subjective science attitude change measures motivation; SSE = science self-efficacy.

*p < .05. **p < .01. ***p < .001.

Table 5

Results of Hierarchical Regression Analyses Predicting Concurrent Validity Outcomes.

	ATM Motivation					Intrinsic Science Motivation					Science Identity					Confidence Learning Science
Variable	b	SE	β	R ²	Δ R ²	b	SE	β	R ²	ΔR ²	b	SE	β	R ²	ΔR ²	b	SE	β	R ²	ΔR ²
Step 1				.202**					.285**					.159**					.215**
Gender	−.27	.39	−.03			−.45	.29	−.06			−.08	.60	−.01			−.13	.30	−.02
MOCC	.98	.45	.10*			.55	.34	.07			.61	.70	.04			.31	.35	.04
FOCC	−.12	.34	−.02			.16	.25	.03			−.47	.52	−.04			.11	.26	.02
MAP	.86	.08	.45**			.83	.06	.53**			1.18	.13	.40**			.71	.06	.46**
Step 2				.323**	.121**				.293**	.008*				.215**	.056**				.218**	.004
Gender	.00	.36	.00			−.47	.29	−.07			.12	.58	.01			−.11	.30	−.02
MOCC	.60	.42	.06			.52	.34	.06			.52	.68	.03			.29	.35	.04
FOCC	−.03	.31	−.00			.17	.25	.03			−.46	.51	−.04			.11	.26	.02
MAP	.56	.08	.29**			.83	.06	.53**			1.14	.13	.38**			.71	.06	.47**
SSACMM	1.75	.20	.38**			—	—	—			—	—	—			—	—	—
ISI	—	—	—			.38	.17	.09*			—	—	—			—	—	—
SCI	—	—	—			—	—	—			1.80	.32	.24**			—	—	—
SSE	—	—	—			—	—	—			—	—	—			.26	.18	.06

Note. ATM = absolute task mastery; FOCC = father's occupation; ISI = intrinsic science interest; MAP = mastery approach goal; MOCC = mother's occupation; SCI = science career identity; SSACMM = subjective science attitude change measures motivation (general factor); SSE = science self-efficacy.

*p < .05. **p < .001.

In the third analysis, science identity was regressed on the set of statistical control variables on Step 1 and SCI on Step 2. The Step 1 model fit the data well, F(4, 445) = 21.04, p < .001, with the covariates accounting for 15.9% of the variance in science identity. Mastery approach goal was the only significant predictor in the Step 1 model (β = .40, p < .001). The variance explained in science identity increased by 5.6% when SCI was added to the equation on Step 2, ΔF(1, 444) = 31.42, p < .001 (β = .24). Finally, the Step 1 model for predicting CLS was significant, F(4, 454) = 31.02, p < .001, as the coefficient for mastery approach goal was significant (β = .46, p < .001), and the entire set of covariates explained 21.5% of the variance. Contrary to expectation, SSE was found to be a nonsignificant predictor of CLS (β = .06) as the variable accounted for only 0.4% of the variation on Step 2, ΔF(1, 453) = 2.08, p = .15.

Discussion

The current study examined the psychometric properties of the motivation-relevant scales of the SSACM using a college sample of science students. Results revealed that these scales possess a bifactor structure and are predictive of theoretically similar constructs. Results of the ESEM analysis indicated that a bifactor model with one general factor and three specific factors outperformed several alternative models, including a bifactor model consisting of a general factor and the two original scales in question—ISM and ISC. Previous research on the validity of SSACM scores had also been severely lacking; therefore, our study addressed a problematic oversight in the literature. Specifically, the hypothesized general and specific SSACM variables were predictive of such outcomes as mastery motivation for research, intention to conduct undergraduate research, and science identity.

Results of the bifactor analysis indicated that all 12 of the items examined loaded rather impressively onto a general science motivation factor, while separate groups of items demonstrated different patterns of loading onto the three hypothesized specific factors. Stake and Mares’ (2001) original conceptualization of ISM called for the measurement of both science identity and science interest in one scale, but our findings suggest that the two should be decomposed into separate constructs. Interest can be fleeting, particularly situational interest because its formation is dependent upon often mutable factors within the external environment (Hidi & Renninger, 2006; Krapp, 2002). In contrast, our results show that SCI can be thought of as a stable aspect of one’s self-concept that is essentially a by-product of the synthesis of interest, goals, and self-efficacy (Eccles, 2009; Osborne & Jones, 2011). The fact that the specific factor loadings for SCI were stronger than the loadings for ISI and SSE fits with the “by-product” conceptualization of SCI because while SCI was found to be an important contributor to general science motivation, it is an entity that is at the same time more independent of motivation than ISI and SSE are. This would suggest that SCI should be a stronger predictor of theoretically related outcomes than either ISI or SSE, and this was found to be the case as the hierarchical regression analyses indicated that SCI predicted science identity more strongly than ISI and SSE predicted their respective outcomes.

Contrary to expectation, SSE was not a significant predictor of CLS. Recall, however, that in bifactor analysis most of the variance in item scores is explained by the general factor, while the remaining variance is accounted for by specific factors and measurement error. This partitioning of variance is thus not unlike the progressive decline in the magnitude of eigenvalues with each factor extracted in traditional EFA. It seems that most of the variance in SSACM scores was extracted by the SSACMM, SCI, and ISI factors and very little extracted by SSE. Moreover, only 2 items loaded meaningfully on SSE, whereas 2 items that were originally theorized by Stake and Mares (2001) to reflect ISC—“Increased my confidence in my ability to do science” and “Increased my confidence that I can succeed in science as a career”—loaded only on SSACMM. One possible reason why SSE extracted the smallest proportion of variance is that one science laboratory course may not be sufficient to make a pronounced impact on one’s confidence. Students may report more interest in science after having taken a class in it, but this does not mean that they will necessarily feel more confident in their science ability. Indeed, it is more likely that students need repeated exposure to science tasks to render the requisite mastery experiences (Bandura, 1997) needed for self-efficacy to take shape.

Implications for Career Counseling

Counselors in university counseling center settings often work with students who are undecided as to what to major in while in college and/or with students who are considering switching majors. In either case, college students may “sample” science classes, particularly lower level courses (e.g., introductory chemistry), to determine whether a particular major may be of (more) interest to them. We suggest that career counselor may use the SSACM as a tool to explore with students what characteristics of a particular science class led them to feel more or less interested in science or confident in their science abilities. Comparing SSACM across classes could also provide valuable differential information about important determinants of interest, self-efficacy, and/or identity development. For instance, a female student may rate a physics class as less interesting than a chemistry class, but upon deeper analysis it is learned that the student attributes her low interest in physics to feelings of isolation and working harder than other students because she had been the only woman in a male-dominated class (e.g., Smith, Lewis, Hawthorne, & Hodges, 2013). In sum, discussion of instructional, contextual, and internal factors using SSACM scores as a guide can assist the career counselor in exploring optimal person–environment fit (Rounds & Tracy, 1990) with the client.

Limitations

Some limitations in the present study warrant mentioning. First, because participant responses were obtained at the midpoint of their science courses, it is possible that the SSACM items did not fully capture changes in science attitudes. Further research in which attitudes are measured after variable-length intervention periods would further substantiate the construct validity of SSACM scores. A second limitation refers to the fact that parental occupation type was collapsed across a variety of STEM disciplines. Further analysis of exactly which of the parent STEM occupations are related to student outcomes would allow researchers to create more precise control variables in future concurrent and/or predictive validity analyses. Perhaps, for example, parents who are employed in the same fields their children are pursuing careers in have more of an influence on the student-related outcomes described in this study. Creating more refined control variables of this sort would lend greater precision to resulting validity coefficients. Finally, the SSACM were originally developed for the purpose of assessing attitude change from the perspective of both students and their parents; however, parents’ perceptions of their children’s science attitudes were not a focus of the current study. This feature could be readily incorporated into future research designs to obtain additional estimates of construct validity.

Conclusion

The present study yielded evidence that the original ISC and ISM scales of the SSACM possess a bifactor structure with all items demonstrating strong convergence on a general science motivation factor yet loading equally well on specific competence-related constructs. Of course, the SSACM were originally developed with the intent of measuring attitude change in adolescents (Stake & Mares, 2001), therefore, we encourage other researchers to test this bifactor structure in this population as well. As noted at the outset of this article, STEM careers are in high demand and will continue to be for the foreseeable future. Perhaps the process of guiding students into and staying with a STEM career path can be facilitated to the extent that intervention programs are implemented on the basis of precise attitude measurement.

Footnotes

Authors’ Note

The data presented and views expressed in this article are solely the responsibility of the authors.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the National Science Foundation (HRD-1036767) to Eric D. Deemer and Jessi L. Smith.

Note

References

Asparouhov

Muthén

(2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397–438. doi:10.1080/10705510903008204

Bandura

(1997). Self-efficacy: The exercise of control. New York: Freeman.

Berlyne

D. E.

(1960). Conflict, arousal, and curiosity. New York: McGraw-Hill.

Bischoff

P. J.

Castendyk

Gallagher

Schaumloffel

Labroo

(2008). A science summer camp as an effective way to recruit high school students to major in the physical sciences and science education. International Journal of Environmental & Science Education, 3, 131–141.

Brophy

. (1999). Toward a model of the value aspects of motivation in education: Developing appreciation for particular learning domains and activities. Educational Psychologist, 34, 75–85. doi:10.1207/s15326985ep3402_1

Brophy

(2008). Developing students’ appreciation for what is taught in schools. Educational Psychologist, 43, 132–141. doi:10.1080/00461520701756511

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

K. A.

Long

J. S.

(Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.

Brunner

Nagy

Wilhelm

(2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796–846. doi:10.1111/j.1467-6494.2011.00749.x

Bureau of Labor Statistics, U.S. Department of Labor. (2013). Occupational Outlook Handbook (2012-13 Ed.). Projections Overview. Retrieved from http://www.bls.gov/ooh/about/projections-overview.htm

10.

Byars-Winston

A. M.

Fouad

N. A.

(2008). Math and science social cognitive variables in college students: Contributions of contextual factors in predicting goals. Journal of Career Assessment, 16, 425–440. doi:10.1177/1069072708318901

11.

Chen

F. F.

West

S. G.

Sousa

K. H.

(2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189–225. doi:10.1207/s15327906mbr4102_5

12.

Deci

E. L.

Ryan

R. M.

(1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press.

13.

Deemer

E. D.

Carter

A. P.

Lobrano

M. T.

(2010). Extending the 2 x 2 achievement goal framework: Development of a measure of scientific achievement goals. Journal of Career Assessment, 18, 376–392. doi:10.1177/1069072710374575

14.

Deemer

E. D.

Thoman

D. B.

Chase

J. P.

Smith

J. L.

(in press). Feeling the threat: Stereotype threat as a contextual barrier to women’s science career choice intentions. Journal of Career Development.

15.

Diemer

M. A.

Blustein

D. L.

(2007). Vocational hope and vocational identity: Urban adolescents’ career development. Journal of Career Assessment, 15, 98–118. doi:10.1177/1069072706294528

16.

Doosje

Ellemers

Spears

(1995). Perceived intragroup variability as a function of group status and identification. Journal of Experimental Social Psychology, 31, 410–436. doi:10.1006/jesp.1995.1018

17.

Durik

A. M.

Harackiewicz

J. M.

(2007). Different strokes for different folks: How individual interest moderates the effects of situational factors on task interest. Journal of Educational Psychology, 99, 597–610. doi:10.1037/0022-0663.99.3.597

18.

Dweck

C. S.

Leggett

E. L.

(1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, 256–273. doi:10.1037//0033-295X.95.2.256

19.

Eccles

(2009). Who am I and what am I going to do with my life? Personal and collective identities as motivators of action. Educational Psychologist, 44, 78–89. doi:10.1080/00461520902832368

20.

Elliot

A. J.

McGregor

H. A.

(2001). A 2 x 2 achievement goal framework. Journal of Personality and Social Psychology, 80, 501–519. doi:10.1037//0022-3514.80.3.501

21.

Elliot

A. J.

Murayama

(2008). On the measurement of achievement goals: Critique, illustrations, and application. Journal of Educational Psychology, 100, 613–628. doi:10.1037/0022-0663.100.3.613

22.

Glynn

S. M.

Koballa

T. R.

(2006). Motivation to learn in college science. In Mintzes

Leonard

W. H.

(Eds.), Handbook of college science teaching (pp. 25–32). Arlington, VA: National Science Teachers Association Press.

23.

Harackiewicz

J. M.

Rozek

C. S.

Hulleman

C. S.

Hyde

J. S.

(2012). Helping parents to motivate adolescents in mathematics and science: An experimental test of a utility-value intervention. Psychological Science, 23, 899–906. doi:10.1177/0956797611435530

24.

Hidi

(2000). An interest researcher’s perspective: The effects of extrinsic and intrinsic factors on motivation. In Sansone

Harackiewicz

J. M.

(Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 309–339). San Diego, CA: Academic Press.

25.

Hidi

Renninger

K. A.

(2006). The four-phase model of interest development. Educational Psychologist, 41, 111–127. doi:10.1207/s15326985ep4102_4

26.

Holden

R. R.

Fekken

G. C.

(1994). The NEO five-factor inventory in a Canadian context: Psychometric properties for a sample of university women. Personality and Individual Differences, 17, 441–444. doi:10.1016/0191-8869(94)90291-7

27.

Hulleman

C. S.

Godes

Hendricks

B. L.

Harackiewicz

J. M.

(2010). Enhancing interest and performance with a utility value intervention. Journal of Educational Psychology, 102, 880–895. doi:10.1037/a0019506

28.

Krapp

(2002). Structural and dynamic aspects of interest development: Theoretical considerations from an ontogenetic perspective. Learning and Instruction, 12, 383–409. doi:10.1016/S0959-4752(01)00011-1

29.

Lent

R. W.

Brown

S. D.

Hackett

(1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45, 79–122. doi:10.1006/jvbe.1994.1027

30.

Marko

K. W.

Savickas

M. L.

(1998). Effectiveness of a career time perspective intervention. Journal of Vocational Behavior, 52, 106–119. doi:10.1006/jvbe.1996.1566

31.

Marsh

H. W.

Hau

K.-T.

Grayson

(2005). Goodness of fit evaluation in structural equation modeling. In Maydeu-Olivares

McCardle

(Eds.), Psychometrics: A festschrift to Roderick P. McDonald (pp. 225–340). Hillsdale, NJ: Erlbaum.

32.

Marsh

H. W.

Liem

G. A. D.

Martin

A. J.

Morin

A. J. S.

Nagengast

(2011). Methodological measurement fruitfulness of exploratory structural equation modeling (ESEM): New approaches to key substantive issues in motivation and engagement. Journal of Psychoeducational Assessment, 29, 322–346. doi:10.1177/0734282911406657

33.

McCrae

R. R.

Zonderman

A. B.

Costa

P. T.

Jr Bond

M. H.

Paunonen

S. V.

(1996). Evaluating replicability of factors in the Revised NEO Personality Inventory: Confirmatory factor analysis versus procrustes rotation. Journal of Personality and Social Psychology, 70, 552–566. doi:10.1037/0022-3514.70.3.552

34.

Mueller

R. O.

(1997). Structural equation modeling: Back to basics. Structural Equation Modeling, 4, 353–369. doi:10.1080/10705519709540081

35.

National Center for Education Statistics, U. S. Department of Education. (2009). Students who study science, technology, engineering, and mathematics (STEM) in postsecondary education (NCES 2009-161). Retrieved from nces.ed.gov/pubs2009/2009161.pdf

36.

Nauta

M. M.

Kahn

J. H.

(2007). Identity status, consistency and differentiation of interests, and career decision self-efficacy. Journal of Career Assessment, 15, 55–65. doi:10.1177/1069072705283786

37.

Nicholls

J. G.

(1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, 328–346. doi:10.1037/0033-295X.91.3.328

38.

Osborne

J. W.

Jones

(2011). Identification with academics and motivation to achieve in school: How the structure of the self influences academic outcomes. Educational Psychology Review, 23, 131–158. doi:10.1007/s10648-011-9151-1

39.

Renninger

K. A.

(2000). Individual interest and its implications for understanding intrinsic motivation. In Sansone

Harackiewicz

J. M.

(Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 373–404). New York, NY: Academic Press.

40.

Rice

K. G.

Lopez

F. G.

Richardson

C. M. E.

(2013). Perfectionism and performance among STEM students. Journal of Vocational Behavior, 82, 124–134. doi:10.1016/j.jvb.2012.12.002

41.

Rounds

J. B.

Tracey

T. J.

(1990). From trait-and-factor to person-environment fit counseling: Theory and process. In Walsh

W. B.

Osipow

S. J.

(Eds.), Career counseling: Contemporary topics in vocational psychology (pp. 1–44). Hillsdale, NJ: Erlbaum.

42.

Ryan

R. M.

Deci

E. L.

(2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25, 54–67. doi:10.1006/ceps.1999.1020

43.

Sansone

Smith

J. L.

(2000). Interest and self-regulation: The relation between having to and wanting to. In Sansone

Harackiewicz

J. M.

(Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 341–372). San Diego, CA: Academic Press.

44.

Sansone

Thoman

D. B.

(2005). Interest as the missing motivator in self-regulation. European Psychologist, 10, 175–186. doi:10.1027/1016-9040.10.3.175

45.

Scherer

K. R.

(2001). Appraisal considered as a process of multilevel sequential checking. In Scherer

K. R.

Schorr

Johnstone

(Eds.), Appraisal processes in emotion: Theory, methods, research (pp. 92–120). New York, NY: Oxford University Press.

46.

Seymour

Hewitt

N. M.

(1997). Talking about leaving: Why undergraduates leave the sciences. Boulder, CO: Westview Press.

47.

Silvia

P. J.

(2005). Emotional responses to art: From collation and arousal to cognition and emotion. Review of General Psychology, 9, 342–357. doi:10.1037/1089-2680.9.4.342

48.

Silvia

P. J.

(2008). Interest—The curious emotion. Current Directions in Psychological Science, 17, 57–60. doi:10.1111/j.1467-8721.2008.00548.x

49.

Smith

J. L.

Lewis

K. L.

Hawthorne

Hodges

S. D.

(2013). When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns. Personality and Social Psychology Bulletin, 39, 131–143. doi:10.1177/0146167212468332

50.

Stake

J. E.

Mares

K. R.

(2001). Science enrichment programs for gifted high school girls and boys: Predictors of program impact on science confidence and motivation. Journal of Research in Science Teaching, 38, 1065–1088. doi:10.1002/tea.10001

51.

Stake

J. E.

Mares

K. R.

(2005). Evaluating the impact of science-enrichment programs on science adolescents’ science motivation and confidence: The splashdown effect. Journal of Research in Science Teaching, 42, 359–375. doi:10.1002/tea.20052

52.

Super

D. E.

(1980). A life-span, life-space approach to career development. Journal of Vocational Behavior, 16, 262–298. doi:10.1016/0001-8791(80)90056-1

53.

Taasoobshirazi

Glynn

S. M.

(2009). College students solving chemistry problems: A theoretical model of expertise. Journal of Research in Science Teaching, 46, 1070–1089. doi:10.1002/tea.20301

54.

Thoman

D. B.

Smith

J. L.

Silvia

P. J.

(2011). The resource replenishment function of interest. Social Psychological and Personality Science, 2, 592–599. doi:10.1177/1948550611402521