Assessment of Scientific Communication Self-efficacy,Interest,and Outcome Expectations for Career Development in Academic Medicine

Abstract

Competency in forms of scientific communication, both written and spoken, is essential for success in academic science. This study examined the psychometric properties of three new measures, based on social cognitive career theory, that are relevant to assessment of skill and perseverance in scientific communication. Pre- and postdoctoral trainees in biomedical science (N = 411) completed online questionnaires assessing self-efficacy in scientific communication, career outcome expectations, and interest in performing tasks in scientific writing, oral presentation, and impromptu scientific discourse. Structural equation modeling was used to evaluate factor structures and model relations. Confirmatory factor analysis supported a 22-item, 3-factor measure of self-efficacy; an 11-item, 2-factor measure of outcome expectations; and a 12-item, 3-factor measure of interest in scientific communication activities. Construct validity was further demonstrated by theory-consistent inter-factor relations and relations with typical communications performance behaviors (e.g., writing manuscripts, abstracts, and presenting at national meetings).

Keywords

scientific communication social cognitive theory SCCT career development academic medicine scale development measurement LISREL

The phrase, “publish or perish,” cuts to the heart of most scientists to aptly describe the pressure in academia for career success. Scientific communication, whether written or oral, is a typical and highly necessary activity of successful scientists. Routinely, scientists must write to publish the results of research studies in scientific journals, formally present results in oral presentations at scientific meetings, and informally communicate findings in conversations with other scientists, the media, or lay individuals. Understanding social and behavioral factors that relate to the motivation and skills needed for excellence in the realm of scientific communication is crucial for the development of effective behavioral interventions to enable success in research careers. The promotion of interventions to increase interest and preparedness for careers in biomedical and behavioral research has become a priority in public health research at the National Institutes of Health, especially in efforts to increase students from underrepresented backgrounds in academic medicine (National Institute of General Medical Sciences, 2013).

Despite the critical need for research in the specific domain of scientific communication, no psychometrically sound measures exist that assess factors contributing to the development of strong writing and speaking skills. Measures to assess key, psychosocial factors that foster interest in these types of activities are needed, such as measures of beliefs about abilities in scientific writing and speaking, and important career outcomes that might result from competency in these behaviors (Bandura, 1986; Lent, Larkin, & Brown, 1989). Such measures will be useful in a variety of educational settings, as educators explore potential differences in trainees’ beliefs, capabilities, and career goals. Developing theoretically driven measures will allow investigators to test and validate models for domain-specific predictors of career goals and, ultimately, career performance and success (Lent & Brown, 2006).

The purpose of this study was to design and psychometrically evaluate a set of three, domain-specific scales assessing scientific communication skills within the framework of social cognitive career theory (SCCT; Lent, Brown, & Hackett, 1994). Using a sample of public health, biomedical, and social science trainees enrolled in doctoral and postdoctoral programs, this article describes the development and psychometric properties of scientific communication self-efficacy, career outcome expectations, and task interest, and reports the relation of these measures with trainee scientific communication behaviors. We anticipate that these measures may encourage a new area of empirical research investigating the role of scientific communication skills in successful research career development.

Theoretical Framework

SCCT (Lent et al., 1994; Lent, Brown, & Hackett, 2000) is an emerging, useful theoretical framework to explain individual differences in career interests and choices across gender and racially diverse groups. Recent research has supported the application of Lent et al.’s SCCT model of academic and career interest and choice to a variety of science career domains, including diverse groups in engineering (Lent et al., 2003, 2005), computing (Lent, Lopez, Lopez, & Sheu, 2008), and biological science (Byars-Winston, Estrada, Howard, Davis, & Zalapa, 2010). Important work among physician-scientists has included the development of an inventory to assess clinical research self-efficacy (Mills, Caetano, & Rhea, 2014; Mullikin, Bakken, & Betz, 2007). The domain of scientific communication represents a new frontier for SCCT. Individual beliefs on key constructs of the SCCT model in regard to scientific communication may have long-term implications, helping to explain the sustained interest, choice intentions, and quality of performance of new scientists in academic medicine over time, as well as the failure of some trainees to follow successful career paths in health science research.

Self-Efficacy and Outcome Expectations

As an extension of Bandura’s (1986) social cognitive theory into the domain of academic and career development, SCCT focuses on the same core constructs considered so crucial to motivation by Bandura, specifically, self-efficacy and outcome expectations. These variables are posited to have a central role in predicting interest in specific tasks, such as academic subjects and career choice goals (Lent et al., 1994, 2000). Self-efficacy (beliefs about one’s ability to perform particular behaviors well) is theorized to influence outcome expectations (beliefs about the consequences of certain actions), which together promote interest in performing certain career-related tasks. Self-efficacy, outcome expectations, and interests are then predicted to jointly affect major career choice goals (e.g., take more math-science courses, remain in a particular science discipline, pursue a research career, etc.). Concerning outcome expectations, although Bandura’s conceptualization included social, material, and self-evaluative outcomes that could be positive or negative, most of the work in SCCT has focused on positive outcomes (Lent & Brown, 2006). Very few studies have included negative outcomes (Hackett, Betz, Casas, & Rocha-Singh, 1992), and recommendations for future research and measurement in career development have called for the measurement of both positive and negative career outcome expectations (Fouad & Guillen, 2006; Lent & Brown, 2006).

To facilitate the application of SCCT (Lent et al., 1994) to the domain of scientific communication, we developed measures of self-efficacy, career outcome expectations, and task interest to assess scientific communication beliefs important to building a research career in academic medicine. The design and format of the measures were guided by measures developed and published by Lent and his colleagues with the content for all three measures tailored to reflect specific scientific communication behaviors in the domains of writing, presenting, and impromptu conversation. Previous work by Lent et al. has used two measures, one measuring task-related self-efficacy (Lent, Brown, & Larkin, 1986) and another measuring barrier-coping self-efficacy (Lent et al., 2003). We included both types of self-efficacy in the development of our scale. For outcome expectations, we addressed shortcomings of previous research by including both positive and negative career outcome expectation items. For task interest, we followed previous work by Lent and colleagues (Lent et al., 2003, 2005) to capture interest in performing scientific communication-related tasks. Finally, criterion-related validity was addressed by examining the relations of the measures to progress in goal-directed activities (Lent & Brown, 2008) in the domain of scientific communication.

Method

Procedure

The Institutional Review Board of The University of Texas M.D. Anderson Cancer Center approved the study protocol. All participants provided online consent following eligibility questions and completed the survey online. They received a US$20 gift card for participation.

Participants

Doctoral students enrolled in a degree program and postdoctoral trainees in basic biomedical sciences, biomedical statistics, epidemiology, behavioral and social science, and related public health disciplines at The University of Texas M.D. Anderson Cancer Center, Houston TX, were recruited for an online survey via institutional e-mail. Of those showing initial interest (N = 514), 411 met eligibility criteria and agreed to participate (247 females and 164 males). Participants ranged in age from 22 to 57 years (M = 31.05, SD = 5.10) and self-identified as 11% Hispanic (89% not Hispanic); 42.6% White, 2.7% Black, 25.6% Asian, and 29.2% other ethnicities. Half of the participants were U.S. citizens (49.9%), half were visa holders (50.1%), and 46.3% reported English as their primary language. Half were enrolled in postdoctoral fellowships (49.5%; see Table 1).

Table 1.

Sample Characteristics.

	N = 411
Trainees
Gender
Female	60.1%
Male	39.9%
Mean age	31.1 years (SD = 5.0)
U.S. citizen status
U.S. citizen or permanent resident	49.9%
Visa holder	50.1%
Ethnicity (%)
Hispanic	10.9%
Non-Hispanic	89.1%
White	42.6%
Asian	25.5%
African American	2.7%
Other or more than one race	29.2%
Highest level of education
Bachelor’s degree	20.3%
Master’s degree	25.9%
PhD	45.7%
MD	6.4%
Other	1.7%
Current academic status
PhD program	47.6%
Postdoctoral fellowship	49.5%
Other	2.9%
Academic rank of current primary mentor
Assistant professor	15.8%
Associate professor	25.4%
Professor	57.1%
Don’t know/other	1.7%

Instrument Development

Several stages comprised the development of the measures including (1) focus groups, (2) item generation, (3) expert review, and (4) pilot testing. A multidisciplinary, collaborative team of population scientists, educators, linguists, social and counseling psychologists, and master’s level staff composed the research team. Each contributed expertise and experience in conducting research, mentoring, teaching, writing, presenting, and working with a wide variety of research trainees and faculty. The main team of five was based at M.D. Anderson and did the primary work on focus groups and item generation. Three consultants were located at other institutions outside of Texas.

Focus groups

To generate the item content and ensure that the content reflected the breadth of scientific communication skills considered relevant to a research career, qualitative data were collected from a group of participants not included in the present study. As previously reported (Cameron et al., 2013), semistructured focus groups and interviews of 43 trainees (postdoctoral fellows and doctoral students) at M.D. Anderson in Houston were conducted. These findings helped to establish the ecological validity of the factorial dimensions of the measures.

Item generation

The research team used the themes and wordings that emerged from the qualitative study and also reviewed multiple measures of self-efficacy, outcome expectations, and interest used in previous studies by the Lent group and Bandura to generate items for each construct. The relevant background literature used in the item development for each construct is described subsequently. Items were reviewed and revised by team members until there was consensus on each item.

Self-efficacy in scientific communication

Based on the focus group results and Lent et al.’s measures of academic task self-efficacy (Lent et al., 1986) and barrier-coping efficacy (Lent et al., 2001, 2003), an initial item pool of 27 items was developed to reflect task-related and coping self-efficacy in scientific writing, presentation, and impromptu conversational speaking. Items were measured on a 5-point Likert-type scale with the anchors ranging from 1 (very insecure) to 5 (very confident). The instruction was “Please rate your level of confidence, even if you have never done it yet, in your ability to . . .,” perform or cope with tasks such as “give an oral presentation at a scientific meeting,” or “deal with a lack of mentor support in scientific writing.”

Career outcome expectations

Based on measures of outcome expectations suggested by Bandura (1997) and previous research by Lent et al. (2008), an initial item pool of 20 items was written to reflect positive and negative self-evaluative, social, and physical categories of career outcome expectations within the domain of scientific communication. Participants rated their belief that, “My work to achieve high performance in scientific writing and speaking will . . .,” on a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Examples of positive items include “make me feel good about myself” and “inspire me to do great work.” Negative item examples include “deprive me of time with family and friends” and “make me become angry and frustrated.”

Interest

A total of 12 items were generated to assess trainees’ interest in performing typical academic scientific writing, presenting, and impromptu speaking tasks during their current training period. This measure was based on previous research by Lent et al. (2003, 2005) and suggestions from the research team. It used a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Participants read the following instruction, “During my current training period, I am interested in . . .,” to rate their interest in items such as “creating a poster of my own work” and “actively participating in group scientific discussions.”

Task participation/goal progress

The survey included four scientific communication tasks or behaviors for use in the criterion validity analyses. Participants were asked to indicate the number of writing and speaking tasks they had completed during their current training period: (1) “Prepared by myself a full first draft of a first-author manuscript,” (2) “Prepared by myself an abstract for a scientific meeting,” (3) “Given a presentation at a scientific meeting,” and (4) “Asked a speaker a question during their presentation at my institution or at a scientific meeting.”

Expert review and pilot testing

Three experts in research training from other institutions were asked to review the item quality and representativeness. These individuals serve as consultants on the project, but were not directly involved in the item generation process. Following feedback and consensus among the research team, paper-and-pencil versions of the scales were pilot-tested by five M.D. Anderson trainees not eligible for the study. The final versions of the scales were then prepared for online access by our institutional research programmers.

Statistical Analyses

The analyses proceeded in three phases. First, structural equation modeling in LISREL 8.80 (Jöreskog & Sörbom, 2006) was used to conduct confirmatory factor analyses (CFAs) to test the hypothesized measurement models for self-efficacy, outcome expectations, and interests. A model was fit for each scale at the item level to assess the reliability of the individual items. If a model failed to reach an acceptable fit, theoretically sound respecifications were made (e.g., theoretically sound alternative models, dropping poorly performing items). Differences between nested models were tested by examining the Satorra–Bentler χ² difference for change in degrees of freedom (Satorra & Bentler, 2001). Second, following the evaluation and satisfactory fit of the item-level measurement models, item parcels (2 or more items averaged together to function as a single aggregate) were used to allow a measurement model that included all three scales (self-efficacy, outcome expectations, and interests) to be fit, given the sample size and complexity of this model, so that the relations between the latent factors of the scales could be estimated (Marsh, Lüdtke, Nagengast, Morin, & von Davier, 2013; Marsh & O’Neill, 1984). Third, the construct validity of the measures was further examined by estimating the relation of the factor scores to the four behavioral outcomes using Pearson correlations.

CFA model fit

The Robust Maximum Likelihood (RML) estimation method in LISREL 8.80 (Jöreskog & Sörbom, 2006) was used to generate the standardized parameter estimates for the structural equation models, since the data were not multivariate normal. PRELIS, the pre-processing program of LISREL, estimated the asymptotic covariance matrix of the sample variances and covariances that the RML method requires and listwise deletion was used for missing data. The Satorra–Bentler (S-B) scaled χ² statistic (Bryant, 2013; Bryant & Satorra, 2012; Satorra & Bentler, 1988) was used to determine the fit of each model to the observed data in order to correct for nonnormality. A χ² that is not significant (p > .05) indicates a good fit, as the model does not differ significantly from the observed data. However, significant χ² that reject the model can frequently occur even when the model fit is relatively good (Hu & Bentler, 1995; Marsh, Balla, & McDonald, 1988); therefore, other fit indices were also used. These included the Comparative Fit Index (CFI; Bentler, 1990), Non-Normed Fit Index (NNFI; Bentler & Bonnett, 1980), Standardized Root Mean Squared Residual (SRMR; Hu & Bentler, 1999), and the root mean square error of approximation (RMSEA; Browne & Cudeck, 1993). RMSEA values of 0.06 or less, SRMR values of less than 0.08, and CFI and NNFI values of 0.95 or higher are recommended to indicate well-fitting models (Hu & Bentler, 1999).

Results

Missing Data

With the focus on the development of new scales in new domains of career development, a conservative approach of listwise deletion was used to handle missing data in the LISREL analyses. Of the 411 participants who responded to the survey, about 10% (N = 33) failed to answer more than 80% of items across sections. PRELIS analyses indicated no substantial or systematic loss of data for the remaining participants, and complete data on all items were available for N = 356 for self-efficacy (87%), N = 350 for outcome expectations (85%), and N = 346 (84.2%) for task interests.

CFAs

Self-efficacy

The initial CFA for self-efficacy examined the fit of the data to the hypothesized, correlated, two-factor model of task self-efficacy and coping self-efficacy used by Lent and colleagues (1986, 2001, 2003). The fit was not satisfactory (RMSEA = 0.15, NNFI = 0.86, and CFI = 0.87), so alternative, hypothesized models that reflected self-efficacy in the specific types of tasks were examined. Models of one factor, two factors (writing and speaking), and three factors (writing, presenting, and conversation) were compared. As indicated in Table 2, the one-factor model (Model 1) showed poor overall fit. The correlated two-factor model (Model 2: writing and speaking) fit the data significantly better than the one-factor model, as shown by the significant χ² difference test. The three-factor model of writing, presenting, and conversation provided the best fit to the data. From the 27-item pool, a total of 22 items were retained in the LISREL analyses. An iterative process was used to delete 5 items due to redundancies, lack of domain specificity (e.g., “Deal with fear of disappointing your mentor”), or loadings below 0.60, our chosen cutoff for a moderate to strong association (Kline, 2005). All models were fit without post hoc model modifications (i.e., no correlated errors), providing the most stringent test of the models. Given these strict criteria, the fit indices for the final three-factor model were quite satisfactory (NNFI = 0.95, CFI = 0.96, and SRMR = 0.065), although the RMSEA value was somewhat high at 0.091. In sum, the final model demonstrated a satisfactory fit and reflected the theoretical constructs that would allow us to investigate self-efficacy within the domain of scientific communication.

Table 2.

Goodness-of-Fit Indicators of Models for Cognitive Factors of Scientific Communication.

Model	χ² _S-B	df	χ²/df	χ² _{S-B diff}	RMSEA [90% CI]	NNFI	CFI	SRMR
Self-efficacy
One factor	2156.01	209	10.32	—	0.160 [0.160, 0.170]	0.85	0.87	0.110
Two factors: writing and speaking	1056.53	208	5.08	31.63***	0.100 (0.100, 0.110)	0.94	0.94	0.076
Three factors: writing, presenting, and conversation	809.29	206	3.93	53.64***	0.091 (0.084, 0.097)	0.95	0.96	0.065
Outcome expectations
One factor	930.83	44	21.16	—	0.240 (0.230, 0.250)	0.54	0.63	0.170
Two factors: positive and negative	78.89	43	1.83	144.10***	0.049 (0.031, 0.066)	0.98	0.99	0.042
Interest
One factor	402.41	54	7.45	—	0.140 (0.130, 0.150)	0.92	0.94	0.086
Two factors: writing and speaking	222.67	53	4.20	81.75***	0.098 (0.085, 0.110)	0.96	0.97	0.061
Three factors: writing, presenting, and conversation	198.12	51	3.88	26.65***	0.093 (0.080, 0.110)	0.96	0.97	0.055

Note. Satorra–Bentler scaled χ² is reported. Scaled χ² difference test for LISREL 8 is reported, computed with Satorra-Bentler scaled correction factors (Bryant & Satorra, 2012; Bryant, 2013). RMSEA = root mean square error of approximation; NNFI = Non-Normed Fit Index; CFI = Comparative Fit Index; SRMR = Standardized Root Mean Squared Residual.

***p < .001.

Table 3 shows the item wordings and completely standardized parameter estimates for each item. Item loadings were significant and substantial, and coefficient α reliabilities were 0.91 for writing, 0.89 for oral presentation, and 0.89 for conversation. Factor means, standard deviations, ranges, and correlations (LISREL Φ values) are shown in Table 6.

Table 3.

Self-Efficacy Scale Items and LISREL Completely Standardized Factor Loadings.

“Rate your level of confidence (even if you have never done it yet) in your ability to . . .”	Factor loading
Scientific writing self-efficacy
1. Excel in scientific writing tasks, e.g., abstracts, manuscripts	0.78
2. Deal with a lack of mentor support in scientific writing	0.67
3. Complete a writing task in the time allowed	0.61
4. Write and submit an abstract to a scientific meeting.	0.70
5. Write a first draft by yourself of a manuscript intended for publication	0.71
6. Write using correct grammar	0.63
7. Manage any anxiety you may have about your writing ability	0.74
8. Use the expected scientific style when writing	0.79
9. Continue to revise a manuscript multiple times after receiving negative feedback from your mentor or reviewers	0.70
10. Need minimal help because my writing skills are strong enough	0.83
Scientific oral presentation self-efficacy
11. Excel in giving scientific presentations (i.e., you usually receive high praise for your presentations from your mentor or the audience)	0.89
12. Give a scientific talk to a lay audience (e.g., high school students, cancer patients)	0.680
13. Give an oral presentation at a scientific meeting	0.86
14. Require little to no assistance with my speaking and presenting skills	0.85
Scientific conversation self-efficacy
15. Defend your point of view convincingly in a scientific discussion, in spite of a negative response from others	0.72
16. Effectively answer questions from the audience at a scientific meeting	0.77
17. Speak using correct grammar without rehearsing	0.69
18. Manage worries you may have about your pronunciation, accent, vocabulary, grammar, or style of speaking	0.71
19. Ask a question or add a comment during a meeting or discussion in your own lab or research group	0.67
20. Ask a question in front of the audience after a presentation at a national scientific meeting	0.65
21. Use the expected scientific style when speaking	0.76
22. Introduce yourself and your research concisely and effectively to other professionals	0.73

Note. Response anchors: 1 = very insecure; 2 = insecure; 3 = neither confident nor insecure; 4 = confident; and 5 = very confident.

Career outcome expectations

The CFA using the 20 items to examine the fit of the data to a correlated, two-factor model (positive and negative outcome expectations), as conceptualized by Bandura (1997), approached criteria for an acceptable fit (RMSEA = 0.080, NNFI = 0.93, CFI = 0.94, SRMR = 0.067). Next, an iterative process was used to delete 9 items that had lower than desired loadings or were redundant, resulting in a two-factor model of 11 items (positive outcomes = 6 items and negative outcomes = 5 items) that provided a good fit to the data, as shown in Table 2 (RMSEA = 0.049, NNFI = 0.98, CFI = 0.99, and SRMR = 0.042). Table 4 shows the item wordings and completely standardized parameter estimates for each item. Item loadings were significant and substantial, and coefficient α reliabilities were 0.84 for positive outcomes and 0.85 for negative outcomes. Factor means, standard deviations, ranges, and correlations (LISREL Φ values) are shown in Table 6.

Table 4.

Outcome Expectations Scale Items and LISREL Completely Standardized Factor Loadings.

“My work to achieve high performance in scientific writing and speaking will . . .”	Factor loading
Positive outcome expectations
1. Allow me to obtain a highly desirable academic faculty position	0.75
2. Be necessary for me to be recognized as an expert in my research area	0.62
3. Be critically important for me to become a successful independent investigator	0.78
4. Make me feel good about myself	0.67
5. Inspire me to do great work	0.60
6. Make me feel confident and secure about my future career	0.72
Negative outcome expectations
7. Deprive me of time with family and friends	0.61
8. Lead to chronic stress, anxiety, and worry in my life	0.87
9. Cause me to lose sleep	0.67
10. Make me become angry and frustrated	0.70
11. Cause my physical health to become poor	0.80

Note. Response anchors: 1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; and 5 = strongly agree.

Interest

The initial CFA of the 12 interest items examined the fit of the data to a one-factor model of interest as used by Lent et al. (2003, 2005). The fit was not satisfactory (RMSEA = 0.14, NNFI = 0.92, CFI = 0.94, and SRMR = 0.086). The two additionally hypothesized models, a two-factor model of interests in writing and speaking, and a three-factor model of interests in writing, presenting, and conversation, were then examined and compared. As shown in Table 2, the three-factor model provided the best fit to the data (RMSEA = 0.093, NNFI = 0.96, CFI = 0.97, and SRMR = 0.056) and reflected the three conceptual subdomains of scientific communication that were of interest in our research. Table 5 shows the item wordings and completely standardized parameter estimates for each of the 12 items. Item loadings were significant and substantial, and coefficient α reliabilities were 0.86 for writing, 0.84 for oral presentation, and 0.85 for conversation. Factor means, standard deviations, ranges, and correlations (LISREL Φ values) are shown in Table 6.

Table 5.

Interest Scale Items and LISREL Completely Standardized Factor Loadings.

“During my current training period, I am interested in . . .”	Factor loading
Scientific writing
1. Writing first-author manuscripts for submission to journals	0.69
2. Writing manuscripts with other authors	0.66
3. Writing and submitting abstracts to scientific meetings	0.92
4. Creating a poster of my work	0.84
Scientific oral presentation
5. Giving an impressive oral presentation at a national scientific meeting	0.77
6. Presenting my poster formally to a seated audience at a scientific meeting	0.75
7. Presenting a summary and leading the discussion of an article for fellows’ “journal club”	0.70
8. Explaining my poster informally during a poster session during a scientific meeting	0.80
Scientific conversation
9. Asking questions of a presenter at a scientific meeting	0.74
10. Actively participating in group scientific discussions	0.79
11. Making an outstanding impression introducing myself and my research to various individuals	0.83
12. Being able to express my ideas eloquently to a variety of audiences	0.70

Note. Response anchors: 1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; and 5 = strongly agree.

Table 6.

Factor Means, Standard Deviations, Ranges, α Reliabilities, and Intercorrelations.


Scale	M	SD	Range	α	1	2	3
Self-efficacy
1. Writing	3.73	0.69	1.40–5.00	0.91	—
2. Presentation	3.64	0.84	1.00–5.00	0.89	.53	—
3. Conversation	3.57	0.74	1.38–5.00	0.89	.65	.77	—
Outcome expectations
1. Positive	4.28	0.55	2.00–5.00	0.84	—
2. Negative	2.58	0.83	1.00–5.00	0.85	−.29	—
Interest
1. Writing	4.52	0.59	1.00–5.00	0.86	—
2. Presentation	4.34	0.66	1.00–5.00	0.84	.74	—
3. Conversation	4.35	0.62	2.00–5.00	0.85	.61	.91	—

^aLISREL Φ values within each model.

Self-Efficacy, Outcome Expectations, and Interest Measurement Model Evaluation

Having established the reliability of the item-level data, item parcels were used next to allow the estimation of a measurement model that included self-efficacy, positive outcome expectations, negative outcome expectations, and interests. In addition to reducing the number of estimated parameters in the model in order to allow the estimation of more complicated models, parcels have been preferred over single items as indicators of latent constructs because they better approximate to normally distributed continuous variables (Bentler & Chou, 1987) and may reduce distortion of estimates (Bandalos, 2002). Parcels were formed on a conceptual basis. As shown in Figure 1, three item parcels were allowed to load on the self-efficacy and interests factors, respectively, representing the subdomains of writing, presentation, and conversation. Two item parcels were allowed to load on the positive outcome expectations factor, representing the positive social and self-evaluative outcomes in the final item pool, and three item parcels were allowed to load on the negative outcome expectations factor, representing negative social, self-evaluative, and physical outcomes.

Figure 1.

Measurement model of cognitive factors in scientific communication. Note. *p < .05. ***p < .001.

Significant, positive correlations were found between self-efficacy and interests, as well as between positive outcome expectations and interests. Self-efficacy was also positively related to positive outcome expectations, but the t value was not significant (t = 1.47, p < .20). Significant, negative correlations were found between positive and negative outcome expectations, and between negative outcome expectations and interests. Factor intercorrelations indicated adequate construct distinction.

Relation of Measures With Scientific Communication Behaviors

Pearson correlations between the eight scientific communication subscales and the trainees’ current progress in scientific communication tasks are shown in Table 7. The self-efficacy subscales of writing, presentation, and conversational speaking were all significantly and positively related to all of the four behavioral outcomes (i.e., preparing a manuscript, preparing an abstract, giving an oral presentation at a national meeting, asking a speaker a question at a national meeting). However, none of the four performance measures were significantly correlated with career outcome expectations or trainee task interest.

Table 7.

Pearson Correlations of Social Cognitive Variables and Trainee Performance.

	Trainee prepared manuscripts	Trainee prepared abstracts	Trainee national meeting oral presentations	Asked a speaker a question during presentation at national or local meeting
Self-efficacy
1. Writing	0.21****	0.23****	0.14*	0.12*
2. Presenting	0.21****	0.30****	0.21***	0.25****
3. Conversation	0.20***	0.30****	0.19***	0.31****
Outcome expectations
4. Positive	0.09	0.07	−0.03	0.01
5. Negative	0.03	0.01	0.06	−0.02
Interest
6. Writing	0.04	0.11	0.07	0.01
7. Presenting	0.04	0.11	0.07	0.08
8. Conversation	−0.03	0.03	0.02	0.11

Note. *p < .05. ***p < .001. ****p < .0001.

Discussion

This research extends prior studies in SCCT to develop measurement scales to assess social cognitive constructs in scientific communication. The primary objective of this work was to evaluate the psychometric properties of three new measures that reflect attitudes of self-efficacy, career outcome expectations, and interests associated with writing, oral presentation, and informal speaking tasks in research careers. The hypothesized underlying factor structures of the measures were tested, and a measurement model examined the factorial relations between measures. The results provide support for a three-factor model of self-efficacy, a two-factor model of career outcome expectations, and a three-factor model of task interests. Consistent with the SCCT’s prediction of academic choice and success, the components of self-efficacy were significantly associated with self-reported progress on typical scientific communication behavioral tasks of trainees.

From a theoretical perspective, these findings support the factorial and construct validity of self-efficacy, career outcome expectations, and task interest within the virtually unstudied domain of scientific communication. Although previous work utilizing the SCCT framework has been most useful in developing inventories of overall research self-efficacy among physician-scientists (Mills et al., 2014; Mullikin et al., 2007), the current study is the first to develop measures tailored to the particular domain of scientific communication. As originally hypothesized by Bandura (1977) and Lent et al. (1994), and empirically supported in multiple, subsequent studies using the SCCT model (Lent et al., 2001; Lent & Brown, 2008), we found a positive relation between self-efficacy beliefs and scientific communication task interests, a positive relation between self-efficacy beliefs and positive outcome expectations, and a positive relation between professionally relevant positive outcome expectations and scientific communication task interests. The current study also extends prior research on SCCT by expanding knowledge concerning outcome expectations. The measure we developed on outcome expectations was constructed to include Bandura’s (1977) original notions of social reactions, self-evaluative, and physical outcomes, including both positive and negative potential outcomes. This approach has been recommended for future research, involving social cognitive models (Fouad & Guillen, 2006; Lent & Brown, 2006). Thus, an important addition to the social cognitive career model is our finding of a significant, negative relation between interest in performing scientific communication tasks and negative outcome expectations, and between positive and negative expected outcomes.

As would be predicted from prior studies on SCCT, self-efficacy yielded the highest correlations with written and oral communication task performance in the trainee sample, providing good evidence of construct validity. In contrast, neither career outcome expectations nor trainee task interest were significantly related to performance. A brief meta-analytic review of theory-relevant research by Lent, Brown, and Hackett (1994) also found weak correlations between positive outcome expectations and performance, as well as between interests and performance. Career outcome expectations, as well as interests, appear to be much more closely related to choice goals in the Lent model (e.g., take another math course, select engineering as a major), rather than to actual academic performance. Recent data from our group were supportive of this prediction (Cameron et al., 2015). In a structural equation choice model with intention to remain in an academic research career as the choice goal among research trainees, all three cognitive variables (scientific communication self-efficacy, career outcome expectations, and task interest) significantly predicted career intentions. Interest exerted a stronger effect on intention than performance, and career outcome expectations exerted a stronger total effect on intention than self-efficacy, interest, or performance. Thus, despite their lack of significant relation with performance in the current study, the newly developed outcome expectations and interest scales appear promising as important facilitators in studying career choice models that include a focus on scientific communication.

This study has important implications for career development, counseling efforts, and research. First, these new measures can be used for individual-level assessment and skill development in writing and speaking. Measurement of communication self-efficacy, career outcome expectations, and task interest could provide timely information to trainees and their mentors before and during training, identifying specific areas in which additional mentoring and focused support are needed. Tailored interventions designed to influence one or more of the components of scientific communication self-efficacy could be used to enhance trainee performance and perseverance in research science (Bandura, 1997; Lent, Brown, & Larkin, 1984). For example, programs could focus on training or mentoring activities, helping trainees to plan, draft, revise, and resubmit papers; plan, practice, and deliver scientific talks in local, regional, and national settings; and informally talk more about their work to research colleagues and the lay public (e.g., an elevator speech). Mentors, who deliver constructive, practical, and frequent feedback to trainees in developing papers and presentations, balancing sensitive and supportive instruction with the application of rigorous scientific standards, will be needed to enhance these training efforts in scientific communication. Second, these measures may be valuable to training program evaluations, where the scholarly products of trainees and graduates provide markers in the accreditation process. Since scientific publications are considered one of the most important research outcomes of graduate training programs (Dores et al., 2006), the assessments we have developed may help programs understand the needs of trainees in order to focus efforts to improve trainee and program outcomes.

The study has several key strengths. The measures were developed from a sound, conceptual framework, and the analyses used structural equation modeling to provide a rigorous evaluation of the measures and their hypothesized interrelations. Furthermore, an ethnically diverse and relatively gender-balanced sample of pre- and postdoctoral biomedical trainees was used to test the models. This research is limited, however, by the geographical location of the sample in one area of the southwestern United States, and the lack of a second, independent sample to confirm the factor structures of the scales developed and presented here. In spite of these limitations, this work represents an important, first step toward providing useful tools in scientific communication. Future research is needed to examine the factor structures in new samples, including multiple group analyses among specific groups of interest (e.g., invariance of the measures across groups), and longitudinal studies designed to evaluate temporal relations.

In summary, three measures of individual beliefs are presented as a valuable methodological contribution to the study of attitudes in the career domain of scientific communication. Reliable, psychometrically sound instruments are needed as the field of academic medical research strives to better understand psychosocial factors that are related to the development of writing and speaking skills among developing scientists. Increasing awareness of trainees’ views and identifying trainees at risk in scientific communication skills will be important to mentoring and counseling efforts to maximize trainees’ research career potential.

Footnotes

Acknowledgments

The authors would like to thank Nikita Robinson, Alicia Bibbs, Candice Collie Greenfield, and Christine Pfund.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research and the preparation of this article was supported by grants from the National Institute of General Medical Sciences (R01-GM085600-01A1) awarded to Shine Chang.

References

Bandalos

D. L.

(2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling, 9, 78–102. doi:10.1207/S15328007SEMO901__5

Bandura

(1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215. doi:10.1037/0033-295x.84.2.191

Bandura

(1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

Bandura

(1997). Self-efficacy: The exercise of control. New York, NY: W.H. Freeman.

Bentler

P. M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. doi:10.1037/0033-2909.88.3.588

Bentler

P. M.

Bonnett

D. G.

(1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606. doi:10.1037/0033-2909.88.3.588

Bentler

P. M.

Chou

(1987). Practical issues in structural modeling. Sociological Methods and Research, 16, 78–117. doi:10.1177/0049124187016001004

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

K. A.

Long

J. S.

(Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.

Bryant

F. B.

(2013). EXCEL macro file for conducting Bryant-Satorra chi-square difference test via LISREL 8. Retrieved from http://www.econ.upf.edu/~satorra/

10.

Bryant

F. B.

Satorra

(2012). Principles and practice of scaled difference chi-square testing. Structural Equation Modeling, 19, 372–398. doi:10.1080/10705511.2012.687671

11.

Byars-Winston

Estrada

Howard

Davis

Zalapa

(2010). Influence of social cognitive and ethnic variables on academic goals of underrepresented students in science and engineering: A multiple groups analysis. Journal of Counseling Psychology, 57, 205–218. doi:10.1037/a0018608

12.

Cameron

Collie

C. L.

Baldwin

C. D.

Bartholomew

L. K.

Palmer

J. L.

Greer

Chang

(2013). The development of scientific communication skills: A qualitative study of the perceptions of trainees and their mentors. Academic Medicine, 88, 1499–1506. doi:10.1097/ACM.0b013e3182a34f36

13.

Cameron

Lee

H. Y.

Anderson

C. B.

Byars-Winston

Baldwin

C. D.

Chang

(2015). The role of scientific communication skills in trainees’ intention to pursue academic careers in biomedical research: A social cognitive analysis. CBE-Life Sciences Education, 14, 1–12. doi:10.1187/cbe.14-09-0152.

14.

Dores

G. M.

Chang

Berger

V. W.

Perkins

S. N.

Hursting

S. D.

Weed

D. L.

(2006). Evaluating research training outcomes: Experience from the Cancer Prevention Fellowship Program at the National Cancer Institute. Academic Medicine, 81, 535–541. doi:10.1097/01.ACM.0000225216.07584.60

15.

Fouad

N. A.

Guillen

(2006). Outcome expectations: Looking to the past and potential future. Journal of Career Assessment, 14, 130–142. doi:10.1177/1069072705281370

16.

Hackett

Betz

N. E.

Casas

J. M.

Rocha-Singh

I. A.

(1992). Gender, ethnicity, and social cognitive factors predicting the academic achievement of students in engineering. Journal of Counseling Psychology, 39, 527–538. doi:10.1037/0022-0167.39.4.527

17.

Bentler

P. M.

(1995). Evaluating model fit. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76–99). Thousand Oaks, CA: Sage.

18.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118

19.

Jöreskog

K. G.

Sörbom

(2006). LISREL 8.80. Chicago, IL: Scientific Software International, Inc.

20.

Kline

R. B.

(2005). Principles and practice of structural equation modeling. 2nd ed. New York, NY: Guildford Press.

21.

Lent

R. W.

Brown

(2008). Social cognitive career theory and subjective well-being in the context of work. Journal of Career Assessment, 16, 6–21. doi:10.1177/1069072707305769

22.

Lent

R. W.

Brown

S. D.

(2006). On conceptualizing and assessing social cognitive constructs in career research: A measurement guide. Journal of Career Assessment, 14, 12–35. doi:10.1177/1069072705281364

23.

Lent

R. W.

Brown

S. D.

Brenner

Chopra

S. B.

Davis

Talleyrand

Suthakaran

(2001). The role of contextual supports and barriers in the choice of math/science educational options: A test of social cognitive hypotheses. Journal of Counseling Psychology, 48, 474–483. doi:10.1037/0022-0167.48.4.474

24.

Lent

R. W.

Brown

S. D.

Hackett

(1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45, 79–122. doi:10.1006/jvbe.1994.1027

25.

Lent

R. W.

Brown

S. D.

Hackett

(2000). Contextual supports and barriers to career choice: A social cognitive analysis. Journal of Counseling Psychology, 47, 36–49. doi:10.1037/0022-0167.47.1.36

26.

Lent

R. W.

Brown

S. D.

Larkin

K. C.

(1984). Relation of self-efficacy expectations to academic achievement and persistence. Journal of Counseling Psychology, 31, 356–362. doi:10.1037/0022-0167.31.3.356

27.

Lent

R. W.

Brown

S. D.

Larkin

K. C.

(1986). Self-efficacy in the prediction of academic performance and perceived career options. Journal of Counseling Psychology, 33, 265–269. doi:10.1037/0022-0167.33.3.265

28.

Lent

R. W.

Brown

S. D.

Schmidt

Brenner

Lyons

Treistman

(2003). Relation of contextual supports and barriers to choice behavior in engineering majors: Test of alternative social cognitive models. Journal of Counseling Psychology, 50, 458–465. doi:10.1037/0022-0167.50.4.458

29.

Lent

R. W.

Brown

S. D.

Sheu

Schmidt

Brenner

B. R.

Gloster

C. S.

Treistman

Dana.

(2005). Social cognitive predictors of academic interests and goals in engineering: Utility for women and students at historically Black universities. Journal of Counseling Psychology, 52, 84–92. doi:10.1037/0022-0167.52.1.84

30.

Lent

R. W.

Larkin

K. C.

Brown

S. D.

(1989). Relation of self-efficacy to inventoried vocational interests. Journal of Vocational Behavior, 34, 279–288. doi:10.1016/0001-8791(89)90020-1

31.

Lent

R. W.

Lopez

A. M.

Jr Lopez

F. G.

Sheu

H.-B.

(2008). Social cognitive career theory and the prediction of interests and choice goals in the computing disciplines. Journal of Vocational Behavior, 73, 52–62. doi:10.1016/j.jvb.2008.01.002

32.

Marsh

H. W.

Balla

J. R.

McDonald

R. P.

(1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103, 391–410. doi:10.1037/0033-2909.103.3.391

33.

Marsh

H. W.

Lüdtke

Nagengast

Morin

A. J. S.

von Davier

(2013). Why item parcels are (almost) never appropriate: Two wrongs do not make a right-camouflaging misspecification with item parcels in CFA models. Psycholgical Methods, 18, 257–284. doi:10.1037/a0032773

34.

Marsh

H. W.

O’Neill

(1984). Self-description questionnaire III: The construct validity of multidimensional self-concept ratings by late adolescents. Journal of Educational Measurement, 21, 153–174. doi:10.1111/j.1745-3984.tb00227.x

35.

Mills

B. A.

Caetano

Rhea

A. E.

(2014). Factor structure of the Clinical Research Appraisal Inventory (CRAI). Evaluation & the Health Professions, 37, 71–82. doi:10.1177/0163278713500303

36.

Mullikin

E. A.

Bakken

L. L.

Betz

N. E.

(2007). Assessing research self-efficacy in physician-scientists: The Clinical Research Appraisal Inventory. Journal of Career Assessment, 15, 367–387. doi:10.1177/1069072707301232

37.

National Institute of General Medical Sciences. (2013). Research to understand and inform interventions that promote the research careers of students in biomedical and behavioral sciences. NIH Guide. Retrieved from http://grants.nih.gov/grants/guide/rfa-files/RFA-GM-14-013.html

38.

Satorra

Bentler

P. M.

(1988). Scaling corrections for chi-square statistics in covariance structure analysis. American Statistical Association 1988 proceedings of the business & economics section (pp. 308–313) Alexandria, VA: American Statistical Association.

39.

Satorra

Bentler

P. M.

(2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507–514. Retrieved from http://www.statmodel.com/chidiff.shtml