A Scale to Measure Science Communication Training Effectiveness

Abstract

The research describes efforts toward developing a valid and reliable scale used to assess science communication training effectiveness (SCTE) undertaken in conjunction with a 4-year project funded by the National Science Foundation. Results suggest that the SCTE scale possesses acceptable psychometric properties, specifically reliability and validity, with regard to responses from graduate students in science, technology, engineering, and math fields. While it cannot be concluded that the SCTE scale is the “be-all-end-all” tool, it may assist investigators in gauging success of science communication training efforts and by identifying aspects of the program that are working or that need improving.

Keywords

scale development survey measurement scale reliability and validity science communication training effectiveness

It’s a miserable time for science. America has an almost non-existent climate policy, its support for basic science research is flagging, and [scientists are harassed] in the name of ideology.

—Robinson Meyer, The Atlantic, April 28, 2017.

Scientists are increasingly realizing the importance of clearly communicating their work outside the sciences to policy makers, taxpayers, and the broader publics (Safford & Brown, 2019). Ensuring evidence-based decision making by individuals, policy makers, and government is a priority for many (Rodari & Weitkamp, 2015). But training for scientific careers rarely, if ever, includes the skills needed for effective communication with diverse audiences outside the discipline. As a result, training programs have proliferated, creating an expanding “scicomm training industry” (Wilhelmson, 2002). There is little consensus on how science communication training should be done, or what it should contain (Baram-Tsabari & Lewenstein, 2016, 2017a, 2017b; Neeley et al., 2014; Newman, 2019; Stocklmayer & Rennie, 2017). Training programs vary in approach, structure, length, target audiences, skills taught, and assessment.

Training programs require time and money; some can take days or weeks and cost tens of thousands of dollars. It is essential, therefore, to know that such training is effective, and what training elements lead to success. Effectiveness can be defined and evaluated in many different ways (see Baram-Tsabari & Lewenstein, 2016; Burns, O’Connor, & Stocklmayer, 2003; Nisbet & Scheufele, 2009). The gold standard of training effectiveness really should be the impact on target audiences (Bray et al., 2012; Rodgers et al., 2018).

There are two other sources of evaluation information that are more frequently pursued: perceptions of the trainees and viewpoints of the trainers (Peterman et al., 2017). The impact of training on trainees’ self-perception is a key influence on eventual outcomes. For example, self-confidence has a major impact on performance before audiences (Woolfolk et al., 2008). And trainers are in a position to determine the degree to which their inputs are realized in trainee performance. Widespread adoption of a reliable and valid evaluation instrument could improve the ability to conduct credible and defensible evaluations to guide science communication initiatives (Baram-Tsabari & Lewenstein, 2013, 2017a).

Having identified training elements and approaches that have demonstrable impacts on target audiences, we report here the development and validation of a science communication training effectiveness (SCTE) scale that assesses the impacts on trainees that are associated with communication successes. The scale was vetted using three cohorts of graduate science, technology, engineering, and math (STEM) students who participated in a science communication training (Rodgers et al., 2018). The research was motivated by a need to provide a measurement tool that can be used by others seeking to identify trainee attitudes and behaviors associated with successful science communication training.

Scale Development Challenges

In developing an evaluative scale, we sought insights into potential gaps or pitfalls from existing science communication education initiatives. First, measurement development has not been a main focus of science communication training programs (Robertson Evia et al., 2018). Second, most training evaluations have relied on qualitative approaches (e.g., Alexander et al., 2011; Silva & Bultitude, 2009), or in cases where quantitative approaches were used, there were a limited number of scale items or a single construct (e.g., self-efficacy; Miller et al., 2009; Robertson Evia et al., 2018; Silva & Bultitude, 2009). The general approach in these studies has been to determine whether scientists felt they benefited in some way from the training, for example, they learned from it, found it useful, or received quality instruction (see Miller et al., 2009; Silva & Bultitude, 2009), rather than to investigate the nature, intensity, and dimensions of SCTE.

Third, wide variations in program characteristics also pose challenges to scale development, including variations in (a) form, for example, workshop or seminar (e.g., Miller et al., 2009), outreach program (e.g., Clark et al., 2016), on-the-job training (e.g., Noblitt et al., 2010), or curricula (e.g., Hryciw & Dantas, 2016); (b) activity, for example, hands-on exercises (O’Leary & Abbaspour, 2015), practice sessions (Silva & Bultitude, 2009); (c) instructional method, for example, case study (Noblitt et al., 2010), inquiry (e.g., Clark et al., 2016), dialogic interaction (e.g., Besley et al., 2015), and social constructivism (Woolfolk et al., 2008); (d) trainee, for example, undergraduate and graduate STEM students (e.g., Alexander et al., 2011), career scientists (e.g., Miller et al., 2009), or others (see Silva & Bultitude, 2009); (e) target audience, for example, lay audience (Besley et al., 2015), policy makers (Bauer & Gregory, 2007), science journalists (Blum et al., 2006), corporate, nonprofit, or government organizations (see Longnecker & Gondwe, 2014); and (f) STEM field, for example, animal science (Hamernik & Johnson, 2015), physics (O’Leary & Abbaspour, 2015), biomedicine (Cameron et al., 2015), biology (Thompson & Blankinship, 2015), neuroscience (Holstein et al., 2015), and plant science (Ward et al., 2014). With so many program variations, no evaluative tool is likely to meet every need (see DeVellis, 1991). Nonetheless, evaluating the value or benefit of any science communication training requires a measurement tool that would aid in reaching accurate conclusions about program success overall and success of individual program components (see Rossi et al., 2003).

Conceptualizing Science Communication Training Effectiveness

A single SCTE scale cannot propose to measure everything. Decisions must be made about the core training impacts one wishes to evaluate. With this in mind, we chose to focus evaluation on these five elements: scientists’ perceived improvements in motivation, self-efficacy, cognition, affect, and behavior resulting from science communication training. We regarded a positive change in these measures as indicators of successful science communication training. Descriptions of and justifications for these five elements follow below.

Motivation

Motivation, a desire to do something (Deci & Ryan, 1985), determines whether individuals take a genuine interest in learning (Pintrich et al., 1993). Intrinsic motivation promotes optimal learning and enhances confidence and personal growth (Deci et al., 1991). Intrinsically motivated students have greater academic performance than extrinsically motivated students (Renninger, 2000). Motivation in science communication training is the degree to which scientists take a genuine interest in learning. Scientists exhibiting intrinsic motivation to learn should perceive greater communication gains and greater program performance than those motivated by external forces, such as research mentors (Besley, 2018; Silva & Bultitude, 2009).

Self-Efficacy

It is well established that self-efficacy (and the related self-confidence) enhances training outcomes and performance. Science communication self-efficacy, or perceived science communication self-efficacy, is a scientist’s belief in his or her ability to control his or her environment (Anderson et al., 2016) and perform communication functions or behaviors to enable effective communication with audiences (modified from Bandura, 1997, 2001). Self-efficacy has motivational effects that influence training responses, trainee satisfaction, and future use of learned skills (Christoph et al., 1998; Mathieu et al., 1993). Self-efficacy predicts career success of scientists (C. B. Anderson et al., 2016) and mediates a number of personal variables, such as trainee satisfaction (e.g., Saks, 1995). Individuals with higher levels of self-efficacy more easily cope with emotional arousal (Bandura, 1997, 2001). Thus, scientists with higher science communication self-efficacy may adapt more easily to stress associated with writing or giving oral presentations, facilitating effective use of science communication training (see Robertson Evia et al., 2018).

Cognition

Cognition refers to the process of acquiring knowledge and understanding (J. R. Anderson, 1983; Cacioppo & Petty, 1981). A main objective of science communication training is to increase scientists’ science communication knowledge. Training effectiveness is influenced by and should be determined by the extent to which the relevant information conveyed via workshops was transferred to trainees. Effective science communication training programs should increase scientists’ knowledge of communication concepts, skills, and tactics for communicating with various audiences. Cognition is reciprocally linked to motivation and self-efficacy. Motivation to learn and perceived self-efficacy contribute to the development of cognitive skills (Bandura, 1993) and ability to acquire and retain knowledge (Deci et al., 1991). And scientists who acquire greater science communication knowledge should have greater self-efficacy and self-confidence in their presentation abilities.

Affect

Affect refers to individuals’ cognitive representation of emotion (Russell, 1980). The influence of positive affect on cognitive organization and social interaction has been well established (see Isen, 1987). Researchers believe that positive emotions created by enjoyable experiences lead to a desire to engage further, although cognitive factors (i.e., learning) may also be relevant (Hidi & Renninger, 2006). For instance, positive affect improves creative problem solving and long-term (episodic) memory (Ashby & Isen, 1999). For a training to be effective, scientists must develop positive affect toward the training and its components, that is, learning, application, and product. Generally speaking, we want scientists to enjoy the training experience and feel satisfied with their abilities when they complete the training (see Miller et al., 2009; Silva & Bultitude, 2009). This positive affect will then be evident and influential in their communication efforts.

Related to this is ones’ attitudes, defined as enduring beliefs and dispositions toward objects or persons (Ajzen & Fishbein, 1980). Our focus on attitudes as an indication of SCTE is based on the important role that attitudes exert on influencing perceptions, organizing information, and guiding different behaviors (e.g., Eagly & Chaiken, 1993; Petty et al., 1994). Attitudes, for example, whether participation is regarded as positive, are a predictor of scientists’ participation training as well as in public engagement activities (Poliakoff & Webb, 2007). Besley et al. (2015) found that scientists’ attitudes toward engagement in science communication training increased the perceived value of the training related to specific program goals. Consistent with perspectives from cognitive psychology, greater learning is associated with more extensive networks in memory, thereby increasing the likelihood that attitudes developed in response to science communication training will be available to maintain scientists’ attitudes over time (see Haugtvedt & Petty, 1992).

Behavior

Behavior is defined as actions that are goal-directed toward enhancing communication functions through science communication training programs. Improved communication behaviors, that is, the ability for scientists to communicate clearly and engagingly with nonscientist audiences, are the main reason to offer science communication training opportunities. Behavior has long been studied in relation to educational training, and most theories on motivation have behavior or behavior intentions as a concept (Deci et al., 1991). Scientists who have higher perceived motivation, higher perceived knowledge, and who formulate more positive attitudes toward the training will be more successful in communication behavior than those with lower perceived motivation and knowledge, and more negative attitudes (see Ajzen & Fishbein, 1980).

Scale Development

We employed the scale development and validation procedures described by DeVellis (1991) and Clark and Watson (1995). Our scale was developed and validated in conjunction with a science communication training program focused on producing short videos about individuals’ research (see Rodgers et al., 2018). Training goals drew from journalism, strategic communication, biology, engineering, theater, and education. The program consisted of four hands-on workshops (learning) each with a different communication goal; four rehearsal sessions with live coaching and individualized feedback (application); and a 3-minute science story captured in video form (product). The reader is referred to a detailed discussion of the program’s components and procedures (see Luisi et al., 2019; Rodgers et al., 2018).

Item Pool Generation

Initial scale items from selected constructs were pooled from various literatures in science communication and social science journals based on recommendations by Longnecker and Gondwe (2014). Concepts outlined by Silva and Bultitude (2009; e.g., interest, usefulness, attitudes) were also used. As recommended by Bunce et al. (2010), existing measures were modified and additional new measures were created to form the SCTE scale. A summary of scales is provided due to space constraints but the full scales can be accessed on Figshare (see Supplemental Tables 1 and 2, available online). Briefly, measures included science communication self-efficacy (n = 7), measured on 4-point scales: 1 = not at all true, 2 = hardly true, 3 = moderately true, 4 = exactly true (e.g., I can always manage to solve difficult problems in science communication if I try hard enough; C. B. Anderson et al., 2016; Schwarzer & Jerusalem, 1995); oral presentation self-confidence (n = 4), measured on 5-point scales ranging from 1 = very insecure to 5 = very confident (e.g., based on the workshop you just completed, how would rate your level of confidence in your ability to . . . Give a scientific talk to a lay audience; C. B. Anderson et al., 2016; Rosenberg, 1979); science communication knowledge (n = 1; i.e., How much do you currently know about science communication? (0 = I know nothing at all 100 = I know all I could possibly know; Griffin et al., 2008; Griffin et al., 2013); attitudes toward workshops (n = 14), measured on 5-point Likert-type scales ranging from 1 = strongly disagree to 5 = strongly agree (e.g., Overall, this workshop . . . helped me make sense of how to communicate science effectively to a nonscience audience; Brownell et al., 2013; Silva & Bultitude, 2009); positive outcome expectations (n = 11), measured 7-point Likert-type scales ranging from 1 = strongly disagree to 7 = strongly agree (e.g., Participating in the workshops will . . . make me feel well prepared for my next presentation; C. B. Anderson et al., 2016; Bandura, 1997; Lent et al., 2008); training satisfaction (n = 7), measured on 7-point Likert-type scales ranging from 1 = strongly disagree to 7 = strongly agree (e.g., On the whole, I am satisfied with myself in this workshop; Rosenberg, 1979); attitudes toward coaching (n = 5), measured on 7-point Likert-type scales ranging from 1 = strongly disagree to 7 = strongly agree (e.g., The coaching session was valuable in polishing my final presentation); presentation preparation (n = 6), measured on 5-point Likert-type scales ranging from 1 = strongly disagree to 5 = strongly agree (e.g., I put a lot of effort into creating my presentation); and likelihood of using learned skills (n = 3) measured on 7-point scales ranging from 1 = not very likely to 7 = highly likely (e.g., Based on the workshop you just took, what is the likelihood that you will use the science communication skills the next time you make a speech?).

SCTE Scale Validity

Based on The Standards for Educational and Psychological Testing (American Education Research Association, American Psychological Association, and National Council on Measurement in Education, 2014), validation involved accumulating evidence from three sources: (1) face/content, (2) internal structure, and (3) relations to other variables.

Face Validity

Face validity was confirmed before any data were collected by a group of seven faculty researchers from different disciplines with backgrounds in communication. Faculty explored whether the proposed dimensions for SCTE were reasonable when considered from the trainee’s perspective. All faculties indicated that SCTE could be measured as a result of the proposed dimensions. Faculty trainers who led the workshops were able to describe their workshop effectiveness in terms of the proposed cognitive dimensions. Consistent with steps described by Bearden et al. (1993), it was concluded that proposed indicators of SCTE had achieved a high degree of face validity.

Content Validity

To assess content validity, four faculty researchers (two from education, one from plant sciences, and one from strategic communication) were asked to critique scale items categorized according to the proposed constructs. All but two measures met content validity criteria, that is, Perceived Science Communication Need and Personal Involvement, and were subsequently dropped to enhance content validity (e.g., Haynes et al., 1995). However, it was decided to still collect data on the dropped measures to be used as external variables in the validation process.

Results

SCTE Scale Reliability

Results suggest the SCTE scale has generally good internal consistency. Specifically, Cronbach’s alpha coefficients ranged from moderate (.555) to high (.962) for each measure of the SCTE scale with an overall average alpha coefficient of .856 based on the developmental sample, Spring 2016 (N = 18), and two validation samples, Fall 2016 (N = 11) and Spring 2017 (N = 14), with alpha coefficients ranging from .535 to .921 and an overall average alpha coefficient of .766.

SCTE Scale Validation

To build validity evidence, a two-step process was undertaken to examine the scale’s: (1) stability and/or sensitivity to change and (2) relations to external variables.

Step 1: Validation Procedure

First, we examined the SCTE scale’s stability and/or sensitivity to change. Using the combined data collected Fall 2016 and Spring 2017, paired t tests were conducted using pre- and postsurveys (Table 1). Results indicated significant positive shifts in Science Communication Self-Efficacy, Oral Presentation Self-Confidence, and Science Communication Knowledge. Scale sensitivity was strongest when trainees were asked about their science communication knowledge, which significantly increased by an average of 1.7 times from the point at which trainees began (M = 48.52) and completed (M = 82.52) the science communication training.

Table 1.

Constructs Measured in Pre- and Post-training Surveys.

Construct	Presurvey		Postsurvey		t	p
Construct	M	SD	M	SD	t	p
Science Communication Self-Efficacy	3.13	0.38	3.44	0.45	−3.889	.001
Oral Presentation Self-Confidence	3.26	0.68	4.17	0.37	−8.408	<.001
Science Communication Knowledge	48.52	18.11	82.52	9.31	−7.144	<.001

Note. N = 25.

Repeated measures analyses of variance were also conducted for SCTE measures using postsurvey workshop/rehearsal data (Table 2). Results showed that trainees’ Perceived Workshop Knowledge was relatively high, ranging from 4.71 to 5.29 on a 1-to-6 scale. Attitudes Toward Coaching was also generally high (with means for the four sessions ranging from 5.95 to 6.44 on a 1-7 scale), and trainees, on average, indicated they had adequate Presentation Preparation (with means for the four workshops ranging from 3.75 to 3.98 on a 1-5 scale). The statistically nonsignificant F tests indicated that the four workshops/rehearsals were comparable in terms of trainees’ affect, measured by Attitudes Toward Coaching, and trainees’ behavior, measured by Presentation Preparation (Table 2). After completing the training program, trainees reported moderate to high, suggesting the scale was sensitive to detecting change for the following: Attitudes Toward the Workshops (M = 3.76, SD = 0.34), Positive Outcome Expectations (M = 5.96, SD = 0.73), Training Satisfaction (M = 6.13, SD = 0.63), and Likelihood of Using Learned Skills (M = 6.67, SD = 0.62).

Table 2.

Constructs Measured After Each Workshop/Rehearsal Session.

Construct	Workshop 1		Workshop 2		Workshop 3		Workshop 4		Repeated-measures ANOVA
Construct	M	SD	M	SD	M	SD	M	SD	F ^a	p
Perceived Workshop Knowledge^b	4.88	0.60	4.71	0.75	5.16	0.58	5.29	0.58
Attitudes Toward Coaching	5.95	0.69	6.10	0.70	6.32	0.62	6.44	0.55	2.37	.105
Presentation Preparation	3.75	0.48	3.86	0.51	3.98	0.47	3.90	0.57	1.52	.242

Note. N = 25. ANOVA = analysis of variance.

N = 21 for both F tests due to missing values. ^bF test was not conducted for “Perceived Knowledge Related to Workshop” because the items are different for different workshops.

Step 2. Validation Procedure

As further validation, we explore the SCTE’s ability to make distinctions among (a) variables that are not part of the scale and (b) variables predicted to be theoretically meaningful. This was accomplished, first, by correlating the SCTE variables with two variables external to the SCTE scale, that is, perceived science communication need, consisting of 7 (1-5) scaled items adapted from Griffin et al. (2008), and personal involvement, consisting of 9 (1-7) scaled items adapted from Zaichkowsky (1994). Using pre- and/or postsurvey data (Fall 2016 and Spring 2017), there was a significant negative correlation between perceived science communication need and science communication self-efficacy, oral presentation self-confidence, and training satisfaction (Table 3). The SCTE scale appears to have consistently identified the nonscale variable, perceived science communication need, in the three SCTE measures used for this portion of the validity test.

Table 3.

Correlations Between SCTE Measures and Perceived Science Communication Need at Pre- and/or Post-survey.

	1	2	3	4	5	6	7	8	9	10	11
1. Perceived Science Communication Need	1
2. Science Communication Self-efficacy (Pre)	−.402^*	1
3. Science Communication Self-Efficacy (post)	−.473^*	.530^**	1
4. Oral Presentation Self-Confidence (pre)	−.400^*	.603^**	.309	1
5. Oral Presentation Self-Confidence (post)	−.419^*	.612^**	.535^**	.381	1
6. Science Communication Knowledge (pre)	−.007	.266	.123	.561^**	−.015	1
7. Science communication knowledge (post)	−.287	.173	.470^*	.106	.279	.017	1
8. Attitudes Toward Workshops (post)	−.300	.162	.692^**	.160	.337	.041	.476^*	1
9. Positive Outcome Expectations (post)	−.358	.472^*	.702^**	.264	.667^**	−.110	.363	.470^*	1
10. Training Satisfaction (post)	−.473^*	.397^*	.630^**	.245	.567^**	−.148	.436^*	.656^**	.651^**	1
11. Likelihood of Using Learned Skills (post)	−.348	.238	.513^**	.042	.323	−.213	.307	.509^**	.611^**	.647^**	1

Note. SCTE = science communication training effectiveness. N = 25; Numbers 2, 4, and 6 were measured at presurvey; Numbers 3, 5, and 7 to 11 were measured at postsurvey.

p < .05. ^**p < .01.

Using postworkshop data, the SCTE measures were correlated with an external variable, personal involvement (see Supplemental Table 3 in Figshare, available online). While the personal involvement variable was significantly correlated with some SCTE measures, no consistent pattern of results was revealed. The same was true of correlations conducted for personal involvement and the three SCTE variables, suggesting that the SCTE measures—perceived workshop knowledge (cognition), attitudes toward coaching (affect), and presentation preparation (behavior)—represent distinct concepts.

A perhaps more generally useful scale should be able to make meaningful theoretical connections with other variables. Earlier, we suggested that scientists who acquired greater science communication knowledge would report having greater self-confidence in their presentation abilities. To examine this, correlations were run between perceived workshop knowledge and oral presentation self-confidence using pre- and/or postsurvey data for the four workshops. Results revealed that across three of the four correlations we see a moderate to large increase in correlation strength from the before test to the after test: Workshop 1 (before: r = .086, p = .684; after: r = .552, p < .01), Workshop 2 (before: r = .427, p < .05; after: r = .390, p = .054), Workshop 3 (before: r = .339, p = .097; after: r = .431, p < .05), and Workshop 4 (before: r = .371, p = .068; after: r = .717, p < .001). However, this pattern fails to materialize for Workshop 2. Variations in participants’ topic knowledge and variations in the role of different trainers may explain results.

We also hypothesized that scientists with higher levels of self-efficacy (motivation) resulting from the training adapt more easily to stress and anxiety (emotional arousal) associated with giving oral presentations to non-expert audiences. To examine this, two correlations were run. First, science communication self-efficacy was correlated with two (1-5) scaled items representing worry/anxiety, that is, “I am worried about my ability to succeed in the science communication workshops,” and stress, that is, “I am stressed out thinking I may fail in the science communication workshops.” Using presurvey data, there was a significant negative correlation between science communication self-efficacy and the worry/anxiety scale item (r = −.439, p < .05), and science communication self-efficacy and the stress scale item (r = −.540, p < .01). The same results occurred for postsurvey data when science communication self-efficacy was correlated with worry/anxiety (r = −.447, p < .05) and stress scale items (r = −.506, p < .05).

Second, we correlated oral presentation self-confidence (motivation) with the same worry/anxiety, that is, “I am worried about my ability to succeed in the science communication workshops,” and stress scaled items, that is, “I am stressed out thinking I may fail in the science communication workshops.” This was based on the theorizing that self-confidence would reinforce a scientist’s ability to manage stress and anxiety associated with giving oral presentations to lay audiences. If this were true, the SCTE scale should show a negative correlation between oral presentation self-confidence and the worry/anxiety and stress scale items. Using pre- and postsurvey data, results confirmed that this was, indeed, the case for both worry/anxiety (pre: r = −.298, p < .148; post: r = −.559, p < .01) and stress (pre: r = −.532, p < .01; post: r = −.518, p < .01). The data indicate a substantial increase in correlation strength for worry/anxiety but correlation consistencies did not materialize for stress, presumably because worry/anxiety is more enduring and stress is more situational.

Taken as a whole, the results of the two-step validation process provide strong supportive evidence for the validity of the SCTE scale. Step 1 validation results showed a consistent pattern in the scale’s ability to produce positive responses in trainees’ motivation, cognition, affect, and behavior pre- and/or posttraining. The scale showed sensitivity to all the reported measures but was perhaps most sensitive to the cognition measures. Results of the Step 2 validation process provided further validity evidence, showing that the SCTE scale accurately identified relationships (and nonrelationships) between the scale’s measures and external variables and variables assumed to be theoretically meaningful. Critical information concerning inferences about trainees’ science communication abilities relative to their affect was also detected after the workshops, giving insights into trainees’ thought processes. Results of the two-step validation process were strongly supportive of the SCTE scale’s validity, suggesting the four constructs could be important aspects of SCTE outcomes.

Discussion

Additional evaluation instrumentation is needed to measure effectiveness of science communication training programs designed to improve scientists’ communication with nonscientists. Such instrumentation would enhance accuracy of evaluation efforts and programmatic goals and advance research. In this study, results were presented on the development and validation steps undertaken for a SCTE scale.

The results of this study provide strong supportive evidence for the SCTE scale’s psychometric properties, specifically reliability and validity. The evidence suggests the scale is reliable, accurate, stable, and sensitive to change. For example, the SCTE scale had generally good internal reliability based on moderate to high Cronbach’s alpha coefficients. One exception was the behavioral measure, Presentation Preparation, which yielded Cronbach’s alpha coefficients in the moderate (.855) to low (.535) range. This is perhaps due to trainee ambivalence, that is, trainees’ mood, the day, the weather, and so on. Alternatively, trainees’ feelings about preparing for presentations may not be as straightforward. For example, trainees may agree with certain aspects of their preparation performance and not others.

The scale also detected trainee response shifts occurring pre- and/or posttraining. Specifically, perceived Science Communication Self-Efficacy, Oral Presentation Self-Confidence, and Science Communication Knowledge showed significant increases from the time trainees began and completed the training. The scale distinguished between two external variables, for example, Perceived Need for Science Communication and Personal Involvement. When SCTE measures were correlated with Perceived Science Communication Need, results showed that trainees who did not understand how the communication skills would advance their careers were less likely to report mastery over those skills. This suggests the scale was accurate in its ability to differentiate between measures that both corresponded with and did not correspond with the conceptual definition of SCTE. In contrast, no consistent pattern of results was revealed when correlating SCTE measures with Personal Involvement, perhaps because the trainees were self-selected and already involved before training.

As predicted, the scale was confirmed to show significant positive relationships between Perceived Workshop Knowledge and Oral Presentation Self-Confidence, and significant negative relationships between emotional arousal (affect) and Science Communication Self-Efficacy and Oral Presentation Self-Confidence. The scale also detected dispositional differences (trainee perceived need) and situational differences (trainee stress/anxiety).

Reliability, accuracy, stability, and sensitivity to change are all desirable qualities in any scale. For a scale to receive broad adoption, it should produce consistent results when used over time. Our study produced relatively consistent results when the scale was repeated under the same conditions (four workshops/rehearsals) over time (three semesters) and with different samples of graduate STEM students. To show reproducibility, the scale would need to be tested under changed conditions and with different and larger samples. Given the hands-on nature of the workshops used in conjunction with our validation study, large samples of trainees were not practical.

We have argued that a single SCTE scale cannot propose to measure everything. To organize the scale and enhance accuracy, we proposed a framework to measure three main program components, i.e., learning, application, and product, in conjunction with a science communication training. Measures for overall training effectiveness were also provided. The five elements composing the SCTE scale, namely, motivation, self-efficacy, cognition, affect, and behavior, showed promise in their ability to produce a nuanced and potentially fuller picture. For example, our results showed that trainees who perceived greater gains in Science Communication Self-Efficacy also reported greater gains in Science Communication Knowledge and exhibited more positive Attitudes Toward the Workshops. An implication is that science communication training evaluations that rely on a single item or even multiple items on a single dimension might miss insights provided by other scale dimensions.

Results also demonstrated that trainees responded more positively to one aspect of the training (e.g., learning) as compared to another (e.g., product). Results showed that trainee responses differed even within a component (e.g., learning) that contained multiple offerings (e.g., four workshops). An implication is that to draw stronger conclusions, measurements can be undertaken for the specific program component(s) and the program as a whole. This may help strengthen evaluation efforts by providing outcome measures that relate to a program’s individual parts, yielding potential insights into the specific aspect(s) of a training that yield desired effects versus those aspects that may need improvement.

In this respect, the SCTE scale serves a descriptive and diagnostic purpose in helping investigators to better understand what is working and/or how to improve current and future training efforts (see Feldon, Maher, & Timmerman, 2010). Descriptively, the SCTE scale can assess program performance on the four dimensions, accomplished with pre- and/or postsurveys. Diagnostically, scale items that measure specific program components can be used to improve low-performing components and subsequent measures taken before and after implementation of such improvements can determine whether desired results were achieved and inform next steps.

Practically speaking, understanding which aspects of science communication training contributed the most/least to trainee success will advance techniques and instructional methods used and improve identification of shared progress in the field of science communication education. Theoretically, the SCTE scale can be used to determine effects of science communication training programs to assess if there were positive gains in trainees’ motivation, cognition, affect, and behavior acquired in response to the training. Tracked over time, results can be used as a benchmark against which to track shifts over time in science communication education.

The SCTE scale can also be used to conduct training needs analysis to assist science communication educators in making curricular decisions and/or to help individual scientists in deciding who needs to be trained and on what topics. Particularly, the Perceived Workshop Knowledge items would assist in gauging scientists’ knowledge on specific communication topics. Individuals who perform low/high on the Perceived Workshop Knowledge scale(s) can use results to diagnose whether additional science communication training is needed and in what areas, saving time and money in the long run. The SCTE scale can also be used to make predictions about the probability that trainees will succeed in a given training and in developing models that predict the impact of training efforts in scientists’ career success (e.g., C. B. Anderson et al., 2016; Baram-Tsabarai & Osborne, 2015; Dudo, 2013).

Limitations and Future Directions for Research

The procedures carried out in developing and validating the SCTE scale were rigorous but were done under specific conditions that may affect scale precision. Graduate students are likely to have a homogenous set of viewpoints and experiences with science communication training so a diverse pool of participants is needed. Science communication success is measured by self-report. Ideally, this should be combined with external performance evaluations. The scale was validated in a naturalistic setting outside the traditional classroom, but we see no reason why it could not be used in a classroom setting. The program used in scale validation identified goals that many science communication trainings have but these are not universal goals and other goals should be tested (see Besley & Tanner, 2011). It is reasonable to assume that instructors also played a role in the development of learning in a science communication–teaching environment. Future studies may evaluate effectiveness at the level of the individual instructor if that is a program goal. Future studies may identify additional scale dimensions to further clarify definitions and measures. Finally, science communication learning opportunities are constantly changing, so to prevent “wear and tear” and further enhance accuracy the SCTE scale can be recalibrated over time to reflect current standards and practices in science communication education (e.g., Trench & Miller, 2012).

Conclusion

Careful empirical research is needed to advance the “science of science communication” training initiatives (Fischhoff & Scheufele, 2014, p. 13583). Such evidence will help to transform STEM graduate education, providing added value to society. We have presented results on initial steps undertaken to develop and test validity and reliability of a science communication training effectiveness (SCTE) scale. While it cannot be concluded that the SCTE scale is a “be-all-end-all” tool, we hope it provides a potentially useful instrument for those who want to measure effectiveness of science communication training programs designed to help scientists communicate scientific discoveries with the general public.

Supplemental Material

SC_Scale_Table_1_Supplemental – Supplemental material for A Scale to Measure Science Communication Training Effectiveness

Supplemental material, SC_Scale_Table_1_Supplemental for A Scale to Measure Science Communication Training Effectiveness by Shelly Rodgers, Ze Wang and Jack C. Schultz in Science Communication

Supplemental Material

SC_Scale_Table_2_Supplemental – Supplemental material for A Scale to Measure Science Communication Training Effectiveness

Supplemental material, SC_Scale_Table_2_Supplemental for A Scale to Measure Science Communication Training Effectiveness by Shelly Rodgers, Ze Wang and Jack C. Schultz in Science Communication

Supplemental Material

SC_Scale_Table_3_Supplemental – Supplemental material for A Scale to Measure Science Communication Training Effectiveness

Supplemental material, SC_Scale_Table_3_Supplemental for A Scale to Measure Science Communication Training Effectiveness by Shelly Rodgers, Ze Wang and Jack C. Schultz in Science Communication

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding is provided by a National Science Foundation Grant No. 1545177, “NRT-IGE: A test bed for STEM graduate student communication training.”

ORCID iD

Shelly Rodgers

Supplemental Material

Supplemental material for this article is available online at

Author Biographies

Shelly Rodgers, PhD, is a professor of strategic communication and Senior Research Advisor for the Health Communication Research Center, School of Journalism, University of Missouri. She is nationally ranked as a leading health and advertising researcher. Her federally funded research, published in leading science, health and advertising journals, has focused on using information and technology to improve individuals’ health. She is the editor-in-chief of the Journal of Advertising, 2019 SEC Faculty Achievement Award winner, 2018 William T. Kemper Fellow, and 2010 President of the American Academy of Advertising.

Ze Wang, PhD, is an associate professor in the Research Methods program, Department of Educational, School and Counseling Psychology, University of Missouri. Her research interests include statistical modeling and psychometrics. She has extensive experience in research design and applied statistical analyses. Techniques used in her published research include a priori and post hoc power analyses, factor analysis, measurement invariance over time and across groups, among others.

Jack C. Schultz, PhD, is Executive Director of Research Development at the University of Toledo, and former director of the interdisciplinary Christopher S. Bond Life Sciences Center at the University of Missouri, where he led efforts to develop science communication training for STEM graduate students and faculty. He is Emeritus Distinguished Professor of Entomology at the Pennsylvania State University. His federally funded research has focused on interactions between plants and their environment, including communication among plants and insects. He was elected Fellow of American Association for the Advancement of Science in 2011.

References

Ajzen

Fishbein

(1980). Understanding attitudes and predicting social behavior. Prentice Hall.

Alexander

Waldron

A. M.

Abell

S. K.

(2011). Science and me: A student-driven science outreach program for lay adult audiences. Journal of College Science Teaching, 40(6), 28-34.

Anderson

C. B.

Lee

H. Y.

Byars-Winston

Baldwin

C. D.

Cameron

Chang

(2016). Assessment of scientific communication self-efficacy, interest, and outcome expectations for career development in academic medicine. Journal of Career Assessment, 24(1), 182-196. https://doi.org/10.1177/1069072714565780

Anderson

J. R.

(1983). The architecture of cognition. Harvard University Press.

Ashby

F. G.

Isen

A. M.

(1999). A neuropsychological theory of positive affect and its influence on cognition. Psychological Review, 106(3), 529-550. https://doi.org/10.1037/0033-295X.106.3.529

Bandura

(1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117-148. https://doi.org/10.1207/s15326985ep2802_3

Bandura

(1997). Self-efficacy: The exercise of control. Freeman.

Bandura

(2001). Social cognitive theory: An agentic perspective. Asian Journal of Social Psychology, 2(1), 21-41. https://doi.org/10.1111/1467-839x.00024

Baram-Tsabari

Lewenstein

B. V.

(2013). Assessing scientists’ written skills in public communication of science. Science Communication, 35(1), 56-85. https://doi.org/10.4135/9781473941618

10.

Baram-Tsabari

Lewenstein

B. V.

(2016). Assessment. In Van der Sanden

M. C. A.

de Vries

(Eds.), Science and technology communication and education (pp. 163-185). Sense.

11.

Baram-Tsabari

Lewenstein

B. V.

(2017a). Preparing scientists to be science communicators. In Patrick

P. G.

(Ed.), Preparing informal science educators (pp. 437-471). Springer International.

12.

Baram-Tsabari

Lewenstein

B. V.

(2017b). Science communication training: What are we trying to teach? International Journal of Science Education, Part B, 7(3), 285-300. https://doi.org/10.1080/21548455.2017.1303756

13.

Baram-Tsabarai

Osborne

(2015). Bridging science education and science communication research. Journal of Research in Science Teaching, 52(2), 135-144. https://doi.org/10.1002/tea.21202

14.

Bauer

Gregory

(2007). From journalism to corporate communication in post-war Britain. In Bauer

Bucci

(Eds.), Journalism, science and society: Science communication between news and public relations (pp. 33-51). Routledge.

15.

Bearden

W. O.

Netemeyer

R. G.

Mobley

M. F.

(1993). Handbook of marketing scales. Sage.

16.

Besley

J. C.

(2018). Understanding scientists’ willingness to engage. Science Communication, 40(5), 559-590. https://doi.org/10.1177/1075547018786561

17.

Besley

J. C.

Dudo

Storksdieck

(2015). Scientists’ views about communication training. Journal of Research in Science Teaching, 52(2), 199-220. https://doi.org/10.1002/tea.21186

18.

Besley

J. C.

Tanner

A. H.

(2011). What science communication scholars think about training scientists to communicate. Science Communication, 33(2), 239-263. https://doi.org/10.1177/1075547010386972

19.

Blum

Knudson

Marantz Henig

(2006). A field guide for science writers (2nd ed.). Oxford University Press.

20.

Bray

France

Gilbert

J. K.

(2012). Identifying the essential elements of effective science communication: What do the experts say? International Journal of Science Education, Part B, 2(1), 23-41. https://doi.org/10.1080/21548455.2011.611627

21.

Brownell

S. E.

Price

J. V.

Steinman

(2013). Science communication to the general public: Why we need to teach undergraduate and graduate students this skill as part of their formal scientific training. Journal of Undergraduate Neuroscience Education, 12(1), e6-e10. https://doi.org/10.1121/2.0000569

22.

Bunce

D. M.

VandenPlas

J. R.

Neiles

K. Y.

Flens

E. A.

(2010). Development of a valid and reliable student-achievement and process-skills instrument. Journal of College Science Teaching, 39(5), 50-55.

23.

Burns

T. S.

O’Connor

D. J.

Stocklmayer

S. M.

(2003). Science communication: A contemporary definition. Public Understanding of Science, 12(2), 183-202. https://doi.org/10.1177/09636625030122004

24.

Cacioppo

J. T.

Petty

R. E.

(1981). Social psychological procedures for cognitive response assessment: The thought listing technique. In Merluzzi

T. V.

Glass

C. R.

Genest

(Eds.), Cognitive assessment (pp. 309-342). Guilford Press.

25.

Cameron

Lee

H. Y.

Anderson

Byars-Winston

Baldwin

C. D.

Chang

(2015). The role of scientific communication skills in trainees’ intention to pursue biomedical research careers: A social cognitive analysis. CBE–Life Sciences Education, 14(4), Article 46. https://doi.org/10.1187/cbe.14-09-0152

26.

Christoph

R. T.

Schoenfeld

G. A.

Tansky

J. W.

(1998). Overcoming barriers to training utilization technology: The influence of self-efficacy factors on multimedia-based training receptiveness. Human Resource Development Quarterly, 9(1), 25-38. https://doi.org/10.1002/hrdq.3920090104

27.

Clark

Russell

Enyeart

Gracia

Wessel

Jarmoskaite

Polioudakis

Stuart

Gonzalez

MacKrell

Rodenbusch

Stovall

G. M.

Beckham

J. T.

Montgomery

Tasneem

Jones

Simmons

Roux

(2016). Science educational outreach programs that benefit students and scientists. PLOS Biology, 14(2), e1002368. https://doi.org/10.1371/journal.pbio.1002368

28.

Clark

L. A.

Watson

(1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319. https://doi.org/10.1037/1040-3590.7.3.309

29.

Deci

E. L.

Ryan

R. M.

(1985). The general causality orientations scale: Self-determination in personality. Journal of Research in Personality, 9(2), 109-134. https://doi.org/10.1016/0092-6566(85)90023-6

30.

Deci

E. L.

Vallerand

R. J.

Pelletier

L. G.

Ryan

R. M.

(1991). Motivation and education: The self-determination perspective. Educational Psychologist, 26(3-4), 325-346. https://doi.org/10.15760/etd.2025

31.

DeVellis

R. F.

(1991). Scale development: Theory and applications (4th ed.). Sage.

32.

Dudo

(2013). Toward a model of scientists’ public communication activity: The case of biomedical researchers. Science Communication, 35(4), 476-501. https://doi.org/10.1177/1075547012460845

33.

Eagly

A. H.

Chaiken

(1993). Psychology of attitudes. Harcourt Brace Jovanovich.

34.

Feldon

D. F.

Maher

M. A.

Timmerman

B. E.

(2010). Performance-based data in the study of STEM Ph.D. Education. Science, 329(16), 282-283. https://doi.org/10.1126/science.1191269

35.

Fischhoff

Scheufele

D. A.

(2014). The science of science communication II. Proceedings of the National Academy of Sciences U S A, 111(4), 13583-13584. https://doi.org/10.1073/pnas.1414635111

36.

Griffin

R. J.

Dunwoody

Zhang

Z. J.

(2013). Linking risk messages to information seeking and processing. In Salmon

(Ed.), Communication Yearbook 36 (pp. 323-362). Routledge/Taylor & Francis.

37.

Griffin

R. J.

Yang

Huurne

Boerner

Ortiz

Dunwoody

(2008). After the flood: Anger, attribution and the seeking of information. Science Communication, 29(3), 285-315. https://doi.org/10.1177/1075547007312309

38.

Hamernik

D. L.

Johnson

K. A.

(2015). Communicating animal science to the public, policymakers, and students. Animal Frontiers, 5(3), 4-5. https://doi.org/10.2527/af.2015-0024

39.

Haugtvedt

C. P.

Petty

R. E.

(1992). Personality and persuasion: Need for cognition moderates the persistence and resistance of attitude changes. Journal of Personality and Social Psychology, 63(2), 308-319. https://doi.org/10.1037/0022-3514.63.2.308

40.

Haynes

S. N.

Richard

D. C. S.

Kubany

E. S.

(1995). Content validity in psychological assessment. A functional approach to concepts and methods. Psychological Assessment, 7(3), 238-247. https://doi.org/10.1037/1040-3590.7.3.238

41.

Hidi

Renninger

K. A.

(2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111-127. https://doi.org/10.1207/s15326985ep4102_4

42.

Holstein

S. E.

Mickley Steinmetz

K. R.

Miles

J. D.

(2015). Teaching science writing in an introductory lab course. Journal of Undergraduate Neuroscience Education, 13(2), A101-A109.

43.

Hryciw

D. H.

Dantas

A. M.

(2016). Scaffolded research-based learning for the development of scientific communication in undergraduate physiology students. International Journal of Innovation in Science and Mathematics Education, 24(1), 1-11.

44.

Isen

A. M.

(1987). Positive affect, cognitive processes, and social behavior. Advances in Experimental Social Psychology, 20, 203-253. https://doi.org/10.1016/S0065-2601(08)60415-3

45.

Lent

R. W.

Lopez

A. M.

Jr. Lopez

F. G.

Sheu

H. B.

(2008). Social cognitive career theory and the prediction of interests and choice goals in the computing disciplines. Journal of Vocational Behavior, 73(1), 52-62. https://doi.org/10.1016/j.jvb.2008.01.002

46.

Longnecker

Gondwe

(2014). Graduate degree programmes in science communication: Educating and training science communicators to work with communities. In Wee Hin

L. T.

Subramaniam

(Eds.), Communicating science to the public (pp. 141-160). Springer.

47.

Luisi

M. L.

Rodgers

Schultz

J. C.

(2019). Experientially learning how to communicate science effectively: A case study on decoding science. Journal of Research in Science Teaching, 56(8), 1135-1152. https://doi.org/10.1002/tea.21554

48.

Mathieu

J. E.

Martineau

J. W.

Tannenbaum

S. I.

(1993). Individual and situational influences on the development of self-efficacy: Implication for training effectiveness. Personnel Psychology, 46(1), 125-147. https://doi.org/10.1111/j.1744-6570.1993.tb00870.x

49.

Miller

Fahy

, & The ESConet Team (2009). Can science communication workshops train scientists for reflexive public engagement? The ESConet experience. Science Communication, 31(1), 116-126. https://doi.org/10.1177/1075547009339048

50.

Meryer

(2017). The case for the politicization of science. https://www.theatlantic.com/science/archive/2017/04/scientists-should-just-be-political/524682/

51.

Neeley

Goldman

Smith

Baron

Sunu

(2014). Mapping the pathways to integrate science communication training into STEM graduate education. COMPASS. https://www.informalscience.org/sites/default/files/GradSciComm_Roadmap_Final.compressed.pdf

52.

Newman

T. P.

(Ed.). (2019). Theory and best practices in science communication training. Routledge, Taylor & Francis Group.

53.

Nisbet

M. C.

Scheufele

D. A.

(2009). What’s next for science communication? Promising directions and lingering distractions. American Journal of Botany, 96(10), 1767-1778. https://doi.org/10.3732/ajb.0900041

54.

Noblitt

Vance

D. E.

Smith

L. D.

(2010). A comparison of case study and traditional teaching methods for improvement of oral communication and critical-thinking skills. Journal of College Science Teaching, 39(5), 26-32.

55.

O’Leary

Abbaspour

(2015). LCphysX: Teaching undergraduate physics majors multi-media science communication skills for public outreach. In Eblen-Zayas

Behringer

Kozminski

(Eds.), 2015 Conference on Laboratory Instruction Beyond the First Year (pp. 72-75). AAPT Advanced Labs. https://doi.org/10.1119/bfy.2015.pr.018

56.

Peterrman

Robertson Evia

Cloyd

Besley

(2017). Assessing public engagement outcomes by the use of an outcome expectations scale for scientists. Science Communication, 39(6), 782-797. https://doi.org/10.1177/1075547017738018

57.

Petty

R. E.

Priester

J. R.

Wegener

D. T.

(1994). Cognitive processes in attitude change. In Wyer

R. S.

Srull

T. K.

(Eds.), Handbook of social cognition (Vol. 2, pp. 69-142). Lawrence Erlbaum.

58.

Pintrich

P. R.

Smith

D. A.

Garcia

McKeachie

W. J.

(1993). Reliability and predictive validity of the motivated strategies for learning questionnaire. Educational and Psychological Measurement, 53(3), 801-813. https://doi.org/10.1177/0013164493053003024

59.

Poliakoff

Webb

T. L.

(2007). What factors predict scientists’ intentions to participate in public engagement of science activities? Science Communication, 29(2), 242-263. https://doi.org/10.1177/1075547007308009

60.

Renninger

K. A.

(2000). Individual interest and its implications for understanding intrinsic motivation. In Sansone

Harackiewicz

J. M.

(Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 373-404). Academic Press.

61.

Robertson Evia

Peterman

Cloyd

Besley

(2018). Validating a scale that measures scientists’ self-efficacy for public engagement with science. International Journal of Science Education, Part B, 8(1), 40-52. https://doi.org/10.1080/21548455.2017.1377852

62.

Rodari

Weitkamp

(2015). Short training courses in science communication. Why? To whom? What? Journal of Science Communication, 14(4), Article E. https://doi.org/10.22323/2.14040501

63.

Rodgers

Wang

Maras

M. A.

Burgoyne

Balakrishnan

Stemmle

Schultz

J. C.

(2018). Decoding science: Development and evaluation of a science communication training program using a triangulated framework. Science Communication, 40(1), 3-32. https://doi.org/10.1177/1075547017747285

64.

Rosenberg

(1979). Conceiving the self. Basic Books.

65.

Rossi

P. H.

Lipsey

M. W.

Freeman

H. E.

(2003). Evaluation: A systematic approach. Sage.

66.

Russell

J. A.

(1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161-1178. https://doi.org/10.1037/h0077714

67.

Safford

Brown

(2019). Communicating science to policymakers: Six strategies for success. Nature, 572(7771), 681-682. https://doi.org/10.1038/d41586-019-02372-3

68.

Saks

A. M.

(1995). Longitudinal field investigation of the moderating and mediating effects of self-efficacy on the relationship between training and newcomer adjustment. Journal of Applied Psychology, 80(2), 221-225. https://doi.org/10.1037/0021-9010.80.2.211

69.

Schwarzer

Jerusalem

(1995). Generalized self-efficacy scale. In Weinman

Wright

Johnston

(Eds.) Measures in health psychology: A user’s portfolio (pp. 35-37). NFER-NELSON.

70.

Silva

Bultitude

(2009). Best practice in communications training for public engagement with science, technology, engineering and mathematics. Journal of Communication, 8(2), A03. https://doi.org/10.22323/2.08020203

71.

Stocklmayer

S. M.

Rennie

L. J.

(2017). The attributes of informal science education: A science communication perspective. In Patrick

P. G.

(Ed.), Preparing informal science educators (pp. 527-544). Springer.

72.

Thompson

Blankinship

L. A.

(2015). Teaching information literacy skills to sophomore-level biology majors. Journal of Microbiology & Biology Education, 16(1), 29-33. https://doi.org/10.1128/jmbe.v16i1.818

73.

Trench

Miller

(2012). Policies and practices in supporting scientists’ public communication through training. Science and Public Policy, 39(6), 722-731. https://doi.org/10.1093/scipol/scs090

74.

Ward

J. R.

Clarke

H. D.

Horton

J. L.

(2014). Effects of a research-infused botanical curriculum on undergraduates’ content knowledge, STEM competencies, and attitudes toward plant sciences. CBE: Life Sciences Education, 13(3), 387-396. https://doi.org/10.1187/cbe.13-12-0231

75.

Wilhelmson

(2002). Science communication: Science’s next frontier. Science. https://www-sciencemag-org-s.web.bisu.edu.cn/careers/2002/05/science-communication-science-s-next-frontier

76.

Woolfolk

Hughes

Wallup

(2008). Psychology in education. Lowman.

77.

Zaichkowsky

J. L.

(1994). The personal involvement inventory: Reduction, revision, and application to advertising. Journal of Advertising, 23(4), 59-70. https://doi.org/10.1080/00913367.1943.10673459

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.01 MB