Science Communication Training in North America: Preparing Whom to Do What With What Effect?

Scientists as Communicators

Although calls for more frequent interaction between scientists and public audiences are not new (Bodmer, 1985), global leaders of the scientific community are increasingly vocal when it comes to improving public communication of science (e.g., Cicerone, 2006, 2010; Jia & Liu, 2014; Leshner, 2015; McNutt, 2016; Napolitano, 2015; The Royal Society, 2004; South Africa Department of Science and Technology, 2014; Thorp, 2020). Their pleas consistently aim to motivate STEM (science, technology, engineering, and mathematics) experts to heighten the volume and quality of their science communication efforts.

For these pleas to be realized scientists must possess the motivations and skills necessary to communicate more often and to do so effectively. Do they? Large surveys of members of groups such as the American Association for the Advancement of Science (AAAS; Rainie et al., 2015) and researchers in the United Kingdom (Hamlyn et al., 2015) suggest that large majorities (at least four out of every five) had personally found themselves involved in some form of public engagement in the previous year.

A growing body of research, however, suggests that scientists think about public communication in relatively simplistic ways. This work has clarified scientists’ attitudes toward engagement and includes a range of qualitative and quantitative research (e.g., Besley, 2015; Davies, 2008; Dunwoody et al., 2009; Jensen et al., 2008; Nisbet & Markowitz, 2015; Peters et al., 2008a; Poliakoff & Webb, 2007). A summary of initial work in this area concluded that scientists have relatively negative opinions about the public and journalists, but see a great need to have a role in public decision making (Besley & Nisbet, 2013). And, as will be discussed in further detail below, recent research shows that scientists often approach their engagement efforts with a narrow set of skills and considerations (e.g., Besley et al., 2018; Dudo & Besley, 2016; Grand et al., 2015; Peters, 2013; Yuan et al., 2018).

Collectively, these data illustrate a critical dilemma: Scientists seem increasingly motivated to partake in public engagement, but these intentions may not be coupled with the expertise needed to maximize effectiveness. In short, scientists need help becoming better communicators. This is not to imply that scientists are “bad” at communicating (e.g., Radford, 2011). It instead acknowledges that achieving effective communication stems from sophisticated communication practices that are rarely innate to anyone, including STEM professionals.

Science Communication Training

Professional science communication training programs are a key conduit through which STEM professionals can strengthen their communication capacity and impact (Besley & Tanner, 2011; Salas et al., 2012). They are positioned to convey insights from communication research to scientists (Miller et al., 2009; Trench & Miller, 2012) and these programs have been growing rapidly in recent years (Baram-Tsabari & Lewenstein, 2017b; Baron, 2016; Brown et al., 2004; Gold, 2001; Heath et al., 2014; Neeley & Goldman, 2013; Peters et al., 2008b; Smith et al., 2013; Stylinski et al., 2018).

Exemplars of science communication training programs include efforts of the Alan Alda Center for Communicating Science (http://www.centerforcommunicatingscience.org/), the Center for Public Engagement with Science & Technology at the American Association for the Advancement of Science (https://www.aaas.org/programs/communicating-science), COMPASS (https://www.compassscicomm.org/) and the Portal to the Public Network (http://popnet.instituteforlearninginnovation.org/). The structure and curricula of these programs vary and include training, and stand-alone workshops with the goal of preparing scientists for interfacing directly with the public or other stakeholders, working with media professionals, or using social media. Most of the trainers interviewed in this study provide courses of a half-day up to a full week or more. In some cases, these trainings might be split over multiple sessions. Consistent with our sense of the field, the trainers interviewed primarily work for organizations that include training as part of their mission (e.g., scientific societies or training-focused organizations), although some were also individuals who regularly conduct training. The training provided was typically funded by a host organization (e.g., a group at a university might pay to bring a trainer to campus to work with its members), although trainees sometimes pay a fee to participate such as when training occurs on the margins of a meeting. Some training organizations receive grant funding to subsidize the cost of providing training. Similarly, trainees sometimes choose to participate and in other cases trainees are asked to participate by their employer as part of professional development.

Existing research on science communication training has mostly examined the character of programs and their effects (e.g., Luisi et al., 2019; Miller et al., 2009; Rakedzon & Baram-Tsabari, 2017; Rodgers et al., 2018). Other research has developed tools to assist training efforts (e.g., Baram-Tsabari & Lewenstein, 2013; Rakedzon et al., 2017; Rowland et al., 2018), while another line of work has explored what science communication researchers think scientists should learn in training (Besley & Tanner, 2011; Yuan et al., 2018). A few studies have focused on trainers’ views of programs and trainees in Europe (Miller et al., 2009; Trench & Miller, 2012), North America (Besley et al., 2016; Cameron et al., 2013), and internationally (Silva & Bultitude, 2009).

In the current study, our conceptualization of science communication training is any activity wherein someone from the scientific community—often a natural scientist—receives structured guidance on how to communicate effectively. “Public engagement” training is considered communication training in the current context given that much of current science communication scholarship emphasizes the value of creating opportunities for dialogue between scientists and their broader communities (Nisbet & Scheufele, 2009) or helping audiences think about scientific issues more deeply than might otherwise be the case (Besley et al., 2019). The current study does not include training taken as course work in a degree program or informal advice given from one colleague to another. We next describe the research questions investigated in the present study.

Trainees

The scientific enterprise has long been characterized by structural inequities and connections to racial injustice (Saini, 2019). This inequality remains evident in the complexion of the STEM workforce, which remains largely composed of White, male scientists (National Science Board, 2020). Research suggests that the practice of science communication exhibits similar structural inequalities. For example, a recent study by Dawson (2018) demonstrates how “science communication practices construct a narrow public that reflects the shape, values, and practices of dominant groups at the expense of the marginalised” (p. 784). This core finding, which stemmed from Dawson’s application of numerous theoretical frameworks including social reproduction (Bourdieu, 1984), social justice (Young, 1990), and intersectionality (Crenshaw, 1990), suggests that the current science communication ecosystem is likely amplifying the long-standing inequities evident in STEM.

Related to the context of the current study, Dawson’s work—and the theories on which it is based—raises at least one urgent question: To what extent are training programs exhibiting and catalyzing diversity and inclusion within science communication? For example, what individual identities (e.g., cultural/racial backgrounds, gender, career stages, etc.) are represented among trainers and trainees? And in what ways are training programs preparing scientist communicators to engage with stakeholders who are not like them? These may seem like simple questions, but examining them is essential to understanding the training landscape’s current and potential contribution to inclusion within science communication. Moreover, these questions have not yet been directly examined.

The closest related research has explored the different types of identities that are drawn to public engagement. Regarding gender, some research suggests that male scientists are more likely to participate in public engagement (Besley, 2015; Kreimer et al., 2011; von Roten, 2010), while other research suggests that outreach spaces may be more commonly occupied by scientists who are female or minorities (Ecklund et al., 2012; Jensen, 2011; Thiry et al., 2007). Other evidence suggests that older scientists are more likely to partake in public engagement than younger scientists (Bentley & Kyvik, 2011; Kreimer et al., 2011; The Royal Society, 2006; Torres-Albero et al., 2011), as are scientists at more advanced career stages (Dudo, 2013; Dunwoody et al., 2009; Jensen, 2011; von Roten, 2010). Studies also suggest that public engagement may be more highly practiced by researchers in the social and environmental sciences (Bentley & Kyvik, 2011; Entradas & Bauer, 2017; Jensen, 2011; Rainie et al., 2015; Torres-Albero et al., 2011) and biology or medicine (Besley et al., 2013; Marcinkowski et al., 2014). Taken together, these considerations lead to our first research question:

Curricula and Strategy

The scientific community’s traditional approach to public communication has been narrow, built on a top-down assumption that providing more scientifically accurate information inspires greater support for science and, in turn, translates into broad “pro-science” outcomes (Bauer et al., 2007; Logan, 2001). The shortcomings of this “knowledge deficit model” approach have been well documented (Allum et al., 2008; Kahan et al., 2009) and have implied the need for scientists to consider communication objectives beyond the transmission of factual knowledge about science (Besley et al., 2018).

Training programs are positioned to play key roles in helping scientists to diversify their communication objectives and develop skills. Are they? The limited scholarship done on this topic suggests ample room for improvement. One recent interview project found that North American–based science communication trainers continue to emphasize knowledge building as a core component of their curricula (Besley et al., 2016). This same project found that training rarely seeks to help scientists identify engagement objectives or goals (Besley et al., 2016) or foster scientist-public dialogues (Yuan et al., 2017).

Considering these findings, we sought to provide a more granular examination of the communication objectives emphasized by training programs, especially the extent to which curricula reflect key theoretical insights from strategic communication. First, we aimed to identify how frequently programs help scientists engender the active listening, dialogic, and audience-centeredness highlighted in public relations’ excellence theory (Grunig et al., 1992; Grunig & Hunt, 1984). Second, we asked how often programs help scientists develop communications mind-sets that prioritize strategic thinking rather than discrete technical abilities, as articulated in Dozier and Broom’s (1995) framework of public relations roles. And, third, we explored how often program curricula address core tenets from public relations campaign strategy (Hon, 1998; Kendall, 1996) by helping scientists anchor their engagement efforts on clearly articulated behavioral goals and communication objectives before choosing specific communication tactics. We therefore posed the following two-part research question:

Beyond choices made about curricula, training programs can also differ in terms of their pedagogical frameworks. We are particularly interested in how often trainers structure their programs to link scientists’ classroom work to external engagement opportunities. This interest stems from the emphasis that the situated learning approach (Lave & Wenger, 1991) and situated cognition theory (Kirshner & Whitson, 1997) have placed on facilitating learning that occurs within real-world contexts and experiences. It also stems from the notion that practicing a behavior can help learners engage in the type of metacognition (e.g., sense-making, self-assessment, reflection on needed improvements) that can more efficiently boost their abilities (National Research Council, 2000).

Evaluation

Considering the financial investments organizations make to give their employees occupational training (Blume et al., 2010), it is not surprising that there is a robust science-of-training literature that has revealed effective ways to evaluate training programs (Salas et al., 2012). For example, Kirkpatrick’s (1967) four-level model, the CIPP (context, input, process, and product) model (Stufflebeam, 2003), and the multidimensional learning model (Kraiger et al., 1993) have all been implemented across fields to help refine training programs.

To what extent have the insights and models from this literature informed the assessment practices of science communication training programs? Do programs have defined learning objectives that inform the creation of measurable assessment categories? Although new methods to evaluate science communication training and engagement efforts are being created (e.g., Bogue et al., 2013; Peterman et al., 2017; Robertson Evia et al., 2017; Rodgers et al., 2020; Sevian & Gonsalves, 2008), little is known about how science communication training programs conduct evaluation (Baram-Tsabari & Lewenstein, 2017a; Barel-Ben David & Baram-Tsabari, 2020). Thus, our third research question asks:

Sampling

This sampling process consisted of a four-step approach: (1) an initial list of trainers created by the training organization COMPASS (Neeley & Goldman, 2013), (2) dialogue with the Kavli Foundation’s Public Engagement Program and the AAAS’s Center for Public Engagement with Science & Technology (both of whom help coordinate public engagement efforts across the broader professional scientific community), (3) snowball sampling (i.e., respondents were asked to identify other potential interview subjects with experience in science communication training; Noy, 2008), and (4) targeted internet searching and monitoring. Although identifying an exhaustive list of trainers and training organizations is impossible, we are confident that our sampling process yielded a comprehensive database of North American science communication trainers.

We e-mailed the individuals (hereafter, trainers) to describe the project and invite them to participate in an interview. In situations where a single organization had multiple people conducting training, only one or two trainers were contacted. Ultimately, we contacted 34 trainers. Of these, 33 responded, and 32 people completed the interviews. One individual declined to be interviewed because of having limited availability; another did not respond to repeated invitations.

The research team determined that they were not hearing additional substantial information after 32 interviews and that it was not useful to contact trainers beyond our initial sampling pool. This decision aligns with standards in qualitative research, whereby investigators determine their sample size inductively based on their collective judgment about when no new information is emerging from the interviews (i.e., data saturation; for more information, see Guest et al., 2011).

There were slightly more female (n = 16) than male (n = 14) trainers and the average age of the interviewed trainers was 43 years (SD = 12). Most interviewees reported being White (n = 29). The majority of interviewees had completed a graduate school degree (PhD = 17; master’s = 11), more than 75% indicated that they spend “all,” “most,” or a “good portion” of their time planning, conducting, or evaluating science communication training. Half of the interviewees (n = 16) reported the biological sciences as being their primary discipline, while the remainder were split across several other scientific disciplines (e.g., 8 interviewees identified themselves as social scientists, 7 as geoscientists, 5 as chemists, 5 as physicists/astronomers, 3 as engineers, and 2 as computer scientists/mathematicians).

Interview Procedures and Coding

Prior to interviews, trainers were asked to consent and provide background information through an online Qualtrics survey. The research team developed a semistructured interview protocol designed to unearth qualitative insights about numerous issues. The interview protocol was revised and developed based on questions in a prior interview study with trainers (Besley et al., 2016). Feedback about the interview protocol was obtained from an expert advisory board associated with the project. The advisory board included a handful of senior academic and professional leaders in the North American science communication landscape. Key feedback from this board included a desire for interviews to investigate the nature of program evaluation and participation, especially in terms of trainee inclusion and accessibility. Questions included (1) trainers’ organization’s origin/evolution story, (2) what they seek to help scientists achieve (e.g., civic impact, individual-level behavior change, etc.), (3) whether-or-not they help scientists identify/seek particular communication goals (e.g., helping policy makers) and objectives (e.g., demonstrating shared values), (4) what makes their approach unique/different, (5) their organization’s experience with diversity (i.e., staff composition and scientists trained), (6) projections for their organization’s priorities, (7) their current sense of “best practices,” and (8) collaborations with other trainers.

The first and second authors each conducted approximately half of the interviews. Telephone interviews were used because the trainers are located across North America. All the interviews were recorded, transcribed immediately by a professional transcription service, and shared with the research team. The interviewers also shared postinterview notes and discussed the ongoing interviews. Interview lengths ranged from 39 to 110 minutes, and all interviews were conducted between July 2017 and September 2017.

Thematic analysis using NVivo software was used to extract key themes from the interview transcripts (Braun et al., 2019; Guest et al., 2011). Because this research is anchored on several theoretical perspectives, we used theory to guide the coding and analytical process. The research team first identified a series of codes from the interview protocols, such as “know your audience,” “facilitate dialogue,” “goal-setting,” and we then merged the codes into several themes. Table 1 lists all the codes and associated themes. The establishment of these codes and themes was performed by the entire research team on a subset of interview protocols (approximately 20%); the first author then performed the remainder of transcript coding in ongoing consultation with the coauthors. As qualitative research with a small sample, the focus was not on performing traditional quantitative intercoder reliability using trained coders (Krippendorff, 2004), but on coming to mutual agreement among experts through discussion about the potential meaning and appropriate categorization of text (Braun & Clarke, 2013; Yardley, 2008). In order to help validate the legitimacy of analytic interpretations, the research team shared penultimate results with research participants (Yardley, 2000), in this case more than 10 trainers. This process yielded no feedback that disputed the veracity of the study findings. The results presented below are organized based on the research questions.

OPEN IN VIEWER

Table 1.

Summary of Interview Themes.

Theme	Description of the theme
Trainees
Trainee characteristics	Responses about demographics and psychographics of trainees
Inclusivity	Responses addressing issues of diversity, equity, and inclusion in science communication
Curricula—Communication skills
Storytelling	Responses addressing the value of using narrative formats in science communication
Brief and clear	Responses focused on the need for clear, concise communication (e.g., removing jargon)
Elevator pitch	Responses about the development and/or value of elevator pitches for science communication
Design & presenting	Responses focused on discrete design aspects of science communication (e.g., presentation slides)
Platform-specific	Responses focused on science communication via specific platforms (e.g., social media, journalists)
Curricula—Communication strategy
Know your audience	Responses focused on the value of understanding an audience, including its prior knowledge and/or worldviews
Goal-setting	Responses regarding the identification and/or development of communication goals
Self-efficacy and confidence	Responses addressing the enhancement of trainee communication self-efficacy
Increase knowledge	Responses addressing science education and increasing public knowledge/awareness about science or specific facts/issues
Foster excitement	Responses focused on enhancing interest or excitement
Build trust & warmth	Responses focused on boosting trust, including respectful/warm behavior, and competence
Listen	Responses focused on listening to others
Facilitate dialogue	Responses about facilitating bidirectional communication with others
Societal impacts	Responses focused on communication seeking to enhance science’s effects on policy and decision making
Curricula—Enact communication behavior
Facilitate practice	Responses describing training that facilitates interactions and practice among trainees
Facilitate experiences	Responses about training that links to real-world communication experiences
Evaluation
Assessment of scientists	Responses addressing the assessment of training programs’ effects on trainees
Assessment of scientists’ audiences	Responses addressing the assessment of training programs’ effects on trainees’ audiences

Who Receives Science Communication Training? (RQ1)

Training programs appear to be serving a broad spectrum of STEM researchers. Most programs work with researchers from varied fields and career stages. The apparent heterogeneity, however, does not extend much further. Trainers commonly lamented that the scientists they train tend already to be inclined toward communication: “The things we do unfortunately are a little bit self-selecting because . . . people who are coming to us are already interested in what we’re doing” (Interview 29).

There seems to be a widespread desire among trainers to connect with respected researchers who are not yet involved with public communication. Likewise, a subset of interviewees emphasized a need for scientific institutions to take more proactive roles connecting trainers with researchers whose voices have been traditionally underrepresented:

[Scientific institutions] have to go seek out people who are good communicators or who can communicate with audiences that you want to tap, who have not raised their hand. And that is also huge for overcoming gender bias and racial bias, and a lot of other forms of discrimination that we wind up perpetuating when we just let people self-select. (Interview 16)

Trainers frequently noted that the scientists they train lack cultural and ethnic diversity. Further, most training programs—with a handful of notable exceptions—do not seem designed to maximize the trainee diversity among or enhance trainees’ abilities to engage with diverse audiences. For example: “In some ways I guess [increasing diversity within the science training communication ecosystem] would be out of my hands, but that’s probably sort of feckless way to think about it. . . . I mean underrepresented groups are underrepresented” (Interview 21).

Many trainers, however, identified diversity as a priority for their program (e.g., Interviews 3, 5, 17). Some interviewees specifically discussed a desire to go beyond their current efforts, which include tactics such as tuition scholarships for underrepresented scientists and ensuring that underrepresented individuals are invited to participate on panels or deliver keynote lectures. There seems to be a specific focus emerging that centers on building community:

We really worked to cultivate, especially with our local workshops, sort of a local community in science communication and I think that’s really valuable to know who in your community knows what, is interested in what, and to come together with people who are geographically in the same place. (Interview 28)

Trainers often said their trainees tend to skew female. They also often noted challenges that uniquely affect female scientists’ communication efforts. Among them, trainers suggested female scientists are more likely to “get backlash and criticism when they engage” (Interview 2), suffer “uncomfortable power dynamics” (Interview 3), and have their credibility unfairly undermined by “being nice or too friendly” (Interview 10).

Trainers also observed an age difference. Junior scientists, they note, seem different than their more senior colleagues with younger scientists seeming more interested and willing to engage (e.g., Interviews 8, 18): “[J]unior scientists, they are all on board. I mean, from my perspective even the ones that are doing theoretical physics and talking about black holes, they are much more on board than senior scientists” (Interview 17).

Some trainers characterized this difference as emblematic of a broader cultural acceptance of science communication among STEM researchers (Interview 16), but trainers also commonly stressed pragmatic reasons for this apparent generational difference, including that junior scientists increasingly “understand that their institutions aren’t necessarily going to be there for them” (Interview 16) and “recognize that a significant part of their success hinges on how well they communicate” (Interview 14).

What Are Scientists Being Trained to Do? (RQ2a)

We asked trainers to describe their programs’ core learning objectives and one clear pattern emerged: Trainers want to help scientists become better storytellers. A handful of interviewees argued that the storytelling’s importance derives from the fundamental role narrative plays in human sense-making (e.g., Interviews 19, 32). However, some others expressed uncertainty about why the training community is so focused on storytelling (e.g., Interview 14).

In addition to storytelling, trainers seem focused on helping scientists communicate clearly; more than half of the interviewees discussed components of their programs designed to help scientists remove jargon. Curricula also frequently address scientists’ interactions with media professionals (e.g., Interviews 2, 12), with some trainers emphasizing the need to help journalists and scientists better understand one another. For example, one trainer mentioned the importance of helping scientists and journalists become more familiar with one another’s “customs” (Interview 26) or clearing up “misconceptions” (Interview 27). Trainers also sometimes discussed improving scientists’ interactions with policy makers and government officials (e.g., Interviews 2, 4).

Our interviews also revealed that training curricula are highly focused on helping scientists with the delivery of information. This includes a focus on making scientists better presenters (e.g., Interviews 15, 19) through discrete skills like creating visuals, mastering nonverbal communication, and gaining comfort on camera or radio. Many trainers described efforts to bolster scientists’ delivery through exercises that help scientists develop elevator pitches for their research (e.g., Interviews 9, 13).

Trainers also overwhelmingly expressed a desire for participants to leave training feeling more skilled as communicators (e.g., Interviews 31, 32). Trainers noted that a scientist who participated in their training “would be much more confident in her ability to communicate her science to any type of audience” (Interview 1), would gain “experience, confidence, and motivation in communicating their research” (Interview 24), and would be “better informed about how as a scientist they can get involved” (Interview 27). Similarly, some trainers discussed a desire to shape researchers into change agents and the “the voices of science” (Interview 24).

We also asked trainers about the degree to which they focus training on specific communication channels (e.g., social media, traditional media, face-to-face, etc.). Overall, it seems curricula often cut across different communication platforms, or, as one trainer put it, that curricula are “platform agnostic” (Interview 4). However, nearly half of the trainers interviewed noted increasing focus on helping scientists communicate through social media because of the “interest and relevance to a lot of the scientists that [they] work with” (Interview 2).

Another subset of trainers described their programs as seeking to help scientists feel comfortable with their (potential) decision to do public engagement (e.g., Interviews 4, 12, 28). Citing a perceived scientific culture that negatively judges public engagement, these trainers seek to help trainees feel that they are part of a growing community of scientific communicators (Interview 9) whose reputations and credibility are enhanced—not harmed—by taking outreach seriously (Interview 32).

How Much Strategy Is Present in the Training? (RQ2b)

We also sought to gauge the extent to which trainers address principles of strategic communication in their curricula. The most widely noted element of strategy raised by the trainers, when asked, was the need for scientists to “know their audience.” When asked to describe what this means, many trainers emphasized the centrality of this concept to the training process (e.g., “Who is your audience and why do they care? Everything should be tailored around that thought” [Interview 23]), and discussed teaching scientists that “there’s no such thing as a homogeneous audience” (Interview 19). Trainers also described the concrete ways they help their trainees learn how to analyze their audiences, which included examining their demographics, cultural backgrounds, belief systems, knowledge levels, information sources, connections to science, and more. A subset of trainers described discrete exercises they use to help convey these audience-profiling capabilities to scientists.

We also asked trainers to describe what they want their training programs to help scientists achieve. The majority noted wanting their efforts to help scientists have broad, positive impacts on society. These impacts, however, were often described somewhat generically, with interviewees noting, for example, that their trainees can make specific scientific disciplines “more relevant for people” (Interview 25), ensure “that science is out there” (Interview 29), and “amplify the impact that scientists are having in society” (Interview 18).

We also sought to learn how trainers teach scientists about behavioral goals and communication objectives. When trainers address goal-setting in their curricula, which does not seem to be done by the majority of trainers, they tend to do so by asking scientists to identify the goal that most interests them. Interviewees expressed that they give scientists an opportunity to “focus on what they want to achieve” (Interview 1), by asking them “what they want to get out of the course” (Interview 9), and getting them to “think about their goals for engagement” (Interview 4). The common assumption among trainers seems to be that scientists can successfully identify their own personal goal (or goals) if they are given time to think about them, and so trainers rarely describe their programs as actively helping scientists identify or prioritize shared goals.

Our discussions with trainers also suggest that they focus on a limited range of communication objectives. First, most trainers—though not all—seek to weaken scientists’ widespread, tacit reliance on a communication deficit model (i.e., a knowledge objective). Trainers remarked, for example, that they do not want scientists to “think of communication as a deficit model” (Interview 8) and to learn that “just throwing information out there” (Interview 27) and “shouting from the rooftop” (Interview 10) do not lead to impact. To combat these tendencies among scientists, nearly half of the trainers we spoke with emphasized the need for scientist communicators to facilitate dialogue with nonscientists. They described having curricula designed to teach trainees that “two-way conversations” are necessary to enable mutually beneficial information exchange (Interview 10).

Whereas dialogue is more of an activity (or tactic) than an objective, programs that emphasize the importance of dialogue often also indicated that they prioritized helping scientists become better listeners (e.g., Interviews 7, 10). Although rarely explicit, such a focus could be understood as helping scientists achieve objectives such as “learning from engagement” or being perceived as willing to listening. Often, however, the emphasis placed on developing listening skills seems rather vague; many trainers had difficulty explaining how listening can help scientists. When interviewees elaborated on this topic, they described how active listening could help scientists identify audiences’ authentic feelings about topics (Interview 16), and help expand scientists’ thinking (Interview 8).

Only a small minority of trainers explicitly addressed communication objectives beyond increasing scientific knowledge or understanding. Among them, a handful described having curricula designed to help scientists build relationships or trust with their audiences. These trainers highlighted the importance of helping scientists learn tactics such as how to introduce themselves, share personal stories, convey warmth and friendliness, and suppress preachiness and condescension (e.g., Interviews 5, 8, 24). A small number linked their focus on teaching improved techniques with building trust (Interview 18). Other trainers explained their decision not to focus on this objective because “it’s not the sort of thing you can turn on and off,” while others noted how “[helping scientists learn how to build trust and relationships] always seems to go to the bottom of the list” (Interview 16).

To What Extent Does Training Help Enact Behavior? (RQ2c)

Overall, most trainers described having curricula that increasingly gives scientists a chance to practice communicating. This more interactive, hands-on approach has evolved from what many trainers described as programs that used to adhere to a more traditional, top-down classroom structure. Seldom, however, did trainers describe their programs as explicitly connecting their trainees’ practice (i.e., rehearsal) with real-world opportunities to engage (i.e., performance).

A subset of trainers stressed that their key desired impact is helping to catalyze scientists’ communication behavior. They noted, for example, that they seek to “drive an increase in public engagement in science communication by helping to give scientists the needed activation energy” (Interview 8). These trainers often noted how good outcomes hinge on giving trainees actual tools/activities to take with them:

And so when you’re able to give something concrete that they have worked on in a training that they can physically take out and bring with them back to their desks and their work, I think [their likelihood of future engagement] increases. (Interview 13)

This same subset of trainers emphasized the importance of providing ongoing support to program alumni. One interviewee described how their program “sends trainees a resource packet after the workshop, that includes information that is more tailored to their location, to their discipline, that can help them locate opportunities to do public engagement” (Interview 4). One noted requiring trainees to articulate communication commitments and buddy systems (Interview 32). Another interviewee detailed how their program gives their alumni “access to a bank of 20 technical assistance hours” to support their resulting communication efforts, noting how this commitment results in “dramatically different, better, deeper incorporation of the [taught communication] principles” (Interview 13). And another trainer echoed the importance of providing tangible posttraining support:

You can look at any field where you are trying to create behavior change, and you know there is lots of information about what it takes. So, I think as a community of practice we can probably learn more from that research on what does it take to reinforce behavior change. (Interview 2)

Only a small handful of interviewed trainers described their programs as being structured to parlay scientists’ communication practice to performance. One trainer described how they “provide their trainees with an audience and an event” (Interview 5), and another noted how completing their program requires at least one public engagement activity (Interview 8).

How Do Trainers Know If Their Training Works? (RQ3)

We asked trainers to discuss how they judge the impacts of their training efforts. Nearly all interviewees recounted anecdotes they regard as showing the efficacy of their programs. For example, one said:

We have just gotten some feedback also from the program leaders at University of [redacted], the folks that went through the program when they put in their dissertation, and you can clearly tell the difference between someone who had been through the program and someone who had not, which I think is a good sign. (Interview 24)

Many trainers expressed having a keen interest in receiving this type of informal feedback from trainees and any stakeholders involved with the scientists who participated in their programming. Although most trainers say they frequently receive positive informal feedback, they also almost unanimously noted a need for more and better assessment:

. . . evaluation is frequently kind of masquerading as a popularity contest. Like, “Oh, I liked doing that!” Or, “Oh, I had fun.” Or, “Oh, this is what I think I should do and therefore . . .” You can check those boxes and that means that it works. And that actually doesn’t mean that it worked. (Interview 13)

This concern highlights what many trainers acknowledge: “missing documentation” that demonstrates training works (Interview 3). One trainer remarked, “To a certain extent I think we all throw spaghetti against the wall to see what sticks” (Interview 25). Other trainers pragmatically acknowledged the increasing need to demonstrate return on investment to their funders and clients: “People are paying for us to come out there [and deliver training] and so they don’t want to waste their money” (Interview 4).

We also asked trainers doing program assessment to describe their procedures. Trainers commonly began with caveats noting, for example, that they are unsure about the rationale behind the creation of their assessment measurements (Interview 5) and see a need for researchers and evaluators to vet these procedures (Interview 4). The most pervasive practice is administering a posttraining survey (e.g., Interviews 6, 19) on trainees’ attitudes about the training experience (i.e., a customer satisfaction survey), not the acquired skills or future engagement plans.

A subset of trainers described doing two trainee assessments, which often meant collecting some sort of pre- and posttraining data (e.g., Interviews 10, 28). Only a small handful of trainers described using more extensive evaluation procedures. For example, one trainer said they “monitor how involved [in public engagement activities] their scientist trainees are over time (5 years)” (Interview 5). A few other trainers (e.g., Interview 9) said their program regularly partners with external experts who provide limited feedback to participating scientists.

Trainers who acknowledged the importance of improving program evaluation, also often pointed to barriers. Chiefly, trainers lamented a dearth of resources, noting “it’s hard to get money to pay for evaluation” (Interview 32). We observed a desire among these trainers for program funders to financially support more systematic, empirical assessment:

It’s really hard because the people who fund us, the first question they ask is evidence of impact and the last thing that they fund is evaluation. . . . That means that any kind of more serious, rigorous thinking that we do on evaluation comes out of our hides and our nights and weekends. (Interview 13)

This lack of perceived support has led some trainers to settle for anecdotal feedback or rely on groups such as participating scientists’ home institutions. Others expressed ideas about how evaluation could be improved. A handful of trainers, for example, want assessment that demonstrates the extent to which programs are empowering the diversification of STEM (Interview 18).

Other trainers noted a desire to receive feedback not only from their scientists but also from the audiences with whom those scientists engage. To wit, one trainer noted the primacy of that feedback, hoping to figure out how to systematically enable “audiences to come directly to us and tell us what’s working or what isn’t” (Interview 29). Another trainer highlighted a need to ensure that trainers equip scientist communicators to assess their own engagement efforts: “What types of tools we can provide to training participants so that when they go out and do public engagement they have the tools to evaluate how well they’re doing” (Interview 4).