The Roles of STEM Faculty Communities of Practice in Institutional and Departmental Reform in Higher Education

Abstract

This study examines how involvement in four cross-institutional STEM faculty communities of practice is associated with local departmental and institutional change for faculty members belonging to these communities. It is informed by the communities of practice and change in higher education literature and utilizes data gathered through a survey of community members (n = 2,503). The findings reveal engagement experiences and aspects of community design associated with departmental and institutional change related to STEM reform. These findings contribute to recommendations for designing future STEM reform initiatives.

Keywords

communities of practice STEM reform faculty issues higher education

The need for reform in undergraduate science, technology, engineering, and mathematics (STEM) education is well-documented in the discipline-based educational research (DBER) literature (Borrego & Henderson, 2014; Henderson, Beach, & Finkelstein, 2011; Singer, Nielsen, & Schweingruber, 2012). Poor instructional practices (especially the lack of student-centered and active learning strategies) adversely affect student learning and retention in STEM majors for majority and especially for underrepresented minority students (Seymour & Hewitt, 1997). The aggregate consequences of these instructional trends in STEM have led to national calls for reform (e.g., American Association for Advancement of Science [AAAS], 2011; President’s Council of Advisors on Science and Technology [PCAST], 2012; Singer et al., 2012) to ensure the future competitiveness of the United States in the global economy. Although research has identified instructional practices that contribute to increased student learning and improved outcomes in STEM education, the adoption of these practices has not widely spread to create large-scale change across the academy (Austin, 2011; Dancy & Henderson, 2010; Fairweather, 2009). Austin (2011) suggests that change efforts in STEM require change agents to take a systems approach to reform. A systems approach acknowledges the important role played by individuals in change efforts (in this case, STEM faculty) while also considering the complex contexts in which they find themselves, including their departments, colleges, institutions, and disciplines, and how these work together to both promote and inhibit change (Austin, 2011; Kezar, 2014).

Most STEM reform efforts have focused on improving the pedagogical practices of individual faculty members (Henderson et al., 2011). Although this focus has enabled isolated individuals to improve their teaching, it has been largely ineffective in fostering broad, scaled-up change in departments and institutions (Austin, 2011; Corbo, Reinholz, Dancy, Deetz, & Finkelstein, 2015; Fairweather, 2009). In particular, reform strategies aimed at individual faculty members often do not account for the broader structures that inhibit the spread of reform, including institutional reward structures that value research productivity over improving teaching, disciplinary cultures that place more emphasis on research, and lack of institutional leadership fostering a culture that values teaching (Austin, 2011; Fairweather, 2009). Additionally, individual-oriented strategies often do not lead to nor do they support the development of learning communities and other networks, which can be crucial in sustaining STEM reforms (Fairweather, 2009; Henderson et al., 2011; Steinert et al., 2006). Sunal and colleagues (2001) suggest that targeting faculty for reform can generate momentum for change that can eventually lead to reforms for departments and institutions, but these broader changes are unlikely to occur without a complementary emphasis on increasing buy-in and support from administration for change as well as developing faculty members’ skills and abilities to facilitate them moving into leadership roles (Kezar & Lester, 2011; Seymour, 2002).

A reform strategy that has gained prominence in recent years is to develop faculty communities, both local and cross-institutional. Specifically, funding agencies have supported cross-institutional communities of practice (CoPs) as ways to spread STEM reforms and innovations. These communities are designed to engage a national community of faculty in learning effective pedagogy, developing leadership skills, and expanding their personal and professional networks for support in STEM reform efforts. These STEM reform CoPs engage faculty members in departments and institutions from across the United States, each with their own norms and cultures.

The extant literature on CoPs focuses predominantly on organizationally situated CoPs, meaning they are coordinated and supported by an umbrella organization or institution with membership in the CoP overlapping with the organizations; in other words, most research on CoPs focuses on communities within a single organization, comprised solely of individuals belonging to that organization, and that tend to be much smaller than the larger cross-institutional CoPs being funded today. This literature highlights both individual and organizational benefits that come from involvement in these organizationally situated CoPs (Wenger, McDermott, & Snyder, 2002); specifically relating to educational reform, CoPs show promise for changing faculty practices and, by extension, influencing broader institutional changes (Hammersmith, 2015). However, the extent to which cross-institution CoPs that engage individuals from multiple organizations, such as those in STEM reform, can effect change in those individuals’ home organizations remains unknown.

As professional development opportunities, cross-institutional STEM reform CoPs can provide faculty with the necessary skills to change their practice (Austin, 2011; Gehrke & Kezar, 2016). However, if CoPs are to contribute to scaling reform in STEM, they must not only develop individual faculty but also contribute to broader organizational outcomes. Despite the faith placed in these CoPs by funding agencies to effect broader change, research by-and-large has not focused on the community aspects of change efforts. While there is a growing body of research on scaling up efforts for institutional change, many of the reports from funded studies through programs such as the NSF’s Widening Implementation and Dissemination of Evidence Based Research (WIDER) and Improving Undergraduate STEM Education (IUSE) focus on various strategies to facilitate change that were identified by participants engaging in communities rather than on how to utilize communities to effect change, which is generally not the focus of this body of research. This is the focus of this exploratory study, which identifies the ways in which individual faculty involvement in four CoPs engaged in STEM reform efforts is associated with perceived benefits for their home departments and institutions. STEM CoPs that exist at the national and regional level, which are the focus of this study, involve thousands of faculty. They host events; have resources such as curricular modules, journals, and newsletters; and provide ongoing networking opportunities for the faculty who participate. Some aspects of the communities, such as newsletters and regional network meetings, can be free, but they often charge for publications, curricular resources, and major events. The four communities in this study involve between 2,000 and 7,000 faculty, each from many institutions, which sets them apart from traditional organizationally situated CoPs, which tend to be smaller.

By focusing on organizationally related outcomes such as department and institutional change, we identify the ways in which individual faculty involvement in these communities is related to localized efforts at STEM reform and can thus be leveraged to scale up reform efforts. Beyond faculty involvement and engagement, we also examine the design of these communities to better understand what future CoPs should consider as they seek to influence STEM reform at the organizational level. Design in the CoP literature refers to the ways that communities are structured to engage participants in order to help the community accomplish its goals; aspects of design include things such as events, leaders, communication strategies, and resources. Therefore, the purpose of this article is to examine how individual faculty engagement in these communities and aspects of how they are designed is associated with departmental and institutional changes related to STEM reform for faculty who belong to them. This inquiry is guided by the following research question:

How are faculty engagement in and perceptions of cross-institutional CoPs’ design characteristics associated with local institutional and departmental change related to STEM reform after controlling for institutional, professional, and personal characteristics?

This study focuses on involvement of individual faculty in cross-institutional CoPs and how this involvement can have benefits to their local departments and institutions. Therefore, it is primarily informed by literature pertaining to (a) CoP design and outcomes and (b) change in higher education. We begin by reviewing the literature on outcomes associated with CoP involvement and considerations for designing CoPs, followed by the literature pertaining to change in higher education, to better understand what contributes to effective change.

CoPs

A CoP is a group of people who share a concern or a passion for something they do and learn how to do it as they interact regularly (Allee, 2000; Lave, 1988; Wenger, 1998, 2007). CoPs are typically located within a single organization and involve people who work in the same or related areas. CoPs may also cross organizational boundaries and be more loosely connected, resembling what is referred to as a distributed CoP (Wenger et al., 2002). While the communities in our study resemble distributed communities, they are also different in that most distributed CoPs are coordinated and supported by an umbrella organization, with members of the community predominantly belonging to that organization (Wenger et al., 2002).

Benefits of CoPs

The benefits of CoPs are frequently documented in the literature (Fontaine & Millen, 2004). One of the foremost purposes of CoPs is to enhance organizational capacity (Brown & Duguid, 2001), and scholars highlight the benefits for departments and organizations due to individuals’ involvement in CoPs. Millen, Fontaine, and Muller (2002) suggest that the most tangible benefits of CoPs are business-related outcomes. These researchers have provided the bulk of the research on CoPs that link involvement to organizational outcomes in the management and business literature (Fontaine & Millen, 2004; Millen & Fontaine, 2003; Millen et al., 2002). The most prevalent organizational benefits of CoP involvement, as cited by individuals in their studies, are operational efficiency, cost savings, and increased quality and speed of service. While these benefits were identified through interviews with CoP members, other research identifies quantitative relationships between exchanging information and networking within a community and organizational benefits in the forms of increased knowledge production and business performance (Zboralski, Salomo, & Gemuenden, 2006). Outside of the business literature, educational researchers have highlighted educationally related benefits for schools and institutions due to CoPs, including improved school culture for teaching (Supovitz, 2002), facilitating adoption of new practices school-wide (McDonald & Star, 2008; Price, 2005), and increased communication, collaboration, and knowledge-sharing (Sànchez-Cardona, Sànchez-Lugo, & Vèlez-Gonzàlez, 2012; van Wyk, 2005).

While research indicates that involvement in traditional CoPs influences organizational outcomes, the work cited above focuses on communities that are organizationally situated. The four communities we studied are not connected to a coordinating organization but rather exist independent of another organization and involve members from many different institutions. This is an important distinction because the literature that examines the organizational-level benefits of CoP involvement is from the perspective of organizationally situated CoPs, in which all of the members in the CoP belong to the same organization. Thus, organizational benefits accrue when members of an organization also belong to an internal CoP, which in turn leads to benefits trickling up to the organization. Our communities exhibit different clustering, in which the members of each community are distributed among numerous higher education institutions. Further, faculty members may be the only individuals at their institutions who belong to a given CoP, or they may be one of a few or many faculty who belong to the same CoP. So the goals and membership of the CoPs in our study are focused on benefits not solely for one institutional community but for many institutions. The way organizationally related outcomes play out among distributed CoPs has not been explored. Our study fills the gap by examining organizational outcomes that can accrue when faculty are involved in CoPs outside the confines of a single organization.

Designing CoPs

The ways in which CoPs are designed vary based on identified goals (Wenger et al., 2002). As a result, scholars do not point to specific blueprints for designing CoPs but rather highlight general principles to consider and that are common across most CoPs. Zboralski and colleagues (2006) suggest that, regardless of a specific topic area or sector, all CoPs are organized around two primary goals: transferring/exchanging knowledge (Lesser & Storck, 2001; McDermott, 1999; Stewart, 1996) and networking (Zboralski et al., 2006). In order to meet these goals, CoPs must consider how to design events, resources, and communications, as well as how to utilize members and leaders of the community, to meet these goals. Wenger and colleagues (2002) highlight seven general principles to consider when designing communities for these functions.

The first principle to consider when developing a CoP is to design the community to evolve naturally, which accounts for the dynamic nature of these communities in which interests, goals, and even members may change (Wenger et al., 2002). The second principle is to create opportunities for open dialog within and with outside perspectives, ensuring that goals are met by capitalizing on expertise of members while seeking that of individuals external to the community. The third principle is to welcome and allow different levels of participation, which allows for individuals with different levels of energy and time to contribute to the goals of the community. The fourth principle, develop both public and private community spaces, allows for both public exchange of ideas among all members but also opportunities for more personal, private interactions in order to fit the different goals and styles of members. The fifth principle is to focus on the value of the community, which provides members a chance to be able to reflect on the value that comes from being involved with the group. The sixth principle, combine familiarity and excitement, is meant to ensure that members can expect predictable events and opportunities while also being engaged by new opportunities that may arise. Finally, the principle of nurture a regular rhythm for the community ensures a continuous cycle of events and involvement opportunities so members can anticipate what is to come through their regular involvement.

While these principles can be a guide, CoP leaders must decide how to fulfill them as they design their communities (Wenger et al., 2002). Up until recently, the research literature did not connect aspects of CoP design to benefits, which could have provided guidance to CoP organizers; this is largely due to the overabundance of descriptive, qualitative research on CoPs as opposed to quantitative research to identify relationships among CoPs and their outcomes (Zboralski et al., 2006). More recently, researchers have begun examining quantitative relationship among CoP involvement, design, and benefits, but these are largely focused on individually focused outcomes (Chang, 2010; Chang, Chang, & Jacobs, 2009). Zboralski and colleagues (2006) produced one of the only quantitative studies examining associations with community involvement and organizational benefits. They found that participants who engaged in more knowledge-sharing practices in the community (i.e., both providing and consuming information in the community) and networked more frequently (i.e., interacted with and sought out community members) reported greater gains for their umbrella organization, which again controlled the CoPs in the study. Zboralski et al. (2006), Chang (2010), and Chang et al. (2009) suggest that the dearth of quantitative studies connecting CoP design to outcomes generally inhibits CoP designers from maximizing the potential of these communities. This study contributes to this gap specifically by identifying aspects of community involvement and design that are quantitatively associated to organizational outcomes in order to identify empirical evidence of their benefits for broader organizations. Additionally, the fact that the CoPs in our study differ from traditional CoPs in that they are national and cross-institutional necessitates exploration of design principles and their association to organizational outcomes, as these communities may behave differently from traditional CoPs. We now turn to the change in higher education literature to better understand how these CoPs may be effecting change and factors that influence change.

Change in Higher Education Institutions

Studies of change demonstrate that leadership at multiple levels facilitates change (Kezar, 2014). We briefly highlight the importance of balancing top-down and bottom-up strategies from the change and reform literature that contribute to our understanding of the type of reform pursued by the communities in our study.

The issue of scaling-up change dominates current thinking about the ways to enact change in higher education (Kezar, 2011). This is largely due to the fact that researchers have found that top-down approaches to reform, which rely on policies being put in place and enforced from administrators, have not been successful at generating wide-spread change (Dede, 2006). Top-down efforts at change tend to reflect an underlying of scientific management theory of change (Corbo et al., 2015; Kezar, 2014), suggesting that change will come through rational processes that either prescribe specific teaching methods developed by outside developers or enact policies and procedures to influence faculty behavior. Critiques of these approaches include that they limit the agency of individual faculty to engage in the reform process (Corbo et al., 2015) and often do not take into account the power of local departmental and disciplinary cultures in which faculty find themselves (Austin, 2011). Henderson and colleagues (2011) suggest that while top-down mandates do not work in isolation, it is still important for administrative leaders to provide support by removing or mitigating barriers to faculty adopting changes, such as altering incentives and reward structures.

Increasingly, the notion of scale-up has been advocated as a strategy to effect change by targeting faculty in reform efforts and providing them with ongoing support to change their practice and spread change from the bottom up (Kezar, 2011; Seymour, 2002). This approach to change reflects more of a social cognition logic theory of change, in which the goal is to enact change by influencing and altering individuals’ thought processes toward practice (Corbo et al., 2015; Kezar, 2014). The underlying logic of this approach is that if you can impact the practices of large numbers of faculty, then change will eventually diffuse through the academy and spread to departments and institutions. This goal is reflected in Henderson and colleagues’ (2011) characterization of change efforts focused on reflective teaching practice, in which the goal of reform is to engage faculty in active reflection to adopt new teaching strategies. In this approach to bottom-up change, scale-up occurs through empowering individual faculty in the hopes of influencing some of the broader systems, cultures, and context in which they find themselves. This is achieved through ongoing support, engagement, and connection to other faculty to spread the change throughout institutions (Henderson et al., 2011; Kezar, 2014). Barriers to this type of reform include the lack of incentives and support in institutional reward structures and policies for faculty to engage in this behavioral change (Henderson et al., 2011), again indicating that these efforts in isolation are often not successful.

While there is greater need for bottom-up buy-in to support change, studies identify how bottom-up approaches to change alone are unlikely to create change and can be very fragile (Kezar & Lester, 2011). Instead, scaled changes require bottom-up buy-in and support for institutional changes in policies and practices that might block or can enable the new pedagogy/curriculum. Fairweather (2009) and Austin (2011) identify how institutional leaders could support changes among faculty by altering incentive structures to reward teaching, making professional development available on campus, fostering a culture within institutions and departments that value teaching, and altering policies for promotion and tenure. While top-down mandates do not support scaled changes, changes in policies and practices, which are often led and supported by administrators, can complement bottom-up efforts. The communities in our study approach change from this perspective, focusing their efforts on faculty members with the hopes that practice and leadership will diffuse into institutions through development in the communities; this study is designed to identify factors that contribute to this spread to larger institutional and departmental changes.

Methods

The paper is part of an overall study that employed an exploratory mixed-methods approach. Studies of CoPs utilize both quantitative and qualitative methods to identify trends and examine underlying mechanisms within the communities (Fontaine & Millen, 2004). In line with exploratory mixed-methods studies (Creswell & Plane Clark, 2011), we began with qualitative data collection through participant interviews (n = 112) of leaders and staff from each of the four communities, observations of signature community events, and document analyses for key documents from the four reform communities that we studied. This allowed us to better understand the design aspects of and the nature of involvement in these reform communities as a means to explore this topic, which has not previously been studied. We utilized findings from this phase of data collection in order to inform the survey design for the second phase; this paper reports on findings from the second phase of data collection: the survey. We first describe our selection of the four CoPs in this study, followed by data collection, sample, and analysis strategies.

Sample Selection

Our interest was to focus on initiatives that had the following key features: (1) STEM education and reform as focus; (2) large in scale and leading to dissemination of best practices; (3) focused STEM reform within the context of postsecondary education; (4) long enough history so we could study not just formation but also outcomes and sustainability; and (5) ability to survey members of the communities. We selected the four communities for this study, then interviewed and surveyed their memberships.

Regarding the four communities, Project Kaleidoscope (PKAL) is a national community of STEM faculty that focuses on creating innovation among faculty so that they change their practices. The national PKAL community has nearly 7,000 members and has been in existence since 1989. The POGIL Project is a national professional development and curriculum reform effort whose mission is to connect and support educators from all disciplines interested in implementing, improving, and studying student-centered pedagogies and learning environments. It involves approximately 6,500 faculty across a range of disciplines and educational sectors and has been in existence since 2002. SENCER is a faculty development and STEM education reform initiative initiated in 2001 that has 2,500 members and approaches STEM education reform through complex, capacious, contemporary, and contested civic challenges and an interdisciplinary approach. The BioQUEST Curriculum Consortium has a 25-year history of supporting undergraduate biology education reform and has 2,000 members.

Data Collection

Data for this study were collected through the Achieving Scale for STEM Reform Survey described below. In the spring of 2014, the survey invitation was sent to 17,868 e-mail addresses.¹ The survey was custom designed for each community’s particular structures (e.g., activities, communication vehicles) but followed a common survey design to allow for comparison across the four communities. For examples, each community has signature events they organize and resources they utilize, so each community’s survey listed the specific name of the signature events and resources. We then grouped responses pertaining to the same phenomenon across the surveys under common variable names, such as “signature events” and “resources.” This allowed us to utilize community-specific language on the survey to better understand broader constructs and community activities.

The 94-item survey instrument included questions eliciting information in the following areas: participants’ involvement in the community over time, such as events attended and activities engaged in; participants’ perceptions of benefits of community involvement for individuals, their departments, and their institutions; perceptions of the importance of community design elements on participants’ professional practice; and individual demographics and professional characteristics. Survey design was informed by the information gathered in the first phase of data collection as well as the literature pertaining to design and outcomes of networks and CoPs. This allowed us to identify the design aspects and involvement opportunities that characterized these types of communities.²

While we did not utilize any predesigned scales or items, we did rely on the literature we highlighted above coupled with the analyses of our interview data to inform survey development. We employed several techniques to ensure content validity of the survey and its items. We had two advisory boards that informed the study design and reviewed results: an external board comprised of national STEM experts, and an internal board comprised of members from each of the four communities. We presented survey drafts, as well as findings, to each board for input. The internal board could register whether the survey adequately accounted for the differing experiences and design facets of the communities. We also piloted the survey with several members of each organization and conducted follow-up interviews with them regarding the constructs in the study and the extent to which the surveys captured those constructs.

A total of 3,927 participants responded to the survey invitation, indicating a 22.0% initial response rate. This response rate is similar to the response rates of other surveys administered to national samples of STEM faculty (e.g., Hurtado, Eagan, Pryor, Whang, & Tran, 2012). The final sample for this study consists of 2,503 participants who completed the entire survey; these participants were distributed among 997 institutions (ranging from 1 to 28 observations per institution) and four CoPs (ranging from 235 to 1,102 observations per community). The Appendix lists the descriptive statistics for the variables in this study, including institutional, professional, and individual characteristics of the final sample.

Variables

Dependent variables

The dependent variables for this study are two outcome constructs representing departmental change and institutional change, all related to STEM reform. These constructs were calculated from among a total of 39 items assessing the extent to which participants perceived that involvement in the CoPs contributed to a host of possible individually focused, departmental-focused, and institutional-focused outcomes, utilizing exploratory principal axis factor analysis with Promax rotation.³ This approach to identifying organizational changes through participant perceptions is reinforced in the literature as a sound strategy for examining organizational outcomes (Zboralski et al., 2006). Items pertaining to organizational outcomes utilized the following question stem: “Please indicate the extent to which institutional involvement (e.g., several members of your campus being involved) and/or your own personal involvement with [community] has. …” Examples of institutionally focused and departmentally focused items, respectively, include indicating the extent to which involvement “led to developing communities of practice at your home institutions” and “led to curricular changes in your department that support student learning and success,” measured on a 5-point Likert-like scale, ranging from 1 = not at all to 5 = to a great extent. These items loaded on two different factors—departmental change (four items) and institutional change (eight items)—with the other items loading on three individual outcomes. We calculated factor/scale scores by averaging the individual items in each scale rather than summing the items (Furr, 2011). We opted to only calculate the average scores for these items so that we could view them in context and comparison to the other outcome variables given the specific measurement and scaling of the survey items and the fact that the different outcome scales in the study (see Note 4) contained a different number of items comprising each scale. Calculating the average of these items for the different scales allowed us to compare scores on a common scale. Factor loadings and reliability analyses (Cronbach α) for the two dependent variables in this study can be found in Table 1.

Table 1

Composite Scales and Descriptive Statistics for Outcomes in Study^a

	Factor Loading	Cronbach α	M (SD)
Departmental change		0.94	2.39 (1.04)
Led to changes in teaching practices in my department	0.96
Led to curricular changes in my department	0.93
Led to changes in educational values in my department	0.90
Informed departmental strategic planning	0.80
Institutional change		0.95	2.05 (0.98)
Led to curricular changes in other departments	0.96
Led to changes in teaching in other departments	0.95
Led to changes in educational values in other departments	0.91
Led to developing campus network for STEM reform	0.82
Led to developing communities of practice at institution	0.80
Led to emergence of new campus leaders for change	0.74
Informed campus strategic planning	0.68
Led to campus workshops and professional development	0.66

Questions for outcome variables asked participants to identify the extent to which participation in the network contributed to the items listed, measured on a 5-point scale: 1 = not at all; 3 = to some extent; 5 = to a great extent.

Other focal variables for this study are items pertaining to design characteristics of these CoPs as well as variables measuring the kinds of engagement by faculty in these CoPs. Participants were asked the following question about different design characteristics: “Please indicate the importance of the following in contributing to your institution’s work related to STEM reform.” These aspects of community design were chosen based on the literature on relevant design considerations for CoPs (reviewed above) and our interview analyses. Design items were measured on a 4-point Likert-type scale, ranging from 1 = not important to 4 = essential, and were explained by definitions if the term had the potential to be misunderstood. Examples of design characteristics in this study include opportunities to be mentored, community culture (described as shared values, beliefs, and practices of the community on the survey), and community resources (described as community-specific publications and libraries of tools and pedagogies—see full list in Appendix). Additional variables utilized in this study related to the extent of involvement among participants, including the years involved in the community, extent of involvement (from one-time to continuous), the frequency of attending events, engaging in community-related activities, and membership in subgroups in the community. These involvement measures emerged from our interviews in conjunction with design characteristics as being important for understanding how design and involvement were related to outcomes of these communities.

Finally, we included a cadre of control characteristics that are frequently cited as influencing involvement and outcomes in CoPs (Wenger et al., 2002) and influencing faculty development in general (Austin, 2011), including personal demographics (e.g., gender, race/ethnicity), professional characteristics (e.g., rank, years of service, academic discipline, motivations), and institutional characteristics (e.g., Carnegie type, control). We also controlled for the community to which participants belong (utilizing dummy variables in the models). However, we do not report the coefficients for these variables for two reasons: (1) We are interested in common experiences across these communities, not in comparing specific communities and the ways in which their community members report specific benefits, and (2) we aim to protect the anonymity of the differences between the communities. Models regressing our dependent variables only on community membership indicate very little variance in our dependent variables accounted for by community (between 3.7% and 6.6%). The relative explanatory power of these community variables did not change with the addition of our control variables in the regression models. However, once we added the design and engagement variables, the effect size of the community variables was reduced, suggesting that the ways in which the participants engage within these communities and exposure to different design elements of them are more important in explaining the variance in our dependent variables than simply the community to which one belonged. Descriptive statistics for all other variables (aside from the dependent variables) can be found in the Appendix.

Data Analysis

Prior to running our analyses, we explored the data to identify any patterns in missing data. The missing values analysis revealed a substantial amount of missing values, ranging from 1.8% to 34.7% of missing values for the continuous variables in the study and between 0.2% and 1.2% of missing values for categorical variables in the study. This is explained by the “Not Applicable” (N/A) option for continuous items on the survey. Given the variety of experience inherent in the sample of study, we included an N/A option as a response for all variables rating the importance of design principles and the extent of reaching outcomes because we did not want to force participants on the study to answer regarding a phenomenon they may or may not have experienced; we coded N/A responses as missing responses. Given that these missing responses were not missing at random, we utilized list-wise deletion for the regression models in the study. Comparing the full sample with the sample utilized in the regression models reveals relatively minor differences between the two. As examples, female faculty are slightly underrepresented in the regression models by 2%, faculty of color are slightly underrepresented by 2%, private institutions are slightly overrepresented by 3%, and the regression sample participants indicate slightly longer involvement by 0.4 years. In most variables, little to no differences are observed. After comparing descriptive statistics for the full sample and subset of complete data, we ultimately made the decision to utilize list-wise deletion because we were interested in gauging the relative importance of these items to participants who had experienced all of these items. We identified relevant engagement and design variables through our qualitative work, and given the exploratory nature of our study, we were most interested in knowing how they compared for participants who had experienced all of them and could make adequate comparisons. If a participant had not attended an annual event, for example, their experience would be drastically different from all of the participants who had. Therefore, we only utilized complete cases so that we would gain a sense of the relative importance of all of these design and engagement characteristics and their association with perceived change. Given the minor differences between the full and regression samples, we feel confident that no groups were excluded or skewed due to this analytic decision.

We utilized ordinary least squares (OLS) regression to examine the extent to which participants’ perceptions of CoP design characteristics and engagement are associated with the two dependent variables in our study. Our participants exhibited clustering by institution and reform community. However, we opted to utilize OLS regression rather than multilevel modeling because our sample contained a large proportion of singletons in institutions as well as institutions with only two participants (35.2% total), threatening the estimates and validity of utilizing multilevel modeling with these data (Rabe-Hesketh & Skrondal, 2012).

Prior to running our regression models, we calculated descriptive statistics and examined histograms for each continuous variable in the study to ensure approximately normal distributions. We also calculated multicollinearity statistics for all variables. Variance inflation factor (VIF) values were low (ranging from 1.10 to 3.85) and well within the acceptable range to indicate no issues with multicollinearity in the analyses (Meyers, Gams, & Guarino, 2006).

We ran regression models that included focal variables (perceptions about design and engagement behavior) and control variables (personal demographics, professional characteristics, and institutional characteristics) to answer our research question. All continuous variables (including the dependent variables) were standardized (i.e., grand-mean centered) prior to their inclusion in the models.

Limitations

This study exhibits several limitations. First, the survey instrument relies on self-report in identifying the extent to which outcomes are met as a result of involvement in these communities and the importance of design principles in contributing to effectiveness in STEM reform. However, given our desire to understand trends in outcomes from the large sample of community members, this is the best approach to examining benefits that come from community involvement. A related limitation is our approach to assessing the importance of design principles on outcomes. We utilize participants’ ratings on the importance of CoP design principles as a proxy for experiencing and valuing various design features. We made this decision because some aspects of design, such as the participation of STEM leaders or inclusive spaces, are difficult to measure direct experiences with. Therefore, our models are not causal models in which experiencing a design principle can be said to contribute directly to achieving outcomes; rather, the relationship of design principles in our models to the outcomes is interpreted as the relationship between participants’ valuing and experiencing a certain design principle and participants’ perceptions of change to their local institutions. The results for this study closely mirror results from other qualitative portions of our study (see Kezar, Gehrke, & Bernstein-Sierra, 2017), which highlight the same general design principles for individual engagement in these communities. Triangulation of data sources helps provide a greater validity and reliability to findings. Finally, as we described above, the regression sample differed slightly from our full sample. While we feel utilizing list-wise deletion is the more sound analytic approach to our study, the results could have varied slightly had we had more complete information regarding involvement across the sample.

Results

Table 1 lists descriptive statistics for the composite scales for departmental and institutional change. These results indicate that participants indicate greater departmental change resulting from their involvement with the CoPs than institutional change.⁴ Results for our regression models can be found in Table 2. After controlling for institutional, professional, and personal characteristics, we identified several involvement and CoP design variables that are significantly associated with departmental change. Three aspects of involvement and activities are positively and significantly associated with participants reporting departmental change due to their involvement in the CoPs—length of involvement, peer involvement, and presenting materials related to community involvement at professional/disciplinary meetings and conferences. In addition to these involvement variables, the salience of three community design variables is also positively and significantly associated with departmental changes—community culture, community leaders, and innovative and new ideas of the community. While these findings point to how faculty involvement in these CoPs is associated with departmental change, we also uncovered other findings regarding faculty characteristics. One’s discipline appears to play a large role in one’s perception of departmental change due to community involvement, as participants from the biological and life sciences reported more departmental changes compared to their colleagues in mathematical sciences, social sciences, and other disciplines, which were the largest effects for departmental change.⁵ Faculty motivation for engaging in the community is also associated with departmental change, as faculty who were motivated to improve teaching and learn strategies for leadership reported more departmental change after controlling for other variables. Finally, faculty perceptions of the climate for STEM reform are associated with changes at the department level, with faculty who perceive a more supportive climate indicating more departmental change. The variables in this model explain a substantial amount of variance in departmental change through these communities (R² = .43).

Table 2

OLS Regression Models for Departmental and Institutional Change

	Departmental Change		Institutional Change
	β	SE	β	SE
Institutional characteristics
Control: Public^a	−.10	.06	.07	.07
Carnegie: Masters institution^b	.12	.08	−.13	.08
Carnegie: Baccalaureate institution^b	.10	.08	−.06	.08
Carnegie: Associates institution^b	.17	.10	−.27**	.10
Personal and professional characteristics, perceptions, and motivations
Female	.04	.05	−.01	.05
Race: Person of color^c	−.04	.08	.10	.08
Rank: Associate professor^d	−.11	.07	−.09	.07
Rank: Assistant professor^d	−.19	.11	−.24*	.11
Rank: Non-tenure-track faculty^d	−.03	.10	.00	.10
Rank: Faculty at nontenure institution^d	−.16	.11	−.18	.11
Rank: No rank^d	.00	.16	−.15	.16
Department: Physical sciences^e	−.08	.07	.12	.07
Department: Mathematical sciences^e	−.28**	.08	.21*	.09
Department: Arts & humanities^e	−.18	.21	.47*	.19
Department: Social sciences^e	−.47***	.11	.32**	.11
Department: Medical sciences^e	−.17	.17	−.16	.17
Department: Other^e	−.34**	.12	.03	.11
Motive: Improve teaching	.15*	.07	.09	.07
Motive: Seeking support for change	.04	.06	.10	.06
Motive: Involve peers in change	.02	.06	.01	.06
Motive: Connect with colleagues	−.08	.06	−.06	.06
Motive: Career development	−.04	.06	.01	.06
Motive: Learn leadership strategies	.16*	.06	.19**	.06
Years teaching undergraduates	−.06	.03	−.03	.04
Perception of climate for STEM reform	.19***	.03	.12***	.03
Engagement and design variables
Years involved with community	.15***	.04	.13***	.04
Number of peers involved with community	.12***	.03	.19***	.03
Characterized extent of involvement	.05	.03	−.05	.04
Attend: National event	.06	.03	.02	.03
Attend: Regional/local event	.05	.03	.06	.03
Activity: Present at community event	−.01	.04	.07	.04
Activity: Author material	.04	.03	.01	.03
Activity: Present material at professional meeting	.11**	.03	.10**	.03
Activity: Publish about community work	−.04	.03	.01	.03
Group: Leadership/board member	−.01	.11	−.15	.10
Group: Project/grant-based	−.07	.07	−.17*	.07
Design: Different involvement opportunities	−.01	.04	.03	.04
Design: Innovative & new ideas	.09*	.04	.04	.04
Design: Community resources	.03	.03	.02	.03
Design: Safe, supportive space	.03	.04	.06	.04
Design: Inclusive practices	.01	.04	.01	.04
Design: Community leaders	.09*	.04	.12**	.04
Design: Connection with other faculty	−.04	.05	−.03	.05
Design: Opportunity for mentoring	−.01	.04	.04	.04
Design: Connection with STEM leaders	.07	.04	−.02	.05
Design: Community culture	.13**	.05	.13**	.05
Design: Community philosophy	.04	.05	.07	.05
R ²	.425		.488

Note. We controlled for the community to which participants belonged by including dummy variables in the model; however, we do not report these coefficients due to the sensitive nature of reporting benefits and comparison across the communities.

Reference group: Public.

Reference group: Doctoral institution.

Reference group: Race: White.

Reference group: Professor/professor emeritus.

Reference group: Biological sciences.

p < .05. **p < .01. ***p < .001.

We observe many of the same relationships of community involvement and design with institutional change as we do for departmental change. Length of involvement, greater peer involvement, and presenting materials related to community involvement at professional/disciplinary meetings and conferences are all positively associated with institutional change after controlling institutional and faculty characteristics. However, not all involvement experiences are positive, as being involved in a project or grant-based group is negatively associated with participants reporting institutional change. Of our design variables, participants perceiving community culture and community leaders as important for their work is positively associated with institutional change. This model departs more drastically from the departmental model when considering professional characteristics. Whereas belonging to biological and life sciences was positively associated with departmental change compared to some other disciplinary areas, participants from these disciplines report less institutional change compared to their colleagues in the mathematical sciences, arts and humanities, and social sciences. Regarding faculty motivation for engaging in these communities, faculty who desire to learn leadership strategies as part of their community involvement reported greater institutional changes from community involvement after controlling for other characteristics. Perceiving a positive climate for STEM reform is also positively associated with institutional change. Finally, we found differences in institutional change by rank and institutional type; assistant professors reported significantly less institutional change compared to full professors as a result of their involvement, while participants from associates institutions also reported less institutional change stemming from community involvement compared to doctoral institutions. These variables explain a substantial amount of variance in institutional change as well (R² = .49).

Discussion

CoPs are increasingly funded and utilized by reformers as strategies to scale up change in undergraduate STEM education, and the findings from our study reveal several factors that contribute to this scale-up by identifying aspects of faculty involvement that are associated with both departmental and institutional change. In doing so, we identify key involvement and design aspects of CoPs that can engage individual faculty to contribute to broader organizational outcomes and provide evidence of the potential for nonorganizationally situated CoPs to impact a broad range of institutions through involving individual faculty in reform. These communities bring faculty together from hundreds of institutions across the United States and through certain key involvement and design aspects are able to effect broader organizational changes in these institutions. Due to space limitations, we review the most important findings below pertaining to engagement and design followed by other factors associated with departmental and institutional change.

Engagement Contributes to Organizational Outcomes

First, our analyses reveal several key engagement experiences that are associated with departmental and institutional change resulting from involvement in these communities. Prolonged involvement and presenting about the community to outside audiences were both associated with our outcomes. These two variables suggest that a deeper engagement in these CoPs can have benefits for individuals hoping to influence broader organizational goals for STEM reform. In order to be able to present about a CoP’s pedagogical or other reform strategy to outside audiences, a faculty member must be able to engage long enough and deeply enough to feel comfortable with the material and be able to communicate the strategy’s nuances to audiences who are less familiar with the work. Engaging in these types of presentation also communicates an expertise that can translate to legitimacy and leadership on one’s campus, which can help these individuals foster more change through their efforts. Additionally, prolonged involvement associated with these changes also reinforces that change is complex and can take time (Austin, 2011; Henderson et al., 2011). CoPs and the faculty they serve cannot labor under the false impression that brief engagement in these CoPs can lead to larger changes, as our findings indicate that those who are involved for longer periods of time report greater change. While this finding could indicate that those who have been involved for longer periods of time with these communities have more history from which to observe and assess the extent of change resulting from their involvement, these findings at least suggest that these efforts must exist long enough to engage faculty over longer periods of time to allow them to see these changes take hold.

Our other key engagement finding points to change being a collective effort, as faculty who reported having more peers involved in the same community with them reported both greater departmental and institutional change. The communities in our study can serve as venues for multiple members of an institution to engage together in the activities of the community, an approach that was reinforced by qualitative data. We observed this as a successful strategy in particular when CoPs encouraged members to engage in their communities as teams; team members would meet in these spaces and engage in conversations pertaining to bringing strategy back to their home campus and how they might work together when they returned. But collective action does not just come through CoPs intentionally focusing on institutional teams. Interviews with active community members revealed that simply having others at their home institutions who had experienced certain workshops or development through the same community acted to give them a sense of shared language and trust even when they had not participated together. As a result, multiple members of these communities in a single institution can contribute to organizational learning within their institution, which is often cited as a benefit of CoP involvement (Allee, 2000).⁶

Key Community Design Characteristics for Departmental and Institutional Change

We also identified several CoP design characteristics that are positively associated with departmental and institutional change resulting from involvement in these communities. Specifically, participants who perceive key community leaders (several key leaders identified by each community were listed as examples in the survey) and community culture (defined as shared values, beliefs, and practices of the community on the survey) as important to their work report greater gains in both institutional and departmental change; in other words, participants who have greater exposure to and experience with these aspects of the communities report they are able to effect greater change at their home institutions. Based on our observations and interviews, we are not surprised to see the positive association of leaders and culture with our outcomes. First, leaders in the community play important roles of influence for community members. The four communities in our study were all founded by visionary individuals who continuously recruit a cadre of leaders to be active in community operations and generally visible at community events. These community leaders model effective leadership styles, which serve as models for community members for how to effectively lead when they return to their campuses, allowing for diffusion of these practices from nonorganizationally situated communities to multiple institutions. These leaders also set the tone for community gatherings and events in which faculty learned from one another in peer-to-peer settings, fostering cultures of active engagement, trust, and support among community members.

These communities also exhibit cultures that value and place personal support at a premium, which are associated with change outcomes on faculty members’ campuses. The culture that is fostered through community events, resources, and communication materials is a supportive one in which faculty are provided valuable feedback and mentorship for seeking changes on their home campus. Faculty further experience community cultures in which sharing strategies and best practices is commonplace as they learn in a peer-to-peer environment (Kezar, Gehrke, & Bernstein-Sierra, 2017). These findings related to culture also make sense in conjunction with the importance of key leaders, who not only set the direction and tone of the community but also serve as mentors and provide support for community members; these leaders can provide guidance and even consult on departmental and institutional changes being sought by community members. The combined value placed on these key leaders and community culture suggests key considerations for CoPs to influence broader change in STEM reform, specifically to be intentional about the ways that leaders represent and model cultures of support and engagement in these types of communities.

Faculty, Institutional, and Disciplinary Factors in Organizational Change

Beyond engagement and design characteristics, several other variables related to leadership and organizational context point to the importance of these communities contributing to top-down change. Our analyses reveal that motivation to lead and the support of institutional leadership play key roles in contributing to broader STEM reform. Specifically, faculty who identified a desire to develop leadership skills and strategies, as well as institutional climates supportive of STEM reform, reported greater gains for their home departments and institutions. Faculty motivations play a key role in their adoption of changes and reforms (Austin, 2011; Blackburn & Lawrence, 1995), and our research suggests that faculty seeking strategies for effective leadership is correlated with change across their institutions and departments. Thus, it appears that in order for bottom-up change strategies to be effective, CoP leaders should consider the reasons faculty are seeking them out for faculty development. Leadership skills can also be useful for effecting change at both the departmental and institutional level. Roughly two out of five participants in our study indicate that they gained leadership skills and that involvement in these communities contributed to their career advancement (Gehrke & Kezar, 2016), which can place faculty in key places in the organizational hierarchy to influence change from the top down. Faculty also indicated greater departmental and institutional changes when they perceived a more supportive climate for STEM reform at their home institutions, climates that are often fostered by institutional leadership (Austin, 2011; Fairweather, 2009); faculty report great gains when their campus climate is supportive of reform. While these communities focus on a bottom-up approach to change and STEM reform, it is important to also consider the important role played by top administrators in setting the tone for campus change by fostering a culture open to reform (Seymour, 2002).

Finally, despite the occasional significant relationship, we generally found that faculty appointment and institutional type were not associated with departmental and institutional change in our study, suggesting the potential for CoPs like those in our study to work around structural barriers to change, such as different reward structures or faculty contracts (Austin, 2011; Fairweather, 2009). However, we found that assistant professors, as well as faculty from associates colleges, reported significantly less institutional change than their peers. While this makes intuitive sense given prior research that shows the different foci of faculty at different points of their careers (Neumann, 2009) and institutional types (Gappa, Austin, & Trice, 2007), it is still worth noting for the purpose of development for these communities. Junior faculty inherently have less years of work at an institution and competing demands pretenure, while faculty from associates institutions face their own challenges and constraints given their nearly sole focus on instruction, which may make it difficult to broaden scale of reform through these individuals. Reform CoPs will likely need to consider ways in which they can help these faculty think about and effect change more intentionally on their home campuses or recognize that their impact will likely be in the long run, not short run, if institutional reform is one of their goals.

Disciplinary Differences

Finally, we also uncovered disciplinary differences associated with our outcomes. Specifically, faculty from the biological and life sciences (who served as the reference group) indicated significantly more departmental change resulting from their involvement compared to several other disciplinary areas, while reporting significantly less institutional change compared to other disciplines. Follow-up analyses comparing participants from the physical sciences also indicate more departmental change and less institutional change compared to the other disciplines.

We can interpret these findings pertaining to disciplines in two ways. First, the communities in our study by-and-large focus on the typical STEM fields, and as our sample indicates, most community members belong to the biological/life and physical sciences. Thus, these reforms are targeting these participants the most, contributing to departmental change in particular for these individuals. In cases where participants from non-STEM disciplines, such as social scientists, are involved, they are usually part of larger teams examining broad institutional reform in interdisciplinary contexts, working with others from the STEM disciplines. Second, as we cited above, disciplines are important influences on the extent to which change is adopted in higher education (Austin, 1994, 2011; Beecher & Trowler, 2001). Our results perhaps suggest that the cultures among faculty in the biological/life and physical sciences may lend themselves to focus more on departmental initiatives. The biological and life sciences education (AAAS, 2011, 2015; Goldey et al., 2012; National Research Council [NRC], 2003) and physical sciences-based research communities (NRC, 2003; Weiman, Perkins, & Gilbert, 2010) also have a history of pursuing national pedagogical reform specifically focused on their own disciplines, which could suggest that these disciplines are more focused on pursuing departmental change as opposed to broader institutional changes. For example, the Vision and Change in Undergraduate Biology Education efforts (AAAS, 2011, 2015) have been far-reaching and ubiquitous in the biological and life sciences, and these results likely speak to the multiple efforts of this disciplinary community in conjunction with the work of the communities in this study. In fact, other cross-institutional communities have emerged to help spread these reform efforts, such as the Partnership for Undergraduate Life Sciences Education (PULSE) Fellows who are tasked with “stimulating department-level implementation” of the reforms outlined in Vision and Change (PULSE Fellows, 2016). These efforts distributed among multiple communities (including those in our study) likely contribute to the ubiquity of departmental change among biological and life science faculty members compared to other disciplines.

Implications for Practice and Research

In the end, this research identifies several key strategies for STEM reformers to engage faculty to contribute to broader organizational change outcomes at their home institutions. Reformers interested in starting or designing CoPs to contribute to STEM reform should (a) provide adequate support to keep faculty involved for longer periods of time, (b) seek involvement from multiple individuals within single institutions in their communities, (c) engage community members in activities and development to help them gain mastery over the material in order to communicate their work beyond the community, and (d) identify key leaders who can both support faculty and foster a culture of engagement. In doing so, these efforts can effectively engage faculty in the kinds of bottom-up change efforts advocated for in the literature and shown to correlate with broader institutional and departmental outcomes for STEM reform.

This study has several implications for both STEM reformers and scholars. First, our findings suggest that the notion of scaling up through involving individual faculty in communities focused on reform is a concrete strategy for effecting organizational and departmental change. In another paper from our study (Gehrke & Kezar, 2016), we highlight the benefits for individual faculty who engage in these communities, but our current analyses reveal that in addition to the individual benefits, involving faculty in these reforms contributes to broader departmental and institutional change. Scholars of change in higher education (Henderson et al., 2011; Kezar, 2011; Kezar & Lester, 2011; Seymour, 2002) have advocated for change that includes both bottom-up and top-down strategies for change. While individual benefits are more prevalent, both in the literature and for the individuals in our study (Gehrke & Kezar, 2016), participants in our study still indicate that some departmental and institutional changes do result from their involvement, and we now know which aspects of their involvement are associated with these changes (e.g., supportive community cultures and leaders, continuous engagement with members from one’s institution).

Our study contributes to the literature on CoPs as it is one of the first to associate aspects of CoP involvement and design with broader outcomes, and it is also one of the first to do so for nonorganizationally situated and cross-institutional CoPs. It is easy to understand how involvement in traditionally oriented CoPs could contribute to organizational learning (Allee, 2000) and other organizational outcomes, because every member who belongs to such a community also belongs to the larger organization. This is not only an intuitive finding, but it is also supported by the research. In the case of these nonorganizationally situated CoPs, members belong to a diffuse network of higher education institutions and are spread out around the country. Despite this, development of individual faculty (and ideally some of their peers) in these communities appears to influence change at a variety of institutions from which the community members hail, despite the fact that the only thing these institutions have in common is the involvement of their faculty in the common community. These communities have found ways to effect change across a host of institutions despite not receiving any direction or tangible support from these institutions in the work they do. In addition to connecting involvement with outcomes, our study also contributes to the literature by specifically examining how motivation of community participants relates to outcomes. While motivation has been studied in relation to fostering change (Austin, 2011; Blackburn & Lawrence, 1995), it is generally not a focus in the CoP literature. This study highlights the importance of considering community members’ motivations for participating when utilizing CoPs for change efforts.

This study also advances the research on CoPs by highlighting how several key aspects of these communities associated with broader outcomes. The dearth in research connecting aspects of CoP involvement and design to outcomes is a hindrance to advancing the scholarship of these communities and to informing the creation of future CoPs to address best practices (Chang, 2010; Chang et al., 2009; Zboralski et al., 2006). Through our study, we identified the important role that leadership and culture in particular play in engaging faculty and their perceptions that involvement in the communities influences departmental and institutional change. Future research should seek to confirm that these aspects of community design are important for other communities as well as identify more factors that were not the focus of this study that contribute to CoPs meeting broader outcomes. These key aspects also provide guidance to funding agencies who should seek to support communities that adequately and intentionally address the ways in which they will foster leadership and a culture of engagement in their community.

Footnotes

Appendix

Descriptive Statistics for Other Variables in Study

	M	SD	Min.	Max.	Description
Community
BioQUEST	0.10		0.00	1.00	1 = BioQUEST; 0 = all others
Project Kaleidoscope	0.44		0.00	1.00	1 = PKAL; 0 = all others
The POGIL Project	0.33		0.00	1.00	1 = POGIL; 0 = all others
SENCER	0.13		0.00	1.00	1 = SENCER; 0 = all others
Institutional characteristics
Control: Public	0.54		0.00	1.00	1 = public; 0 = all others
Control: Private	0.46		0.00	1.00	1 = private; 0 = all others
Carnegie: Doctoral institution	0.22		0.00	1.00	1 = doctoral; 0 = all others
Carnegie: Masters institution	0.34		0.00	1.00	1 = masters; 0 = all others
Carnegie: Baccalaureate institution	0.29		0.00	1.00	1 = baccalaureate; 0 = all others
Carnegie: Associates institution	0.14		0.00	1.00	1 = associates; 0 = all others
Personal and professional characteristics
Female	0.55		0.00	1.00	1 = female; 0 = all others
Male	0.45		0.00	1.00	1 = male; 0 = all others
White	0.84		0.00	1.00	1 = White; 0 = all others
Person of color	0.16		0.00	1.00	1 = person of color; 0 = all others
Rank: Professor	0.37		0.00	1.00	1 = professor; 0 = all others
Rank: Associate professor	0.28		0.00	1.00	1 = associate professor; 0 = all others
Rank: Assistant professor	0.09		0.00	1.00	1 = assistant professor; 0 = all others
Rank: Non-tenure-track faculty	0.12		0.00	1.00	1 = non-tenure-track faculty; 0 = all others
Rank: Faculty at nontenure institution	0.08		0.00	1.00	1 = faculty at nontenure institution; 0 = all others
Rank: No rank	0.06		0.00	1.00	1 = no rank; 0 = all others
Department: Biological sciences	0.34		0.00	1.00	1 = biological sciences; 0 = all others
Department: Physical sciences	0.35		0.00	1.00	1 = physical sciences; 0 = all others
Department: Mathematical sciences/engineering	0.12		0.00	1.00	1 = mathematical sciences; 0 = all others
Department: Arts & humanities	0.02		0.00	1.00	1 = arts & humanities; 0 = all others
Department: Social sciences	0.08		0.00	1.00	1 = social sciences; 0 = all others
Department: Medical sciences	0.02		0.00	1.00	1 = medical sciences; 0 = all others
Department: Other	0.07		0.00	1.00	1 = other department; 0 = all others
Motive: Improve teaching	0.74		0.00	1.00	1 = selected; 0 = not selected
Motive: Seeking support for change	0.25		0.00	1.00	1 = selected; 0 = not selected
Motive: Involve peers in change	0.27		0.00	1.00	1 = selected; 0 = not selected
Motive: Connect with colleagues	0.47		0.00	1.00	1 = selected; 0 = not selected
Motive: Career development	0.30		0.00	1.00	1 = selected; 0 = not selected
Motive: Learn leadership strategies	0.29		0.00	1.00	1 = selected; 0 = not selected
Years teaching undergraduates	16.81	8.66	1.00	31.00	Units in years
Perception of climate for STEM reform	4.07	0.81	1.00	5.00	5-point scale; 1 = completely closed and unsupportive; 5 = completely open and supportive
Engagement and design variables
Years involved with community	6.38	5.47	1.00	31.00	Units in years
Number of peers involved with community	1.85	0.55	1.00	3.00	3-point scale; 1 = no one is involved; 2 = some people are involved; 3 = many people are involved
Characterized extent of involvement	2.27	0.89	1.00	4.00	4-point scale; 1 = newcomer; 4 = continuous involvement
Attend: National event	2.03	0.89	1.00	4.00	4-point scale; 1 = never; 4 = many times
Attend: regional/local event	1.66	0.82	1.00	4.00	4-point scale; 1 = never; 4 = many times
Activity: Present at community event	1.46	0.83	1.00	4.00	4-point scale; 1 = never; 4 = many times
Activity: Author material	1.48	0.86	1.00	4.00	4-point scale; 1 = never; 4 = many times
Activity: Present material at professional meeting	1.36	0.76	1.00	4.00	4-point scale; 1 = never; 4 = many times
Activity: Publish about community work	1.13	0.46	1.00	4.00	4-point scale; 1 = never; 4 = many times
Group: Leadership/board member	0.07		0.00	1.00	1 = leadership/board member; 0 = all others
Group: Project/grant-based	0.23		0.00	1.00	1 = project/grant-based group member; 0 = all others
Design: Different involvement opportunities	2.25	0.89	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Innovative & new ideas	2.58	0.86	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Community resources	2.21	0.91	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Safe, supportive space	2.39	0.95	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Inclusive practices	2.27	0.93	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Community leaders	2.21	0.96	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Connection with other faculty	2.43	0.93	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Opportunity for mentoring	2.31	0.94	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Connection with STEM leaders	2.49	0.95	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Community culture	2.36	0.88	1.00	4.00	4-point scale; 1 = not important; 4 = essential
Design: Community philosophy	2.41	0.91	1.00	4.00	4-point scale; 1 = not important; 4 = essential

Notes

SEAN GEHRKE is the director of institutional research at Everett Community College, 2000 Tower Street, Everett, WA 98201, USA; e-mail: sgehrke@everettcc.edu . He is primarily a survey researcher with experience designing and administering surveys for a variety of stakeholders in higher education, including college students, faculty, academic and student affairs administrators, and alumni, in addition to his training in social network analysis and mixed-methods research. His research focuses on organizational issues in higher education relating to social networks, leadership, organizational change, and STEM reform as well as how the college environment and student experiences influence learning and development.

ADRIANNA KEZAR is a professor of higher education and co-director of the Pullias Center for Higher Education at the University of Southern California, WPH Room 701, 3470 Trousdale Parkway, Los Angeles, CA 90089, USA. She is a national expert of change, governance, and leadership in higher education. Her research agenda explores the change process in higher education institutions and the role of leadership in creating change. Kezar also regularly consults for campuses and national organizations related to her work on non-tenure-track faculty, STEM reform, collaboration, leadership development, and change.

References

Allee

(2000). Knowledge networks and communities of learning. OD Practitioner, 32(4), 4–13.

American Association for Advancement of Science. (2011). Vision and change in undergraduate biology education: A call to action. Washington, DC: American Association for Advancement of Science. Retrieved from http://visionandchange.org/files/2013/11/aaas-VISchange-web1113.pdf

American Association for Advancement of Science. (2015). Vision and change in undergraduate biology education: Chronicling change, inspiring the future. Washington, DC: American Association for Advancement of Science. Retrieved from http://visionandchange.org/files/2015/07/VISchange2015_webFin.pdf

Austin

(2011). Promoting evidence-based change in undergraduate science education. Washington, DC: National Academies National Research Council.

Austin

A. E.

(1994). Understanding and assessing faculty cultures and climates. In Kinnick

M. K.

(Ed.), Providing useful information for deans and department chairs: New directions for institutional research (pp. 47–63). San Francisco, CA: Jossey-Bass.

Blackburn

R. B.

Lawrence

(1995). Faculty at work: Motivation, expectation, satisfaction. Baltimore, MD: The Johns Hopkins University Press.

Beecher

Trowler

P. R.

(2001). Academic tribes and territories: Intellectual enquiry and the culture of disciplines. Philadelphia, PA: The Society for Research into Higher Education and Open University Press.

Borrego

Henderson

(2014). Increasing the use of evidence-based teaching in STEM higher education: A comparison of eight change strategies. Journal of Engineering Education, 103(2), 220–252.

Brown

J. S.

Duguid

(2001). Knowledge and organization: A social-practice perspective. Organization Science, 12(2), 198–213.

10.

Chang

(2010). The relationships among participants’ characteristics, perceptions, nature of involvement, and outcomes in strategic community of practice programs in a large electric utility company (doctoral dissertation). Available from ProQuest Dissertations and Theses database (UMI No. 3429660)

11.

Chang

Jacobs

(2009). Relationship between participation in communities of practice and organizational socialization in the early careers of South Korean IT employees. Human Resource Development International, 12, 407–427.

12.

Corbo

J. C.

Reinholz

D. L.

Dancy

M. H.

Deetz

Finkelstein

(2015). Sustainable change: A model for transforming departmental culture to support STEM education innovation. Retrieved from the arXiv database.

13.

Creswell

J. W.

Plano Clark

V. L.

(2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: SAGE.

14.

Dancy

M. H.

Henderson

(2010). Pedagogical practices and instructional change of physics faculty. American Journal of Physics, 78, 1056–1063.

15.

Dede

(2006). Scaling up: Evolving innovations beyond ideal settings to challenging contexts of practice. In Sawyer

R. K.

(Ed.), Cambridge handbook of the learning sciences (pp. 1–37). New York: Cambridge University Press.

16.

Fairweather

(2009). Linking evidence and promising practices in science, technology, engineering and mathematics (STEM) undergraduate education. Paper for the National Academies National Research Council Board of Science Education. Retrieved from http://www.nsf.gov/attachments/117803/public/Xc-Linking_Evidence-Fairweather.pdf

17.

Fontaine

M. A.

Millen

D. R.

(2004). Understanding the benefits of communities of practice. In Hildreth

Kimble

(Eds.), Knowledge networks: Innovation through communities of practice (pp. 1–13). Hershey, PA: Idea Group Publishing.

18.

Furr

(2011). Scale construction and psychometrics for social and personality psychology. London: Sage.

19.

Gappa

J. M.

Austin

A. E.

Trice

A. G.

(2007). Rethinking faculty work: Higher education’s strategic imperative. San Francisco, CA: Jossey-Bass.

20.

Gehrke

Kezar

(2016). STEM reform outcomes through communities of transformation. Change: The Magazine of Higher Learning, 48(1), 30–38.

21.

Goldey

E. S.

Abercrombie

C. L.

Ivy

T. M.

Kusher

D. I.

Moeller

J. F.

Rayner

D. A.

Smith

C. F.

Spivey

N. W.

(2012). Biological inquiry: A new course and assessment plan in response to the call to transform undergraduate biology. CBE-Life Sciences Education, 11(4), 353–363.

22.

Gorsuch

R. L.

(1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

23.

Hammersmith

(2015, February). Advancing scholarship on active learning spaces through faculty learning communities. Poster presented at the Educause Learning Initiative annual meeting, Anaheim, CA.

24.

Henderson

Beach

Finkelstein

(2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48, 952–984.

25.

Hurtado

Eagan

M. K.

Pryor

J. H.

Whang

Tran

(2012). Undergraduate teaching faculty: The 2010–2011 HERI Faculty Survey. Los Angeles, CA: Higher Education Research Institute, UCLA.

26.

Kezar

(2011). What is the best way to achieve broader reach of improved practices in higher education? Innovative Higher Education, 36(4), 235–247.

27.

Kezar

(2014). How colleges change: Understanding, leading, and enacting change. New York, NY: Routledge.

28.

Kezar

Gehrke

Bernstein-Sierra

(2017). Designing for success in STEM communities of practice: Philosophy and personal interactions. The Review of Higher Education, 40, 217–244.

29.

Kezar

A. J.

Lester

(2011). Enhancing campus capacity for leadership: An examination of grassroots leaders in higher education. Stanford, CA: Stanford University Press.

30.

Lave

(1988). Cognition in practice: Mind, mathematics and culture in everyday life. New York, NY: Cambridge University Press.

31.

Lesser

E. L.

Storck

(2001). Communities of practice and organizational performance. IBM Systems Journal, 40(4), 831–841.

32.

McDermott

(1999). Learning across teams: The role of communities of practice in team organizations. Knowledge Management Review, 2, 32–36.

33.

McDonald

Star

(2008). The challenges of building an academic community of practice: An Australian case study. In Engaging communities: Proceedings of the 31st HERDSA Annual Conference, Rotorua, July 1-4, 2008 (pp. 230–240). Rotura, New Zealand: HERDSA.

34.

Meyers

L. S.

Gams

Guarino

A. J.

(2006). Applied multivariate research: Design and interpretation. Thousand Oaks, CA: Sage Publications, Inc.

35.

Millen

D. R.

Fontaine

M. A.

(2003, November). Improving individual and organizational performance through communities of practice. In Proceedings of the 2003 International ACM SIGGROUP conference on supporting group work (pp. 205–211). New York: ACM.

36.

Millen

D. R.

Fontaine

M. A.

Muller

M. J.

(2002). Understanding the benefit and costs of communities of practice. Communications of the ACM, 45(4), 69–73.

37.

National Research Council. (2003). Improving undergraduate instruction in science, technology, engineering, and mathematics. Washington, DC: National Academies Press.

38.

Neumann

(2009). Professing to learn: Creating tenured lives and careers in the American research university. Baltimore, MD: The Johns Hopkins Press.

39.

President’s Council of Advisors on Science and Technology. (2012). Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Washington, DC: Executive Office of the President. Retrieved from http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_2-25-12.pdf

40.

Price

(2005). Assessment standards: The role of communities of practice and the scholarship of assessment. Assessment & Evaluation in Higher Education, 30, 215–230.

41.

PULSE Fellows. (2016). About the partnership for undergraduate life science education. Retrieved from http://www.pulsecommunity.org/page/about

42.

Rabe-Hesketh

Skrondal

(2012). Multilevel and longitudinal modeling using stata, volume 1: Continuous responses (3rd ed.). College Station, TX: Stata Press.

43.

Sànchez-Cardona

Sànchez-Lugo

Vèlez-Gonzàlez

(2012). Exploring the potential of communities of practice for learning and collaboration in a higher education context. Procedia – Social and Behavioral Sciences, 46, 1820–1825.

44.

Seymour

(2002). Tracking the processes of change in US undergraduate education in science, mathematics, engineering, and technology. Science Education, 86, 79–105.

45.

Seymour

Hewitt

N. M.

(1997). Talking about leaving: Why undergraduates leave the sciences. Boulder, CO: Westview Press.

46.

Singer

S. R.

Nielsen

N. R.

Schweingruber

H. A.

(2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. Washington, DC: National Research Council of the National Academies of Science.

47.

Steinert

Mann

Centeno

Dolmans

Spencer

Gelula

Prideaux

(2006). A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education. Medical Teacher, 28(6), 497–526.

48.

Stewart

(1996, August 5). The invisible key to success. Fortune, pp. 173–176.

49.

Sunal

D. W.

Sunal

C. S.

Whitaker

K. W.

Freeman

L. M.

Odell

Hodges

Edwards

Johnston

R. A.

(2001). Teaching science in higher education: Faculty professional development and barriers to change. School Science and Mathematics, 101(5), 246–257.

50.

Supovitz

J. A.

(2002). Developing communities of instructional practice. Teachers College Record, 104, 1591–1626.

51.

Van Wyk

. (2005, July). Communities of practice in academic library: A run on the wild side? In Proceedings of the 5th International Conference of Animal Health Information Specialists (pp. 97–102).

52.

Weiman

Perkins

Gilbert

(2010). Transforming science education at large research universities: A case study in progress. Change: The Magazine of Higher Learning, 42(2), 6–14.

53.

Wenger

(1998). Communities of practice: Learning as a social system. Systems Thinker, 9(5), 2–3.

54.

Wenger

(2007). Communities of practice: A brief introduction. Retrieved January 14, 2009, from http://www.e.com/theory/

55.

Wenger

McDermott

Snyder

(2002) Cultivating communities of practice: A guide to managing knowledge. Cambridge, MA: Harvard Business School Press.

56.

Zboralski

Salomo

Gemuenden

H. G.

(2006). Organizational benefits of communities of practice: A two-stage information processing model. Cybernetics and Systems: An International Journal, 37, 533–555.