The Operational Impact of Organizational Communities of Practice

Abstract

Organizations are increasingly making use of communities of practice (CoPs) as a way of leveraging the dispersed knowledge and expertise of their employees. One important way in which CoPs are predicted to benefit organizations is by facilitating the transfer of best practices. In this study, we examined the impact of the introduction of global CoPs on changes made to operational procedures in three refineries operated by a multinational company over a period of more than 5 years. We used a Bayesian change point detection model to assess the probability that changes in the rate of adoption of new and revised operational procedures occurred following the introduction of CoPs. The results confirmed our predictions, providing support for the idea that CoPs benefit organizations by contributing to the development of better operational routines and demonstrating the utility of Bayesian techniques for assessing the impact of complex organizational change.

Keywords

communities of practice groups organizational change knowledge management organizational capital

As developments in information technology and systems have enabled faster, richer, and more expansive communication networks, organizations are witnessing the emergence of new, dynamic collaborative structures (Dibble & Gibson, 2013; C. Gibson & Dibble, 2013; Tannenbaum, Mathieu, Salas, & Cohen, 2012). One collaborative form that is attracting increasing interest in contemporary organizations, largely as a consequence of improvements in connectivity and data management capabilities, is the community of practice (CoP). Traditionally, CoPs have existed as diffuse, informal, voluntary, organizationally nonaligned networks of people, linked by a common interest, passion, or purpose (Brown & Duguid, 1991; Lave & Wenger, 1991). In the contemporary organizational context, however, they are increasingly appearing as organizationally bounded collaborative structures, purposefully created in order to generate improvements in organizational performance (Dubé, Bourhis, & Jacob, 2005; Kirkman, Cordery, Mathieu, Kukenberger, & Rosen, 2013; Kirkman, Mathieu, Cordery, Rosen, & Kukenberger, 2011; Lesser & Storck, 2001; McDermott & Archibald, 2010; Wenger & Snyder, 2000; Yamklin & Igel, 2012).

Theory on team processes, knowledge creation, and innovation indicates that specific characteristics of CoPs as collaborative forms might be conducive to building human, social, and organizational capital. For example, researchers have proposed that flexible structures (Adler & Borys, 1996; Bresman & Zellmer-Bruhn, 2013), autonomy (C. Gibson & Vermeulen, 2003; Haas, 2010), and permeable boundaries (Dibble & Gibson, 2013) are of great value for combining and capitalizing on the individual knowledge stores held by members.

Despite their increasing popularity and potential as structures for accessing, developing, and capitalizing on knowledge, evidence for the beneficial impact of CoPs on organizations is scant and largely derived from descriptive case studies (e.g., Koliba & Gajda, 2009). Given that many organizations deliberately set up and support such communities for the specific purpose of improving how organizations operate (McDermott & Archibald, 2010; Saint-Onge & Wallace, 2003; Yamklin & Igel, 2012) and invest considerable amounts of time and resources in doing so (Scarso, Bolisani, & Salvador, 2009), the current lack of concrete evidence of their capacity to deliver tangible organizational benefits is of concern. Without such evidence, scholars and practitioners have little to go on in terms of whether and how to assemble such communities to enhance organizational performance. In response to these limitations, we investigate the impact that organizational CoPs had on the operations of minerals-processing plants operated by a large U.S.-based multinational. Our particular focus is on the extent to which the introduction of CoPs corresponded with changes in operational procedures, that is, the extent to which they altered the established ways of doing things within the plants.

Our study makes several significant contributions to knowledge regarding the value of CoPs in organizational settings and to theory regarding collaborative processes. First, we test the proposition, derived from the work of Wenger and colleagues (Wenger, 1998; Wenger & Snyder, 2000; Wenger, McDermott, & Snyder, 2002), that CoPs benefit organizations by acting as a mechanism for the identification and subsequent adoption of new and improved operational procedures (i.e., best practices). While anecdotal evidence of productivity improvements and cost savings generated as a result of CoP-initiated operational improvements abound within the management literature (Lesser & Storck, 2001; McDermott & Archibald, 2010; Yamklin & Igel, 2012), empirical research has yet to demonstrate that CoPs, which are generally not part of the formal authority structure of the organization, are able to manifestly influence operational practices within organizations as a direct result of new knowledge creation. Our study thus examines whether the introduction of CoPs has a tangible impact on the rate at which operational procedures are introduced or revised within a production system and, in doing so, provides greater understanding of the value and role of the peripheral, autonomous nature of collaborative structures for reaping organizational benefits and generating learning-based outcomes.

Second, most of the evidence relating to the value created by CoPs at the organizational level, in addition to being predominantly based on qualitative research, is based on snapshots at just one or two points in time. Importantly, this makes it difficult to judge whether or not CoPs are able to generate significant and lasting benefits. To address this limitation, we employ a quasi-experimental research design to assess the impact of the introduction of CoPs, and we use a time series of operational data collected weekly over a span of more than 5 years. In this way, we are able to assess the extent to which the communities produce both tangible and sustainable operational benefits and hence represent more than just another transient management fashion or “fad” (J. Gibson & Tesone, 2001). If, despite their flexible and permeable nature, CoPs are able to continue to consistently generate performance improvements, this constitutes important evidence of the ability of such structures to be maintained over time, without declining returns.

Third, a number of different theoretical mechanisms have been proposed to account for the potential value contributed by CoPs to organizations at an operational level. These include increasing the knowledge and skill levels of the workforce, speeding up problem solving, and improving operational routines by combining knowledge across sites in unique and valuable ways (Wenger & Snyder, 2000). In this study, we seek empirical evidence that CoPs can improve operational routines within an organization. We propose that the sharing, codification, and transfer of best-practice ideas that occurs within CoPs will result in fewer new operational procedures being implemented but that these will tend to be better practices, requiring less subsequent revision. Such a dynamic perspective, which enables the understanding of the impact of knowledge, rather than just its introduction, is infrequent in the literature. Hence, evidence in this regard extends theory, which has predominantly focused on new knowledge created, as opposed to the trajectory of that knowledge in generating value for the organization.

Finally, we make a contribution to organizational research more generally by demonstrating how Bayesian approaches can be effectively used to evaluate complex organizational change. Bayesian techniques have been applied to a range of change-point detection problems, for example, the detection of abrupt changes in the frequency of coal-mining disasters, increases in crop yield following agricultural field trials, and changes in real wage growth as a function of political changes within Organization for Economic Cooperation and Development countries (e.g., Carlin, Gelfand, & Smith, 1992; Chib, 1998; Raftery, 1994; Western & Kleykamp, 2004). However, Bayesian change-point models have yet to be applied to the analysis of organizational change, where, we suggest, they may prove theoretically and practically useful.

The advantages Bayesian approaches provide in general terms are well documented (see Zyphur & Oswald, 2015, for a comprehensive overview and literature review). We highlight here the particular advantages provided by a Bayesian change-point analysis of the impact of an organizational intervention. These advantages arise because a Bayesian change-point analysis “treats the timing of the change as uncertain and the location of a change point as a parameter to be estimated” (Western & Kleykamp, 2004: 355), with two “states” or models that describe the data before and after the change point. In our context, this is particularly important because many organizational changes involve multiple elements, which are rolled out over time, and hence the intervention can rarely be said to have taken place at a unique point in time. Furthermore, it is often difficult to predict a priori how long the effect of an intervention will take to be noticed and have impact. A Bayesian analysis is able to deal with many of these complexities and uncertainties in a straightforward manner. By treating the change point as a random variable, allocating a prior distribution in some way to this unknown quantity and then updating this prior with additional data, a Bayesian analysis provides the posterior probability that change occurs at each time point in the observational period. This posterior distribution then provides statistical inference via probability statements about the time of change (as well as other parameters in the model), given the data set we have observed and the initial prior assumptions. In our case, we are interested in whether the time interval with the highest probability of transition from one state to another coincides with the time that an organizational intervention (the formation of a number of CoPs) is believed to have taken place and whether the parameters of each state support the hypothesis that the intervention has indeed reduced the number of new/revised procedures. We turn next to a more detailed description of organizational CoPs.

Organizational CoPs

Broadly defined, CoPs are “groups of people who share a concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis” (Wenger et al., 2002: 7). Such communities operate in all walks of life—within, outside, and indeed spanning the boundaries of formal organizations (Koliba & Gajda, 2009; Lave & Wenger, 1991; Wenger, 1998; Wenger & Snyder, 2000). CoPs have a long history of operating in organizational settings. For example, Lee and Neff (2004) describe the operation of communities of iron makers within multinational BHP-Billiton’s steel division in the 1950s and the subsequent formation of global communities of maintenance engineers in the 1990s. Today, CoPs have become a recognizable feature of organizations around the globe (e.g., IBM, Rio Tinto, Shell, Alcoa, Siemens, World Bank) and are a common manifestation of strategic approaches to knowledge management and performance improvement.

How do organizational CoPs differ from other collective structures that are found in organizational settings, such as work groups or teams? Brown and Duguid (1991) make the following distinction between groups and communities as organizational forms:

Group theory in general focuses on groups as canonical, bounded entities that lie within an organization and that are organized or at least sanctioned by that organization and its view of tasks…. The communities we discern are, by contrast, often noncanonical and not recognized by the organization. They are more fluid and interpenetrative than bounded, often crossing the restrictive boundaries of the organization to incorporate people from outside.…And, significantly, communities are emergent. That is to say, their shape and membership emerges in the process of activity, as opposed to being created to carry out a task. (p. 49)

Similarly, Raven (2003) suggests that CoPs that operate within organizations differ from other common forms of collaboration (e.g., work or project teams) in that they tend to be characterized by (a) emergent, rather than mandated, task missions; (b) voluntary, as opposed to assigned, membership; (c) naturally evolving and often shared leadership; (d) relatively low task interdependencies; (e) fluid internal structures; (f) accountability to internal, as opposed to external, stakeholders; and (g) resources supplied by the community itself, rather than the parent organization. Therefore, it is safe to say that CoPs are multifaceted entities that are not uniform in structure or nature and may be designed for a variety of different purposes.

Increasingly, however, the perceived potential for such communities to create value for an organization is leading to CoPs being intentionally created by organizations as teamlike structures that operate alongside the formal organizational structure and hierarchy of authority and whose primary objective is to generate improvements in business performance (Cordery, Soo, Kirkman, Rosen, & Mathieu, 2009; Yamklin & Igel, 2012). As such, CoPs come to resemble other collaborative forms that have received increasing attention in the literature, such as temporary teams with fluid, dynamic structures (Kane, Argote, & Levine, 2005; Bechky, 2006) or virtual teams with geographically dispersed members (C. Gibson & Gibbs, 2006; C. Gibson, Huang, Kirkman, & Shapiro, 2014). Hence, theory pertaining to these forms may be applicable to CoPs that take on such characteristics, and our investigation contributes to the growing literature in this arena.

Recent empirical research into these organizational CoPs has mainly focused on factors that differentiate between communities in terms of their capacity to function effectively internally and not on the communities’ impact on long-term organizational performance. For example, researchers have examined how different levels of empowering leadership, diversity of membership, networking, and information sharing influence assessments of how well individual communities are performing relative to one another (e.g., Hemasi & Csanda, 2009; Kirkman et al., 2011; Zboralski, Salamo, & Gemuenden, 2006). Next we discuss the potential theoretical explanations for the benefits generated by organizational CoPs.

Theoretical Mechanisms for Organizational Benefits

Generally speaking, the three main theoretical mechanisms by which CoPs deliver organizational benefits are by enhancing (a) human, (b) social, and (c) organizational capitals. Rather than viewing these as unique separate forms of capital, however, we submit that the introduction of CoPs may have a synergistic or “gestalt” influence by promoting their collective effects on organizational effectiveness.

Enhancing human capital

The introduction of CoPs may yield organizational benefits, in part, by enhancing an organization’s human capital—that is, the stock of knowledge that resides in an organization’s employees. Lave and Wenger (1991) coined the term “legitimate peripheral participation” to describe the process by which less experienced CoP members gradually learn from their more expert counterparts and, as a result, become more proficient in what they do. CoPs may also assist organizations to recruit and retain talent by providing interesting challenges and professional development opportunities for employees that may otherwise be lacking in a job. Evidence for this mechanism accrues when new knowledge is exchanged and accumulated in the CoP.

Yet, there is growing evidence that such processes are fleeting and difficult to sustain in more flexible, informal collaborative structures. For example, Bresman and Zellmer-Bruhn (2013) found that at the level of the local collaborative subunit within an organization, more (rather than less) structure is associated with internal learning (among core members) as well as more external learning from those outside the core membership. When such local structure was lacking, structure provided by the organization was beneficial. Similar findings have been obtained in numerous studies (e.g., Bundersen & Boumgarden, 2010; Willem & Buelens, 2009). For example, in a comprehensive analysis of two firms in the energy and finance industries, Willem and Buelens (2009) found no evidence for a general rule that less formalization and centralization leads to more knowledge sharing. Hence the free-form nature of CoPs may not be as conducive to generation of new knowledge as originally conceived.

Enhancing social capital

The introduction of CoPs may also generate organizational benefits through improvements in social capital (i.e., resources that derive from the network of relationships possessed by individuals or groups). For example, it has been argued that CoPs speed up problem solving and decision making, because they provide a readily accessible network of relevant expertise for individuals to consult whenever they encounter operational difficulties (Zboralski et al., 2006). Again, such a mechanism implies that new knowledge will be created in the CoPs as diverse members’ expertise is called upon when new problems or operational anomalies are faced.

Yet, when collaborations are fluid and boundaries are permeable, as is the case in most CoPs, it may be unlikely that such relational mechanisms are prominent. Using the example of emergency response groups, Majchrzak, Jarvenpaa, and Hollingshead (2007) suggest that under such conditions, coordination that develops for the team centers not around people but on action-based scenarios that either have been or might be carried out. The scenarios are not scripts, because they do not define the roles that people play; instead, the scenarios are patterns of actions strung together to be matched with events (Dyer & Shafer, 2003; Vera & Crossan, 2005). Thus, when expertise is constantly changing, the focus of the CoPs efforts may not be on agreeing who has expertise in a task-related topic; rather, more significant performance benefits are realized by developing the patterns of action through trial and error.

Enhancing organizational capital

The introduction of CoPs may also yield value in terms of generating organizational capital—that is, knowledge-based resources that reside in technology, systems, routines, and databases. Wenger and Snyder (2000) argue that CoPs provide a practical mechanism for the development, and subsequent identification and transfer of, best practices from one location to another. The term practice refers to know-how that is embedded in descriptions of requisite procedures within an organization (Kogut & Zander, 1992), and best practice transfer refers to “the firm’s replication of an internal practice that is performed in a superior way in some part of the organization and is deemed superior to internal alternate practices and known alternatives outside the company” (Szulanski, 1996: 28). In CoPs, members discuss with others how they do things in practice across different locations, share ideas about what constitutes best practice, codify these as generalizable best practices, and then promote the spread of superior operational procedures across the organization. Adopting a social-practice perspective, Brown and Duguid (2001: 202) describe CoPs as contributing to organizational capital by acting as “privileged sites for a tight effective loop of insight, problem identification, learning and knowledge production” and as “significant repositories for the development, maintenance, and reproduction of knowledge.”

Contextualized CoPs

Importantly, the organizational context is likely to place differential premiums on the benefits of enhancing different forms of capital. In some instances, the introduction of CoPs may be primarily designed to promote individual employee learning and development. This may be particularly important when employees are dispersed and knowledge is tacit (Bertels, Kleinschmidt, & Koen, 2011). In other instances, CoPs may be introduced as a forum to share joint interests, promote collaborations, and build employee networks in an organization. In still other instances, CoPs may be introduced as incubators or vehicles for “thought trials” or the development of prototypes or to “game play” different ideas among a cross-section of experts without encumbering costly and time-involving changes to organizational practices (Hienerth & Lettl, 2011). CoPs may involve all three facets, but the context or raison d’être for the introduction of CoPs in any particular circumstance will likely emphasize one of the factors more so than the others. In our instance, as is true in many manufacturing applications, CoPs were designed primarily to promote process improvements, and to vet and mature them, prior to introducing them in the production process. Indeed, at the launch of the CoP initiative, corporate management evidenced this multifaceted approach while underscoring the primary goal for the program introducing it as follows:

This program enables groups of people who have similar roles to deepen their expertise through ongoing interaction and apply this to alter operating and maintenance routines to our current best method. The initial problem was of a lack of a formal best practice transfer process. Communities are the structure we have put in place to address this issue. (internal company communication; emphases in the original)

Accordingly, from a scientific standpoint, the introduction of CoPs is not like an experimental manipulation by which one attempts to isolate a specific change while holding all other factors constant. Rather, the introduction of CoPs is more akin to action research, in which change agents—including organizational members—introduce multifaceted organizational interventions designed to have a collective impact (Marshall, 2011). Such interventions roll out over a period of time and also take time to gain acceptance and use (Armenakis & Bedeian, 1999). Once implemented, their impact on subsequent organizational functioning and effectiveness may also be lagged and take time to develop (Langley, Smallman, Tsoukas, & Van de Ven, 2013). Consequently, the exact causal nature of such interventions may be somewhat ambiguous, and the temporal relationships may be unclear. Accordingly, our challenge, and associated research question, is, can we discern whether the introduction of CoPs has lagged effects on important organizational functioning?

Whereas CoPs may be introduced for a multitude of purposes, in our case, they were focused upon process improvement. Changes to operational procedures are disruptive and costly. Naturally, such changes are intended to improve processes—to make work safer, more efficient, less costly, and/or perhaps more enjoyable. Yet, benefits are realized only if the changes that are introduced are indeed advantageous. Moreover, to the extent that changes are designed optimally in the first place, it minimizes costly rework and later modifications, thereby yielding even further benefits. Therefore, we anticipate that the benefit of introducing CoPs in our context is likely to be realized mostly through enhanced organizational capital in the form of improved operational procedures, and this is where the focus of our research lies. The link between CoPs and enhanced organizational capital is consistent with the dominant narrative in the theoretical and qualitative literature on CoPs, which focuses on their role in developing, evaluating, refining, and improving existing operational practices (Brown & Duguid, 2001; Kogut & Zander, 1992; McDermott & Archibald, 2010). It is this dynamic capability to deliver organizational benefits by delivering improved operational procedures (Zollo & Winter, 2002) that is the focus of the present study.

One method of determining the quality of operational procedure interventions is to track the extent to which an organization finds it necessary to later alter or augment them. We submit that CoPs that cultivate organizational capital through best-practice transfer will benefit the organization by minimizing the need to change or modify process improvement changes. They do so by increasing the likelihood that any new or revised procedure that is introduced is superior, that is to say, more fit for purpose, to those it replaces or those available elsewhere. In other words, in accordance with the enhancement-of-organizational-capital perspective, if new operational procedures identified and transferred by CoPs are indeed “best practice,” then this should subsequently result in less proliferation of ineffective operational procedures. Stated differently, to the extent that CoPs serve to promote “thought trials” and otherwise vet process improvement ideas from a variety of angles before they are introduced, the organization will likely derive benefit from minimizing the number of such changes and later modifications of them. Perhaps counterintuitively, then, to the extent that the procedures identified and transferred by CoPs turn out to be highly effective and superior, this should reduce the rate at which an organization’s existing operational procedures need to be added to over time. Similarly, as operational procedures are improved as a consequence of the influence of CoPs, then it would seem to follow that they are also likely to require less frequent revision in future. Hence, we hypothesize the following:

Hypothesis 1: The introduction of CoPs will be associated with a reduction in the rate at which new operational procedures are introduced.

Hypothesis 2: The introduction of CoPs will be associated with a reduction in the rate at which existing operational procedures are revised.

Method

Setting

The study took place within a single division of a large multinational company operating in the metals industry. This division operates multiple refineries in the United States, South America, the Caribbean, Europe, and Australia. In late 2004, the company set about implementing organizational CoPs, whose specified primary goal was to identify best-practice operational procedures that could then be transferred across the various refineries. Our study examines the impact of that intervention on operations within three refineries located in a single country. Plants in this country were selected for study because they were the only ones that had maintained continuous logs of the introduction of new and revised operational procedures over the period spanning the introduction of the CoPs. Each refinery, though differing in age and size, uses an identical process to extract the same mineral used in the production of a metal. Our study focuses on the impact of the deployment of 10 CoPs on operational procedures deployed within the “core” parts of this refining process.

The CoPs

The 10 CoPs were set up for the express purpose of identifying best practices and transferring them across the refineries (internally, they were called “communities of best practice”). Each CoP represented a different core part of the refining process, and the CoPs themselves were named after the core element of the refining process their members were associated with (e.g., CoP X contains engineers across the refineries, who are all working in the section of the refinery that deals with core process X). An external “sponsor,” a senior manager within the organization, and an internal facilitator were appointed for each community. The CoPs’ members interacted virtually, communicating with one another online and via teleconferences, and included members from nine different locations spread across six different countries located in both northern and southern hemispheres. Membership of the CoPs was restricted to those working in the company.

Measures

We measured the frequency with which (a) new and (b) revised operational procedures were introduced into the refining process over time. The organization concerned defined an operational procedure as a document containing “a work instruction that standardizes how all personnel interact with a process.” As an example, it might be an instruction to change from one pump to another at a certain point and time in the process. According to a senior manager within the firm,

the procedure guides people doing that work in what to do, when, why, how long it should take and how to know if you have been successful. New procedures are written for when we substantially change a process or we introduce new equipment or technology. Procedures are updated when we make minor changes or improvements to processes.

We extracted our data directly from operational logs that were maintained by the company and which spanned a period of just less than 6 years (298 weeks). Each time a new operational procedure was introduced into a refinery, or an existing procedure was revised, supervisors were required to register this change in an operational log as a document. These logs were updated weekly, and so the data thus comprised weekly counts of the number of new and revised operational procedures registered over a period of 298 consecutive weeks.

The data series thus constructed spanned the period where the organization commissioned the CoPs, which took place over a 33-week period, commencing in Week 36 and ending in Week 69.

Analyses

To assess our hypotheses, we employed a Bayesian change-point detection model similar to that described by Carlin et al. (1992), Chib (1998), and Raftery (1994). As indicated previously, the attraction of using a Bayesian model to evaluate change resulting from a complex organizational intervention is that it can account for uncertainty in knowledge of when precisely a change point occurs, a problem notoriously difficult in frequentist statistics (Western & Kleykamp, 2004). For example, in our situation, each of the 10 CoPs associated with the core processes we studied was initiated at different times across a 33-week period, and a number of a priori predictions are possible regarding when the systemwide effect would be likely to emerge. With a Bayesian approach, there is no need to assign in advance an arbitrary point in time when the intervention would be deemed to have “taken.” We describe the change-point detection approach we use in this study below. For those readers less familiar with Bayesian approaches, we note that the method described below can also be thought of as the simplest (intercept-only) case of a mixture latent growth model with a log-linking function. The Bayesian analysis incorporates the dynamic nature of the rate of implementation of CoPs by allowing the mixture part of our method to model probabilistically the uncertainty about when a change occurs and averaging over all possible weeks of change.

Since the data were recorded as weekly counts of new or revised procedures, we modeled the data as observations of Poisson random variables. Poisson random variables are described by a rate parameter that represents the expected number of counts per unit of time. In order to test our hypotheses, we first assessed whether this rate parameter had changed at some point within the 298-week observational period. Our model stipulated that each observation (weekly counts of new and revised procedures) was generated under two potential “states,” the observations in each state associated with its own Poisson random variable and rate parameter. The states were sequential in that State 1 occurs before the change point and State 2 occurs after the change point. We denoted S_t = 1 if at time t the data were better described by the model according to State 1 and S_t = 2 if at time t the data were better described by the model according to State 2. Inference about a change point lay in the posterior distributions Pr(S_t = 1|Data) and Pr(S_t = 2|Data) = 1 − Pr(S_t = 1|Data) for t = 1, 2, . . ., 298. For example, if Pr(S₂₀ = 2|Data) = 0, we would infer that a change had not occurred prior to time t = 20; if Pr(S₅₀ = 2|Data) = 0.8, there was a 0.8 probability that a change had occurred before time t = 50; and if Pr(S₁₀₀ = 2|Data) = 1, then we would conclude a change had indeed occurred before time t = 100. Finally, if Pr(S₂₉₈ = 1|Data) = 1 (t = 298 was the final observation), this would indicate that a rate change had not occurred. Once it has been established that a change in the rate at which procedures were introduced had occurred, we then examined whether this change coincided with the implementation of the CoPs. By plotting Pr(S_t = k|Data) for k = 1, 2 across all time points, we inferred whether the most probable time interval over which this change occurred coincided with the period over which the CoPs were introduced.

The second stage to test the hypotheses was to examine whether the change implied a reduction in new/revised procedures. This involved examining the posterior distribution of the rate parameters. We denoted the rate parameter for State 1 as λ₁ and the rate parameter for State 2 as λ₂. If the posterior densities p(λ₂|Data) and p(λ₁|Data) suggest that it is more probable that λ₂ < λ₁, and the change coincided with the introduction of the CoPs, we inferred that the CoPs reduced the rate of new or revised procedures.

The model described above is outlined in more detail in the appendix, with explicit descriptions of the priors for all parameters. For the more interested reader, we refer to Chib (1998) and Martin, Quinn, and Park (2011) for more detail. The model was implemented using the statistical package MCMCpack available in the open-source statistical software R (R Development Core Team, 2013) and described in Martin et al. (2011: 12-16). The appendix also includes the commands in MCMCpack and R we used to implement the model and produce the plots and summary statistics.

Results

New Operational Procedures

Figure 1 shows a scatterplot of the number of new procedures introduced within the production system over time, with the vertical lines indicating when each of the 10 CoPs began formally operating. The scatterplot suggests that the number of new operational procedures logged fell following the CoP implementation. If correct, then this would indicate that those operating the production system were changing their operational routines less frequently following the introduction of CoPs.

Figure 1

Plot of the Number of Weekly New Operational Procedures Over 298 Weeks

To establish whether or not this is in fact the case, we report the results of our analysis with a plot of the posterior probability of a change in the rate of new operational procedures being registered. As indicated above, and in the appendix, Pr(S_t = 1|Data) is the posterior probability that at time t, the data are allocated to State 1; and Pr(S_t = 2|Data) is the posterior probability that at time t, the data are allocated to State 2. Figure 2 shows the evolution of both of these probabilities over time.

Figure 2

Plots of the Posterior Regime Probabilities State 1 (solid line) and State 2 (dotted line) for New Procedures Over 298 Weeks

Careful inspection of Figure 2 reveals that Pr(S₂₉₈ = 1|Data) = 0, indicating a change point had indeed occurred. Furthermore, prior to Week 57, the data are best described by State 1 with Pr(S₅₇ = 1|Data) ≈ 1. Between Weeks 57 and 64, however, the posterior probability of the data being allocated to State 1 starts to decrease, while the probability of it being allocated to State 2 increases. This indicates that there is a change in the rate of new procedures being logged over this 7-week period. By Week 69, the posterior probability that the remaining data is allocated to State 2 becomes (effectively) 1, Pr(S₆₉ = 2|Data) ≈ 1. Importantly, this state shift coincides with the precise period over which the CoPs became operational.

To assess the direction of the change in the state shift (i.e., did the numbers of new procedures increase or decline?), we subsequently examined the posterior densities of λ₁, the rate parameter prior to a change point, and λ₂, the rate parameter after the change point. The estimated posterior mean and standard deviation of λ₁ are 36.42 and 0.85, respectively, while those of λ₂ are 17.17 and 0.29, confirming a significant drop in the rates of new operating procedure implementation. Table 1 contains a numeric summary of the posterior distribution of both parameters.

Table 1

Quantiles of the Posterior Distribution of the Rate Parameters State 1 and State 2 (λ₁ and λ₂, respectively) for the Number of New Operational Procedures

	Quantile
Rate Parameter	2.5%	25%	50%	75%	97.5%
λ₁	34.81	35.90	36.41	36.95	38.15
λ₂	16.62	16.99	17.16	17.37	17.72

Note that the 97.5th percentile for λ₂ is 17.72, and the 2.5th percentile for λ₁ is 34.81. There is no overlap between the two posterior densities, and we can therefore conclude that the difference between the two rates is highly significant. Therefore, not only has the rate of the number of new procedures changed during the period when the CoPs were introduced, but also the change resulted in a significant decline in new procedures being logged. Therefore, Hypothesis 1 was fully supported.

Revised Operational Procedures

The scatterplot of the numbers of operational procedures being revised, again with vertical lines denoting when the CoPs were formally introduced, is presented in Figure 3.

Figure 3

Plot of the Number of Weekly Revised Operational Procedures Over 298 Weeks

In comparison with the rate at which new procedures were introduced, the effects are far less apparent. Following the same analytical procedure as outlined earlier, we also detected a state shift post–CoP implementation (see Figure 4), although this did not occur until much later—between Weeks 203 and 209.

Figure 4

Plots of the Posterior Regime Probabilities State 1 (solid line) and State 2 (dotted line) for Revised Procedures Over 298 Weeks

The estimated posterior mean and standard deviation of λ₁ are 48.78 and 0.48 and of λ₂ are 36.03 and 0.62. Table 2 shows a summary of the posterior distribution of both parameters. Note that the 97.5th percentile for λ₂ is 37.26, and the 2.5th percentile for λ₁ is 47.89. Therefore, similar to the new procedures, a change has occurred that corresponds to a decline in the number of revised procedures. Therefore, Hypothesis 2 was fully supported. Noticeably, for the revised procedures, this change has occurred substantially after the period when the CoPs were introduced.

Table 2

Quantiles of the Posterior Distribution of the Rate Parameters State 1 and State 2 (λ₁ and λ₂, respectively) for the Number of Revised Operational Procedures

	Quantile
Rate Parameter	2.5%	25%	50%	75%	97.5%
λ₁	47.89	48.44	48.79	49.11	49.74
λ₂	34.83	35.61	36.02	36.44	37.26

Discussion

We predicted that the introduction of CoPs within an organization would result in fewer new operational routines being introduced and less need for revision of those routines within associated parts of a production system. This prediction was derived from the perspective that CoPs in organizations generate improvements in organizational capital, stemming from their key role in the development and transfer of best-practice operational routines (Wenger & Snyder, 2000). In support of our predictions, we found evidence of a contemporaneous reduction in the number of new operational procedures and a subsequent decrease in the number of revised procedures being logged.

What we believe is happening here is that the CoPs trigger an improvement in operating procedures in associated areas, based on discourse within the communities that centers around what others are doing in their locations and what seems to work best. The CoP acts as a virtual location where current local practices are exposed to broader examination by a global community and where members commit to taking the best of what others have and implementing it where they work. By formalizing knowledge sharing and innovation in this way, the organization is ensuring that any proposed new operational procedure is subjected to careful and rigorous review by a group of subject matter experts before it is determined to be of benefit. In other words, the CoP processes act to screen out innovations that are likely to prove ineffective and mean that less local adjustment to operational procedures is needed over time. We discuss theoretical implications, limitations, managerial implications, and future research next.

Theoretical Implications

Our research makes a number of significant contributions to research relating to organizational CoPs and to collaborative organizational forms more generally. Our study is the first to provide clear evidence of a link between CoP-based collaborations and changes to the way an organization operates. These findings are particularly noteworthy, given the nonauthoritative status of CoPs in most organizations, including this one. While CoPs are increasingly formally recognized by organizations as structural entities (McDermott & Archibald, 2010; cf. Brown & Duguid, 1991), it is still the case that they do not form part of the formal authority structure of an organization and lack any specific power to have their ideas implemented. Any transfer of best practices occurs mostly through informal influences and must be negotiated with line managers who are responsible for operations in the different locations. Our research demonstrates that CoPs are able to be influential in shaping how an organization operates.

A second key contribution is the finding that the effect of organizational CoPs on operations persisted over many years. Indeed, the CoPs themselves, despite having no formal place in the authority structure of the organization, were still operating some 5 years after their initial formation. This stability and resilience, especially as they are not part of the formal hierarchy of the organization, is remarkable. This may reflect the fact that organizational CoPs are hybrids of traditional virtual teams and CoPs (Kirkman et al., 2011), with a vigor that may derive synergistically from the combination of the key features of each, for example, the relatively low formality combined with common member interests, the ability to interact over distance, and high autonomy over task missions.

Third, our research has highlighted that there are different ways of looking at the value that is created by CoPs for organizations. In much of the previous literature on CoPs in organizations, the emphasis has been on the new knowledge that is generated by the communities (Lesser & Storck, 2001) and not on how that knowledge is subsequently assimilated, transformed, and exploited for benefit by the wider organization. Our research, by contrast, has shown that the contribution of CoPs can also be assessed in terms of how the knowledge they share, discuss, and create translates into improvements in an organization’s operations.

Managerial Implications

From a management perspective, this study has suggested that organizational CoPs are able to deliver concrete performance benefits to organizations and hence should be actively encouraged. Importantly, they also suggest that these benefits can be long lasting, if they are embedded in the organizational capital of the firm through operational procedures and systems. This strongly suggests that, although fashionable, CoPs cannot be dismissed as just another management “fad” (J. Gibson & Tesone, 2001). However, a question arises with respect to the extent to which increasingly formal integration and active management of CoPs within organizations, a trend noted by McDermott and Archibald (2010), might in fact reduce their capacity to thrive. While research has found that some degree of formal structuring of CoPs helps them become established in organizational settings (Dubé et al., 2005; Scarso et al., 2009; Yamklin & Igel, 2012), the degree to which CoPs should remain relatively informal entities (e.g., permitted to self-organize, identify own task missions, and have voluntary membership) has been a matter of some debate within the literature (Koliba & Gajda, 2009).

Limitations and Future Research

The evidence is fairly convincing that changes took place in the rate at which operational routines were being introduced and revised, but how confident can we be that those changes were due to the influence of CoPs? As with all quasi-experimental research, it is necessary to consider possible alternative explanations for what we found (Grant & Wall, 2009). First, it is conceivable that other factors, unrelated to the introduction of CoPs, could account for what we observed. For example, there may have been variations in the way in which changes to operational routines were recorded, a major program of plant maintenance, or less capital available to upgrade equipment or processes. Mindful of these possibilities, we questioned senior technical managers with detailed knowledge of the plants and their operations, probing whether or not there was any particular event (planned or otherwise) that took place over the period of time in question, which could have accounted for the drop in the number of new and revised documents being logged. The only substantive change that these experts could identify as occurring at this time was the rollout of the CoP initiative. This is not to say that other influences were not at work, but neither the managers nor ourselves as researchers were able to identify any that could have had the observed effect.

Second, if we accept that the CoP intervention was indeed responsible for the observed changes at an operational level, it is important to acknowledge that the Bayesian change-point analysis is focused on operational outcomes and cannot directly illuminate the precise mechanisms whereby change in those outcomes was achieved. As we have noted previously, the introduction of these CoPs represents a complex, multifaceted organizational change. In our introduction, we indicated that we believed that the most likely mechanism to account for a change in operational routines was through improvements to organizational capital brought about through the activity of the CoPs. Indeed, we noted that this was their primary organizational mission. However, we also recognized that all three theoretical mechanisms that have been advanced to account for the impact of CoPs on organizational effectiveness (human capital, social capital, and organizational capital) are likely to have been brought into play by the creation of the CoPs and that the changes we have observed are attributable to some combination of enhanced human, social, and organizational capital. For example, improvements in employee knowledge and skill brought about by participation in the CoPs may have led to greater stability in the operating system, reducing the need for changes to operational routines. It is also possible that the observed changes in operational performance arose directly from improvements in social capital, that is to say, from more rapid decision making that has been facilitated by the networks created through the communities.

Our findings suggest a number of potentially fruitful avenues for future research and theorizing. First, there is a need to explore multiple mechanisms by which CoPs influence the effectiveness of an organization’s operations. The holistic nature of the introduction of CoPs in an organization precludes a clear accounting of the underlying dynamics responsible for the changes. Clearly, this suggests the value of more detailed process studies that may reveal the influential processes. Such factors could be introduced as time-varying covariates into the current Bayesian change-point model to highlight such relationships. We encourage and look forward to investigations employing such designs.

As discussed earlier, CoPs have been portrayed as having an impact on the human, social, and organizational capital available to the firm. These effects are likely to have a different temporal character, with some evident immediately and others taking time to emerge, manifesting in different aspects of an organization’s effectiveness. There is also a need to adopt multilevel perspectives on CoP impact. Clearly, some CoP impacts occur at the level of the individual community member, others at the level of the community itself, and still others at the level of the organization. Future research needs to account for these multiple levels of influences and outcomes. Next, there is the matter of the impact of formalization on the capacity of CoPs to deliver organizational benefits. How formalized and structured do such entities need to be before they cease to become entities that are capable of attracting the interest, passion, and participation of people? Future studies also need to examine human capital, social capital, and organizational capital outputs simultaneously and to map the relationship between the various forms of intellectual capital development. We believe that doing so is well worth the substantial impact that CoPs can impart.

Footnotes

Appendix

The model in this paper uses the method of Chib (1998) by reformulating the change-point model into a Markov mixture model with latent states. In our case, we have a state space {1, 2} and discrete random variables are defined as S_t = 1 if the observation at time t is generated by State 1 and S_t = 2 if observation at time t is generated by State 2 for t = 1, 2, . . ., 298. The evolution of the states over time is described by a discrete-space, discrete-time Markov process. That is, the transition matrix is given by

P = (\begin{matrix} P_{11} & P_{12} \\ 0 & 1 \end{matrix})

where P₁₁ = Pr(S_t = 1|S_t _{− 1} = 1) and P₁₂ = (1 − P₁₁) = Pr(S_t = 2|S_t _{− 1} = 1). Notice that the lower left element of P is 0 and Pr(S_t =2|S_t _{− 2} = 2) = 1 so that once in state 2, the Markov chain remains in this state and cannot revisit State 1, thereby enforcing the sequential ordering of the change-point problem.

Let the observations of counts for week t be yt. Conditioned on the states across time, the model is then

y_{t} | S_{t} = k \sim Poisson (λ_{k}), k = 1, 2

λ_{k} \sim G (a, b), k = 1, 2

P_{11} \sim Beta (1, 1),

where Poisson(λ) denotes the probability mass function for a Poisson random variable with rate parameter λ. Beta(1, 1) denotes the beta density with parameters (1, 1). This is an uninformative prior density implying we have no useful prior knowledge concerning the transition probabilities or the position of the change point. G(a, b) denotes the gamma density with parameters a and b. We follow the method of Martin, Quinn, and Park (2011) by using an empirical Bayes approach and set the mean of the prior gamma density to be equal to the average of the entire sequence of observations in the case of the new and revised procedures, such that a = $\bar{y}$ and b = 1.

The marginal posterior distribution Pr(S_t = k|Data) and the marginal posterior densities p(λ₁|Data) and p(λ₂|Data) are easily obtained from a Markov chain Monte Carlo (MCMC) scheme and are used for inference in this article. For more technical details of MCMC construction for this model, the reader is directed to Chib (1998). However, an MCMC scheme for this model has already been made publicly available in the MCMCpack package, in the open-source statistical software R by Martin et al. (2011). The below syntax describes how to implement the model in our case and produce the plots and summary statistics discussed in this manuscript.

Acknowledgements

This project was supported under the Australian Research Council’s Linkage Projects funding scheme. John Cordery, Christine Soo, and Cristina Gibson received project funding from the Australian Research Council (Australian Government National Competitive Grants Program).

References

Adler

P. S.

Borys

1996. Two types of bureaucracy: Enabling and coercive. Administrative Science Quarterly, 41: 61-89.

Armenakis

A. A.

Bedeian

A. G.

1999. Organizational change: A review of theory and research in the 1990s. Journal of Management, 25: 293-315.

Bechky

B. A.

2006. Gaffers, gofers, and grips: Role-based coordination in temporary organizations. Organization Science, 17: 3-21.

Bertels

H. M. J.

Kleinschmidt

E. J.

Koen

P. A.

2011. Communities of practice versus organizational climate: Which one matters more to dispersed collaboration in the front end of innovation? Journal of Product Innovation Management, 28: 757-772.

Bresman

Zellmer-Bruhn

2013. The structural context of team learning: Effects of organizational and team structure on internal and external learning. Organization Science, 4: 1120-1139.

Brown

J. S.

Duguid

1991. Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2: 40-57.

Brown

J. S.

Duguid

2001. Knowledge and organization: A social-practice perspective. Organization Science, 12: 198-213.

Bundersen

J. S.

Boumgarden

2010. Structure and learning in self-managed teams: Why “bureaucratic” teams can be better learners. Organization Science, 21: 609-624.

Carlin

B., P.

Gelfand

A., E.

Smith

1992. Hierarchical Bayesian analysis of changepoint problems. Applied Statistics, 41: 389-405.

10.

Chib

1998. Estimation and comparison of multiple change-point models. Journal of Econometrics, 86: 221-241.

11.

Cordery

J. L.

Soo

Kirkman

B. L.

Rosen

Mathieu

J. E.

2009. Leading parallel global virtual teams: Lessons from Alcoa. Organizational Dynamics, 38: 204-216.

12.

Dibble

Gibson

C. B.

2013. Collaboration for the common good: An examination of challenges and adjustment processes in multicultural collaborations. Journal of Organizational Behaviour, 34: 764-790.

13.

Dubé

Bourhis

Jacob

2005. The impact of structuring characteristics on the launching of virtual communities of practice. Journal of Organizational Change Management, 18: 145-166.

14.

Dyer

Shafer

R. A.

2003. Dynamic organizations: Achieving marketplace and organizational agility with people. CAHRS Working Paper Series, Paper 27. Ithaca, NY: Cornell Center for Advanced Human Resource Studies.

15.

Gibson

C. B.

Dibble

2013. Excess may do harm: Examining the diminishing returns of external adjustment in team-based collaborations. Organization Science, 24: 687-715.

16.

Gibson

C. B.

Gibbs

J. L.

2006. Unpacking the concept of virtuality: The effects of geographic dispersion, electronic dependence, dynamic structure, and national diversity on team innovation. Administrative Science Quarterly, 51: 451-495.

17.

Gibson

C. B.

Huang

Kirkman

B. L.

Shapiro

2014. Where global and virtual intersect: The value of examining both in 21st century teams. Annual Review of Organizational Psychology, 1: 217-244.

18.

Gibson

C. B.

Vermeulen

2003. A healthy divide: Subgroups as a stimulus for team learning behavior. Administrative Science Quarterly, 48: 202-239.

19.

Gibson

J. W.

Tesone

D.V.

2001. Management fads: Emergence, evolution, and implications for managers. Academy of Management Executive, 15: 122-133.

20.

Grant

Wall

T. D.

2009. The neglected science and art of quasi-experimentation: Why-to, when-to, and how-to advice for organizational researchers. Organizational Research Methods, 12: 653-686.

21.

Haas

2010. The double-edged swords of autonomy and external knowledge: Analyzing team effectiveness in a multinational organization. Academy of Management Journal, 53: 989-1008.

22.

Hemasi

Csanda

C. M.

2009. The effectiveness of communities of practice: An empirical study. Journal of Managerial Issues, 21: 262-279.

23.

Hienerth

Lettl

2011. Exploring how peer communities enable lead user innovations to become standard equipment in the industry: Community pull effects. Journal of Product Innovation Management, 28:175-195.

24.

Kane

A. A.

Argote

Levine

J. M.

2005. Knowledge transfer between groups via personnel rotation: Effects of social identity and knowledge quality. Organizational Behavior and Human Decision Processes, 96: 56-71.

25.

Kirkman

B. L.

Cordery

J. L.

Mathieu

J. E.

Kukenberger

Rosen

2013. Nationality diversity and organizational community of practice performance: The moderating effects of psychological safety and media richness. Human Relations, 33: 333-362.

26.

Kirkman

B. L.

Mathieu

J. E.

Cordery

J. L.

Rosen

Kukenberger

2011. Managing a new collaborative entity in business organizations: Understanding organizational communities of practice effectiveness. Journal of Applied Psychology, 96: 1234-1245.

27.

Kogut

Zander

1992. Knowledge of the firm, combinative capabilities, and the replication of technology. Organization Science, 3: 383-397.

28.

Koliba

Gajda

2009. “Communities of practice” as an analytical construct: Implications for theory and practice. International Journal of Public Administration, 32: 97-135.

29.

Langley

Smallman

Tsoukas

Van de Ven

A. H.

2013. Process studies of change in organization and management: Unveiling temporality, activity, and flow. Academy of Management Journal, 56: 1-13.

30.

Lave

Wenger

1991. Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.

31.

Lee

L. L.

Neff

2004. How information technologies can help build and sustain an organization’s CoP: Spanning the socio-technical divide? In Hildreth

Kimble

(Eds.), Knowledge networks: Innovation through communities of practice: 165-183. London: Idea Group.

32.

Lesser

E. L.

Storck

2001. Communities of practice and organizational performance. IBM Systems Journal, 40: 831-841.

33.

Majchrzak

Jarvenpaa

S. L.

Hollingshead

A. B.

2007. Coordinating expertise among emergent groups responding to disasters. Organization Science, 18: 147-161.

34.

Marshall

2011. Images of changing practice through reflective action research. Journal of Organizational Change Management, 24: 244-256.

35.

Martin

A. D.

Quinn

K. M.

Park

J. H.

2011. MCMCpack: Markov chain Monte Carlo in R. Journal of Statistical Software, 20: 1-22.

36.

McDermott

Archibald

2010. Harnessing your staff’s informal networks. Harvard Business Review, 88: 82-89.

37.

R Development Core Team. 2013. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.

38.

Raftery

A. E.

1994. Change point and change curve modeling in stochastic processes and spatial statistics. Journal of Applied Statistical Science, 1: 403-424.

39.

Raven

2003. Team or community of practice: Aligning tasks, structures, and technologies. In Gibson

C. B.

Cohen

S. G.

(Eds.), Virtual teams that work: Creating conditions for virtual team effectiveness: 292-306. San Francisco: Jossey-Bass.

40.

Saint-Onge

Wallace

2003. Leveraging communities of practice for strategic advantage. New York: Butterworth Heinemann.

41.

Scarso

Bolisani

Salvador

2009. A systematic framework for analysing the critical success factors of communities of practice. Journal of Knowledge Management, 13: 431-447.

42.

Szulanski

1996. Exploring internal stickiness: Impediments to the transfer of best practice within the firm. Strategic Management Journal, 17: 27-43.

43.

Tannenbaum

S. I.

Mathieu

J. E.

Salas

Cohen

2012. Teams are changing: Are research and practice evolving fast enough? Industrial and Organizational Psychology, 5: 2-14.

44.

Vera

Crossan

2005. Improvisation and innovative performance in teams. Organization Science, 16: 203-224.

45.

Wenger

1998. Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press.

46.

Wenger

McDermott

Snyder

W. M.

2002. A guide to managing knowledge: Cultivating communities of practice. Boston: Harvard Business School Press.

47.

Wenger

Snyder

W. M.

2000. Communities of practice: The organizational frontier. Harvard Business Review, 88: 139-145.

48.

Western

Kleykamp

2004. A Bayesian change point model for historical time series analysis. Political Analysis, 12: 354-374.

49.

Willem

Buelens

2009. Knowledge sharing in inter-unit cooperative episodes: The impact of organizational structure dimensions. International Journal of Information Management, 29: 151-160.

50.

Yamklin

Igel

2012. Communities of practice purposely designed for improving business performance. Knowledge and Process Management, 19: 189-202.

51.

Zboralski

Salamo

Gemuenden

H. G.

2006. Organizational benefits of communities of practice: A two-stage information processing model. Cybernetics and Systems: An International Journal, 37: 533-552.

52.

Zollo

Winter

S. G.

2002. Deliberate learning and the evolution of dynamic capabilities. Organization Science, 13: 339-351.

53.

Zyphur

M. J.

Oswald

F. L.

2015. Bayesian estimation and inference: A user’s guide. Journal of Management, 41: 390-420.