The Science of Organizational Change and the Art of Changing Organizations

Theory

There is nothing as practical as a good theory. (Kurt Lewin, 1946)

Many years ago, Lewin made his own attempt to bridge the gap between our science and our art by reminding his colleagues that theories have immense practical utility in the world of action. The psychological sciences posit that people carry implicit theories in their minds concerning almost all aspects of their reality and their behavior is based (at least in part) on these theories or “understandings” about how things are related to each other, how the world works, why others behave as they do, and so on (e.g., Frith & Frith, 2012; Holyoak & Cheng, 2011). In everyday language, we have developed habits of using the word theory in a manner far different from what is intended in the sciences. For example, it is common to hear people say things such as, “That sounds good in theory, but it will never work in practice,” or responding to some assertion from another, “What you say is just ‘theory’ not proven fact.” When used in everyday speech in this fashion, there is an implied separation between theory and reality as though they were intended to exist in different worlds. This is harmless enough as far as it goes. It becomes less harmless in the hands of OD professionals, change agents, organizational scientists, doctoral students, and others who should know better. For years, I have railed at my graduate students, in curmudgeonly fashion, with regard to understanding the role of theory and applying it appropriately in the organizational sciences in general and the change arena in particular.

As noted above, only in the world of common speech (and, sadly, sometimes in the world of university students) is theory considered to be something separate from reality. In science, theory is always of reality. That is, a theory is simply our best current explanation of some particular phenomenon and/or a summary of knowledge about some aspect of our reality. The great theoretical physicist, Stephen Hawking, beautifully and succinctly captured the relationship between a theory and the reality it is designed to explain. Dr. Hawking has visited my university on several occasions. When asked how close he thought his theory of black holes was to the actual reality of black holes, Hawking was observed to hesitate before answering the question. He eventually stated that he had difficulty responding to a question phrased in that manner. He explained, “I have no concept of reality apart from my theory.”

But Hawking goes further in his formal writing: He argues that it is impossible to have a concept of reality that is independent of theory (Hawking, 1994; Hawking & Mlodinow, 2010)—there is no such thing (in his view) as a “model-independent reality.” All understanding of reality, thus, can be said to be dependent on the model used to explain or describe it. Hawking argues that, for example, two theories might offer their own versions of the same reality. “Model-dependent realism” means that this is acceptable as long as their predictions agree whenever they can both be applied (Hawking & Mlodinow, 2010). Model-dependent reality has profound implications for both our science and our art. For example, in both the social and organizational sciences there is a plethora of theory that has been advanced to explain almost every phenomenon of any significance. New graduate students may be dismayed by these multiple explanations. “Don’t tell me about ten theories of motivation,” the MBA student says, “Just tell me which one of them works.”

The value of the concept of model-dependent reality to our field can readily be seen. For example, Jean Bartunek and I recently used this concept (Woodman & Bartunek, 2013) to explain why a selection of articles published together as a collection (i.e., Oreg, Michel, & By, 2013), while seemingly wildly disparate in focus and treatment of a set of explanatory variables, is more coherent than it appeared at first blush. In the articles we were commenting on, it was quite noticeable how the same construct was, in one article, an independent variable while being used as a dependent variable in another, followed by a third article where the same explanatory construct was now advanced as a moderator or mediator. Drawing on Hawking’s perspective, we argued that this disparity spoke to the perceived importance of the construct to the change arena and the possible complexity and richness of the phenomena under discussion. Furthermore, as long as the theories being advanced did not disagree in their predictions each could capture some aspects of a particular reality. However, it may require some art to sort through the differing perspectives and to appreciate how they complement and reinforce one another, coupled with our science to move on from an inadequate theory to a superior one.

A fundamental duality exists within change theory. In a seminal paper, Porras and Robertson (1987) suggested that theories within the OD arena fell into a dichotomy of “change process theory” and “implementation theory.” As is well known, change process theories attempt to explain the dynamics of organizational change processes by identifying the manipulated variables in the change effort, the intended outcomes of the change program, and the relationships between manipulated and outcome variables including mediators—the mechanism through which the changes are effected. Change process theory also typically includes variables that moderate these relationships. Implementation theory, in contrast, is focused on strategies, procedures, and change techniques—in other words, the specific activities that change agents engage in to create the desired changes in the organization. Change process theory is the scholarly side of OD, that which (when tested) generates fundamental knowledge of change phenomena and processes. Implementation theory reflects the practice side of OD. Both types of theories are crucial for the advancement of knowledge and the effectiveness of action based on that knowledge. Bridging this duality requires both our art and our science.

Evaluation Research in Organizational Change

A significant percentage of research studies in change and development, change management, and OD take the form of evaluation research—the assessment of an intervention or change program to determine its effectiveness. In addition, considerable research takes a more “basic” form in the sense of theory testing designed to develop insights into and advance theory about processes and dynamics of organizational change. Practically from the field’s beginnings, there have been expressed concerns about the quality of research in organizational change in general, and OD in particular. These concerns are certainly not unique to the change and development arena, and the broader organizations literature is full of similar concerns and debates. Indeed, I cannot name an area of the organizational sciences where similar concerns have not been expressed from time to time.

Importantly, scholars have not just voiced concern but have frequently engaged in various analyses designed to investigate and draw conclusions about research quality typically with the goal of encouraging the field to do a better job. A prime example of such analysis is provided by a line of inquiry focused on the possibility of a “positive-findings bias” in published studies of organizational change (e.g., Bass, 1983; Bullock & Svyantek, 1983; Terpstra, 1981; Woodman & Wayne, 1985). The notion of a positive findings bias is the concern that there might be an inverse relationship between the quality or rigor of an evaluation and the reported success of the intervention. That is, the less rigorously a change intervention is assessed, the more likely it will be reported to be successful. Results of this line of inquiry were mixed. On balance, the collected body of research in OD did not appear to be suffering from a positive-findings bias. On the other hand, contrasting the fundamental dichotomy (another duality!) of process versus technostructural interventions (cf., Friedlander & Brown, 1974), indicated that process work (e.g., team building, process consultation, survey feedback, etc.) was more susceptible to a positive-findings bias probably because of the heavier use of perceptual measures of change.

Another good example of concern about the quality of change research can be found in the classic alpha, beta, gamma change typology advanced by Golembiewski, Billingsley, and Yeager (1976), which received the Douglas McGregor Memorial Award for the “Best Paper” in JABS. As the years have gone by, references to this change typology have sometimes been a bit, to put it gently, inaccurate or misleading if not completely wrong. As such, it might be of value to revisit Golembiewski et al.’s (1976) contribution in the context of our discussion here. Alpha change was originally defined as variation in some state (or variable) given a “constantly calibrated measuring instrument” that can be related to a consistent conceptual domain. Beta change, in contrast, is variation in some state or variable “complicated by the fact that some intervals of the measurement continuum associated with a constant conceptual domain have been recalibrated” (p. 135). In other words, while the conceptual domain remains the same for the change respondent or recipient, he or she has “recalibrated” the scaling of the measure being used. For example, the same level of cooperation or teamwork in the group that was previously judged to be acceptable is now perceived to be inadequate, and as a result, a survey respondent would now provide a different “score” or assessment to what is, essentially, the same objective reality. Gamma change represents the most problematic condition in terms of assessing change in that the change “involves a redefinition or reconceptualization of some domain.” Here the change recipient has not simply redefined some measuring scale, but has, in effect, redefined the construct or frame of reference within which the construct is understood. For example, the construct of “team” has changed in the mind of the participant such that some “before” and “after” measure of collaboration or teamwork or effectiveness simply cannot be meaningfully compared. Golembiewski et al. (1976) further describe gamma change as a “basic redefinition of the relevant psychological space as a consequence of an OD intervention” (p. 138). Golembiewski et al. observed that most evaluation research in OD seems to be set up to examine only alpha change. As such, when beta or gamma changes occur following an intervention, there are serious implications for conclusions that might well be erroneously drawn. Golembiewski et al. (1976) stated: “An immediate payoff of making such distinctions is more definite reliance on existing research findings, whose interpretation is necessarily related to their underlying concept of change” (p. 133). This change typology has received quite a bit of attention from researchers over time (e.g., Armenakis, 1988; Terborg, Howard, & Maxwell, 1980; Zmud & Armenakis, 1978) and provides an excellent example of how quality concerns about change research have had a positive effect on research design and methodology.

The most recent example of concern about the quality of research in the change and development arena appeared in the first issue of JABS in 2014. Barends, Janssen, ten Have, and ten Have (2014b) raised questions about the quality of evidence that exists in the literature in support of the efficacy of organizational change interventions. They argued that, while many scholars have expressed concern about the quality of research in the field, these concerns have typically been raised and addressed in narrative review form rather than by systematic empirical analyses of the “quality” of the research itself.

Barends et al. (2014b) conducted an extensive examination of the change literature and selected (using well-reasoned criteria) some 563 studies for systematic examination. Their findings will not encourage change scholars to be sanguine about the research quality in change and development. For example, they reported that only 13% of the studies utilized control groups and only 11 (2%) of them involved the use of random assignment. The vast majority of examined studies had weak internal validity providing little confidence in their results. Importantly, they also reported that the overall quality of extant research appeared to be growing worse rather than better. For example, the proportion of “controlled” studies dropped from approximately 30% of all studies in the early 1980s to less than 5% by 2010. Barends et al. concluded that, in a cumulative sense, the internal validity of empirical work in the field is low. Indeed, Barends et al. (2014b) asserted that “scholars and practitioners should be skeptical regarding the body of research results in the field of organizational change management published to date” (p. 5). Importantly, the authors suggested a number of ways that improvements could be made. A full accounting of all aspects of the examined studies (variables, measures, etc.) is beyond the scope of the current article, so the reader is referred to Barends et al. (2014b) for the details of their analyses.

The topic of research quality in organizational change has been of significant interest to the field for a long time and, not surprisingly, the Barends et al. (2014b) findings quickly attracted considerable attention. In fact, the editor of JABS, William Pasmore, orchestrated a point–counterpoint exchange to discuss their study and its findings. Beer (2014), for example, argued that it is important to understand why weak research designs are used, not merely to lament their use. Among other ideas, Beer pushed for the use of more collaborative practitioner–researcher teams to strengthen research results in a wide variety of real-world settings. Schwarz and Stensaker (2014) criticized aspects of the Barends et al. (2014b) analysis, arguing that their approach takes too narrow a view of what constitutes evidence in evaluating change. Furthermore, Schwarz and Stensaker (2014) view the Barends et al. (2014b) study as operating from a negative perceptual bias that conflates an examination of how evidence is collected with the more fundamental issue of the quality of that evidence. While basically agreeing with the findings of the study, Woodman (2014) reminded the readers of JABS that the field actually has a long history of “self-examination” of its research results and methodology, most of which was overlooked by the current study. As do Barends et al. and the other commentators, Woodman suggested several ways that the field could improve the quality of its research. Most notably, he argued for a wider use of quasi-experimental design, a research approach that is seriously underutilized in field research in the organizational sciences (cf., Grant & Wall, 2009). Barends, Janssen, ten Have, & ten Have (2014a) had an opportunity to reply to the comments and concerns raised by the commentaries, and again, the reader is referred to the original articles for additional details. In passing, I would like to point out how useful and valuable such a detailed examination of the field’s research can be. Furthermore, that value is increased by such an exchange as we see JABS providing in this instance. This is in the finest tradition of JABS that, in its 50-year history, has frequently provided such point–counterpoint exchanges along with a number of special issues focused on topics that have served to advance theory, research, and practice in OD and organizational change.

Organizational Diagnosis

To this point, we have focused on the science and art of theory and research. Perhaps where the tensions lie in these arenas is clearer than it is in the area of practice. However, an aspect of practice where science and art come together is found in the “act” of organizational diagnosis. At some level of abstraction, the act of diagnosis almost seems a metaphor (or perhaps a “poster child”) for the essential necessity of blending science and art in our field.

Imagine a complex social system. Now imagine an “actor” in that system who is going to become an agent of change. How does such a thing occur? What conditions are necessary for the human being to imagine something that could be different from what currently exists and behave in such a way as to bring that about? Why would this individual even start down the mental path toward a possible change in the status quo? I argue that there is always a “diagnosis” whether or not the actor recognizes that is what is being done. This is true in the same sense that there is no such thing as a model-independent reality—there is always a framework, or template, or understanding, or “theory” if you will, underlying all action. In other words, the actor must develop some basis on which to behave and some rationale for doing so even if it is nothing more than the thought: “Things could be better.” Before our actor can make that judgment, she or he must have some concept of effective organizing, no matter how implicit or ill-formed, to which the current state of affairs can be compared. Without such, the status quo prevails.

At its most basic, diagnosis might be considered as the starting point to all organizational change. At the risk of seeming to slip something by the careful reader, let me point out how controversial the previous sentence is. Various schools of thought and approaches to organizational change take dramatically different positions on the role and importance of valid diagnosis in effective change. For example, the degree of employee involvement and collaboration in diagnosis varies from typically being quite high in OD approaches to change to much lower in change management approaches that emphasize the use of outside “experts” to guide (or in some cases to dictate) the change. In those circumstances, top management is likely to treat data collected in a diagnostic effort as proprietary with little sharing of such data or change plans with the rank-and-file in the organization. As is well known, OD approaches typically involve large numbers of employees in the diagnostic effort in terms of data collection, data sharing, and action planning. However, many change management approaches, even outside the rubric of OD, do consider valid diagnosis, in one form or another, to be a critical step in the change process. Most published models of the organizational change process will contain a “diagnosis” step or phase. The differences tend to involve who performs the diagnosis and who “owns” the data on which the planned change will be based.

There is, however, an even more fundamental difference in the perceived importance or emphasis on the act of diagnosis between the “new” dialogic OD and what is being called the “old” diagnostic OD (Bushe & Marshak, 2009, 2014). Dialogic OD is an exciting development in the field, and indeed Gervase Bushe and Bob Marshak received the Douglas McGregor Memorial Award for the best paper following their 2009 publication in JABS. The reader may notice that the 2009 article (Bushe & Marshak, 2009) was also followed by a valuable point–counterpoint exchange similar to the one reviewed earlier. As mentioned before, JABS has done a good job orchestrating such exchanges over the years.

Dialogic and diagnostic OD differ on a number of important dimensions including epistemology, data gathering, the focus of the change intervention, and the desired outcomes of this intervention (cf., Bartunek & Woodman, in press). A particularly sharp difference (to my mind) exists with regard to the focus of data gathering. Diagnostic OD has traditionally focused on gathering information about problems facing the organization, striving to develop an accurate picture of how these problems came to be, and the consequences of leaving the problems unresolved. Dialogic OD is not keen on focusing on organizational problems per se but rather, through conversation and dialogue, seeks to develop a “collective awareness of the multitude of perspectives and discourses at play in the system” (Bushe & Marshak, 2009). Dialogic OD, in contrast to diagnostic OD, views the diagnosis as less of a separate step or stage in the change process than as “meaning making” that occurs continuously throughout the change effort. One can observe the separate epistemological assumptions (e.g., positivist vs. constructionist) that are in play in the respective approaches or philosophy of diagnosis.

I find the discussions and arguments surrounding differences in diagnostic focus between diagnostic and dialogic OD to be somewhat reminiscent of a distinction drawn many years ago between fundamental strategies for changing complex human systems ( Chin & Benne, 1976). One strategy, known as the empirical-rational, is based on the premise that human actors are rational beings who will behave in their own self-interest if they are able to ascertain what that course of action consists of. Thus, the collection of empirical data to share among the actors is logical in a change effort, and once we all have the same “facts” on the table, so to speak, then reasonable people will adopt the same course of action or behavior. A second strategy, known as the normative-reeducative, does not deny the value of knowledge but suggests that it is insufficient to effect collective action. Human beings are guided in their choices of behavior by social meaning attached to the information that they have available. In other words, we could agree about the “objective” reality of some knowledge (“yes, we all agree that this value is a 5”) but disagree about how important that information is in terms of needed action and/or what it implies we should do about it. Values, attitudes, judgment, norms, expectations, past experience, and so on, all come into play, and an effective strategy for collective action must take such considerations into account. OD has long been considered as a normative-reeducative approach to changing organizations—one that builds on the empirical-rational, which the field considers necessary, but not sufficient, for behavior change. In like manner, we need both the science of organizational change and the art of changing organizations to succeed in creating more effective organizations. Organizational diagnosis, as is practically everything else that we do in an applied behavioral science, is both an art and a science. Perhaps diagnostic OD leans a little more toward the science; perhaps dialogic OD leans a little more toward the art. Perhaps.

Organizational Creativity

In addition to the broad arenas of theory, research, and practice, one can find both science and art in specific areas of study or lines of inquiry in the field. A good example is found in the topic of organizational creativity. Organizational creativity is commonly defined as the creation of valuable new products, services, ideas, processes, or procedures by individuals working together within complex social systems (cf., Amabile, 1988; Woodman, Sawyer, & Griffin, 1993). There is a natural linkage between creativity and change, and at some level of abstraction, creativity can be considered as a “special case” of organizational change (Woodman, 2008). Underlying processes of creativity and change in organizations are quite similar on some dimensions. For example, studies that investigate factors related to resistance to change (Cummings & Worley, 2015, pp. 183-184; Piderit, 2000) and studies devoted to understanding barriers to creativity (Kilbourne & Woodman, 1999) have been found to identify many of the same explanatory factors in terms of individual differences, situational antecedents, and the like.

The tension between science and art can be seen even within basic definitions of creativity. There are a plethora of definitions of creativity or creative behavior in both the psychological and organizational literatures, which often involve multiple dimensions. However, these definitions commonly deconstruct into two basic dimensions that are always present: (1) originality or invention and (2) value or utility. The science in the first dimension seems clearer—when measuring creativity perhaps we simply need some “counting” mechanism. “Organization A produced 27 new products during the second quarter last year.” “Team C developed six changes in their work processes that had never been used before, and this was more than any other production team in the facility.” And so on. What is commonly recognized is that the second dimension causes the most construct validity issues in creativity research as assessing value or usefulness is often quite problematic. This is basically for two reasons—it is difficult to make this assessment without some value judgment, and (particularly organizationally relevant) we often have to make a projection or judgment about an unknown future. Indeed, Csikszentmihalyi (1990) argued that “creativity is not an attribute of individuals but of social systems making judgments about individuals” (p. 198). It seems that there is science (originality) and art (value) embedded within the very definition of creativity.

I have advocated a deeper exploration of the links between organizational creativity and organizational change (cf., Woodman, 2008) and believe that these two literatures could inform each other to great benefit. Let me provide an example of how that might be done, specifically how creativity theory might inform change theory.

Milgram (1990) developed theory designed to explain the abilities that contribute to individual creativity. He identified the four categories of “giftedness” shown in Table 1. General intellectual ability, in this context, refers to the ability to think abstractly and to solve problems logically. Specific intellectual ability is intellectual capacity in a specific area such as music or mathematics. General creative thinking refers to the ability to generate high quality, unique solutions to problems. Specific creative talent identifies the ability to generate original, valuable products or ideas in specific domains such as art, physics, or business. As shown in Table 1, each of these categories can be conceptualized as having an organizational analogue. For example, the general capacity of the organization to change provides an organizational analogue for the first category, while the capacity to change in specific ways is analogous to the second. In like manner, Category 3 represents a general organizational capacity to create, while Category 4 reflects the ability of the system to create in specific domains.

OPEN IN VIEWER

Table 1.

Categories of Giftedness and Their Organizational Analogues.

Milgram’s (1990) individual categories	Organizational analogue of giftedness
1. General intellectual ability	General capacity to change, adapt
2. Specific intellectual ability	Capacity to change in specific ways
3. General creative thinking	General capacity to create
4. Specific creative talent	Capacity to create in specific domains

Source: Adapted from Woodman, 2008, p. 287.

Milgram’s (1990) model of giftedness is more complex than the single dimension shown in Table 1 as he included ability levels and different types of learning environments. His goal was to create a model of intelligence and creativity that would be generative of theory and research as well as of practical value in an educational sense. In a similar vein, the notions contained in the organizational analogues shown could have both heuristic and practical value as they indicate potentially meaningful and logical linkages between change and creativity. The general capacity to change is a common change target and will be a function of the organizational culture, flexible designs and procedures, slack resources, time availability, managerial and change leader talent, and so on. The capacity to adapt in specific ways (e.g., redesign jobs, design and implement a new appraisal system, improve team decision making, etc.) requires domain-specific skills and knowledge in addition to supportive context (culture, structure, procedures) and other resources. Similar assertions can be made about a general organizational capacity to create and an ability to create in specific domains of organizational functioning. Managers and change agents often think about building both general and specific change capacity into their organizations but seldom consider the importance of building creative capacity into the system. There is some evidence, both empirical and logical, that organizations not only must have the capacity to change in order to thrive but also sometimes (perhaps more frequently than not) need the ability to create and innovate (Woodman et al., 1993).

Closing Comments

I have reflected on the science and art inherent in organizational change theory, research on organizational change, and the practice of OD and other forms of change management. Further, I briefly explored one specific research and writing domain (organizational creativity) as an example of how science and art interface in many specific areas of study. In addition, I have attempted to draw some attention to the contributions of JABS in the theory, research, and practice arenas. The collective contributions of JABS to our field over the past 50 years in these domains are without equal in the change and development field.

This article is far from a value-free or unbiased look at the importance of both our science and our art. Both are essential for the continued health of the field and the advancement of OD, lest someday the ubiquitous announcements of its death actually become true.

. . . OD appear(s) to be on its “second cat.” (Bartunek & Woodman, 2012, p. 728)