Designed Organizational Search: A Comparative Analysis of Alternative Procedures for Learning from Success

Abstract

Sampling on the dependent variable is unlikely to be an effective way to learn and develop the strategy. Even so, organizations spend millions of dollars on processes such as Appreciative Inquiry that make inferences about how to adapt their strategies, routines, and practices based upon only successful examples. Two techniques that are common to this kind of learning process are searching solely for successful solutions and reframing search problems (e.g., unconditionally positive questions). We build a computational model by formalizing appreciative inquiry and comparing it with other, similar processes to understand their relative effectiveness. We find that the organizations simulated in our computational model almost always improved performance over time, despite learning solely from successful observations. Their relative effectiveness depended on the complexity of the problems, the number of iterations of learning, and how much the learning process preserved variety in potential solutions. These findings suggest that appreciative inquiry may be most effective when people take the cost and complexity of organizational problems into account before engaging in the learning process and adapt the process accordingly. These findings contribute to research on organizational learning by explaining why learners may benefit from structuring the way they communicate as they search, why reframing performance measures may dissolve search problems, and how designed organizational search enables managers to be more deliberate about organizational learning.

Keywords

organizational learning appreciative inquiry success positive organizational scholarship strategy formulation computational modeling

Organizational search, the process of seeking solutions to existing or anticipated problems (Argote et al., 2021), can take very different forms. Consider a medical device manufacturing company that seeks to develop a strategic capability for introducing innovative products into the market. It could try to develop this capability by selecting a random sample of attempts at product innovation; collecting data regarding the context and outcomes of these attempts; analyzing the data to see which factors have the greatest impact and using the factors that have the greatest impact to create strategies, routines, or practices for introducing innovative products. Alternatively, the company could ask employees to share stories about only the strategies, practices, and routines that have generated their most innovative products; identify themes across these stories; use these themes to create new strategies, routines, or practices; and perhaps repeat this process periodically to make further improvements in their capability for innovation. Although firms sometimes engage in the second approach (Bushe & Kassam, 2005; Cooperrider & Whitney, 2005; Eisenberg, 2011), scholars tend to discourage approaches to search that sample on the dependent variable, pointing out its many shortcomings (e.g., Denrell, 2003; Freeman & Hannan, 1983; Rosenzweig, 2009).

Sampling on the dependent variable occurs when organizations make inferences about what factors lead to desired outcomes using only observations or experiences in which those outcomes were achieved. However, samples in which all of the observations achieve the desired outcome do not accurately represent their underlying populations (e.g., Argote et al., 2021; Denrell, 2003; Rosenzweig, 2009) and as a result, conclusions drawn from such a sample tend to be biased (Freeman & Hannan, 1983) are unlikely to be generalizable (Christmann, 2000), and may hurt rather than help that organization (Denrell, 2003; Rosenzweig, 2009). For example, an organization that makes inferences about how to innovate based solely on cases of successful innovation may be surprised that they are not able to replicate the desired outcomes because they failed to account for factors that would have surfaced with a broader sample (e.g., specific resources, timing, or human error).

Nevertheless, despite the limitations of sampling on success, organizations often use designed processes for seeking solutions to existing or anticipated problems that draw inferences from samples that only include successful cases, encoding their inferences into strategies, routines, and other courses of action. Sometimes they explicitly choose to adopt specific, formal processes for doing this. We refer to these as designed processes to contrast them with the common descriptions of search in organizational learning theory, which describes search processes as relatively automatic, reflexive, problem-driven efforts to find good-enough solutions. Designed search processes are intentionally mapped out from beginning to end with the goal of finding exceptional solutions. One prominent example of this type of search process is Appreciative Inquiry (AI; a method for change management developed in the field of organization development). Appreciative inquiry is a form of organizational search that elevates a system's capacity to learn from successes and apply its strengths and to achieve desired outcomes (Cooperrider & Fry, 2020) and has been used to make strategic changes in organizations such as John Deere, Verizon, British Airways, NASA, and the United Nations (Bushe & Kassam, 2005; Cooperrider & Whitney, 2005). In many well-established and successful organizations, leaders spend millions of discretionary dollars and devote substantial resources to sampling-on-success processes such as AI.¹ Given what we know about sampling on the dependent variable, is this process mass folly, or is there more involved than appears on the surface?

To examine a designed organizational search that samples the dependent variable, we model AI, a well-known organizational change intervention that seeks to learn solely from success (Cooperrider et al., 2005). Our intention is not to advocate for AI over other search processes. Rather, we seek to gain practical insights about what makes a process such as AI effective and to generate theoretical insights by identifying the basic, abstract components of this search process and comparing it with other search processes that also learn solely from success, albeit in different ways.

In their review of the field of organizational search, Posen and colleagues (2018) identified one of the shortcomings of current research as an overly simplistic view of search that treats the search process as a black box, largely examining only antecedents and outcomes. Organizational search is conceptualized as overly routinized and assumes a high degree of automaticity (e.g., Greeve, 2003). These conceptualizations of search do not reflect the choices that organizations can make in selecting a search process and the choices they can make during a search process. Our paper helps address these shortcomings by relaxing assumptions of automaticity and unpacking the so-called black box by comparing three different processes for seeking solutions. In addition, while studies have contrasted learning from success versus learning from failure (e.g., Deichmann & Ende, 2014; Greve & Seidel, 2015; KC et al., 2013; Lapré, & Cravey, 2022; Madsen & Desai, 2010), we are not aware of any studies that have compared different processes for learning from success. By comparing different forms of learning from success, organizations can make better decisions about how and when to apply a designed search that focuses on learning from success. Finally, scholars have suggested studying the process of search as an evolutionary approach, helping us further understand how organizations learn from success (Lee et al., 2020); this study includes such an approach as one of the analyzed processes.

Specifically, the search processes to which we compare AI in this study are those of natural selection and regression analysis of samples using only successful observations. Natural selection is an evolutionary process of success-oriented learning because at the population level, only the fittest survive natural selection, and fitness is a type of success. We modeled natural selection as a model of success-based learning using the genetic algorithm, which is also employed in other computational models of organizational learning (e.g., Bruderer & Singh, 1996; Crowston, 1996). For learning based on regression, we also modeled success-based search as a regression with samples that included only successful observations and, for comprehensiveness, also included a model of regression using random samples as well. Regression, a statistical method of calculating the relationships between independent and dependent variables, used commonly in data analytics and data mining within organizations, provides a useful baseline for comparison. We did not examine every detail of these differing models of learning from success. Rather, our goal is to acquire insights by comparing the general effectiveness of different structured approaches to success-based search.

Using our model to compare three forms of success-driven organizational search, we found that the relative effectiveness of each form of search depended on the complexity of the problems, the number of iterations of learning, and how much the learning process preserved variety in potential solutions. Surprisingly, our findings support the notion that AI may be the most effective form of search when organizations account for the cost and complexity of the problem. Our findings also contribute to research on organizational learning by explaining why reframing performance measures may dissolve search problems, and how designed organizational search enables organizations to be more deliberate about learning. We found that if managers reframe the performance measure that they want to improve to make it broader and more positive, they will often improve the performance of the old performance measure as well as the new performance measure. These findings respond directly to Posen and colleagues’ (2018) call for research that questions assumptions about the automaticity of the search process. Using these findings, we also make five contributions to the theory of organizational search. First, we explain when and why search processes that focus solely on success enable organizations to improve performance despite their members’ bounded rationality. Second, we show how proactively selecting broader and more positive types of performance improves narrower and less positive forms of performance as well. Third, we use the results to demonstrate how conversational structure influences search effectiveness. Fourth, we explain when and why the sacrifice of short-term improvements enables organizations to reap the benefits of learning solely from success. Finally, we illustrate how performance improvements may depend more on iterative learning than on the way people make inferences.

We begin by situating the concepts of designed search, success-based search, and proactive reframing more fully in the literature on organizational search and introduce AI as an appropriate process to model. We then describe how we operationalized AI as a computational model and compare it to the genetic algorithm and regression-based search. We then present the results of our simulation analysis, concluding with an explanation of how our results influence the theory about organizational search and learning.

Theoretical Background

Early organizational learning scholars assumed that actors are boundedly rational and take a largely behavioral approach to organizational search (Cyert & March, 1963). As such, they argued that (1) organizational search occurs when performance falls below aspirations (problemistic search), (2) they search for solutions that are similar to previous solutions before examining dissimilar solutions (local search), and (3) they usually adopt the first solution that solves the problem rather than trying to find optimal solutions (satisficing). As such, organizational search is generally conceptualized as a reactive process that minimizes cognitive effort (Posen et al., 2018). Using assumptions such as these, scholars have developed models that explained either flawed organizational decision-making (e.g., Cohen et al., 1972) or how organizations were able to perform well in spite of these limitations (e.g., Cohen, 1981).

Eventually, scholars introduced more complex assumptions into their models, such as organizational structures and cognition (e.g., Billinger et al., 2014; Gavetti, 2005: Joseph & Gaba, 2020; Nigam et al., 2016), such as simplified mental representations of potential solutions (e.g., Csaszar & Levinthal, 2016), or preferred directions in which actors think they should search (e.g., Winter et al., 2007). Scholars have introduced cognition into accepted search processes and then examined the impact of these changes (e.g., Csaszar & Levinthal, 2016; Winter et al., 2007).

Designed Organizational Search

However, cognition also occurs when organizations deliberately decide on the specific process for organizational search. For example, when managers encounter problems, rather than immediately searching for solutions in an autonomous or reflexive way, they may instead hire consultants who persuade them to use specific, designed search processes (e.g., Bogan & English, 1994; Lilien et al., 2002). People design search processes using scientific research and professional experience to guide their design choices. When organizations adopt designed search processes, they may shop around, selecting from an array of search processes before ever selecting from an array of potential solutions. This approach is still boundedly rational, but it is also more deliberate than most of the other existing models of organizational search.

Some designed organizational processes differ from the predominant/current models of organizational search in the literature by prescribing reframing a goal or a problem (e.g., Ackoff, 1981; Cooperrider et al., 2005; Fritz, 1989). Reframing a goal or a problem introduces cognition and choice into search models that conceptualize organizational search as automatic. Reframing is important because it can change the array of potential solutions. For example, when Roberto Goizueta and his colleagues changed Coca-Cola's frame from “How do we beat Pepsi?” to “What is our share of the stomach?” the array of potential solutions available to Coca-Cola expanded from those involved with producing and selling carbonated sodas to those involved in the production and sale of all nonalcoholic drinks (Charan & Tichy, 1998). The second set of strategic choices included, but was also much larger than, the more constrained conception of Coke's strategy. Goizuetta and his colleagues may not have known what all of the possible solutions were in either frame—they were still boundedly rational—but the array of possible solutions expanded no matter what mental representations they built upon. This distinction is crucial, because the idea of changing the array of possible solutions is highly relevant to strategic management since premeditated approaches to performance improvement involve managerial discretion and strategic choice (Hambrick et al., 2004; Montenari, 1978; Wangrow et al., 2015).

Another feature of designed organizational search processes is that many of them seek to learn solely from successful solutions, and some of them do so intentionally (e.g., Bogan & English, 1994; Cooperrider et al., 2005; Pascale et al., 2010; von Hippel et al., 1999). Learning from success is a growing area of interest for organizational scholars (e.g., Deichmann & Ende, 2014; Gong et al., 2019; KC et al., 2013; Kim et al., 2009). When scholars have studied organizational learning they have often compared learning from success versus learning from failure. Comparing learning from success and failure is important because it helps understand the benefits of learning from success as compared to learning from failure. For example, learning from success versus learning from failure influences the premium a firm is willing to pay for future acquisitions (Gong et al., 2019), and the likelihood and effectiveness of sustained, radical initiative taking (Deichmann & Ende, 2014).

However, when comparing learning from success to learning from failure, scholars have neglected to acknowledge and examine different strategies for learning from success. Learning from success has been examined in a monolithic and sometimes generic form, ignoring the nuance of alternative learning strategies. For example, Kim and colleagues (2009) conceptualized learning from success as a firms’ cumulative history of periods of exceptionally strong performance. This is consistent with those that have studied learning from success at the individual level. Deichmann & Ende (2014) conceptualize learning from success as an individual's prior experience of success. From organizational practice and organizational learning literature, we know there are differing strategies for learning from success.

Many of these benefits and drawbacks were discovered by comparing learning from success against learning from failure or against learning from samples that include both success and failure (e.g., Deichmann & Ende, 2014; Gong et al., 2019; Heckman, 1979; KC et al., 2013; Kim et al., 2009). We know of no research that compares different approaches to learning from success. Some approaches may do a better job of using success’ advantages and avoiding its drawbacks than others. Designed organizational search processes give us an opportunity to compare and contrast different approaches.

Toward a Model of Designed Organizational Search: The Example of AI

An example of a type of designed organizational search that learns solely from success is AI. AI is often used by organizations around the world (Cooperrider et al., 2005), with some organizations even adopting it formally into their structure and routines (e.g., May et al., 2011). Other success-only search processes also exist (e.g., Lilien et al., 2002; Pascale et al., 2010) and could be examined as well. AI is commonly used and relies intentionally on the assumption that much of the world is socially constructed. Social constructionism assumes that some facts—such as the definition of an organizational problem or goal and the solutions that are relevant to that goal—depend as much—if not more—on the agreement of social actors as they do on physical reality (Berger & Luckmann, 1990). Other success-focused search processes also draw on social constructionism but do so less explicitly, so AI provides us with a useful source for building a model that includes problem reframing.

The first step in AI is the reframing of the search problem to be unconditionally positive. For example, if an organization has severe conflicts between management and labor, then a problemistic search will typically assume that the goal of a search may be to reduce severe labor conflicts. In contrast, AI users might reframe this problem as “How do we build ongoing, mutually-beneficial relationships between management and labor?” This new problem statement subsumes the original problem (you cannot have ongoing, mutually beneficial relationships if those relationships are characterized by severe conflict), but it is also broader, more positive, and more ambitious. Once people reframe the problem, the remaining steps include (a) collecting data about times when the new, positive goal was achieved (Discover); (b) inferring ideas from the data that can be used in other places throughout the organization (Dream); (c) coming up with specific experiments to try (Design); (d) implementing those experiments; and (e) learning from the implementation and repeating the process as appropriate (Destiny).

Practitioners and scholars make claims about the effectiveness of AI, but their evidence consists primarily of successful case studies (Fry et al., 2002; Ludema & Barrett, 2007). A review of these case studies compares more successful cases with less successful cases to identify some interesting patterns (e.g., Bushe & Kassam, 2005), but the lack of case studies that report unsuccessful efforts with AI limits our ability to make generalized inferences about its effectiveness as a search process. In contrast, a computational model can simulate AI under multiple conditions, and also compare it with other search processes that learn only from success.

There is no existing computational model of AI. To build such a model, we followed the example of scholars such as Sastry (1997), who build computational models by translating verbal descriptions of processes into formal mathematics and computational commands. The verbal description we used was the Appreciative Inquiry Handbook (Cooperrider et al., 2005), which may be the most accepted description of how to conduct an AI. In other words, we broke down the semantics of the Handbook to create a formal model of AI. However, similar to Sastry's (1997) formalization of organizational change as punctuated equilibrium (Tushman & Romanelli, 1985), verbal texts are seldom precise enough to make a complete formal model. Therefore, similar to Sastry, where the Handbook was insufficient, we used sensitivity tests, logic, and comparisons with descriptions of empirical processes to complete the model.

We began the process of formalization by reading through the Handbook, extracting statements about search, implementation, and feedback, which are categories of activity that scholars identify as necessary for learning (e.g., Cyert & March, 1963; Zollo & Winter, 2002). Examples of statements that fit these categories include “… what are the best stories, practices and wishes that came out of the interviews?” (p. 98) and “Self-selected groups plan the next steps for institutionalization” (p. 176). The first statement describes feedback because identifying the “best” stories implies the assessment of past performance. The second statement describes implementation because it involves introducing new solutions into the organization. We excluded statements that described elements that do not contribute to learning, such as “begin with some kind of energizing activity” (p. 113). Energizing activities may help people to stay alert and engaged while they go through the search process, but they do not search for, implement, or generate feedback on solutions to search problems. Table 1 contains additional examples of statements we did and did not extract.

Table 1.

Text-Coding Categories and Examples*.

	Topic choice	Search	Implementation	Acquiring feedback	Excluded from analysis
Explanation	Descriptions of how to define or re-define what type of performance the learning process was trying to improve	Descriptions of how to discover courses of action that can be pursued as a means to improve a particular type of performance	Descriptions of how to put a given course of action into effect	Descriptions of how to gather data on the performance of a different courses of action	Text from the Handbook that is related to organizational learning, but does not define the basic steps of the learning process.
Examples	"… the heart and spirit of the AI process starts … with the crafting of engaging questions. The questions are fateful, in that they ultimately determine the change in direction for the organization” (88)	"A typical question asked is ‘how many interviews should we conduct?’ The answer: ‘Do as many as possible.’ Why? For one thing, it is clear that inquiry and change are a simultaneous event and the changes begin at the moment of inquiry” (93).	"Self-selected groups plan the next steps for institutionalization …” (176).	"… what are the best stories, practices, and wishes that came out of the interviews?” (98).	"The group creatively depicts what they have seen. The more playful the depictions, the better” (113).
	"State questions in the affirmative. … Give a broad definition to the topic. … Value ‘what is’" (88).	"… there must be a commitment to let go of the negative discourse categories…” (96).	"… the team … even created new concepts they believe captured the meaning of [their topic choice]. … Then the group took time to reflect on the list and make changes, additions, and alterations” (146).	"The goal is to identify themes in order to know how to do more of what worked well in a situation” (98).	"In the early years of AI work, the fourth D was called discovery” (177).

*Contains page numbers from the Handbook after each quote.

As we extracted statements, it became clear that reframing the problem—choosing the type of performance one wishes to improve, or “topic choice” as it was called in the Handbook—should be included as its own step in the learning process even though it has not been included in other scholarly models of organizational search (e.g., Cohen, 1981; Cyert & March, 1963; Knudsen & Levinthal, 2007; Rivkin, 2000; Siggelkow & Rivkin, 2006). For example, in problemistic search, performance measures are given because problems are predefined: a solution fails to achieve an aspiration level along an already-existing measure of performance. Thus, in these models, when performance fails, managers simply search for ways to improve performance as currently defined (Cyert & March, 1963). However, if managers change the search topic, or in other words, the type of performance they are trying to improve, that change in the topic also changes what kinds of solutions are relevant for inclusion in the search. These models take topic choice for granted.

Once we extracted statements from the Handbook, we examined them further to break them down into steps that were discrete enough to use in a computational model. Statements about topic choice implied two steps, as shown in Table 2. Step 1 is reframing performance (as suggested in the Handbook by statements about how topics “represent what people want to discover or learn more about,” p. 32). Step 2 is defining the types of solutions for which people are searching (suggested by statements such as “choice of topic defines the scope of the inquiry,” p. 32). Search and feedback broke into interrelated steps in Table 2. A good example of this interrelation comes from statements about activities such as interviewing stakeholders “from many levels and perspectives in the organization” (Cooperrider et al., 2005, p. 88), about “peak” experiences with the topic (p. 89), and about identifying “themes” in these peak experiences (p. 112). The first step of the search process involves activities such as drawing experiences “from many levels and perspectives.” We state this step abstractly in step 3 in Table 2 as “identifying possible solutions from which to learn.” The first step of the feedback process (step 4 in Table 2) is also implied in this description, requiring AI participants to select “peak” experiences when they talk about solutions. Stated abstractly, this involves “evaluating the performance of each solution.” Once the performance of each solution is known, the second step of the search process (step 5 in Table 2) can be completed: “selecting the highest performing solutions.” Participants do this when they focus on only the stories of peak experiences that they hear from each other. Within these peak experiences, participants can perform the second step in the feedback process (step 6 of Table 3) by “identifying information from within the peak experiences from which to learn.” They do this by identifying “themes” that the peak experiences have in common. Finally, we broke statements about implementation into three steps (steps 7, 8, and 9 in Table 2). Examples of such statements include the counsel to use the common themes identified in the feedback process as “design elements” (p. 143) for new solutions or the suggestion that “Individuals and small groups self-organize to implement the Design statements” (p. 176). The steps we derived from statements such as these included (1) removing lower-performing solutions, (2) inferring new solutions, and (3) altering inferences to fit the specific situations.

Table 2.

Translation of Relevant Handbook Descriptions into a Computational Model*.

Sample handbook quotes	Learning steps	Computational model
"…affirmative topic choices encourage people to select topics they want to see grow and flourish in their organizations” (p. 32).	1. Define performance	Goal: Y'_i(t+1) > Y'_i(t), given that Y'_i = g(x₁, x₂, …, x_N, x_N+1, x_N+2, …, x_N’)
"Careful, thoughful, and informed choice of topics defines the scope of the inquiry, providing the framework for subsequent interviews and data collection” (p. 32)	2. Define the performance function	g(Y, x₁, x₂, …, x_N, x_N+1, x_N+2, … x_N’) = ([Y ± Y * ƒ(x₁, x₂, …, x_N’)]/2) * (1.0 ± u), where K’ > K, and u is a random number.
"A typical question asked is ‘how many interviews should we conduct?’ The answer: ‘Do as many interviews as possible'“ (p. 93).	3. Identify an initial population of strings	Identify M strings (B_i), where B_i = [x₁ x₂ … x_N], and the x_i have values of 0 or 1.
"Describe a ‘peak experience’ or ‘high point'“ (p. 89).Specifically, what are the best stories, practices, and wishes that came out of the interviews? (p. 98).	4. Evalute each string's performance	Calculate Y'(B _i ) for each B _i .
	5. Select the highest-performing strings	Select M-R higher-performing B_is upon which to focus.
"…listen for information about what the structure was like, as well as the systems, rewards, and so on” (p. 96)."The goal is to identify themes in order to know how to do more of what worked well in a situation” (p. 98).	6. Identify information in the strings from which to learn	Compare each x _i acoss the M-R higher-performing strings. If more than P percent of the values are the same for a given x _i , record that value for that x _i .
"Specifically, what are the best stories, practices, and wishes that came out of the interviews?” (p. 98).	7. Information deletion	Delete the R lowest-performing strings.
"Move into groups [and] share fully what they have seen and imagined and … create a collective verbal picture of the desired future” (p. 113). “…images of the future emerge out of grounded examples from the positive past” (p. 39). "Self-selected groups plan the next steps for institutionalization and continued vitality” (p. 39).	8. Inference	Replace deleted B_is with B_is that have the recorded values of the x _i from step five, and the values of all other x _i are randomly generated with an equal chance of 0 or 1.
"Select Design Elements” (pp. 143).	9. Specific alterations	Based on q, a probability set before the simulation runs, the model makes specific, random, alterations to an x _i in one or more of the new B _i s.
"Destiny is full of continuing dialogue. …It is a time of continuous organizational learning, adjustment, and improvisation” (p. 113).	10. Repetition	Return to step #4 and repeat

*Contains page numbers from the Handbook after each quote.

Table 3.

Results of Paired Sample T Tests of a Smooth Solution Landscape (N = 10, K = 1) Versus a Rugged Solution Landscape (N = 20, K = 9), Comparing the Mean Performance of the Same Learning Procedure at the Same Number of Iterations.

Learning procedure and number of iterations	Difference in means	Lower bound of confidence interval	Upper bound of confidence interval	t	p
Genetic algorithm, one iteration	0.025	−0.004	0.054	1.70	.09
Random regression, one iteration	0.130	0.088	0.172	6.12	.00
Success regression, one iteration	0.108	0.067	0.149	5.28	.00
Appreciative inquiry, one iteration	0.086	0.052	0.120	5.02	.00
Genetic algorithm, five iterations	0.111	0.075	0.148	6.11	.00
Random regression, five iterations	0.135	0.093	0.177	6.37	.00
Success regression, five iterations	0.118	0.077	0.159	5.71	.00
Appreciative inquiry, five iterations	0.100	0.063	0.137	5.35	.00
Genetic algorithm, 100 iterations	−0.003	−0.045	0.039	−0.15	.88
Random regression, 100 iterations	0.138	0.097	0.180	6.59	.00
Success regression, 100 iterations	0.126	0.086	0.167	6.25	.00
Appreciative inquiry, 100 iterations	0.073	0.035	0.111	3.82	.00

Table 2 displays examples of quotations from the Handbook that we used to develop substeps of the learning process (column 1), the substeps as we arranged them in the model (column 2), and the formalization of these steps into the mathematics and computational commands (column 3). Mathematics is a language, so the formalization of these steps is a type of translation. For example, topic choice (row 1) involves identifying the things that people “want to see grow and flourish” (Handbook, p. 32), or in other words, a topic is a dimension of performance (Y_i, where Y_i = ƒ(x₁, x₂, …, x_N)), a new topic is a different-but-related dimension of performance (Y’_i, where Y’_i = g(x₁, x₂, …, x_N’)), and if people want more of that performance (“grow and flourish”) then they want Y’_i(t+1) > Y’_i(t). Similarly, instructions such as “Do as many interviews as possible” implies that there is a population of solutions that people have experienced (row 3) from which they can draw peak experiences and best practices (row 4). The final column in row 3 suggests that we can model solutions as strings (B_i) of zeroes and ones (x_i) with the values of the x_i in the initial strings being assigned randomly in the. Appreciative Inquiry participants must evaluate the performance of strings (i.e., solutions) if they are to describe a “peak” experience, so row 4 suggests that the computational model should use the formula in row 2 to calculate the performance of the strings in row 3. We provide a more comprehensive explanation of each step shortly.

In a few cases, as we translated from the verbal language of the Handbook, the description was not as precise as needed to express mathematically or there were multiple legitimate ways to perform a step. Also, some simplification was required to make the model comprehensible (which is true of any model). For example, one simplification is that the Handbook suggests having multiple, iterative conversations to identify themes in the experiences that people share and to use those themes to make plans. Rather than model every possible conversation, and preserve the essence of the process without creating code for every possible application of the procedure, we coded the outcomes of these conversations (such as the new topic, rather than the process for coming up with a new topic), which accomplishes the same purpose. In cases where the language was not precise, we built parameters into the model that could take on different values, and then conducted sensitivity tests using the different values. For example, the Handbook tells people to look for common themes across success stories, but it was not clear what made a theme “common.” Therefore, we treated commonality as the percentage of stories that contained the same value for a given theme. We simulated the model with multiple permutations of the various parameter values, including but not limited to the percentage of stories that contained the same value for a given theme. We found that if the value was much higher than 60%, then themes occurred too rarely to be useful for learning, and if the value was much lower than 60%, then every element became a theme. Therefore, when we conducted our experiments, we held the common percentage fixed at 60%. We examined each unclear parameter from the Handbook this way. Some parameters only worked at small ranges of values, so we picked a value in that range and used it in all simulations. Other parameters worked at multiple values, so we manipulated that parameter to have multiple values in our experiments. We identify fixed and experimental values in our description of the model.

Steps 1 and 2: Modeling Topic Choice

The model begins with topic choice. As row 1 in Table 2 shows, the first step of topic choice is transforming a problem into an unconditionally positive question. Typically, problems occur when some desired outcome did not turn out as hoped, so people want to improve their performance on this outcome (Cyert & March, 1963). The outcome can be anything that people want to improve, whether financial (e.g., profit, ROI), operational (e.g., productivity, efficiency), a vague-but-desirable outcome (e.g., innovation), or even a subjective outcome (e.g., inclusivity). We model the outcome as a dependent variable, Y. However, in AI, topic choice involves transforming problems (e.g., “How do we fix customer dissatisfaction?”) into unconditionally positive questions (e.g., “How could we inspire our customers to be intrinsically motivated to engage with us in our products’ development and promotion as well as in their consumption?”). Examples might be Coca-Cola ceasing its efforts to win tenths of 1% of share of the cola market from Pepsi (Y) in favor of expanding the market to include all nonalcoholic drinks (Y’) (Charan & Tichy, 1998), or Avon changing its goal of eliminating sexual harassment (Y) to creating high-quality, cross-gender professional relationships (Y’) (Cooperrider & Whitney, 2005).

We capture this transformation by modeling new dependent variables with Y’, recognizing that achieving Y’ typically may also involve achieving Y (e.g., the nonalcoholic drinks market includes colas, and people in high-quality, cross-gender, professional relationships do not engage in sexual harassment). This observation has at least two implications for modeling. First, the independent variables that affect Y are also likely to affect Y’, but because Y’ is a more encompassing goal than Y, there are likely to be more variables that affect Y’ than Y. We model this as follows: Y = ƒ(x₁, x₂, …, x_N), Y’ = g(x₁, x₂, …, x_N, x_N+1, x_N+2, …, x_N’), where N and N’ are the number of variables in each function. (For simplicity, we use only zeroes or ones for the values, consistent with many other computational models; e.g., Gavetti & Levinthal, 2000.) Second, causal relationships in Y’ are likely to be more complex than in Y (see row 2 of Table 2). We capture this complexity using an adapted version of Kaufman’s (1993) NK function to model performance as a landscape. This approach—common in models of organizational search (e.g., Knudsen & Levinthal, 2007; Rivkin, 2000)—treats Y and Y’ as altitudes in the landscape and x_is as landscape coordinates. A landscape is rugged when it has many peaks and valleys, which suggest that solutions with similar values in their x_is have dramatically different outcomes because of complex interactions between the x_is. Landscapes are less rugged when solutions with similar values in their x_is have similar levels of performance.

In an NK model, K refers to the number of other independent variables (not x _i ) that interact with a given independent variable (x _i ) to influence Y. For example, if N equals 5 and K equals 3, then the impact of x₁ on Y is also influenced by x₂, x₃, and x₄; the impact of x₂ on Y is also influenced by x₃, x₄, and x₅; the impact of x₃ is also influenced by x₄, x₅, and x₁; and so on. In this example, if the x_is in each solution (B_i) take on the values x₁ = 0, x₂ = 1, x₃ = 1, x₄ = 0, x₅ = 0, then the impact of x₁ on Y would also be influenced by x₂, x₃, and x₄ (i.e., 0|1 1 0). If K = 3, then there are 16 possible permutations of interactions that could define the impact of x_i on Y. Thus, once the computational model is told the value of K for a simulation run, it assigns each possible permutation of zeroes and ones a random decimal value as the impact of x_i on Y. For example, the permutation (0,0,0,0) may be assigned 0.243, the permutation (0,0,0,1) may be assigned 0.911, and so on. The overall performance of the solution is the average of the assigned values of each of the N x _i s. In NK landscapes, the higher K is, the more rugged the landscape will be.

We can use the NK model to describe topic choice by defining Y’ in terms of Y, as follows:

Y^{'} = [Y \pm Y * f (x_{1}, x_{2}, \dots, x_{N}, x_{N + 1}, x_{N + 2}, \dots, x_{N^{'}})] / 2

In this formula, f(x₁, x₂, …, x_N, x_N+1, x_N+2, …, x_N’) represents the NK procedure but involves N’ variables instead of N variables, and the question of whether Y*f(x₁, x₂, …, x_N, x_N+1, x_N+2, …, x_N’) is added to or subtracted from Y is a uniform 50% draw for each B_i’.

This formula meets our conditions that Y’ should subsume Y (it is partially a function of Y) but should also involve more variables and a more complex relationship between variables. Specifically, x₁ through x_N represent the same variables in Y and Y’, but x_N+1 through x_N’ are unique to Y’. The possibility of adding or subtracting the second term in the formula implies that changes in performance for a broader aspiration will affect performance in the less broad aspiration as well, sometimes for better, sometimes for worse. Multiplying Y by f(x₁, x₂,…,x_N’) ensures that the value that is added to or subtracted from Y will not exceed Y. Also, Y and Y* f(x₁, x₂,…,x_N’) both range from 0 to 1, so the lowest value produced by adding these numbers will not fall below zero or exceed two. We divide by 2 so that the value of the equation for Y’ has the same range [0, 1] as the formula for Y. Finally, the decimal score of Y’ will usually be a lower performance score for Y’ (because it involves multiplying decimals and dividing by two). Even though this score will be lower, it may represent a higher total performance in comparison to Y. For example, if Y = 0.600 and Y’ = 0.500, but the size of the cola market is measured in hundreds of millions of dollars and the size of the nonalcoholic drinks market was measured in billions of dollars, then Coca-Cola's performance in the cola market might be 0.600 * $100,000,000 = $60 million, while their performance in the non-alcoholic drinks market might be 0.500 * $1,000,000,000 = $500 million.

Steps 3 Through 6: Search and Feedback

The next four steps model the “Discovery” phase of AI. They involve searching for solutions and getting feedback on the courses selected (Cyert & March, 1963; Zollo & Winter, 2002). Step 3 involves identifying a sample of M solutions within which the organization will search. The computational model generates the initial version of this sample by using strings (B _i s) to represent stories/solutions, in which each string is composed of N x _i s (i = 1 to N) and each x _i is an element of that solution that is present when x _i = 1 and absent when if x _i = 0. For example, if Y represents customer satisfaction, and B _i s represents the solution used to deliver a product to customers, then the x _i s might represent elements such as statistical process control, just-in-time delivery, adaptation to local tastes, and so forth. The computational model assigns random values of zero or one to each of the x _i s.

The remaining three steps proceed as follows. In step 4, the model evaluates the performance of each B _i using the NK function. In step 5, the model identifies the highest-performing M-R of the B _i s identified in step 3. In step 6, the model compares each x _i across the M-R B_is to see whether more than 60% of each x_i have the same value (zero or one). If an x_i has the same value in more than 60% of the M-R B_is, the model records the value of that x_i, creating a template for making new solutions. In new solutions, some of the x_is are held constant and others are not, according to this template. For example, if the higher-performing B_is have five x_is each, x₁ had a zero in over 60% of the B_is, and x₄ had a one in over 60% of the B_is, then the template used for creating new B_is for the organization would be [x₁ = 0, x₂ = ?, x₃ = ?, x₄ = 1, x₅ = ?].

Steps 7 Through 9: Implementation

The model next begins implementation (Cyert & March, 1963; Zollo & Winter, 2002), which corresponds to the “Dream” and “Design” stages of AI. In the seventh step of Table 2, the model eliminates the R lowest-performing B _i s from the simulated organization. The Handbook does not state this step explicitly, but it is implied in the instructions and in the examples of AI projects.

The computational model creates R new solutions (B _i s) in the eighth step by replacing the ones that were removed. It does this by introducing values (0 or 1) into the x _i s that did not have consistent themes identified in step six. For example, if the template created was [x₁ = 0, x₂ = ?, x₃ = ?, x₄ = 1, x₅ = ?], then for each new B_i, the model would use a random uniform draw of 0 or 1 to select values for x₂, x₃, and x₅ for each B_i. For example, if the computational model replaces B₂, B₇, and B₁₀, then it might generate the strings B₂ = [0,1,1,1,1], B₇ = [0,0,1,1,0], and B₁₀ = [0,0,0,1,1] for implementation in the organization. This is consistent with the instruction from the Handbook to design solutions by grounding future solutions in peak examples from the past. We specify values for R at the beginning of each simulation.

After creating new solutions, the model may make another change in the ninth step. People participating in AI may alter specific elements in new solutions, perhaps because their understanding of specific organizational circumstances suggests poor perceived fit between an element of a solution and those circumstances, or perhaps because they are boundedly rational and make mistakes. Therefore, based on a probability (q) that is set before the model runs, the model makes a random, single-element alteration to one or more of the new solutions. For example, if q = 10%, then with 10% probability, the value of an element (e.g., x₄ in string B₂) may change (e.g., from one to zero).

Repetition

Finally, the computational model may repeat or not repeat the AI process, consistent with descriptions of the “continuous organizational learning, adjustment, and improvisation” (p. 113) of the Destiny stage of an AI. Some organizations only go through one learning cycle; others engage in a few cycles of learning; and others engage in the process indefinitely. For example, in one health system of which we are aware, the top management team created a “Center of Appreciative Practice” that was responsible for regularly executing many interventions as a way of creating an appreciative learning organization (see Cooperrider et al., 2005). In our computational model, if the learning process occurs more than once, subsequent iterations of the process begin at step 4 instead of step 1.

Assessing Effectiveness

One way to assess the effectiveness of AI as an organizational learning process is to examine how it behaves across performance landscapes of different sizes and complexities. When does it exhibit a general trend of performance improvement over time, and when does it not? Another way is to compare its performance with other learning processes. We used both approaches.

We compared AI to natural selection and statistical regression of successful observations because these learning processes are well-understood and share similarities with AI. We do not describe these two methods in the same level of detail as we did AI because they are already formalized and documented in the scientific literature. We present a comparison of the steps in the processes for AI, natural selection, and statistical regression in Table 4.

Table 4.

Comparison of Computational Models.

Simulation steps	Genetic algorithm	Regression analysis, random	Regression analysis, success-only	Appreciative inquiry
1. Define performance	Goal: Y_i(t+1) > Y_i(t), given that Y_i = ƒ(x₁, x₂, …, x _N ).			Goal: Y'_i(t+1) > Y'_i(t), given that Y'_i = g(Y, x₁, x₂, … x_N’)
2. Define the performance function	f(x₁, x₂, … x_N) = (Σ_{(i, 1 to N)}x_i) * (1.0 ± u), where the value of each x_i depends on the ordered values of x_i and the K-1 elements to the right of x_i. K ranges between 1 and N-1. The value of each permutation is pre-assigned as a random, uniform, decimal value drawn between zero and one. u is pre-specified and ranges between zero and 0.4.			g(Y, x₁, x₂, … x_N’) = ([Y ± Y * ƒ(x₁, x₂, …, x_N’)]/2) * (1.0 ± u), N’ > N and K’ > K, and u is a random number.
3. Identify an initial sample of strings	Identify M strings (B_i), where B_i = [x₁ x₂ … x_N], and the x_i have values of 0 or 1, assigned randomly with equal probability.
4. Evalute the performance of each string	Calculate Y(B _i ) or Y'(B _i ) for each B _i .
5. Select the highest-performing strings	Select B_i's two at a time, using p_i(select B_i) = Y_(t)(B_i) / Σ_{(i=1 to n)} Y_(t)(B_i). Do this R times, with replacement. R is the number of strings to be replaced.	Not applicable	Select the higher-performing M-R B_i.
6. Identify the information from which to learn		Regress Y on all x _i . If the effect of x _i on Y is significant and positive, record the value of 1 for that x _i . If the effect of x _i on Y is significant and negative record the value of 0 for that x _i .	Regress Y on x_i. If the effect of x _i on Y is significant and positive, record the value of 1 for that x _i . If the effect of x _i on Y is significant and negative record the value of 0 for that x _i .	Compare each x _i acoss the strings. If more than P percent of the values are the same for a given x _i , record that value for that x _i .
7. Information deletion	Delete the R lowest-performing strings.
8. Inference	Crossover: Create R new strings. Each new string, B _k , is created from a pair of strings selected in step 5. Each x _i in the the new string comes from the value of the same x_i in one of the two selected strings, such that the value of x₁ in B _k is equal to the value of x₁ in B _i approximately 50% of the time, and to the value of x₁ in B _j the rest of the time. The same is true with x₂, x₃, and so forth for each new B_i.	Replace deleted B_is with B_is that have the recorded values of the x _i from step five, and the values of all other x _i are randomly generated with an equal chance of 0 or 1.
9. Specific alterations	Based on q, a probability set before the simulation runs, the model makes specific, random, alterations to an x _i in one or more of the new B _i s.
10. Repetition	Return to step #4 and repeat

For natural selection, we used a model called the genetic algorithm (Holland, 1992), which has been used in organizational learning research (Bruderer & Singh, 1996; March, 1991) and in solving practical problems such as forensic face-matching (Stockdale, 2008), airline booking (George et al., 2012), or creating popular music (BBC News, 1998). As Table 4 suggests, there are differences between AI and the genetic algorithm. First, unlike AI, when organizations learn through natural selection they do not change the topic of the learning process (steps 1 and 2). Second, while AI infers new solutions by looking at building on common themes from high-performing solutions, the genetic algorithm infers by recombining the elements from two higher-performing solutions (similar to how sexual reproduction crosses over the DNA of parents) (steps 5–8). Thus, when organizations in our model learn through natural selection, the model builds a performance landscape for Y (not Y’), identifies solutions (B _i s) from which to learn, evaluates the performance of each B _i , replaces the R lowest-performing B _i s by randomly selecting two (at least probabilistically) higher-performing B _i s at a time and randomly recombining their elements (x _i s) to make two new B _i s, altering some x _i s with probability q in the same was as we do with AI, and perhaps repeating the process.

Learning through statistical regression also has similarities with and differences from AI. First, when people learn using statistical regression, they do not usually change the topic. Second, instead of inferring new solutions using themes, they regress the performance of a solution on the elements that make up that solution and retain the values of those elements that have a significant impact on performance. Thus, when organizations in our model learn through regression, the model builds a performance landscape for Y (not Y’); identifies solutions (B _i s) from which to learn; evaluates the performance of each B _i ; replaces the R lowest-performing B _i s by regressing the performance of a solution on the elements that make up that solution (x _i s), retaining the values of those elements that have a significant impact on performance, and replacing lower-performing B _i s with the new B _i s, altering some x _i s with probability q, and perhaps repeating the process.

We examine regression that samples only on successful solutions because AI focuses only on successful solutions and our research question focuses on learning from success. We also examined regression with random samples, for the sake of comprehensiveness. Previous research suggests that regression that focuses only on successful solutions should bias results (e.g., Heckman, 1979; Rosenzweig, 2009), making learning based on regression in success-only samples less effective. However, because sampling on the dependent variable does not cause estimation problems if performance is deterministic (Heckman, 1979), we adapted our NK model to be stochastic. We introduced randomness into the model by multiplying each solution (i.e., each point in the landscape) by 1.0 ± u, where u is a number randomly drawn from a pre-specified range (e.g., [0, 0.1] or [0, 0.4]). Even though the error we introduce comes from a random distribution, it introduces error into the learning process because, in each iteration of the learning process, the error term has the potential to (mis)inform all the methods for selection of which solutions to adopt in subsequent iterations. If organizational participants adopt a solution based on errors, then each new iteration of learning is affected by that prior random error. The error is random, but the selection of which path to follow is nonrandom, so the error term can lead to alternate solutions (better or worse) being pursued. In other words, if learners make a 10% positive error while making inferences in one iteration of the model, and then make a 10% negative error in the iteration of learning, learners have not necessarily achieved zero error in their learning process. The first error led participants to adopt a particular solution, and then the next error depends on what path was chosen after the first error was made.

Analytical Strategy

We used a simulation to evaluate the effectiveness of AI in three stages. First, we ran our model through hundreds of thousands of sensitivity tests, using hundreds of different parameters, to understand its general behavior, and to discover which input values would reveal the most if manipulated or held constant in our experiments. We ran multiple permutations of values for all the parameters of our model, such as N, K, N’, K’, the range of error, the number of strings, the rate of replacement of strings, and so on. Then we conducted two experiments, one which focuses on the impact of different forms of inference in the learning process, and the second which examines the impact of choosing different topics/changing the definition of performance. We refer to them as experiments because the manipulations generate results, not because the results are empirical. Our model and data are available upon request.

Experiment 1

In the first experiment, we manipulated two features of our computational model: (1) learning procedures (i.e., AI, the genetic algorithm, regression with random samples, and regression that samples on the dependent variable) and (2) the features of the solution landscape (N and K). We manipulated N and K because our sensitivity tests suggested that the complexity of the solution landscape has one of the most significant effects on the performance of the different learning procedures. K had a more significant impact than N, but N and K are not independent: N serves as an upper limit for K. Therefore, we present the results of the two illustrative combinations of N and K: N = 20, K = 9 (a rugged performance landscape) and N = 10, K = 1 (a smooth landscape).

We hold all other parameters constant in this experiment. We examine the effect of topic choice in our second experiment, not this one. Focusing solely on differences in the inference processes of the different learning procedures enables us to isolate their effects. We held the threshold for commonality constant at 60% in AI, as explained above. We held measurement error constant at 40% because sensitivity tests suggested it was a high value that almost always generated meaningful patterns in learning procedures. Lower values did not alter patterns much. We held the number of strings (B _i s) constant at 60, but this number was somewhat arbitrary because it did not appear to affect results much unless the numbers were extreme. We replaced 95% of B _i s at each iteration of the simulation because this was consistent with standards from previous research, in which a high number maximizes learning while still preserving the best of what was learned in previous iterations (Manikas et al., 2016). This parameter had a limited impact on outcomes in sensitivity tests.

For each permutation of manipulations in our experiment, we ran the simulation at least 100 times. We also examined results at zero, one, five, and 100 iterations of the learning process. Zero iteration provided a baseline. We examined one, five, and 100 iterations because some organizations only use these learning procedures one time, some use them only long enough to meet predetermined targets, and some treat them as ongoing processes of organizational learning. The output we examine for each simulation is the average performance of all strings in a simulated organization.

Results

Figure 1a and 1b depicts the performance of the four learning processes for all simulations at zero, one, five, and 100 iterations of the model. One observation of interest that is immediately apparent is that as organizations learn over time, average performance improves for all four learning procedures. In fact, in hundreds of thousands of sensitivity tests, decreases in performance over time were exceedingly rare for any learning procedure and only occurred when random error was high (40%). Even when learning was based on regression with success-only samples, performance never worsened, except in those exceptionally rare cases.

Figure 1.

(a) Average performance of solutions for each learning procedure at N = 10, K = 1. (b) Average performance of solutions for each learning procedure at N = 20, K = 9.

Performance almost never decreased, but the rate of performance improvement for each learning procedure varied significantly, and the complexity of the solution space (N and K—especially K) also significantly altered the rate of improvement of each learning procedure. For example, if we compare Figure 1a with Figure 1b we see that when N and K are low, AI, regression on successful observations, and regression on random observations who steep performance improvements after a single iteration of learning, and then taper off, while the genetic algorithm improves more slowly but achieves much better performance by the end. In contrast, when N and K are high, performance improvement occurs from one iteration of learning, but the rate of improvement is much less dramatic than when N and K are low. The rate of performance improvement remains small for both types of regression through subsequent iterations, but the learning rate is higher for AI as organizations iterate through the learning process. Dramatic improvement takes longer in the genetic algorithm, but it occurs and exceeds that of AI. Overall average performance is lower for most procedures when the problem space is more complex (see Table 4). This suggests that managers should base their decision about which learning procedure to use on how complex the problem is and how long they intend to engage in the learning process.

When comparing average performance, it is important to remember that performance in our model is an abstract percentage-based scale ranging from no benefit [0.0] to maximum possible benefit [1.0]. The meaning of this scale depends on the type of performance the scale represents. For example, the scale could represent billions of dollars or invaluable social benefit, or it could represent thousands of dollars or minor social benefits. The cost of going through multiple iterations of a learning procedure for a complex problem that will only make thousands of dollars or produce minor social benefits is unlikely to be worth it. However, multiple iterations of a learning process may be worthwhile to get a small improvement when the benefits are measured in millions or billions of dollars, or for important social benefits.

Table 3 displays significance tests for differences between means in the performance of the four learning procedures at one, five, and 100 iterations. Twenty-six out of the 36 of these differences in means are significant at p < .05. One intriguing observation from these tests is that, for simple solutions (N = 10, K = 1), regression analysis using only successful observations often outperforms the other procedures, and the only time it is significantly lower than any learning process is the genetic algorithm at 100 iterations. In complex searches (N = 20, K = 9), neither form of regression performed as well as AI nor the genetic algorithm. This is at least partly because the regression analyses in our model are underspecified for problems with complex interactions between variables. However, in our model-building and sensitivity tests, when we tried to create higher-order interactions in regression analysis, the models became so unwieldy that the interactions were not interpretable, and it was not possible to code meaningful inference processes for implementing solutions. Regression is useful for identifying relatively simple relationships, but not for finding higher-performing solutions to problems that involve complex interactions.

Experiment 2

Next, we examined how topic choice influences model behavior, introducing Y’ to our model in addition to Y. The formula, Y’ = [Y ± Y*f(x₁, x₂,…,x_N’)]/2, implies that if an organization tries to improve Y’ instead of Y, it may still influence Y while seeking to improve Y’. We ran the simulation with a topic choice 100 times for each permutation of the values N’ = 20 and 30; K’ = 3, 9, and 19; N = 10 and 20; and K = 1, 3, and 9, always with N’ > N and K’ > K. We examine these permutations for all four learning procedures using Y, for AI using Y’, and for the impact on Y when AI is applied to Y’ (See Table 5). For this experiment, we held all other parameters at the same levels used in the previous experiment.

Table 5.

Results of Paired Sample T tests of Comparing the Mean Performance of Different Learning Procedures at the Same Number of Iterations.

Comparison of performance between learning procedures	Difference in means	Lower bound of confidence interval	Upper bound of confidence interval	t	p
N = 10, K = 1
One Iteration
Genetic algorithm—Random regression	−0.102	−0.127	−0.077	−8.12	.00
Genetic algorithm—Success regression	−0.080	−0.105	−0.055	−6.42	.00
Genetic algorithm—Appreciative Inquiry	−0.060	−0.074	−0.047	−8.73	.00
Random regression—Success regression	0.022	0.010	0.034	3.61	.00
Random regression—Appreciative Inquiry	0.042	0.025	0.058	4.95	.00
Success regression—Appreciative Inquiry	0.020	0.002	0.038	2.19	.03
Five Iterations
Genetic algorithm—Random regression	−0.014	−0.025	−0.002	−2.40	.02
Genetic algorithm—Success regression	0.005	−0.008	0.018	0.71	.48
Genetic algorithm—Appreciative Inquiry	−0.009	−0.020	0.001	−1.74	.08
Random regression—Success regression	0.018	0.009	0.028	3.91	.00
Random regression—Appreciative Inquiry	0.005	−0.011	0.020	0.58	.56
Success regression—Appreciative Inquiry	−0.014	−0.029	0.002	−1.78	.08
100 Iterations
Genetic algorithm—Random regression	0.063	0.048	0.078	8.52	.00
Genetic algorithm—Success regression	0.076	0.061	0.091	10.12	.00
Genetic algorithm—Appreciative Inquiry	0.068	0.054	0.082	9.63	.00
Random regression—Success regression	0.013	0.005	0.021	3.13	.00
Random regression—Appreciative Inquiry	0.005	−0.011	0.020	0.60	.55
Success regression—Appreciative Inquiry	−0.008	−0.024	0.007	−1.09	.28
N = 20, K = 9
One Iteration
Genetic algorithm—Random regression	0.003	0.001	0.005	3.70	.00
Genetic algorithm—Success regression	0.003	0.001	0.005	2.78	.01
Genetic algorithm—Appreciative Inquiry	0.001	−0.001	0.003	0.81	.42
Random regression—Success regression	0.000	−0.002	0.001	−0.46	.65
Random regression—Appreciative Inquiry	−0.002	−0.005	0.000	−1.90	.06
Success regression—Appreciative Inquiry	−0.002	−0.004	0.000	−1.60	.11
Five Iterations
Genetic algorithm—Random regression	0.010	0.008	0.012	8.60	.00
Genetic algorithm—Success regression	0.011	0.009	0.014	9.86	.00
Genetic algorithm—Appreciative Inquiry	−0.021	−0.028	−0.013	−5.65	.00
Random regression—Success regression	0.001	0.000	0.003	1.59	.11
Random regression—Appreciative Inquiry	−0.031	−0.038	−0.023	−8.34	.00
Success regression—Appreciative Inquiry	−0.032	−0.039	−0.025	−9.22	.00
100 Iterations
Genetic algorithm—Random regression	0.205	0.195	0.215	40.11	.00
Genetic algorithm—Success regression	0.206	0.195	0.216	39.28	.00
Genetic algorithm—Appreciative Inquiry	0.144	0.129	0.158	19.71	.00
Random regression—Success regression	0.001	−0.001	0.003	1.24	.22
Random regression—Appreciative Inquiry	−0.061	−0.072	−0.049	−10.62	.00
Success regression—Appreciative Inquiry	−0.062	−0.073	−0.050	−10.93	.00

Results

The patterns of learning were similar across all but the most complex permutations because the simpler permutations were similar to the permutations in our first experiment. Although we found a few significant differences in a few permutations, these differences were uncommon and, among them, few of the patterns of learning were visibly different. Therefore, rather than present learning patterns from different permutations, Figure 2 displays average results. Figure 2 also displays the genetic algorithm and the two regression models as undifferentiated gray lines so that readers have a point of reference for them. In this experiment, the significance of the differences between Y and Y’ is not meaningful because they are different measures of performance. The mathematics of Y’ dictate that Y’ will be less than Y, even though real performance may be higher for Y’ when its decimal score is applied at scale, as explained previously.

Figure 2.

Comparison of average performance of solutions for three approaches to appreciative inquiry.

As with our first experiment, the two new versions of AI also almost always increased performance with more iterations of learning. This was true even when the problem represented by the solution landscape was more complex. This is not surprising for Y’, because it is the same AI procedure applied to a modified search problem. More surprising, however, is that when the model uses AI to improve Y’, Y improves as well. In fact, at one iteration, AI for Y’ led to higher performance in Y than the genetic algorithm; at five iterations, AI for Y’ led to higher performance in Y than the genetic algorithm or either type of regression; and at 100 iterations, AI for Y’ led to higher performance in Y than either type of regression or even than AI for Y. This relatively high performance for Y when trying to improve Y’ at five and 100 iterations occurred because the learning procedure, by focusing on searching the N’K’ landscape, converged less quickly on combinations of x _i s that focus on the first high-performing solutions discovered, and continued searching until even higher-performing solutions were found.

Discussion

We built a computational model of AI, the genetic algorithm, and regression (using both success-only and random samples). We did this to examine features of designed organizational search, including problem reframing and learning solely from success. We found that no matter which method was used, performance almost always improved as organizations iterated through each of the learning procedures, even with regression analysis of only successful observations. This was a repeated finding using systematic search processes. We also found that complexity influences the effectiveness of each of these procedures. Regression on success-only samples increased performance most when searching for solutions to relatively simple problems and only engaging in one or a few iterations of learning; AI increased performance most when searching for solutions to relatively complex problems and only engaging in one or a few iterations of learning; and the genetic algorithm increased performance most when engaging in many iterations of learning, no matter how complex the problem involved. When we added topic choice to AI, not only did the potential for performance increase because of the broader aspirations being pursued, but performance in the narrower aspiration also achieved impressive performance.

One of the reasons these results are interesting is because they contrast different approaches to learning from success, rather than comparing success and failure (e.g., Madsen & Desai, 2010; Muehlfeld et al., 2012) or simply cataloging the benefits and drawbacks of learning from success (e.g., Denrell, 2003; Heckman, 1979; Weick, 1984). We explain when and why different approaches to learning from success are more or less effective. These results do not undermine previous research. For example, in a single iteration of learning, inferences made solely from samples with only successful observations are still likely to be biased estimators of causal relationships (Denrell, 2003, 2005; Denrell et al., 2013; Heckman, 1979). However, our results add nuance to these findings. For example, people interested in improving organizational performance may be less interested in identifying accurate causal relationships than they are in finding higher-performing solutions, even if they do not understand the reasons why performance is better. Organizations that iterate through successful solutions, retaining the most successful ones, and ignoring unsuccessful ones, are likely to improve performance over time in spite of problems with the inferences they make from success-only samples in a single iteration of learning. Success-based learning, systematically implemented and repeated, may avoid some of the downsides associated with success-based search. Taking this a step further, even managers’ interest in others’ success may not always be misguided (e.g., Rosenzweig, 2009). For example, there may be ways to iterate systematically through imitative learning that might also increase performance.

Another significant insight from our results is the finding that re-framing problems (i.e., topic choice) may increase the performance of an organization relative to both an original goal and a new, broader goal. Of course, our model of topic choice was just one of many ways to model topic choice. Even so, it suggests that it is at least possible that learning how to improve a broader and more complex type of performance will also improve a narrower and less complex type of performance, sometimes even beyond that which would have been achieved by focusing on the narrower type of performance alone.

Organizational Learning

These results have implications for the theory of organizational learning. Identifying procedures and conditions under which learning from success is effective advances theoretical knowledge (see Harris et al., 2013). For example, a common goal of computational models of organizational learning is to examine how organizations can learn effectively despite their members’ bounded rationality. Scholars have shown that cognitive structures such as a moderate obsession (Winter et al., 2007), or social structures such as hierarchy (Cohen, 1981; Knudsen & Levinthal, 2007), help organizations take effective action despite people's cognitive limitations. Our model adds to these insights by suggesting that managers may also help organizational members overcome bounded rationality by structuring their communication. Our model suggests three ways in which people can structure their communication to learn from success. First, they can look for themes in stories of achieving broad, positive goals (AI). Second, they can hypothesize and test variables that might affect performance and require people to implement the results of their tests (regression). Third, they can mix and match ideas from different successful solutions and try these combinations experimentally (genetic algorithm). Each of these is more nuanced and deliberate than simply imitating or repeating past successes (Denrell, 2003; Kim et al., 2009; Muehlfeld et al., 2012). These approaches suggest the beginnings of a typology of communicative patterns for learning from success. People can add these structures of search communication to the cognitive and social structures other scholars have identified. When they use these structures, they should use the ones that are appropriate for the complexity of their problems and the number of times they intend to iterate through the learning cycle. By identifying communicative structures for searching, we answer Posen and colleagues’ (2018) call for research on organizational search that is more deliberate than traditional models, as each of these structures implies deliberate, designed, and social cognition.

Our analysis of the Appreciative Inquiry Handbook also suggested the possibility that learners can reframe or redefine what performance is after encountering problems but before searching for solutions. Typically, models of organizational learning assume that organizational search begins with some kind of stimulus, without reflection on the aspiration (a full critique of this approach can be found in Posen et al., 2018). Many models include cognitive elements, such as forward-thinking (Gavetti & Levinthal, 2000) or goal-oriented search (Winter et al., 2007), but most modelers treat solution landscapes as given and fixed (Nelson, 2006). Our research introduced topic choice, or the reframing of performance, into a model of organizational search and found that it can have a significant impact on learning outcomes. We focused on reframing performance in AI but reframing performance need not be limited to AI. Learners may do better if they use reframing in regression or in evolutionary approaches to learning as well. For example, if an organization intends to search for relatively simple solutions and only intends to iterate through the learning procedure one or a few times, it may be most effective for people to reframe performance and then to conduct regression analyses to find a solution. Of course, we only examined one approach to problem reframing, and future research should examine additional ways, including the possibility that the landscape may change along with the learning process itself.

Implications for Practice

Our model and findings are significant for anyone interested in these applied processes of search. The computational model is helpful to practice because it provides some generalizable results and implications. Our results show that AI, an example of success-based search, is in fact systematically effective as a search process, although its degree of effectiveness depends on factors such as the complexity of the problem and the cost of searching. These findings suggest that we might understand success-based search processes in a new way. Notwithstanding the perceived drawbacks of sampling on success only, organizations can effectively learn through processes such as AI. Once more, our results suggest that AI is more effective at a lower number of learning iterations than natural section (GA). This is interesting considering that evolutionary algorithms set high standards for systematic learning.

Our research highlights the importance of notable features of AI. For example, in AI the topic choice involves transforming problems (e.g., “How do we fix customer dissatisfaction?”) into unconditionally positive questions (e.g., “How might we engage and delight our customers in new and exciting ways?”). We provide preliminary validation for crafting unconditional positive questions, showing that it changes how performance is conceived and also how it may increase the original type of performance under some circumstances, although more research is needed to fully understand when and how this will be the case. Our research also suggests the importance of a systematic approach to AI that includes steps such as defining performance, defining the performance function, identifying an initial population of solutions, evaluating the solutions’ performance, selecting the highest performing solutions, making alterations, and repetiting the cycle (see Table 2). Practitioners should be careful about changing or dropping these steps, if they do so at all, or they may lose the benefits of using designed search processes over automatic, problem-driven satisficing search processes.

Our model helps managers make informed decisions about the systemic method for learning that they should implement. For example, when the problem is complex and it is only possible to iterate a few times, AI is the most effect form of learning. Although overall average performance is lower for most procedures when the problem space is more complex (see Table 4), Appreciate Inquiry demonstrated the highest performance. This suggests that managers should base their decision about which learning procedure to use on how complex the problem is, the cost of the learning procedure relative to the benefit, and how long they intend to engage in the learning process.

Appreciative Inquiry exhibited significantly better performance improvement than regression when searching for solutions to complex problems at five and 100 iterations of learning, but the genetic algorithm performed best out of all learning procedures at 100 iterations. Thus, it appears that success provides useful information about what combinations of factors (x _i s) are likely to exhibit higher performance, but too much reliance on this information may lead organizations to converge too quickly on specific combinations of factors, rather than preserve variety and keep learning. The genetic algorithm, AI, and regression on success-only samples all rely on information from success, but the genetic algorithm preserves more variety in this information than AI, and AI preserves more variety than regression as these learning procedures as applied over and over across time. If practitioners wish to preserve the variety of possible solutions to complex problems as a way of maximizing learning over time, then experimenting with partially random recombinations is a more effective way to learn than looking for themes or trying to identify causal relationships.

Moving Forward

There are multiple adaptations scholars could make to our model to examine other, related research questions. For example, are there other ways to structure communication for learning from success that could be built into this model? Future research would also benefit from different research methods. For example, ethnographic observation of learning methods such as these could reveal more nuances in the learning procedures that could be included in models of organizational learning. Experimental research could test the cognitive and behavioral outcomes of using these different learning procedures in groups.

Continued research on the relative effectiveness of different procedures for learning from success is critical because, as Denrell (2003) pointed out, many samples of organizational phenomena no longer have unsuccessful examples in them. This may create bias in learning in some circumstances, but as our research has shown, this bias may be surmountable. In fact, learning from success may reduce defensiveness and increase confidence and trust (Cannon & Edmondson, 2001). If organizations can learn effectively from success, there may be potential for deriving many other benefits as well.

Footnotes

Acknowledgments

The authors would like to thank Yael Goel, Scott Johnson, Peter Madsen, Andrew Manikas, Leslie Perlow, Hart Posen, William Ocasio, and Kyle Oyama for their helpful feedback on this paper, as well as participants in research seminars at Brigham Young University, Northwestern University, Simon Frasier University, Purdue University, Western Ontario University, and the University of Louisville. The code for the computational model used for this research is available from the lead author upon request.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Bret Crane

Notes

References

Ackoff

R. L.

(1981). The art and science of mess management. Interfaces, 11(1), 20–26. https://doi.org/10.1287/inte.11.1.20

Argote

Lee

Park

(2021). Organizational learning processes and outcomes: Major findings and future research directions. Management Science, 67(9), 5399–5429. https://doi.org/10.1287/mnsc.2020.3693

BBC News (1998). To the beat of the byte.

Berger

P. L.

Luckmann

(1990). The social construction of reality. Anchor Books.

Billinger

Stieglitz

Schumacher

T. R.

(2014). Search on rugged landscapes: An experimental study. Organization Science, 25(1), 93–108. https://doi.org/10.1287/orsc.2013.0829

Bisits Bullen

P. A.

(2011). The positive deviance/hearth approach to reducing child malnutrition: Systematic review. Tropical Medicine & International Health, 16(11), 1354–1366. https://doi.org/10.1111/j.1365-3156.2011.02839.x

Bogan

C. E.

English

M. J.

(1994). Benchmarking for best practices: Winning through innovative adaptation. McGraw-Hill.

Bruderer

Singh

J. V.

(1996). Organizational evolution, learning, and selection: A genetic-algorithm-based model. Academy of Management Journal, 39(5), 1322–1349. https://doi.org/10.2307/257001

Bushe

G. R.

Kassam

(2005). When is appreciative inquiry transformational? A meta-case analysis. Journal of Applied Behavioral Science, 41(2), 161–181. https://doi.org/10.1177/0021886304270337

10.

Cannon

M. D.

Edmondson

A. C.

(2001). Confronting failure: Antecedents and consequences of shared beliefs about failure in organizational work groups. Journal of Organizational Behavior, 22(2), 161–177. https://doi.org/10.1002/job.85

11.

Charan

Tichy

N. M.

(1998). Every business is a growth business: How your company can prosper year after year. Crown Business.

12.

Christmann

(2000). Effects of” best practices” of environmental management on cost advantage: The role of complementary assets. Academy of Management Journal, 43(4), 663–680. https://doi.org/10.2307/1556360

13.

Cohen

M. D.

(1981). The power of parallel thinking. Journal of Economic Behavior and Organization, 2, 285–306. https://doi.org/10.1016/0167-2681(81)90011-1

14.

Cohen

M. D.

March

J. G.

Olsen

J. P.

(1972). A garbage can model of organizational choice. Administrative Science Quarterly, 17(1), 1–25. https://doi.org/10.2307/2392088

15.

Cooperrider

D. L.

Fry

(2020). Appreciative inquiry in a pandemic: An improbable pairing. The Journal of Applied Behavioral Science, 56(3), 266–271. https://doi.org/10.1177/0021886320936265

16.

Cooperrider

D. L.

Whitney

(2005). A positive revolution in change: Appreciative inquiry. In Cooperrider

D. L.

Whitney

Stavros

J. M.

(Eds.), Appreciative inquiry handbook: The first in a series of AI workbooks for leaders of change (pp. 317–335). Berrett-Koehler Publishers.

17.

Cooperrider

D. L.

Whitney

Stavros

J. M.

(2005). Appreciative inquiry handbook: The first in a series of AI workbooks for leaders of change. Berrett-Koehler Publishers.

18.

Crowston

(1996). An approach to evolving novel organizational forms. Computational and Mathematical Organization Theory, 2(1), 29–47. https://doi.org/10.1007/BF00125762

19.

Csaszar

F. A.

Levinthal

D. A.

(2016). Mental representation and the discovery of new strategies. Strategic Management Journal, 37(10), 2031–2049. https://doi.org/10.1002/smj.2440

20.

Cyert

R. M.

March

J. G.

(1963). A behavioral theory of the firm. Prentice Hall.

21.

Deichmann

Ende

J. V. D.

(2014). Rising from failure and learning from success: The role of past experience in radical initiative taking. Organization Science, 25(3), 670–690. https://doi.org/10.1287/orsc.2013.0870

22.

Denrell

(2003). Vicarious learning, undersampling of failure, and the myths of management. Organization Science, 14(3), 227–243. https://doi.org/10.1287/orsc.14.2.227.15164

23.

Denrell

(2005). Should we be impressed with high performance? Journal of Management Inquiry, 14(3), 227–243. https://doi.org/10.1177/1056492605279100

24.

Denrell

Fang

Zhao

(2013). Inferring superior capabilities from sustained superior performance: A Bayesian analysis. Strategic Management Journal, 34(2), 182–196. https://doi.org/10.1002/smj.2001

25.

Eisenberg

(2011). Lead-user research for breakthrough innovation. Research-Technology Management, 54(1), 50–58. https://doi.org/10.1080/08956308.2011.11657673

26.

Freeman

Hannan

M. T.

(1983). Niche width and the dynamics of organizational populations. American Journal of Sociology, 88(6), 1116–1145. https://doi.org/10.1086/227797

27.

Fritz

(1989). The path of least resistance: Learning to become the creative force in your own life. Fawcett.

28.

Fry

Barrett

Whitney

(2002). Appreciative inquiry and organizational transformation: Reports from the field. Quorum.

29.

Gavetti

(2005). Cognition and hierarchy: Rethinking the microfoundations of capabilities’ development. Organization Science, 16(6), 599–617. https://doi.org/10.1287/orsc.1050.0140

30.

Gavetti

Levinthal

D. A.

(2000). Looking forward and looking backward: Cognitive and experiential search. Administrative Science Quarterly, 45(1), 113–137. https://doi.org/10.2307/2666981

31.

George

Rajakumar

B. R.

Binu

(2012). Genetic algorithm based airlines booking terminal open/close decision system. Paper presented at the International Conference on Advances in Computing, Communications and Informatics.

32.

Gong

Zhang

Xia

(2019). Do firms learn more from small or big successes and failures? A test of the outcome-based feedback learning perspective. Journal of Management, 45(3), 1034–1056. https://doi.org/10.1177/0149206316687641

33.

Greve

H. R.

(2003). Organizational learning from performance feedback: A behavioral perspective on innovation and change. Cambridge University Press.

34.

Greve

H. R.

Seidel

M. D. L.

(2015). The thin red line between success and failure: Path dependence in the diffusion of innovative production technologies. Strategic Management Journal, 36(4), 475–496. https://doi.org/10.1002/smj.2232

35.

Hambrick

D. C.

Finkelstein

Cho

T. S.

Jackson

E. M.

(2004). Isomorphism in reverse: Institutional theory as an explanation for recent increases in intraindustry heterogeneity and managerial discretion. Research in Organizational Behavior, 26, 307–350. https://doi.org/10.1016/S0191-3085(04)26008-7

36.

Harris

J. D.

Johnson

S. G.

Souder

(2013). Model-theoretic knowledge accumulation: The case of agency theory and incentive alignment. Academy of Management Review, 38(3), 442–454. https://doi.org/10.5465/amr.2011.0141

37.

Heckman

J. J.

(1979). Sample selection bias as a specification error. Econometrica: Journal of the Econometric Society, 47, 153–161. https://doi.org/10.2307/1912352

38.

Holland

J. H.

(1992). Genetic algorithms. Scientific American, 267, 66–72. https://doi.org/10.1038/scientificamerican0792-66

39.

Joseph

Gaba

(2020). Organizational structure, information processing, and decision-making: A retrospective and road map for research. Academy of Management Annals, 14(1), 267–302. https://doi.org/10.5465/annals.2017.0103

40.

Kaufman

(1993). An algorithm for the banded symmetric generalized matrix. SIAM Journal on Matrix Analysis and Applications, 14(2), 372–389. https://doi.org/10.1137/0614026

41.

Staats

B. R.

Gino

(2013). Learning from my success and Others’ failure: Evidence from minimally invasive cardiac surgery. Management Science, 59(11), 2435–2449. https://doi.org/10.1287/mnsc.2013.1720

42.

Kim

J.-Y.

Kim

J.-Y.

Miner

A. S.

(2009). Organizational learning from extreme performance experience: The impact of success and recovery experience. Organization Science, 20(6), 958–978. https://doi.org/10.1287/orsc.1090.0439

43.

Knudsen

Levinthal

D. A.

(2007). Two faces of search: Alternative generation and alternative evaluation. Organization Science, 18(1), 39–54. https://doi.org/10.1287/orsc.1060.0216

44.

Lapré

M. A.

Cravey

(2022). When success is rare and competitive: Learning from others’ success and my failure at the speed of Formula One. Management Science, 68, 874–8756. https://doi.org/10.1287/mnsc.2022.4324

45.

Lee

J. Y.

Jiménez

Devinney

T. M.

(2020). Learning in SME internationalization: A new perspective on learning from success versus failure. Management International Review, 60(4), 485–513. https://doi.org/10.1007/s11575-020-00422-x

46.

Lilien

G. L.

Morrison

P. D.

Searls

Sonnack

von Hippel

(2002). Performance assessment of the lead user idea-generation process for new product development. Management Science, 48(8), 1042–1059. https://doi.org/10.1287/mnsc.48.8.1042.171

47.

Ludema

J. D.

Barrett

(2007). Appreciative inquiry summit. In Holman

Devane

Cady

(Eds.), The change handbook: The definitive resource on today's best methods for engaging whole systems (pp. 47–58). Berrett-Koehler.

48.

Madsen

P. M.

Desai

(2010). Failing to learn? The effects of failure and success on organizational learning in the global orbital launch vehicle industry. Academy of Management Journal, 53(3), 451–476. https://doi.org/10.5465/amj.2010.51467631

49.

Manikas

A. S.

Kroes

J. R.

Gattiker

T. F.

(2016). Metro meals on wheels treasure valley employs a low-cost routing tool to improve deliveries. Interfaces, 46(2), 154–167. https://doi.org/10.1287/inte.2015.0835

50.

Marra

A. R.

Guastelli

L. R.

de Araújo

C. M. P.

dos Santos

J. L. S.

Lamblet

L. C. R.

Silva

, … Neto

M. C.

(2010). Positive deviance: A new strategy for improving hand hygiene compliance. Infection Control, 31(01), 12–20. https://doi.org/10.1086/649224

51.

May

Becker

Frankel

Haizlip

Harmon

Plews-Ogan

, … Whitney

(2011). Appreciative inquiry in healthcare. Crown Custom Publising.

52.

Montenari

J. R.

(1978). Managerial discretion: An expanded model of organization choice. Academy of Management Review, 3(2), 231–241. https://doi.org/10.2307/257664

53.

Muehlfeld

Rao Sahib

van Witteloostuijn

(2012). A contextual theory of organizational learning from failures and successes: A study of acquisition completion in the global newspaper industry, 1981–2008. Strategic Management Journal, 33, 938–964. https://doi.org/10.1002/smj.1954

54.

Nelson

R. R.

(2006). Bounded rationality, cognitive maps, and trial and error learning. Journal of Economic Behavior and Organization, 67, 78–89. https://doi.org/10.1016/j.jebo.2007.06.002

55.

Nigam

Huising

Golden

(2016). Explaining the selection of routines for change during organizational search. Administrative Science Quarterly, 61(4), 551–583. https://doi.org/10.1177/0001839216653712

56.

Pascale

R. T.

Sternin

(2010). The power of positive deviance: How unlikely innovators solve the world's toughest problems. Harvard Business Publishing.

57.

Posen

H. E.

Keil

Kim

Meissner

F. D.

(2018). Renewing research on problemistic search—A review and research agenda. Academy of Management Annals, 12(1), 208–251. https://doi.org/10.5465/annals.2016.0018

58.

Rivkin

J. W.

(2000). Imitation of complex strategies. Management Science, 46(6), 824–844. https://doi.org/10.1287/mnsc.46.6.824.11940

59.

Rosenzweig

(2009). The halo effect: And the eight other business delusions that deceive managers. Free Press.

60.

Sastry

M. A.

(1997). Problems and paradoxes in a model of punctuated organizational change. Administrative Science Quarterly, 42, 237–275. https://doi.org/10.2307/2393920

61.

Siggelkow

Rivkin

J. W.

(2006). When exploration backfires: Unintended consequences of multilevel organizational search. Academy of Management Journal, 49(4), 779–795. https://doi.org/10.5465/amj.2006.22083053

62.

Stockdale

C. A.

(2008). A (R)evolution in Crime-Fighting. Forensic Magazine(June/July), 30–34.

63.

Tushman

M. L.

Romanelli

(1985). Organizational Revolution: A Metamorphisis Model of Convergence and Reorientation . In Cummings

L. L.

Staw

B. M.

(Eds.), Research in organizational behavior (pp. 171–222). JAI Press.

64.

von Hippel

Thomke

Sonnack

(1999). Creating breakthroughs at 3M. Harvard Business Review, 77(5), 47–57.

65.

Wangrow

D. B.

Schepker

D. J.

Barker

V. L.

III . (2015). Managerial discretion: An empirical review and focus on future directions. Journal of Management, 41(1), 99–135. https://doi.org/10.1177/0149206314554214

66.

Weick

K. E.

(1984). Small wins: Redefining the scale of social problems. American Psychologist, 39, 40–49. https://doi.org/10.1037/0003-066X.39.1.40

67.

Winter

S. G.

Cattani

Dorsch

(2007). The value of moderate obsession: Insights from a new model of organizational search. Organization Science, 18(3), 403–419. https://doi.org/10.1287/orsc.1070.0273

68.

Zollo

Winter

S. G.

(2002). Learning and the evolution of dynamic capabilities. Organization Science, 13(3), 339–351. https://doi.org/10.1287/orsc.13.3.339.2780