Abstract
Decision making requires managers to constantly estimate the probability of uncertain outcomes and update those estimates in light of new information. This article provides guidance to managers on how they can improve that process by more explicitly adopting a Bayesian approach. Clear understanding and application of the Bayesian approach leads to more accurate probability estimates, resulting in better informed decisions. More importantly, adopting a Bayesian approach, even informally, promises to improve the quality of managerial thinking, analysis, and decisions in a variety of additional ways.
Keywords
Being able to effectively think in probabilistic terms is a skill highlighted by decision makers as diverse as political forecaster Nate Silver, the second half of the Berkshire Hathaway team Charlie Munger, and professional poker player Annie Duke. 3 And, facility with this type of decision-making approach is becoming increasingly important as predicting outcomes becomes even harder due to growing levels of uncertainty in a wide variety of areas. Technology advancements foster competitive disruptions and rapidly shifting customer preferences; global economic policy uncertainty has more than quadrupled in the two decades since 2000; trade uncertainty in 2019 has reached levels not seen over the previous three decades; and “deep uncertainty is ubiquitous in connected interdependent economies experiencing rapid technological change.” 4 Making good decisions in increasingly uncertain times requires thinking in probabilistic terms. And obviously, the quality of choices resulting from probabilistic decision-making approaches depends on the accuracy of the probability estimates themselves. The goal of this article is to describe an approach that will help you improve your ability to accurately estimate probabilities across a wide range of contexts, making you a better decision maker more equipped to deal with the need to make choices in a world of increasing uncertainty.
For events like coin flips or dice rolls, estimating accurate probabilities is pretty trivial. Unfortunately, most managerial decisions do not involve this sort of objective, easily estimated probability. Estimating probabilities for managerial decisions requires forming initial subjective estimates based on existing knowledge and prior experience, even when data are sparse, while systematically updating those estimates as you learn more. The importance of the updating process makes a lot of sense once you realize that we all do this implicitly. How likely do you think it is that you would enjoy that local restaurant you’ve been hearing so much about? You have a baseline probability estimate based on what you’ve heard and what you know you like. When friends mention their most recent experience or you run across an online review, you update that belief. But the problem with simply relying on implicit processes like this is that they are subject to significant error and bias. It’s also not obvious in this intuitive process how much you should adjust estimates when encountering new information.
This article introduces you to a more systematic approach that will lead to more accurate estimates and better decisions. Bayesian 5 updating provides a formal way to update probability estimates in the light of new information. It involves combining existing estimates with an assessment of the strength of new evidence. The approach has demonstrated its value across an incredibly wide range of contexts: it helped crack the Enigma code in World War II; it reduces the spam we all receive in our email inboxes; and it is fostering the development of artificial intelligence. 6 Work with superforecasters, or individuals with exceptional skills of predicting the probability of a variety of future outcomes, has further demonstrated the importance of the Bayesian approach, as one key distinguishing feature of superforecasters is that they “are markedly better Bayesians than regular forecasters.” 7
Taking a more explicit Bayesian approach when thinking probabilistically promises to help managers make better decisions in at least two fundamental ways. First, the process results in better probability estimates where “better” means that your estimates are more accurate. Inaccurate probability estimates lead to both Type I and Type II errors, and each type of error is costly; overestimates lead to investing in or backing alternatives when you should not; and underestimates lead to withholding investment when you should commit. Better estimates mean fewer errors of both types. Second, even if calculating exact probability estimates is not critical to you, being a more explicit Bayesian makes you a clearer thinker in a number of significant ways. As I discuss in more detail later, adopting a Bayesian approach fosters open-mindedness, helps reduce the effect of cognitive biases, provides guidance on the value of evidence, and fosters clarity of communication, all of which promise to improve the quality of your managerial decisions made under uncertainty. An important starting point to Bayesian thinking is viewing probability in a new way.
Objective versus Subjective Probability
Probability statements are an incredibly common part of managerial conversations and decisions. They are easy to recognize when stated with exact numbers, but words like “usually,” “likely,” “probably,” “possibly,” and the like all reflect probability statements too. But what exactly does it mean when someone says there’s a 50% chance of a particular outcome occurring? We can think of this statement in at least two different ways depending on the types of events being considered.
In the first view, probability quantifies the expected frequency of a particular outcome over a large number of repetitions of an event. In this objective (or frequentist) view, probability statements express properties of long-run outcomes of well-defined, repeatable processes. A gambler would assess the probability of a fair coin flip landing on tails to be 0.50. The underlying idea is that if a coin were flipped a hypothetically infinite number of times, the coin would land on tails one-half of the time. This perspective on probability is likely to be the first, and perhaps only, view most managers have been exposed to (side note: use of the word likely indicates this sentence itself is a probability statement). Flipping coins, rolling dice, and drawing colored balls from urns are the workhorses of this view of probability.
The second view of probability aligns much better with the types of issues and decisions that managers face in the real world. It recognizes that managers must make forecasts and decisions even when they do not have exact statistics to calculate an objective, frequency-based probability. In this view, probability is associated with degrees of belief or confidence in one’s knowledge. It is an expression quantifying how sure an individual is that a particular belief is true; it provides a language for expressing uncertainty. Consider a product manager saying she thinks a proposed product has a 75% chance of success. This estimate quantifies how confident the manager is—in this case, about halfway between totally certain (100%) and thinking success and failure are equally likely (50%). Imagine someone offered that manager a lottery ticket that paid $500 in the case of product success. Given her confidence level, she should be willing to pay no more than $375 for that ticket. For any amount less than $375, she would expect to win money, 0.25 × −375 + 0.75 × (500 – 375) = 0, on average. This is a subjective (or Bayesian) view of probability; probability assessments vary among people depending on differences in information and interpretations of information. For example, another manager might assess the success probability of the same product to be 60% based on access to other information or different assessment of the same information.
The subjective view is a more expansive definition of probability, as it can be applied to many different types of events (including those covered by the objective view). Subjective probabilities apply to anything that can be the subject of belief or knowledge while objective probabilities apply to the relative frequency of events that are repeated many times. The fact that subjective probabilities depend on an individual’s knowledge has significant implications; among the most important is that if knowledge changes via encountering new information or evidence, probability estimates should change, too. For example, consider the product manager with the 75% degree of confidence in a potential new product. Suppose she learns that a competing company is considering the launch of a similar product. How should that affect her probability estimate? Intuitively, it seems obvious that the estimate should change, but it’s not exactly clear how it should be revised. The Bayesian approach tells you what information you need and how to use it to update your existing probability estimate.
Bayesian Updating
Bayesian updating involves combining existing or prior beliefs with an assessment of the strength of new evidence. Here’s an example of how it works. Say you’re considering entry into a new geographic market. Based on your prior experiences and general knowledge of the opportunity, you have an estimate of how confident you are that the entry will be profitable. That’s the prior probability or p(b), where b represents your belief (entry will be profitable). Imagine you then engage in some focus group testing of your products with customers from that geographic market. The information from the focus group represents the evidence (e). You need two estimates associated with the evidence. First, you have to estimate how likely it would be that the focus group would have produced that evidence if your belief were true; stated another way, you have to estimate how likely it is that a profitable entry case would be preceded by a positive focus group result—this is represented by p(e|b); this number will be less than 1.0 assuming focus groups sometimes fail to predict profitable market entry. You also need to estimate how likely it is that the focus group would have produced the evidence if your belief were incorrect (i.e., a false positive)—represented by p(e|~b). These two pieces of information, p(e|b) and p(e|~b), indicate the strength of the evidence generated by the test. Evidence is stronger when it is much more likely to be observed when a belief is true compared with when the belief is false, that is, p(e|b) is much larger than p(e|~b).
Once you have estimates of p(b), p(e|b), and p(e|~b), you can calculate your updated beliefs using Bayes’s rule. You can implement the calculation in a variety of ways, but the underlying idea remains the same. Your updated (or “posterior”) belief is a combination of your prior belief, p(b), and the strength of the evidence, determined by p(e|b) and p(e|~b). Here is one way to calculate the updated belief given the evidence you have encountered:
Although the formula looks a bit complicated, it requires only the three pieces of information mentioned above. I did not mention p(~b), but it is pretty simple to calculate. It represents the prior probability that your belief is false. P(b) and p(~b) add to 1.0 since you are either right or wrong, so just subtract p(b) from 1 to get p(~b).
To see how this formula works, let’s continue with the example above of entering a new geographic market. Say your p(b) estimate is 0.2, meaning you think there is a 20% chance of entry being profitable; this means p(~b) = 0.8. How might you estimate p(e|b) and p(e|~b)? One way to help think through this is to imagine a set of 100 hypothetical entry cases. Based on the p(b) estimate, 20 of those cases would be profitable and 80 would be unprofitable. Think first about those 20 profitable cases: how many of the focus group tests would have produced positive feedback? Let’s say we estimate positive focus group tests would occur in 15 of them, so p(e|b) equals 15 / 20 or 0.75. Next, consider the 80 unprofitable cases: how many of the focus group tests would have produced positive (false) feedback? Say we estimate positive focus group feedback (false positives) in 12 of them, so p(e|~b) = 0.15. Bayes’s rule indicates that we should update our probability estimate to 0.56 ([.75 × .2] / [.75 × .2 + .15 × .8] = 0.56).
If you’re like many people, you’re probably thinking at this point: “well, I suppose that’s interesting but I’m never going to remember that formula.” Fair enough. Fortunately, you’ve got a few alternatives. You can find plenty of Bayesian calculators online, and it would be pretty easy to create your own in a spreadsheet program like Excel (see supplemental material). But you don’t even have to go that far. There are some pretty straightforward options that only require a bit of simple math.
To apply the first, use a visualization of the problem. One option is a frequency tree. Let’s break down our 100 hypothetical entry cases as shown in Figure 1. The first split captures p(b), as we estimated a 0.2 chance of profitable entry. Continuing down the left-hand side of the tree, we estimated 15 focus groups would be positive in those cases (and five negative). On the right-hand side, 12 of the 80 unprofitable cases have positive focus groups (and 68 negative). To calculate our updated belief of p(b|e), we simply ask how likely is it that we are in a profitable case given that we have a positive focus group result. Of the 27 positive focus groups, 15 are in profitable cases—15/27 = 0.56, same as the earlier calculation.
Frequency Tree
A second approach requires you to think in terms of odds rather than probabilities (see the appendix for a quick review on odds). In this approach, you simply multiply your prior odds by a scaling factor to get the updated or posterior odds. The scaling factor represents the strength of the evidence; as mentioned above, strength of evidence indicates how much more likely it would be to see the evidence if you are right about your belief compared with if you are wrong. It’s calculated by dividing p(e|b) by p(e|~b). This quantity is also called the “Bayes’s factor.” If the Bayes’s factor is less than 1, it’s evidence that your belief is wrong, and your confidence level should decrease; if it is greater than 1, your confidence level should increase. The magnitude of the Bayes’s factor tells you how strong the evidence is: a factor of 2 provides fairly weak evidence while a factor of 10 is much stronger evidence. And, again, you need only the three pieces of information mentioned above to apply this approach. In our market entry case, divide 0.2 by 0.8 to get the prior odds of 0.25 (that is, 0.25:1, or equivalently 1:4 if you prefer your odds in whole numbers) and the Bayes’s factor is 5 (0.75 / 0.15). The updated odds of success are now 1.25:1 (which is the same as a probability of 0.56). So, our evidence in this case is pretty strong, and it resulted in more than doubling our existing prior probability estimate.
In essence, Bayes’s formula is simply a mathematical expression of how to quantify learning from experience. It “expresses how a subjective degree of belief should rationally change in light of evidence or data.” 8 Here’s one final important implication of Bayesian updating. The point of evidence is to adjust existing beliefs; the underlying prevalence of a phenomenon, as reflected in prior beliefs, is a critical input that should not be discounted in the face of evidence. For example, the numbers estimated above stated that focus groups generate positive tests for profitable entries 75% of the time. If a focus group generates positive results, you might be tempted to estimate the probability of entry success as 75%. But, you have to account for the prior. Sure, the test results are a positive signal, but it’s also important to realize that successful entries are somewhat rare (p = 0.2 in the example). The updated estimate reflects an increase to 0.56 but not all the way to 0.75, due to the low prior probability. You may have heard the phrase “extraordinary claims require extraordinary evidence.” This phrase reflects this very point—low probability claims don’t become high probability claims unless there is very strong evidence in favor of those claims.
Estimating the Inputs
Bayesian updating is obviously subject to the garbage in, garbage out principle. If the estimates of the prior and evidence strength are low quality, the updated estimate will also be low quality. So, where do these numbers come from? Let’s start with the prior. Remember, this represents the probability you’d assign before you hear specific evidence for or against the belief. One obvious source here is prior experience and existing knowledge. Imagine your company is considering an acquisition. What do you know about acquisitions in general—how successful do they tend to be? Of the acquisitions conducted by your company in the past, what has been the success rate? You can also look to outside sources of information in forming your prior estimates. Is there expert opinion available? Is there dispersed information among a lot of different people that you can aggregate via some sort of wisdom of crowds or prediction market approach?
Evaluating evidence strength draws on the same sources of information. Say the company you are thinking about acquiring releases an earnings forecast greater than Wall Street expectations. Based on your prior experiences and any expert knowledge you can draw upon, how likely is it that you would see such a release if it is a “good” target and how likely is that you would see such a release if it is a “bad” target?
Another way to get some insight into these probabilities is to imagine a lottery scenario like the one mentioned earlier. Imagine someone is offering a lottery ticket that pays $500 in the case your belief turns out to be correct. What is the maximum you are willing to pay for that ticket? Your confidence level equals price / 500. If your price is $250, you’re 50% confident, that is, you expect to win half the time when you’re right (and net $250) and lose half the time when it turns out you were wrong. Higher prices mean higher confidence levels; on the flip side, if you’re only willing to pay something like $125, that means your confidence level is 25%.
The Benefits of Being More Bayesian
The beauty of the Bayesian approach is its generality; it offers benefits in a wide variety of areas of managerial practice. For example, the push for more of an evidence-based approach to management obviously requires managers to be able to intelligently interpret evidence, and Bayes’s rule has been suggested as an effective tool to evaluate evidence. 9 Lean startup approaches in entrepreneurship advocate treating new business ideas as hypotheses to be validated by rapid experimentation in the marketplace. This approach clearly requires a Bayesian updating approach, as an entrepreneur will begin with a certain degree of confidence in his or her product and then systematically update that estimate in the face of new evidence from customer testing. Marketing researchers have pointed to the potential of Bayesian approaches to improve decisions related to pricing, distribution logistics, new product development, and promotional campaigns. 10 A significant challenge in strategic management is how to balance exploration and exploitation activities. Addressing this question obviously involves estimating probabilities of uncertain outcomes associated with each type of activity and updating those estimates over time as the firm accumulates experience. Real options approaches have also been suggested in the strategy literature as an effective tool to deal with uncertainty; however, this approach still requires “managers to act on their subjective beliefs about the asset’s value,” which clearly links to the importance that “Bayesian rational decision makers update their beliefs to make the best possible use of all available information.” 11
Turning to examples of specific decisions benefiting from Bayesian approaches again highlights how broadly applicable the approach is. A fundamental principle that superforecasters draw upon is “Bayes’s core insight of gradually getting closer to the truth by constantly updating in proportion to the evidence.” 12 Bayesian approaches have helped firms operating in the U.S. movie industry learn as they enter multiple international markets; it informs decisions of whether to initiate phase three testing of new drug candidates at pharmaceutical company Amgen; Amazon uses machine learning informed by Bayesian techniques to improve customer recommendations and optimize logistics; staffing company Triplebyte appeals to principles of Bayesian updating to motivate its use of a series of coding tests and interviews to screen technical job candidates; and the Pennsylvania Investment Network advocates for a Bayesian approach to improve outcomes in angel investing. 13
In many respects, it’s not really necessary to argue that you should be a Bayesian. Nearly all of us are already operating as implicit updaters. We have beliefs about the world that we update as we receive more information. This implicit process, however, is subject to a lot of error, and these errors affect even expert decision makers. 14 For example, people tend to ascribe too much weight to new information causing it to skew probability estimates; they are also more likely to update beliefs after encountering favorable information compared with negative information. But, errors like these can be reduced by taking a more explicit approach; as Kahneman, Lovallo, and Sibony recently argue, “reducing errors in judgment requires a disciplined process.” 15 Being a more disciplined explicit Bayesian has a number of specific benefits, both direct and indirect.
To start with, if you’re going to update, you might as well do it right. Following the Bayesian updating approach also helps you achieve finer granularity in your probability estimates, another factor associated with higher forecast accuracy. 16 And, in some cases, calculating more accurate explicit probability estimates will have a major impact. Expected value calculations that rely on underlying probability estimates are obviously sensitive to the accuracy of those estimates. For decisions of a yes/no or go/no-go variety, being able to distinguish between success probability of 0.45 (failure more likely) and 0.55 (success more likely) fundamentally changes the decision that will be made. This ability is particularly valuable if you think about the longer-term implications. Being able to distinguish between projects with a 55% chance of success and a 45% chance of success may not have a huge amount of value for any individual choice between two projects, but being able to pick the better project consistently compounds those smaller one-off gains into a larger cumulative gain over time. In sum, applying a Bayesian approach is indispensable in domains where accuracy of probability estimates is critical, especially given that “evidence is accumulating that formal analysis—and quantification in particular—improves decision making.” 17
What’s truly remarkable about shifting your thinking and decision-making approach to a more explicit Bayesian approach is that it provides a host of benefits even if you rarely engage in exact calculation of probability estimates.
It helps you be more open-minded and intellectually humble. Bayesians don’t think of beliefs and knowledge in dichotomous terms (beliefs being either true or false). Instead, they think in continuous terms of degrees of confidence in the truth of a particular belief. This makes it much easier to adjust beliefs in the face of new evidence. You don’t have to go all the way to totally changing your mind; you just need to adjust a probability estimate. A debate about a new product launch isn’t necessarily about whether to launch the new product or not; it’s about how confident you should be regarding the success of the product. Of course, this confidence estimate then needs to be translated into a decision, but the debate isn’t focused on the yes/no determination. Greater openness to information and evidence should foster learning and more accurate beliefs over time.
Bayesian thinking can also help reduce the effect of cognitive biases. Perhaps most obviously, it reduces the impact of outcome bias, which refers to the fact that individuals tend to evaluate the quality of decisions based on outcomes rather than the processes that produce them. The problem with doing this is that sometimes decision makers get lucky (and unlucky, too). If I offer you a bet on a coin flip that pays $10,000 if it comes up tails but requires you to pay $50,000 if it comes up heads, is it a good decision to take this bet? I suspect most of us would agree this would be a bad decision. Even if someone took the bet and the coin came up tails, it’s still a bad decision. In her fascinating book 18 on decision-making skills learned over her professional poker career, Annie Duke refers to focusing on outcomes as the “resulting fallacy.” Decisions should be evaluated based on the quality of the process used in making them, not solely on the outcomes or results they produce. This point is so critical that one decision-making textbook claims that its most important message is that “the most fundamental distinction in decision making is that between the quality of the decision and the quality of the outcome.” 19 And Bayesian thinking emphasizes this with its requirement to consider that not only can good outcomes follow good processes, but also follow from bad ones—that is, p(good outcome | ~good process) ≠ 0. Outcomes certainly are not irrelevant—they provide evidence that should be used to update your assessment of how confident you are that you have a good decision-making process. In a related way, thinking like a Bayesian is also an effective tool to combat confirmation bias, which refers to the tendency to seek out and pay greater attention to information and evidence that is consistent with our current beliefs being true. The Bayesian approach requires you to consider that evidence and outcomes may occur if your beliefs are incorrect. Being a good Bayesian forces you to acknowledge that any particular piece of evidence could have many different possible causes. Even if information is consistent with your prior belief, you have to consider whether that information might exist even if you’re wrong.
Adopting a Bayesian approach provides insight into what distinguishes evidence of greater value. Strong evidence is evidence that is much more likely if a belief is true compared to if a belief is false. And, it’s this second part that’s easy to forget. Let’s go back to the focus group example. Say one of your managers explains there are two different types of focus group approaches to testing new products. Past experience indicates that successful products that have gone through the first approach saw positive testing results in 90% of the cases—that is, p(positive test | good product) = 0.9. The second approach has generated positive results for good products in 80% of the cases, p(positive test | good product) = 0.8. The first approach is obviously the way to go, right? A positive result from that approach would be stronger evidence, right? Not necessarily. By now, you should know that the strength of evidence, or how much it affects the degree to which prior beliefs should be updated, also depends on p(positive test | bad product). Say this false positive rate is 20% for the first approach and 10% for the second. This means the Bayes’s factor for the first approach is 4.5 while the Bayes’s factor for the second is 8. A positive test from the second approach is much stronger evidence in favor of the product.
Being a more explicit Bayesian fosters clarity of communication, a critical consideration for group-based decisions. As mentioned earlier, managers constantly communicate in probability terms (“probable,” “possible,” “likely,” etc.). The problem is that these are poorly defined terms that can mean very different things to different managers. A recent survey asking people to define phrases like these in numerical terms shows the wide differences. 20 For example, an event with a “real possibility” of occurring spanned a probability range of 0.2 to 0.8 across respondents. The fact that one manager using the word likely may have a degree of confidence of 0.6 while another using the same word has a degree of 0.8 can be problematic. Communicating in actual numbers, even if they are rough estimates, at least forces everyone to clearly specify estimates.
Conclusion
Bayesian approaches are most helpful in uncertain situations that call for probabilistic thinking where beliefs need to be updated regularly as evidence is encountered and learning occurs. This description applies to a wide range of managerial decisions that often need to be made in the face of uncertainty. We’ve considered several obvious examples of areas where Bayesian thinking applies, such as new product development, entry into new geographic markets, and mergers & acquisitions. But, perhaps the most valuable aspect of thinking in Bayesian terms is its wide applicability—it applies to just about any sort of managerial issue that involves the formation and updating of beliefs. The more accurate these beliefs are, the better your managerial decisions will be.
Becoming a more explicit Bayesian requires following a number of relatively simple steps. Begin by thinking probabilistically, shifting your view of probability to see it as a means to quantify strength of subjective beliefs. Bayesians think probabilistically rather than in black-and-white terms. Next, start assigning exact estimates to your degrees of belief. As you encounter new evidence, update those prior estimates and base the magnitude of update on the strength of the evidence. This step requires you to think about not only how likely it would be to observe that evidence if your beliefs are correct, but also how likely it would be to observe that evidence if your beliefs are incorrect. Finally, combine prior probabilities and evidence strength estimates using Bayes’s rule. But, even if you never calculate an exact updated probability using Bayes’s rule, adopting a more explicit Bayesian approach can improve your thinking in a variety of ways. You’re likely already updating implicitly—why not make it more explicit and do it better?
Supplemental Material
Bayes_Calculator – Supplemental material for Using Bayesian Updating to Improve Decisions under Uncertainty
Supplemental material, Bayes_Calculator for Using Bayesian Updating to Improve Decisions under Uncertainty by Brian T. McCann in California Management Review
Footnotes
Appendix
Acknowledgements
I am very grateful for the advice and suggestions received from two anonymous California Management Review (CMR) reviewers. I also appreciate helpful feedback and comments on earlier drafts from Spencer Betts, Justin Fox, Spencer Greenberg, Michael Mauboussin, and Richard Willis.
Supplemental Material
Supplemental material for this article is available online.
Notes
Author Biography
Brian T. McCann is the David K. Wilson Chair and Associate Professor of Strategic Management at the Owen Graduate School of Management, Vanderbilt University (email:
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
