Abstract
Sales managers are unlikely to reap the benefits of implementing predictive analytics applications when salespeople show aversion to or lack understanding of these applications. For managers, it is essential to understand which factors mitigate or exacerbate these challenges. This article investigates these factors by studying the implementation of an application that predicts customer churn. Using 9.7 million transactions from a business-to-business company, the authors develop a predictive model of customer churn, implement it in a field experiment, and study its treatment effects using causal forests. Furthermore, the authors manipulate one specific mitigation strategy proposed by prior literature: the fostering of users’ realistic expectations regarding the accuracy of an algorithm. The results show that the effectiveness of the churn prediction application strongly depends on customer characteristics (most importantly the predicted churn probability and prior revenue) and salesperson characteristics (technology perceptions, abilities, and selling orientations). Fostering realistic expectations improves the effectiveness of the churn prediction only under very specific circumstances. Two follow-up stimuli-based experiments conceptually replicate key results of the field study. Therefore, this article helps build theory on predictive sales analytics and provides specific guidance to managers aiming to increase their return on analytics investments.
Predictive analytics is a cutting-edge business trend with the potential to contribute more than $9 trillion a year in value to the global economy (Chui et al. 2018). Predictive analytics support decision making by providing a quantitative estimation of variables for observations not incorporated in the data set on which the estimation is based (Shmueli and Koppius 2011; Wedel and Kannan 2016). Owing to its data richness, the sales function in particular provides a multitude of opportunities for predictive analytics (Habel, Alavi, and Heinitz 2023), such as predictions of lead conversion likelihoods to prioritize prospects, next-best offers to enable cross-selling, and customer churn likelihoods to improve retention.
However, companies face persistent challenges when implementing predictive sales analytics (Alavi and Habel 2021; Harvard Business Review Analytic Services 2021). One challenge is that employees tend to be averse to predictive analytics (Ammanath, Hupfer, and Jarvis 2020) and lack the necessary skills or understanding to effectively apply the new tools. As a result, enhanced salesperson productivity is inconsistent when adopting such tools. This is emphasized in a 2021 survey we conducted with 189 managers (Web Appendix A describes the sample): although 77% expected the importance of predictive sales analytics to strongly increase within five years, more than half (55%) reported that salespeople harbor concerns about using predictive analytics. Many salespeople tend to mistrust the technology (46%) or lack the ability to use predictive analytics effectively (48%).
To benefit from predictive analytics, sales managers require an intricate understanding of the factors that exacerbate or mitigate these challenges, but to date, these factors are poorly understood (for a literature review, see Table 1). Conceptual works have underlined the importance of predictive analytics for marketing research and practice and have significantly advanced knowledge of its theoretical foundations (Wedel and Kannan 2016). Empirical research, however, has largely focused on the performance consequences of marketing analytics at the firm level, adopting survey-based designs (Germann, Lilien, and Rangaswamy 2013). These papers examined the deployment of analytics in general but not specifically predictive analytics. Recently, a few papers assessed specific artificial intelligence–based predictive analytics tools in sales and marketing contexts, such as automated sales and service coaches (Kim et al. 2022; Luo et al. 2021) or lead qualification systems (D’Haen et al. 2016). Although these works provide valuable insights that predictive analytics tools can indeed leverage productivity in different domains, they (1) rarely focused on the sales context, even though sales organizations are a highly relevant area for predictive analytics tools, (2) scarcely examined contingencies determining adoption and effective use of these tools, (3) rarely adopted a longitudinal adoption perspective, and (4) scarcely examined strategies to mitigate aversion to and misuse of predictive analytics tools.
Selected Literature on Analytics in Marketing and Sales.
Notes: DSS = decision support system.
To help close these prevailing gaps in the literature, our article investigates how the implementation of a predictive sales analytics tool affects customer sales revenue—and which factors mitigate or exacerbate potential challenges. We initially conducted a field experiment implementing a customer churn prediction tool at a business-to-business (B2B) distributor. The tool provides salespeople with monthly estimated churn probabilities of their customers. To reduce salespeople's aversion to algorithms, we test emergent theory and manipulate salespeople's expectations regarding the accuracy of the churn prediction (Burton, Stein, and Jensen 2020). After the implementation of the tool, we monitor customer sales revenue over 12 months, yielding a data set of 234,505 customer–month observations. Results show that the effects of the implementation are highly heterogenous and depend on a broad set of customer and salesperson characteristics, the latter of which can be further classified into (1) technology perceptions, (2) abilities, (3) and selling orientations. Fostering realistic expectations improves the effectiveness of the churn prediction only under specific circumstances—for example, if salespeople harbor low trust in algorithms but are highly oriented toward learning. We subsequently conceptually replicate key results in two stimuli-based experiments with salespeople (Study 2 in the main text and a supplemental study in Web Appendix B).
These findings have important implications for the academic discipline and sales practice. Notably, prior research suggests that predictive sales analytics applications should improve salespeople's decision making and, ultimately, customer sales revenue (e.g., Agnetis, Messina, and Pranzo 2010; Caigny et al. 2020; Kumar, Leszkiewicz, and Herbst 2018). However, we find that such beneficial effects do not unconditionally materialize. Instead, the value salespeople derive from these applications depends on customer characteristics; the interplay of salespeople's technology perceptions, abilities, and selling orientations; and the way companies manage salespeople's expectations of predictive sales analytics applications. We thus contribute to marketing research by uncovering a rich set of moderators from different conceptual categories that govern the effects of implementing predictive sales analytics.
Sales managers can employ our taxonomy of moderating factors to optimize outcomes of implementing predictive sales analytics tools. In essence, sales managers have three pathways to mitigate key implementation challenges: (1) providing an environment that gives salespeople the opportunity to learn how to benefit from a predictive analytics tool, (2) taking salespeople's likelihood to benefit into account when deciding for whom and how to implement a predictive analytics tool, and (3) carefully deciding when to couple predictions with decision rules.
Mitigating Aversion to and Misuse of Predictive Analytics
Previous research has reported persistent issues regarding salespeople's adoption of new technologies and effective usage (Bohling et al. 2006; Speier and Venkatesh 2002). Consequently, our basic proposition rests on the idea that to leverage the potential of a churn predictive analytics tool, companies have to mitigate salespeople's potential aversion to and misuse of the tool (Kim and Kankanhalli 2009). In what follows, we propose salesperson and customer characteristics that might be contingencies of the impact of predictive sales analytics on customer sales revenue.
Salesperson Characteristics as Contingency Factors
We synthesize three clusters of salesperson characteristics from prior literature: salespeople's technology perceptions, abilities, and selling orientation. We elaborate on each in the following subsections.
Technology perceptions
Prior research across disciplines converged on the notion that individuals’ perceptions of technology strongly shape their reactions to it (Shibl, Lawley, and Debuse 2013; Speier and Venkatesh 2002). To be more precise, upon encountering a new technology in their organizations, salespeople appraise this technology's potential benefits and threats for themselves (Bala and Venkatesh 2015; Beaudry and Pinsonneault 2005). Salespeople's perceptions of these benefits and threats in turn determine their attitude toward the technology, which manifests in salespeople's adoption behavior. The technology acceptance model, which is a technology-specific adaptation of the well-established theories of reasoned action and planned behavior, represents a seminal framework regarding such salesperson evaluations (Davis, Bagozzi, and Warshaw 1989).
Inspired by developments of innovative, cutting-edge technologies, recent frameworks focus less on conventional factors that constitute assets for employees, such as technical functions or ease of use, and more on employees’ perceived threat by technologies that may impair technology adoption (Burton, Stein, and Jensen 2020; Castelo, Bos, and Lehmann 2019). Employees perceive such threats, for example, if newly implemented technologies interfere with employee autonomy and work processes (Alavi and Habel 2021). In addition, the prominent literature stream of algorithm aversion indicates that individuals quickly lose trust in algorithms that deliver superior but imperfect predictions (Dietvorst, Simmons, and Massey 2015, 2018). One major driver of this phenomenon is false expectations as to what an algorithm can achieve, which has led researchers to propose the development of algorithmic literacy as a mitigation strategy (Burton, Stein, and Jensen 2020). Such algorithmic literacy explains that “a decision maker has to be able to tolerate error as inherent to any decision task” (Burton, Stein, and Jensen 2020, p. 223). Consequently, mitigating salespeople's aversion to the churn predictive analytics tool might include creating realistic expectations for the tool. 1
Abilities
Different fields of research accept the idea that salespeople's abilities should influence their reactions to the implementation of a churn predictive analytics tool (Bala and Venkatesh 2015; Beaudry and Pinsonneault 2005). Salespeople's abilities should (1) foster the perceived usefulness of new technologies and (2) moderate the effects of the adoption on customer sales revenue (Habel, Alavi, and Heinitz 2023).
First, salespeople need to be endowed with basic abilities to use and control the technology (Bala and Venkatesh 2015). A lack of these abilities will pose barriers to technology adoption because salespeople are bound to anticipate few benefits from the technology without the required technical skills. Thus, learning the required technical skills is essential for promoting the productive usage of predictive sales analytics tools. However, naturally, learning such skills may take time. Consequently, increases in customer sales revenue may not immediately occur for salespeople after the implementation of predictive analytics tools but may do so only after a transition period (Ahearne et al. 2008; Kayande et al. 2009).
Beyond technical skills, salespeople may need proficient selling skills to perceive the benefits of a customer churn prediction tool. For instance, top-performing salespeople may have a large portfolio of customers and, thus, a greater need to prioritize customers, which entails a more pronounced demand for timely information on customer churn. However, the relationship between salespeople's abilities and the perceived benefits of the tool may be complex: although previous research has found that experience with algorithmic decisions increases the utilization of an algorithm, domain-specific expertise reduces it (Montazemi 1991; Whitecotton 1996).
Second, adopting a new churn predictive analytics tool should not automatically increase customer sales revenue, but this effect should depend on salespeople's ability to effectively use the information provided by the tool. While new analytics technologies may deliver superior information about customers to salespeople, salespeople may lack the necessary selling skills to leverage such superior information (Burton, Stein, and Jensen 2020; Habel, Alavi, and Heinitz 2023). Most prominently, obtaining a signal that a customer is likely to churn may be futile if a salesperson lacks the ability to devise an effective strategy to recapture the customer's loyalty.
Selling orientations
Salespeople's behavioral orientations govern their (1) adoption and (2) effective use of newly implemented technologies (Goodhue and Thompson 1995; Hunter and Perreault 2007; Speier and Venkatesh 2002; Tornatzky and Fleischer 1990). First, if salespeople are oriented toward behaviors related to serving and retaining customers, they should value accurate customer information, increasing salespeople's perceptions of tool usefulness (Galbraith 1974; Goodhue and Thompson 1995). Second, salespeople's orientation may also directly influence how effectively they can use the churn prediction. For example, if salespeople tend to adapt their behaviors based on novel technologies, they might be able to improve their decision making with the churn prediction (Habel, Alavi, and Heinitz 2023).
Customer Characteristics as Contingency Factors
A churn prediction application promises value to salespeople by equipping them with the potential to serve customers more effectively and efficiently (Guenzi and Habel 2020; Habel, Alavi, and Heinitz 2023). The extent to which salespeople perceive this promise as attractive and draw value from it can be predicted through the expected value theorem, according to which salespeople will evaluate (1) the possible outcome from using the churn prediction and (2) the probability that they will achieve this outcome. We expect that both evaluations will crucially depend on the characteristics of the specific customer whose churn probability is predicted (Galbraith 1974; Goodhue and Thompson 1995). First, for example, retaining a sizable customer that would otherwise churn would exhibit a higher impact on a salesperson's sales revenue (and thus commission) than retaining a smaller customer. Likewise, using the churn prediction to extend the relationship with a sizable customer (e.g., through upselling and cross-selling) should yield higher possible outcomes. Second, customer characteristics may determine salespeople's perceived probability that they will achieve these outcomes. For example, if a customer relationship is characterized by high uncertainty (e.g., due to highly volatile sales revenues), salespeople may perceive uncertainty about their ability to use the churn prediction effectively (Achrol and Stern 1988). Thus, in summary, a customer's characteristics might determine how a salesperson adopts and draws value from adopting the churn prediction.
Study 1: Field Experiment
Research Context
Our data are from a national B2B wholesaler of construction supplies, such as paint, wallpaper, and insulation. Twelve regional organizations participated in the experiment. This context is well suited for our study for three reasons. First, the company sells to customers such as construction businesses and workshops through salespeople who hold the primary responsibility for the customer relationships within their territories. Second, despite decentralized sales activities across the regional organizations, the company centrally records and stores data in a consistent format. Third, the company considers customer churn to be an important challenge due to high competition and low barriers to changing the supplier.
Customer Churn Prediction
We developed a tool that estimates the churn probabilities of individual customers through extreme gradient boosting (XGBoost; details in Web Appendix C). We classify customers as churned when they do not buy a product in the next month and/or the month after. We adopted this classification from the company's management—it is based on the short purchase cycles in the industry, with customers buying on average every 11 days. Throughout the experiment, we predicted approximately 13,000 customer churn probabilities each month. On average, the monthly customer churn prediction models achieved accuracy, recall, and precision of 80%, 79%, and 29%, respectively, which aligns with prior studies (e.g., Gordini and Veglio 2017). 2
Experimental Procedure
Before the experimental phase, we collected an online survey from salespeople and focused on potential sources of heterogeneity in the treatment effect (see the “Selection of Covariates” section). We received 130 complete responses, for a response rate of 83%. We then assigned regional organizations to the experimental conditions using a random number generator. That is, within each of the 12 regional organizations, all salespeople were assigned to the same condition to avoid treatment diffusion. The experimental conditions are the following:
The experimental phase lasted for 12 months. At the beginning of each month, salespeople in the two treatment conditions (churn prediction only and churn prediction with EXM) received the predicted churn probabilities for each customer. We did not prescribe actions (e.g., how often to call on customers with a high churn probability) because our goal was to investigate the implications of predictive rather than prescriptive sales analytics (Appelbaum et al. 2017).
We combine our survey data with monthly repeated measures on a customer level 12 months before and 12 months after the implementation of the customer churn prediction. The data set contains 3,316 customers and 43 salespeople in the churn prediction only condition, 3,194 customers and 41 salespeople in the churn prediction with EXM condition, and 3,648 customers and 46 salespeople in the control condition. The average monthly number of customers with transactions per salesperson is 63 (SD = 30). On average, a salesperson achieves monthly sales revenues of €150,197 (SD = €80,508).
Empirical Strategy and Identification Concerns
We analyze the effect of our treatment using the causal forest methodology (Athey and Wager 2019). A casual forest combines causal inference with random forests, estimating the conditional average treatment effect (CATE), τ, on an outcome variable, Yi, conditional on the treatment assignment, Wi, and a vector of covariates, Xi:
Nonrandom selection of salespeople
As discussed previously, we assign regional organizations rather than salespeople to the experimental conditions to avoid treatment diffusion. In case there are systematic differences between the regional organizations, our data may be subject to a selection bias. We thus initially compare the sales revenue per salesperson and customer across regional organization experimental conditions (see Table 2). The results suggest that the regional organizations are largely comparable (further analyses in Web Appendix D).
Study 1: Comparability Between Treatment Conditions and Control Condition.
In 12 months before experiment (indexed).
Heterogeneity in the treatment effects at the customer level
Although the treatments are administered at the salesperson level, the treatment effects should vary at the customer level. This is because salespeople receive the churn probabilities every month for every individual customer in their portfolio and subsequently adapt their approaches to their customers. To uncover the effect of these adaptations, we analyze the CATE at the customer–month level and allow it to covary with a customer's predicted churn probability in the respective month.
Nonrandom selection of customers to salespeople
Customers may be assigned to salespeople for unobserved strategic reasons, which are known to salespeople and shape salesperson effort and outcomes in the treatment conditions (e.g., Nair, Manchanda, and Bhatia 2010). To mitigate biases, we employ a wide variety of customer-related and salesperson-related covariates of the treatment effect (see the “Selection of Covariates” section).
Salesperson strategic changes
Salespeople may strategically change their behaviors to suit the firm's taste (i.e., stopping retention), so that estimated treatment effects stem from unobserved salesperson behavior changes over time rather than from the intervention. We use three empirical measures to mitigate this issue. First, as outlined previously, we employ a wide variety of covariates for the estimation of heterogeneous treatment effects, aiming to pick up intrinsic differences in the way salespeople use the churn predictions. Second, we specify a customer's predicted monthly churn probability as a covariate of the treatment effect, aiming to partial out customer quality. Third, to capture learning dynamics, we add a month count variable as a covariate of the treatment effect, allowing the treatment effect to vary across months.
Unobserved salesperson heterogeneity
Despite the wide variety of covariates, unobserved heterogeneity at the salesperson level might cause omitted variable bias. We control for such heterogeneity using an approach developed by Jens, Page, and Reeder (2021). This approach comprises estimating salesperson fixed effects in a regression (first step) and adding these fixed effects as a covariate to the causal forest estimation (second step). Jens, Page, and Reeder show that if unobservables predict the outcome (i.e., customer sales revenue), their approach is effective at recovering both the ATE and heterogeneity in the treatment effect. Furthermore, their approach outperforms alternative approaches, such as including dummy variables in the causal forest. We specify the following first-step fixed-effects regression, estimated for the full 24 months of data (12 months before and 12 during the experiment):
Salesperson departures
It is possible that salespeople who decided to leave the firm during our experimental phase increasingly disengaged and thus neglected customer churn. Using data from these salespeople might thus create identification confounds (Schmitz et al. 2020). Therefore, we drop customers whose salespeople departed during the experiment.
Nonresponse bias
We measured several of the covariates in an employee survey prior to the implementation of the customer churn prediction (response rate of 83%). We compared respondents’ and nonrespondents’ age (t = 1.75, p = .09), company tenure (t = 1.56, p = .13), and cumulative sales revenues in the 12 months before the implementation (t = −.02, p = .99). Because these do not differ significantly, nonresponse bias is unlikely to bias our data.
Selection of Covariates
We already mentioned two covariates essential for our identification strategy: a customer's predicted churn probability in each month as well as a month count variable. In addition, we adopt potential covariates from the literature streams outlined previously.
Customer characteristics
Customer characteristics may determine the salesperson's demand to receive accurate customer information, increasing salespeople's use of and benefits from the prediction (Goodhue and Thompson 1995). Demand for information should increase for more sizable customers, leading us to include a customer's mean sales revenue levels (in €) across the 12 months prior to the experiment. We also include the heterogeneity of a customer's sales revenue, operationalized as the variation coefficient across the 12 months.
Salesperson characteristics: Technology perceptions
In our preexperimental survey, we measured a set of technology perceptions that we derived from two streams of the literature. First, building on the technology acceptance model (Davis, Bagozzi, and Warshaw 1989), we measure expected usefulness (four items, e.g., “I would find the customer churn prediction useful for my work”; full measurements for all survey constructs in Web Appendix E) and expected ease of use (three items, e.g., “I think the customer churn prediction would be easy to use”). Second, building on the more recent literature on algorithm aversion (Dietvorst, Simmons, and Massey 2015), we measure general trust in algorithms (three items, e.g., “In general, I trust automatically generated computer predictions”) and expected error in churn prediction (three items, e.g., “I think that the churn prediction would often be wrong for my customers”). Because individuals are more likely to utilize algorithms if they feel they have control over the results (Dietvorst, Simmons, and Massey 2018), we also measure expected constraint through churn prediction (three items, e.g., “Through the churn prediction I would feel more strongly controlled”).
Salesperson characteristics: Abilities
We derive four sets of ability-related covariates from the literature. First, sales literature often conceives the level and heterogeneity of salesperson performance as indicators of their ability (e.g., Boichuk et al. 2019; Bommaraju and Hohenberg 2018). Therefore, we extract from company records salespeople's mean sales revenue levels (in €) and churn level (as the percentage of customers churned) for the 12 months prior to the experiment. To measure sales revenue heterogeneity and churn heterogeneity, we estimate the variation coefficients of these variables across months. Second, because ability should correlate with experience (e.g., Habel, Alavi, and Linsenmayer 2021b), we measure sales experience as the number of years salespeople have worked in sales.
Third, whether salespeople use and benefit from predictive analytics should depend on their ability to carry out the activities that predictions aim to support (Bala and Venkatesh 2015; Habel, Alavi, and Heinitz 2023). In our context, the predictions aim to enable salespeople to recognize which customers will churn, prioritize customers accordingly, and retain them. Accordingly, we measure customer churn prediction efficacy (three items, e.g., “I am good at assessing whether I will lose a customer”), customer prioritization efficacy (three items, e.g., “I think about which customers I need to prioritize so they keep purchasing from us”), and customer retention efficacy (three items, e.g., “I can influence whether my customers do or do not churn”).
Fourth, how salespeople use analytics applications may interact with their intuition as well as their grasp of mathematical concepts (Burton, Stein, and Jensen 2020). We thus measure intuitive efficacy (three items, e.g., “I believe that I make good decisions when relying on my intuition”) and mathematical efficacy (three items, e.g., “I am mathematically talented”).
Salesperson characteristics: Selling orientations
Lastly, we measure four sets of selling orientations that may affect salespeople's information demand and thus their usage of and benefits from the churn prediction. First, ample research (e.g., Franke and Park 2006) has established the importance of salespeople's adaptive selling orientation (five items, e.g., “I use an individual sales approach for every customer”) and customer orientation (five items, e.g., “I try to find out the customer's needs”). Second, sales research often quantifies the extent to which salespeople emphasize building and maintaining relationships with customers (e.g., Cron et al. 2021; Habel, Alavi, and Linsenmayer 2021a). We borrow two corresponding constructs (DeCarlo and Lam 2016), hunting orientation (three items, e.g., “The best part of my job is acquiring new customers”) and farming orientation (three items, e.g., “The best part of my job in spending time with my existing customers”). Third, sales literature (e.g., Kohli, Shervani, and Challagalla 1998; Sujan, Weitz, and Kumar 1994) has established the importance of learning orientation (three items, e.g., “An important part of my job as a salesperson is continuous improvement of my selling skills”) and performance orientation (three items, e.g., “I want my coworkers to see me as a good salesperson”). Fourth, adoption may hinge on innovation orientation (three items, e.g., “I see myself as very innovative regarding new technologies”).
Controls
We control for salespeople's gender, age, and the number of weekly working hours (e.g., Habel, Alavi, and Linsenmayer 2021b). Web Appendices F and G report correlations and descriptives. All survey measures are discriminant according to the Fornell–Larcker criterion (Fornell and Larcker 1981). Furthermore, following Lindell and Whitney (2001), we examined common method variance based on the lowest positive correlation in our data (rgeneral trust in algorithms, hunting orientation < .01), concluding that a common method bias is unlikely.
Uncovering the Average Treatment Effect
Model-free analysis
In the 12 months of our experiment, the mean monthly customer sales revenue in the churn prediction only condition is not significantly different from customer sales revenue in the control condition (Mchurn only = 2,036.323, Mcontrol = 1,836.863; t = −1.868, p = .062). Similarly, the change in mean monthly customer sales revenue for the 12 months of our experiment relative to the 12 months before does not differ between the churn prediction only and the control condition (Mchurn only = −8.217, Mcontrol = −22.311; t = −.348, p = .728).
Conversely, the churn prediction with EXM condition has a higher mean monthly customer sales revenue than the control condition (Mchurn with EXM = 2,056.024, Mcontrol = 1,836.863; t = −2.3, p = .021), though the change in customer sales revenue relative to before the experiment is not significantly different in the conditions (Mchurn with EXM = −86.065, Mcontrol = −22.311; t = 1.384, p = .166). Furthermore, when comparing the two treatment conditions, we find no significant differences for either absolute levels of mean monthly customer sales revenue (Mchurn only = 2,036.323, Mchurn with EXM = 2,056.024; t = −.175, p = .861) or the change relative to before the experiment (Mchurn only = −8.217, Mchurn with EXM = −86.065; t = 1.596, p = .111).
Model-based analysis
We estimate the CATE using two causal forests—one for the effect of the churn prediction only versus control and one for the churn prediction with EXM versus control. We specify the monthly sales revenue with a customer throughout the experimental phase as the outcome variable and enter all covariates discussed previously. We cluster the analysis to account for the nesting of customers in salespeople (Athey and Wager 2019). Web Appendix H provides details on the parameter tuning.
Figure 1 shows the distribution of the CATE. Both ATEs are nonsignificant, suggesting that the monthly sales revenue with a customer is independent of the treatment condition the corresponding salesperson was assigned to. 3 This corroborates our model-free analysis. However, the figure also shows heterogeneity in the treatment effect, which we analyze next.
Study 1: Distribution of CATE.
Understanding Heterogeneity in the Treatment Effect
We initially inspect the importance of the covariates extracted from the causal forest estimation (Web Appendix I). For both the churn prediction only (vs. control) and the churn prediction with EXM (vs. control), the most important covariates are the prior mean sales revenue level generated with a customer as well the heterogeneity of that sales revenue. Furthermore, the predicted churn probability emerges as highly important, which suggests that salespeople made case-by-case decisions depending on the size and churn probability of a customer.
How do the covariates specifically shape the CATE? Answering this question is not straightforward due to the nonparametric estimation of the causal forest—that is, the covariates are likely to exhibit nonlinear relationships and form higher-order interactions with each other. To find tendencies in these relationships, we follow prior literature and inspect the effects of covariates on the estimated CATE using parametric models (e.g., Chen et al. 2020; Guo, Sriram, and Manchanda 2021). Thus, we specify the following model:
Main results
Table 3 provides the results of the estimation. Models 1 (with fixed effects) and 2 (without fixed effects) explain the CATE for the churn prediction only (vs. control) and Models 3 (with fixed effects) and 4 (without fixed effects) explain the CATE for the churn prediction with EXM (vs. control). For the sake of parsimony and to reduce the likelihood of false positives, the following discussion focuses on highly significant interactions (p < .01).
Study 1: Sources of Heterogeneity in Customer–Month-Level CATE.
*p < .05. **p < .01. ***p < .001.
Notes: Standard errors are in parentheses and clustered in salespeople.
Several insights emerge. First, for the churn prediction only, the month positively affects the CATE (Model 2: β = .03045, p < .001). Thus, the treatment effect grows more positive over time, suggesting that salespeople increasingly adopt the churn prediction or learn how to utilize it effectively. Interestingly, however, this effect is negative in the churn prediction with EXM condition (Model 4: β = −.15299, p < .001). It seems that the EXM manipulation accomplished the opposite of what we intended: Rather than mitigating aversion, on average EXM seems to have deteriorated adoption and learning over time. This finding might be explained by two unintended consequences of the EXM manipulation: (1) Expecting errors may have led salespeople to perceive the churn prediction as less instrumental to reaching their goals; thus, they adopted it less as time progressed (Davis, Bagozzi, and Warshaw 1989). (2) Expecting prediction errors may have sensitized and triggered salespeople to be more likely to spot errors (Habel et al. 2016), which may have undermined trust, leading them to more quickly stop using the churn prediction (Davis, Bagozzi, and Warshaw 1989). Given the surprising nature of this finding, we replicated the negative effect of EXM in a supplemental study in Web Appendix B.
Second, the CATE for the churn prediction only is more positive the higher the predicted churn probability (Model 2: β = .10790, p < .001). This is plausible, as salespeople likely prioritize customers with high churn probabilities aiming to either retain them or skim remaining opportunities. Again, the effect is reversed for the churn prediction with EXM (Model 4: β = −.21417, p < .001), possibly because the EXM led salespeople to mistrust high churn probabilities and thus draw less value from them (see also the supplemental study in Web Appendix B).
Third, the CATE for the churn prediction only is more positive for customers with a high prior sales revenue level (Model 2: β = .66201, p < .001) and less positive for customers with high prior sales revenue heterogeneity (Model 2: β = −.12840, p < .001). Thus, salespeople use churn prediction to prioritize particularly big and stable customers. Again, EXM leads to a lower CATE for customers with a high prior sales revenue level (Model 4: β = −.86756, p < .001), again pointing to salespeople potentially mistrusting these predictions qualified by EXM.
Fourth, the CATE for the churn prediction only is more positive for salespeople with a high level of prior sales revenue (Model 2: β = .08915, p < .01). This supports the importance of salesperson ability when implementing the churn prediction.
Fifth, the constant is negative in Model 2 (β0 = −.12840, p < .001) but positive in Model 4 (β0 = .11213, p < .001). As we standardized all covariates, this suggests that at mean (i.e., zero) values of all covariates, the CATE is negative for the churn prediction only and positive for the churn prediction with EXM. Naturally, this estimate differs for other values. For example, at a prior customer sales revenue level one standard deviation above the mean, the constant is positive in Model 2 (β0 = .53361, p < .001) but negative in Model 4 (β0 = −.75543, p < .001).
Exploratory interaction effects
At first glance, it might seem surprising that only a few of our covariates have effects on the CATE, as their theoretical rationale seems compelling. Notably, though, owing to the nonparametric estimation of the causal forest, their impact may simply not be “linear enough” to be picked up by our linear regression. Therefore, we take an exploratory approach to probe for interaction effects between covariates (Chen et al. 2020), building on Models 2 and 4 from before. Table 4 provides the estimated interactions.
Study 1: Exploratory Interactions Affecting Customer–Month-Level CATE.
*p < .05. **p < .01. ***p < .001.
Notes: Standard errors are in parentheses. Full results table in Web Appendix J.
We extract five insights from this exploratory analysis. First, the predicted churn probability has a quadratic rather than a linear effect on the CATE (Model 1: βquadratic = .11303, p < .001; Model 2: βquadratic = .09467, p < .001), which interacts with a customer's prior sales revenue (Model 1: β = .39473, p < .001; Model 2: β = −.19966, p < .001). Figure 2 plots this effect and reveals that for the churn prediction only (Panel A), the CATE is positive for large customers who have a very low or very high predicted churn probability. Conversely, the CATE is barely affected for small customers. A plausible explanation is that the churn prediction led salespeople to prioritize large customers with low churn probabilities (to extend the relationship) and high churn probabilities (to save the relationship or skim remaining opportunities). For the churn prediction with EXM (Panel B), the CATE is negative for large customers. Perhaps the EXM confused salespeople and led them to make disadvantageous decisions regarding these customers (Xu et al. 2022).
Study 1: Plot for Insight 1 (Predicted Churn Probability × Prior Customer Sales Revenue Level).
Second, similarly, the month count has a quadratic rather than a linear effect on the CATE (Model 1: βquadratic = .04932, p < .001; Model 2: βquadratic = .08623, p < .001), which for the churn prediction only interacts with salespeople's experience (Model 1: β = .00723, p < .05) and learning orientation (Model 1: β = .01467, p < .001). Figure 3 provides corresponding interaction plots, showing that the CATE decreases in early months and only increases in later months, reflecting a learning curve. The dip in the CATE is less pronounced and passes faster for salespeople with high experience (Panel A) and high learning orientation (Panel B). For the churn prediction with EXM, the quadratic effect of the month points to an even more pronounced dip in the CATE (see the plot in Web Appendix K), but we do not find evidence of interaction effects with sales experience and learning orientation.
Study 1: Interaction Plots for Insight 2 Regarding Churn Prediction Only.
Third, we find positive interaction effects between technology perceptions and selling orientations on the CATE of the churn prediction only. Specifically, expected usefulness interacts with adaptive selling orientation (Model 1: β = .05730, p < .01) and general trust in algorithms interacts with learning orientation (Model 1: β = .04925, p < .05). Figure 4 suggests that utilizing the churn prediction effectively requires salespeople to harbor positive perceptions of its effectiveness (Panel A) and be ready to learn and adapt their behavior (Panel B).
Study 1: Interaction Plots for Insight 3 Regarding Churn Prediction Only.
For the churn prediction with EXM, general trust in algorithms and learning orientation exhibit a negative interaction effect on the CATE (Model 2: β = −.02684, p < .01; see interaction plot in Web Appendix K). A potential explanation is that when general trust in algorithms is low, EXM addresses salient concerns and thus fosters salespeople's acceptance of the churn prediction (Burton, Stein, and Jensen 2020; Dietvorst, Simmons, and Massey 2015), leading them to benefit from the prediction if their learning orientation is high. Conversely, when trust in algorithms is high, EXM makes salespeople uncertain and impedes the value of the prediction.
Fourth, for the churn prediction only, the interaction between customer churn prediction efficacy and customer retention efficacy has a negative effect on the CATE (Model 1: β = −.03551, p < .01; see Figure 5, Panel A). This finding aligns with the theoretical proposition that predictive analytics creates lower value for salespeople who already possess the skills for which the prediction aims to enable them (Habel, Alavi, and Heinitz 2023; Luo et al. 2021).
Study 1: Interaction Plots for Insights 4 and 5 Regarding Churn Prediction Only.
Fifth, the interaction between customer orientation and farming orientation has a positive effect on the CATE (Model 1: β = .06027, p < .01; Model 2: β = .02633, p < .01). Figure 5, Panel B, shows the corresponding interaction plot for the churn prediction only condition (for the churn prediction with EXM, see Web Appendix K). This suggests that salespeople who focus on the needs (i.e., high customer orientation) of existing customers (i.e., high farming orientation) benefit more from the churn prediction. Farming orientation may lead salespeople to intensively utilize the churn prediction application because preventing customer churn constitutes an essential task of farming salespeople. Combined with customer orientation, it may allow them to create value specifically for these customers at risk.
Simulation of Ideal Allocation to Treatment Groups
The heterogeneous CATE suggests that firms should not implement a churn prediction across the board, but they need to carefully decide which predicted churn probabilities for which customers to show to which salespeople. To examine the potential impact of such a nuanced approach, we simulated the assignment of salesperson–customer–months to the ideal treatment group and examined the resulting uplift in the firm's sales revenue (Chen et al. 2020). Specifically, salesperson–customer–months for whom neither the churn prediction only nor the churn prediction with EXM had a positive CATE were assigned to the control group (without a churn prediction). Other salesperson–customer–months were assigned to the treatment condition that improved their CATE the most. Table 5 shows the recommended allocation, also broken down by quantiles on our most important covariate, prior customer sales revenue level. The simulation reveals that 42% of salesperson–customer–months should have been allocated to one of the two treatment conditions, while 59% should have been allocated to the control group. This procedure would have resulted in a 3.1% increase in sales revenue—which corresponds to over €7 million for the 12 months of our experiment.
Study 1: Simulation Results.
Study 2: Stimuli-Based Multiround Experiment
The field experimental Study 1 shows that the effect of a churn prediction tool on customer sales revenue depends on various contingencies. In Study 2, we conceptually replicate our study in a controlled environment to test the generalizability of our findings.
Procedure and Design
We conducted a stimuli-based experiment with two groups in a one-factorial between-subjects design, comprising a treatment condition (with a churn prediction tool) and a control condition (without the tool). We focus on a churn prediction without EXM in this replication for the sake of parsimony. We recruited 200 salespeople via an online panel provider. The salespeople have an average age of 39.8 years, 52.5% are male, and they exhibit on average 9.5 years of experience in sales, ranging from novices to salespeople with 44 years of experience. They come from industries such as trade (22%), health care (16%), and professional services (10%)
The salespeople were randomly allocated to the treatment condition (n = 101) or the control condition (n = 99). They received a dashboard with information about four customers and were instructed to develop a strategy for serving these customers during the next month, with respect to their time investment and the discount offered to each customer. The salespeople were to make these decisions four times (for four simulated months) and received updated dashboards after the completion of every round. Such multiround simulations are well established in the sales literature (Boichuk et al. 2019; Habel, Alavi, and Linsenmayer 2021b).
In both conditions, the dashboard listed each customer's ID, size, and prior sales revenue. Two of the customers were small, with a prior revenue of $2,000, and two were large, with a prior revenue of $30,000. In the treatment condition, salespeople additionally received customers’ churn probabilities. Two customers (one small, one large) had a churn probability of 10%, and the other two had a churn probability of 90%. To update the dashboard for salespeople's decisions in the subsequent month, we calculated each customer's new churn probability and sales revenue as functions of these values in the previous month as well as salespeople's decisions. That is, higher time investment and discounts decreased the churn probability and increased the sales revenue relative to the prior month (details in Web Appendix L).
Measures
Dependent variables
For every month, we asked salespeople how much time they would invest in each of the four customers (timeit), employing a constant sum scale. This required trade-off decisions because investments in one customer implied deinvestments in another customer. Such decision trade-offs are typical for salespeople in their daily work, as they are frequently short on time (Cron et al. 2021; Cron, Alavi, and Habel 2022). For discounting, such trade-offs are less likely; thus, we asked salespeople for the discount level offered to each customer (discountit) on a seven-point scale ranging from −3 (“much lower discounts than previously”) to +3 (“much higher discounts than previously”). Based on extensive pretesting (see Web Appendix L), we then simulated customer i's churn probability in month t (pit) as:
Covariates
We surveyed salespeople on the same covariates that emerged as most informative in Study 1: expected usefulness, general trust in algorithms, sales experience, customer churn prediction efficacy, customer retention efficacy, adaptive selling orientation, customer orientation, farming orientation, and learning orientation. Furthermore, given that salespeople in our sample come from a variety of contexts, rather than collecting salespeople's prior sales revenue level, we asked them to evaluate their sales skills on a seven-point scale (1 = “Far below average,” 4 = “Average,” and 7 = “Far above average”). We also collected age and gender to control for demographics. Lastly, because Study 1 showed that the predicted churn probability and prior customer sales revenue level matter, we included both as covariates. Prior customer sales revenue level is a dummy variable (0 = low, 1 = high).
We organized the data in a salesperson–customer–month panel (n = 3,200, that is, 200 salespeople × 4 customers × 4 months). Web Appendix M provides correlations and psychometrics of the survey variables, and Web Appendix N provides descriptive statistics. All survey measures are discriminant according to the Fornell–Larcker criterion (Fornell and Larcker 1981). In addition, correcting the correlations by the lowest positive correlation (rexpected usefulness, sales skills = .02) suggests that a common method bias is unlikely (Lindell and Whitney 2001).
Analytical Approach and Results
We initially verified that our manipulation worked as intended. Participants in the treatment condition scored significantly higher on the seven-point item “I was shown each customer's probability to stop purchasing from me” (Mtreatment = 6.29, Mcontrol = 3.02; t = −15.57, p < .001). Following our approach in Study 1, we proceeded to estimate causal forests with the three outcome variables of time investment, discount, and sales revenue. In all causal forests, we clustered for the nesting of observations in salespeople. Web Appendix O provides details on parameter tuning.
Figure 6 shows the distributions of the CATE for the three dependent variables. In line with Study 1, the ATE on all outcome variables is nonsignificant.
Study 2: Distribution of the CATE.
To understand how the covariates shape the CATE we inspect the importance of covariates (Web Appendix P) and estimate the effects of covariates on the estimated CATE using parametric models (e.g., Chen et al. 2020; Guo, Sriram, and Manchanda 2021), replicating the specification from Study 1. Table 6 reports the results. In the following, we first examine the main effects (three models on the left) and then turn to interactive effects (two models on the right) of the covariates on the CATE.
Study 2: Sources of Heterogeneity in Customer–Month Level CATE.
*p < .05. **p < .01. ***p < .001.
Notes: Standard errors in parentheses and clustered in salespeople.
Main effects
The three models on the left in Table 6 include main effects only and complement Study 1 in three ways. First, like in Study 1, the month positively affects the CATE on time investment (β = .35869, p < .001) and sales revenue generated (β = .37217, p < .001). Thus, as the experiment progressed, salespeople in the treatment group increasingly benefited from the churn prediction by improving their customer time allocation. Interestingly, the month count variable negatively affects the CATE on discount (β = −.09704, p < .001). This means that with time passing, salespeople in the treatment group tended to rely less on discounts. Perhaps they used discounts initially to help reduce the high initial churn probabilities of 90% for two of the customers and, once successful, preferred to rely on allocating time instead.
Second, the predicted churn probability positively affects the CATE on time investment (β = .54760, p < .001), discount (β = .89764, p < .001), and sales revenue (β = .29793, p < .001). This result fully aligns with Study 1 (and the supplemental study in Web Appendix B) and indicates that the treatment group improved their service to customers with high predicted churn probabilities to retain them.
Third, prior customer sales revenue (a dummy indicating whether the customer initially had revenue of $2,000 or $30,000) negatively affects the CATE on time investment (β = −.19122, p < .001), discount (β = −.19699, p < .001), and sales revenue (β = −.11358, p < .001). This result marks an interesting difference from Study 1, where the prior customer sales revenue positively affected the CATE. Plausibly, low prior customer sales revenue might have enhanced participants’ perception that a customer is close to churning, while customers with a high prior sales revenue might suggest a higher likelihood of retainment (Xu et al. 2022). In practice, though, prior customer sales revenue might be more indicative of the importance of a customer for a salesperson. We also note that the revenue generated with a customer in the previous month, entered as a control variable in our regressions, is positively related to the CATE on time investment (β = .25236, p < .001), discount (β = .05130, p < .001), and sales revenue (β = .36561, p < .001), which is consistent with the findings of Study 1.
Exploratory interactions
The fourth model in the table replicates our analysis of exploratory interactions from Study 1 in shaping the CATE on sales revenue, and the fifth model adds further exploratory interactions. Because the results of the models are very similar, in the following we refer to the fifth, more comprehensive model.
First, consistent with Study 1 (and the supplemental study in Web Appendix B), the predicted churn probability has a quadratic effect on the CATE (βquadratic = .21254, p < .001), which positively interacts with the customer prior sales revenue level (βquadratic = .12608, p < .001). Figure 7, Panel A, shows a plot, which conceptually aligns with that from Study 1.
Study 2: Diagrams of Core Interactions.
Second, consistent with Study 1, the month has a quadratic effect on the CATE (βquadratic = .15003, p < .001), which interacts with sales experience (β = −.02908, p < .05). Figure 7, Panel B, shows that the interaction pattern conceptually replicates Study 1, such that salespeople with high experience benefit from the churn prediction sooner. Furthermore, the quadratic effect of the month interacts with learning orientation (β = .03999, p < .05), though the effect sign differs from the one in Study 1. Figure 7, Panel C, shows that high learning orientation reduces the CATE in earlier months and pays off only in the last month. A plausible explanation is that salespeople high in learning orientation initially experimented with different behaviors to understand how these affect sales revenues—a strategy less likely to be found in the field.
Third, in contrast to Study 1, we do not find significant interaction effects between expected usefulness and adaptive selling orientation, between general trust in algorithms and learning orientation, between customer churn prediction efficacy and customer retention efficacy, and between customer orientation and farming orientation. The fact that a stimuli-based experiment does not fully replicate the complex pattern of interactions found in the field-based Study 1 is not surprising (Golder et al. 2022). For example, take the interaction between customer churn prediction efficacy and customer retention efficacy. In the present simulation, salespeople were not able to predict customer churn without the customer churn prediction. In contrast, in a field setting, skilled salespeople might be able to predict customer churn even without access to predictive analytics (Habel, Alavi, and Heinitz 2023), thus giving rise to the interaction found in Study 1. Similarly, because salespeople in this study could not engage with customers’ needs, the nonsignificant effect involving customer orientation is not surprising.
Having said this, we uncovered two further exploratory interactions related to technology perceptions and selling orientations. First, expected usefulness interacts with the predicted churn probability (see Figure 8, Panel A), such that lower expected usefulness decreases the CATE for medium predicted churn probabilities. Low expected usefulness and ambiguous signals such as a medium predicted churn probability may compound each other, fostering aversion against the churn prediction. Second, adaptive selling orientation interacts with farming orientation (see Figure 8, Panel B), such that if farming orientation is high, adaptive selling increases the CATE. Conceptually, this finding is not too dissimilar from the interaction between farming orientation and customer orientation revealed in Study 1.
Study 2: Diagrams of Additional Exploratory Interactions.
Discussion
Research Issues
Our article contributes to the literature in several ways. First, our findings emphasize that mitigating employees’ adverse reactions toward predictive sales analytics is of high importance. Recall that we found no ATE of the implementation of a customer churn prediction on salespeople's sales revenue with customers. This nonfinding is counterintuitive for us and our collaborating firm's management. It seems that for the average customer, salespeople either did not employ the churn prediction tool, or, when they did, they made ineffective decisions based on it. The fact that the average salesperson–customer relationship did not benefit from the churn prediction confirms the notion within managerial (Harvard Business Review Analytic Services 2021) and academic (Dietvorst, Simmons, and Massey 2015, 2018) literature that the implementation of analytics might fail given employees’ aversion to algorithms, constituting a key hurdle to adoption (Ammanath, Hupfer, and Jarvis 2020).
Second, we provide an intricate account of factors mitigating or exacerbating these challenges through a machine-learning-based “empirics-first” approach (Golder et al. 2022). Specifically, we synthesize prior literature into customer characteristics and salesperson characteristics, the latter of which we divide into (1) perceptions of technology characteristics (Davis, Bagozzi, and Warshaw 1989), (2) abilities to effectively employ the technology (Bala and Venkatesh 2015), and (3) selling orientations (Hunter and Perreault 2007; Speier and Venkatesh 2002). We empirically test a wide variety of variables within these conceptual categories. The findings can be condensed into five conclusions.
Conclusion 1
The salesperson's effective employment of the churn prediction crucially depends on the characteristics of the specific customer whose churn probability is predicted. That is, salespeople are more likely to increase their sales revenue with customers with high churn probabilities as well as high and stable prior sales revenue. We hereinafter label these types of variables “characteristics of the predicted object” (COPO). They account for more than 38% of the heterogeneity in the treatment effect (Web Appendix I). Our discovery of the high importance of COPO variables is noteworthy because prior literature typically focuses on non-COPO drivers of sales technology adoption, such as a tool's perceived usefulness and ease of use (Davis, Bagozzi, and Warshaw 1989). We furthermore find nonlinear interactions between these variables, such that for large customers, salespeople benefit particularly from a churn prediction when predicted churn probabilities are very low or very high.
Conclusion 2
Non-COPO variables are less important than COPO variables, but collectively they still account for almost 62% of the heterogeneity in the treatment effect (Web Appendix I). They mainly affect the CATE in interaction with each other and with COPO variables. For example, variables such as expected usefulness (Davis, Bagozzi, and Warshaw 1989), general trust in algorithms (Dietvorst, Simmons, and Massey 2018), sales experience, adaptive selling orientation (Franke and Park 2006), learning orientation (Sujan, Weitz, and Kumar 1994), and farming orientation (DeCarlo and Lam 2016) form higher-order interactions in shaping the CATE. We summarized these in the five insights presented as part of Study 1.
Conclusion 3
The benefits salespeople draw from predictive analytics are highly dynamic over time—and to our best knowledge, ours is the first study to examine these dynamics (see Table 1). The dynamics are shaped by sales experience and learning orientation, suggesting that benefiting from the churn prediction requires time, experience, and learning.
Conclusion 4
Predictive sales analytics can have nuanced effects on salespeople's decisions about how to serve customers. In Study 2, salespeople tended to allocate more time and grant deeper discounts to customers with high churn probabilities, and this effect further interacted with a customer's prior sales revenue level. These findings reveal how the implementation of a predictive sales analytics application induces adaptive selling (Alavi, Habel, and Linsenmayer 2019; Weitz, Sujan, and Sujan 1986).
Conclusion 5
Managing salespeople's expectations about the predictive validity of predictions can foster adoption (Burton, Stein, and Jensen 2020; Castelo, Bos, and Lehmann 2019)—however, only under very specific circumstances. For example, Study 1 suggests that if salespeople generally harbor low trust in algorithms but are highly oriented toward learning, managing their expectations can foster effective adoption. However, managing expectations seems to be a double-edged sword, causing uncertainty and harming effective adoption for many salespeople. Our supplemental study in Web Appendix B corroborates this finding. Specifically, it shows that predicted churn probability on average exhibited a j-shaped effect on salespeople's time investment. However, this j-shaped, quadratic effect particularly emerged for the churn prediction only condition, but not for the control or churn prediction with EXM condition. Thus, in line with Study 1, such EXM does not seem conducive to increasing the beneficial effects of employing the churn prediction. This provides support for some authors’ conjecture that “it seems unlikely that an algorithmic literacy program can suffice as a standalone intervention for solving algorithm aversion” (Burton, Stein, and Jensen 2020, p. 223).
Third, our study contributes to the predictive sales analytics literature by developing a customer churn prediction in a B2B context. While various academic articles have studied the prediction of customer churn (e.g., Ascarza 2018; Gordini and Veglio 2017; Lemmens and Croux 2006), academic studies examining customer churn prediction in a B2B setting are scarce (e.g., Gordini and Veglio 2017; Tamaddoni, Stakhovych, and Ewing 2017). Our study can provide practical guidance for future studies predicting B2B customer churn. For example, the predictive model reveals that purchase recency, the previous number of purchases, and seasonality indicators are most relevant to predict customer churn (see Web Appendix C).
Fourth, our study contributes to marketing research employing the causal forest methodology. Specifically, we control for unobserved salesperson heterogeneity using a novel approach by Jens, Page, and Reeder (2021). This approach entails estimating fixed effects in a parametric model and adding these fixed effects as covariates to causal forests. We thus provide a template for future studies that aim to control for group-level heterogeneity in causal forests.
Managerial Implications
Many salespeople do not adopt predictive analytics tools or fail to effectively base decisions on them. Our study provides four recommendations for mitigating these challenges. First, managers need to provide an environment that gives salespeople the opportunity to learn how to use a predictive analytics tool effectively. This is because as our studies show, the treatment effect of implementing the churn prediction is zero on average and only grows with time. Maybe initially, salespeople simply observe whether the tool's predictions are correct. As they begin to trust the tool and aim to use it to make decisions, salespeople need to learn which strategies are successful. To illustrate, building on Study 2, maybe salespeople initially focus on granting deeper discounts to customers with a high churn probability. Only over time do they learn when discounts are (not) an effective measure to retain customers. In addition, salespeople learn that the tool helps not only reduce customer churn but also target loyal customers for cross-selling. Again, only with time do salespeople learn which of these strategies is effective under which circumstances. Thus, managers might encourage exchanges between salespeople about their experiences and effectiveness when utilizing a predictive analytics tool. In addition, managers could support novice salespeople by dedicating time to one-on-one or team meetings.
Second, when deciding for whom and how to implement a predictive sales analytics tool, managers should account for salespeople's likelihood to benefit from such a tool. Salespeople are especially likely to benefit if they perceive the tool as useful, have high abilities that the tool can complement, and exhibit orientations that have a high fit with the tool, such as adaptive selling and learning. Managers might then focus the implementation of predictive analytics tools on these specific salesperson segments. For other salesperson segments, managers might either refrain from implementing the tool or accompany the implementation with additional training or change management measures. For example, managers might educate salespeople on the usefulness of such tools and help them realize how the tool can support their success.
Third, managers need to consider that salespeople's decisions based on a predictive analytics tool are nuanced and highly contingent on situational factors. Specifically, salespeople reprioritize their efforts in multiple ways, such as through discounting and devoting more or less time to certain customers, depending on these customers’ predicted churn probability and sales revenue. Is such heterogeneity in salesperson decisions desirable? Our conversations with managers and academics during this study revealed two schools of thought. Some managers favor a harmonized market approach and thus a combination of predictions with decision rules. However, to implement decision rules, managers need to know which courses of action are conducive to customer sales revenue in certain situations. Moreover, salespeople feel threatened by decision rules, owing to a perceived loss of control. Therefore, other managers favor refraining from decision rules (Burger and Habel 2020) and instead give salespeople the freedom to combine predictions with their own experience and intuition. Managers should carefully evaluate which of these two schools of thought to follow. We hope that future research will give them specific guidance in this respect.
Fourth, managers need to be careful when managing salespeople's expectations about potential errors of predictive tools. This is particularly important because some might advocate such management of expectations as a crucial step toward reducing algorithm aversion or creating “algorithmic literacy” (Burton, Stein, and Jensen 2020). However, managing expectations improved outcomes only under certain conditions—and these conditions vary substantially from the conditions under which predictive tools without managing expectations are most effective. For example, managers might particularly resort to EXM shortly after introducing a predictive tool for rather small customers and for salespeople who tend to mistrust algorithms but are generally willing to learn.
Limitations and Future Research
Our study has several limitations that suggest avenues for future research. First, an interesting investigation would be to study whether the implementation of predictive sales analytics tools other than customer churn predictions will result in similar effects. For example, are our findings still valid if a predictive sales analytics tool is implemented for lead conversion instead of customer churn? Second, in our study salespeople were free to use the customer churn prediction however they saw fit. However, predictive sales analytics tools can also be introduced to prescribe activities. Thus, a productive inquiry might consider whether and when the effectiveness of predictive sales analytics tools differs depending on the degree of guidance provided by such tools. Third and last, our study investigates one specific mitigation strategy to address the prevalent challenges when implementing predictive sales analytics tools. An interesting avenue of inquiry would be to test other strategies, such as involving sales employees in the tool development process (Burton, Stein, and Jensen 2020).
Supplemental Material
sj-pdf-1-mrj-10.1177_00222437221151039 - Supplemental material for Effective Implementation of Predictive Sales Analytics
Supplemental material, sj-pdf-1-mrj-10.1177_00222437221151039 for Effective Implementation of Predictive Sales Analytics by Johannes Habel, Sascha Alavi and Nicolas Heinitz in Journal of Marketing Research
Footnotes
Special Issue Editor
Vikas Mittal
Associate Editor
Girish Mallapragada
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
