From Free to Paid: Monetizing a Non-Advertising-Based App

Online Content Monetization

Research has shown that consumers are reluctant to pay even small monetary amounts for online content (Ascarza, Lambrecht, and Vilcassim 2012; Chiou and Tucker 2013; Papies, Eggers, and Wloemert 2011; Shampanier, Mazar, and Ariely 2007). The literature indicates that the use of a free online service leads consumers to become accustomed to the free status and induces an unwillingness to pay when the service becomes fee-based. Lambrecht et al. (2014) conducted a retrospective review of the revenue models usually adopted by online content providers: selling digital content, brokering consumer information, or displaying advertisements. While they suggest that related monetization research in the domain of the mobile internet is scant, some studies have explored the monetization strategy of online content using an ad-based model (Godes, Ofek, and Sarvary 2009; Kumar 2014; Kumar et al. 2012; Lambrecht and Misra 2016; Lee, Kumar, and Gupta 2015; Li and Cheng 2014; Pauwels and Weiss 2008; Prasad, Mahajan, and Bronnenberg 2003). A stream of research also examined how the monetization strategy affects user retention and engagement on the platform in an ad-based context (Choi and Mela 2019; Gu et al. 2022; Kanuri, Mantrala, and Thorson 2017; Pattabhiramaiah, Sriram, and Manchanda 2019; Yan, Miller, and Skiera 2022; Zubcsek, Katona, and Sarvary 2017).

Our research substantively differs from previous research in the literature. We explore the monetization strategy of a firm in an ad-free context. Existing literature on online-content monetization strategies is based on an advertising revenue model, or at least takes place in an ad-based context. The ad-free model is fundamentally different from the ad-based one, as noted previously, due to the distinct revenue model in which the only revenue for the content provider comes from the content itself. With different revenue sources, findings from the context of an ad-based model may not be applicable to an ad-free model. Empirical studies have yet to explore ad-free apps’ monetization strategies for services that are initially free. Moreover, in contrast to observational studies, we adopt randomized field experiments to assess the causal impact of monetization strategies on the purchases/conversion rate. We can therefore observe and track the entire process of an app switching from free to paid and examine users’ responses to the change.

App Freemium Pricing

Our research is also related to a stream of studies on freemium pricing, in which the firms offer both free and paid versions of a digital service (Appel et al. 2020; Arora, Hofstede, and Mahajan 2017; Deng, Lambrecht, and Liu 2022; Goli, Reiley, and Zhang 2022; Gu, Kannan, and Ma 2018; Halbheer et al. 2014; Lee, Zhang, and Wedel 2021; Li, Jain, and Kannan 2019; Runge, Nair, and Levav 2021). Freemium is mostly seen in digital content with an ad-based business model. Though it does not charge users directly, the free version can still earn revenue from ads through the traffic generated. However, in the ad-free model, without revenue from advertisements, free usage does not bring any direct revenue to the firm (beyond the learning motive and future revenues that we discuss in this article). By contrast, free users entail an operating cost to firms, and in the short run, firms incur a net loss. Thus, we rarely see firms offer both free and paid versions concurrently in an ad-free model. Different from the existing freemium literature, our study explores the monetization strategy of a free service in an ad-free context with the app switching from free to paid.

Further, the context we study uses a different business model, which sets different expectations for users. In a freemium model, a user can see both the free and paid service when starting the service and expects to pay premium fees to upgrade the service. Different from previous studies, we focus on comparing switching from “free” to “freemium” (soft landing) to a strategy of changing from “free” to “premium” (hard landing). In our context, a user only sees free full services at the start. After the service starts to monetize (switching from free to a paid regime), existing users must pay a fee to continue the full service; new users get a limited-time free trial, but they would still pay to continue the service after the free trial. Moreover, most current studies have been conducted with secondary data from mobile app markets such as the App Store, which can potentially lead to endogeneity concerns and survivor bias. In our study, we adopt a field experiment approach to better address these issues. Because the existing literature does not provide clear answers to our research questions, we examine them empirically via field experiments.

Hard- Versus Soft-Landing Effectiveness

In the hard-landing case, the consumer is choosing between (1) paying and accessing the content and (2) not paying and quitting (choosing the “outside” option). In the soft-landing case, the consumer is instead choosing among three options: (1) paying for content, (2) not paying and quitting, and (3) not paying and enjoying some free content. Drawing on utility theory (Anderson, Hansen, and Simester 2009; Fishburn 1968; Hauser and Wernerfelt 1990), we suppose that the utility associated with option 1 (paying and accessing the content) is the same under both hard landing and soft landing. Since quitting the platform implies no utility, the utility associated with option 2 is 0 (or can be normalized to 0 with no loss in generality). Option 3 under soft landing provides some positive utility to the existing users who do not want to pay. We expect the utility associated with this option to be higher than that from quitting but lower than the utility from accessing all the content, and for some users it may exceed the utility of option 1, net the price to be paid. Thus, the presence of the free option 3 can decrease the perceived value of the paid option 1 in the soft-landing condition. So the probability of subscribing and paying is likely to be smaller under soft landing. ⁸

Soft landing can be operationalized in two ways. In one way, users obtain time-unlimited access to a limited subset of the app's content (the limited-free-content scenario). In another way, users gain full but time-limited access to all content on the app (the limited-free-time scenario). We test the effects of these two forms of the soft-landing approach in our two field experiments.

Exclusive Secondary Offerings (ESO) Effectiveness

ESO design is related to the bundling literature (Bakos and Brynjolfsson 2000; Derdenger and Kumar 2013; Stremersch and Tellis 2002). Specifically, bundling refers to the selling of two or more products and/or services at a single price (Yadav and Monroe 1993). In our context, the provision of ESO along with the main service (regardless of soft landing or hard landing) can be construed as a bundle offer. From utility theory (Anderson, Hansen, and Simester 2009; Fishburn 1968; Hauser and Wernerfelt 1990; Roberts and Lattin 1991), given the same price, the option with more offerings is objectively more valuable and will be chosen (i.e., “more is better”). In a separate stream of behavioral research on choice reversal, Hsee (1998) (see also Hsee and Zhang 2010; List 2002) suggests that when consumers do not see both options (in our case, this would be providing users the options of subscribing to the service with and without ESO separately and simultaneously), they could end up picking the option without ESO (i.e., “less is better”). Hsee has provided the theoretical rationale for this behavior with lab-based experiments. He finds that an objectively less valuable product can be evaluated more favorably than an objectively higher-value option when consumers are in a “separate evaluation mode” and are in the low-evaluability state for the important attributes of the option. In other words, the easy-to-evaluate characteristic, which is not necessarily the most important one and may have little relationship with the core value of the product, is used when an individual evaluates the two options separately. When the two options are evaluated jointly, the “less-is-better” effect is reversed, and the objectively higher-value option would be preferred.

In our case, when faced with having to pay in the separate evaluation mode, consumers may have difficulty identifying the value for the main content but may perceive the ESO to be inferior for achieving the app's goals, even though ESO has some value to users alone. Thus, including ESO in a package can backfire by lowering perceived value, even though the whole package is objectively more valuable. In other words, one potential mechanism for the negative effect of ESO is the high evaluability of the low value of the ESO. Simonson and O’Curry (1994) find that when a new product feature is perceived to be of little or no value, it decreases the brand's overall attractiveness. A related explanation for our expectation on the ESO's effectiveness is that when consumers are expected to pay, they might attribute the payment to the ESO rather than to the main service of interest. Because the ESO is not the main benefit consumers are seeking in the app, associating the ESO with the transition to a paid regime might lower the attractiveness of the app to consumers. Thus, whether consumers prefer the “objectively more valuable” option of the ESO (in addition to the access to the content) or show a “choice reversal” is an empirical question.

Exclusive Secondary Offering in the Presence of Soft Landing

Following the previous discussion on preference reversal with ESO, we examine further whether preference reversal would disappear in certain situations. The executive survey indicated divergent perspectives on the effectiveness of ESO at monetization. The marketing literature does not provide direct guidance on the effect of ESO in the presence of soft-landing monetization. A separate body of literature has identified consumers’ exclusivity-seeking behavior: drawing on utility theory, it suggests that the exclusivity of a product delivering uniqueness and status benefits can create extra value for consumers and, thus, enhance their willingness to pay (Amaldoss and Jain 2005a, b; Arifoğlu, Deo, and Iravani 2020; Tereyağoğlu and Veeraraghavan 2012). Behavioral research also shows that consumers are more likely to seek exclusivity in product domains seen as symbolic of identity, such as music or hairstyles (Berger and Heath 2007). Firms provide ESO to leverage its potential exclusivity value to paying users, as there is no access to ESO for the nonpayers.

Therefore, in our context, it is possible that the ESO provided in the soft-landing condition can be viewed as an exclusive benefit. In other words, we expect that the monetization strategy (hard landing vs. soft landing) would moderate the effect of ESO due to a different realization of exclusivity under these two strategies. When an app takes the hard-landing approach, users choose to either pay or leave. Therefore, the users who remain are all paying users. In this condition, “exclusivity” has no real meaning. However, in the soft-landing situation, some users can still use the app for free, as both free users and paying users coexist on the app. In this case, the ESO for paying users provides them with exclusivity, leading to a higher perceived value for the paid package. In our context, we expect that users will prefer having access to ESO when such offerings provide them with exclusivity value relative to free users (i.e., in the soft-landing condition).

Experiment Design

Context

We designed and implemented field experiments with an app firm that offers articles, e-books, and audiobooks (see Web Appendix B1, Figure WB1). Mobile app is the only channel used by the firm. All the firm's main competitors also use mobile apps as their channels. The app is free to download and served over 4 million registered users across China in January 2019. There is no in-app social function, and users engage in self- and independent learning on the app. The reading materials on the app are not exclusive, and users can also find them on other platforms. Several characteristics make the focal app a top player with respect to its competitors. First, the ample content on the app makes it easy for users to find materials interesting to them at a low search cost without having to search multiple sources. Second, the sophisticated presentation of reading materials makes the app attractive: there are beautiful covers for each chapter or “episode” as well as well-designed, cute animal icons. Third, the highly interactive product design makes the reading experience more engaging than that of its competitors. Furthermore, after a user installs and registers the app, they are invited to take a reading-ability test and receive a reading-ability score. In the app's library, most books have a corresponding “text difficulty” score marked on the corner of the icon. Users can thus choose books that fit their reading ability, or they can opt out of the test. The reading-ability test in the app provides users a clear evaluation of their reading ability. At the same time, the text difficulty scores offer a useful reference for users to make appropriate selections given the vast content. The test also helps users gauge their reading status and progress. At the time of the field experiment, about 80% of the registered users had taken the test.

Figure 1 shows the total number of free users over time across all the cities that were part of Field Experiment 1 (hereinafter, Experiment 1) before monetization. The figure suggests a slowdown in growth around the time of monetization. In other words, indications are that under the free regime, the app was reaching saturation in terms of number of users. This saturation served as the motivation for the firm to focus the experiment on the monetization of existing users and not on new-user acquisition. The customers in Experiment 1 were existing users who had used the app for at least two days prior to the intervention, with the average user having used the app for about two weeks and therefore possessing knowledge of the app's content. The treatments in our field experiments were only applied to existing users of the app and did not affect any new users who might have joined after implementation of the monetization intervention. New users who registered on the app after monetization were provided with free trial offers.

OPEN IN VIEWER

Figure 1.

Number of Registered Free Users in All Cities: Field Experiment 1.

In terms of concomitant changes that could affect the results of our experiments, we considered the influences of content quality and competition. No substantive changes were made to the content, and therefore no changes to app quality took place during the experiment. We also checked for any changes to the competitive landscape during the experiment. The firm identified 12 other platforms that were potential competitors. We focused on the following aspects: (1) whether there is any substantively new service added, and (2) whether there is any significant promotion activity, including price discount and media advertising. We observed no major changes in the services or prices of those apps during the period of the field experiments. Moreover, the prices that the focal app chose to charge fell in the industry’s typical range and were comparable to similar offerings in the market (see Table WB2 in Web Appendix B1).

The experiment

Prior to the experiment, users could read all content for free and could play a basic dress-up game (see Web Appendix B1, Figure WB1). In the game, users choose a virtual figure to represent themselves and select clothes, costumes, accessories, backgrounds, and so forth to dress up the figure for personalization. This is a very popular function in the app. Once the field experiment began, the company started charging existing users for previously free app content. The price change was not preannounced. For both new and existing users, the app was free to download after it started to charge, but the paid content required a subscription. We applied the treatment for each user without the need for a user-initiated app update: after the paid version was assigned, the app's page automatically changed on the next refresh. The company could apply a targeted treatment by toggling a certain “switch” in its background management system. When the switch was on for a user, that user’s app interface changed from the free to the paid version. The contents of the two versions (free vs. paid) were kept the same, but a paywall was added to limit access (indicated by a “VIP” mark on each article or book icon). The different treatments applied to various users correspond to the two goals of this study: soft landing and ESO. The presence or absence of these two conditions forms the basis for a 2 (soft landing vs. hard landing) × 2 (ESO available: yes vs. no) between-subjects experimental design.

We manipulated the free-content variable by providing either some or no free content. We term users provided with limited free content as soft-landing groups and those not provided free content as hard-landing groups. For the soft-landing users, the app included a “Free Zone” on the top of the app homepage, providing complete versions of some medium- to high-quality articles, book chapters, and audio tracks (see Web Appendix B1, Figure WB2). This free content represented less than 1% of the app's entire library—not coincidentally, the approximate amount read by an average user over (the previous) two weeks. Free users saw only an introduction to the remaining content. For users in the hard-landing groups, the app did not provide any free content after the switch to a paid model (no free zone provided). Users could read the content only with a paid subscription.

For the design of ESO, the firm wanted to build on the existing dress-up game function in the app to enhance the perceived value of the subscription package and relieve users of the potential “pain” of paying. When the app was free, the company observed a high rate of usage of this dress-up game, with over 90% of users actively playing. When the app switched from free to paid, it retained the basic features. In the field experiment, a series of newly designed fancier features added to the dress-up game were offered as ESO along with the paid subscription (see Figure WB2 in Web Appendix B1). The ESO was provided as bonus features, including fancier clothes and more appealing decorations in the dress-up game. The basic dress-up game under the free regime was still available to existing users, regardless of experimental condition. To test the impact of providing ESO on subscription behavior, ESO was available only to paying users in some groups and not in others. In the groups with ESO, the membership-benefits page explicitly highlighted that access was restricted to paying users, and many better-designed clothes, costumes, and other add-ons were marked with a VIP-exclusive icon. As there is no in-app social network, the ESO here is not social in nature. For the no-ESO groups, the dress-up game design in the app remained the same as before the switch. Figure 2 shows the experimental design and differences across groups. Apart from the aforementioned manipulations, all other design features remained consistent across the four groups, including the color palette, content, and banner information. The treatment was applied without prior notice and simultaneously to all four groups, all of which faced the same set of pricing options.

OPEN IN VIEWER

Figure 2.

Experimental Conditions in Different Groups: Field Experiment 1.

Online services typically set different prices based on the duration of the subscription. ⁹ Firms offer discounted prices for monthly or annual autorenewal subscription plans and set a higher price for the shorter duration plans. Autorenewal subscription choice can assist in more long-term subscriptions, help keep a stable cash flow, and generate more predictable revenue for businesses. Before the firm set the price, it researched the prices of similar offerings in the market. Table WB2 in Web Appendix B1 lists the main platforms that provide similar services and their pricing options. Taking into account the app's rich services and unique characteristics relative to other apps, the firm decided on three pricing options based on subscription-length-based pricing: (1) 1 CNY for the first month and 30 CNY/month autorenewal (a user could cancel anytime), (2) 40 CNY for a one-month subscription, and (3) a 298 CNY annual subscription. We summarize the rationale for the interventions design in Experiments 1 and 2 in Web Appendix B3.

Treatments might be contaminated if a user registered two devices and could compare versions, or if two users who knew each other got different versions and exchanged information. We wanted to ensure that the users in each treatment group were only exposed to a single paid version. Thus, we randomized treatments at the city level, taking advantage of geographic isolation to minimize treatment contamination. Specifically, we randomly allocated the 99 cities (and the 1.9 million existing users living there) into four treatment groups. We verified that the four groups were comparable before the experiment with a series of balance tests. Both the location and the size of the 99 cities varied greatly. We also randomly drew a few subjects from each treatment group and ensured that their interface had changed to match their assigned condition. To further check randomization, we conducted analysis of variance (ANOVA) tests across the four groups on a series of demographic information (number of existing users, number of Android users, gender, and age), and existing users’ pretreatment behavior information (reading ability, customer tenure, and usage time). The ANOVA results show that intergroup differences are not significant with respect to these variables (see Web Appendix B1, Table WB4). We plot the distributions of users’ reading abilities and preexperiment usage on the app (see Figure WB3 in Web Appendix B1) and notice significant overlap in the distributions across conditions. This further supports our randomization of treatments. In checking the sample size of each group, we found that the difference in size between the largest and smallest groups was less than 5%. In principle, the focal firm also tracks the registration and subscription behavior of new users after the app switched to the paid regime. However, for this study, we are more interested in the conversion rate of existing users. Thus, in the field experiment, our results focus on analyzing existing users. We ran this experiment for 80 days from its launch in January 2019 to allow users sufficient time to make a decision and to look at the unsubscription behavior of those who subscribed to the service. During the field experiment, the company did not launch any user-targeted promotions. Moreover, the firm did not manipulate prices across groups: all users in the four treatment groups received the same set of price plans.

Model-Free Evidence on User Conversion

Table 1 reports the summary statistics for the data from the experiment. The mean user conversion rate for first-time subscribers was about 1.03% in 80 days, consistent with the range of conversion rates observed in previous studies (Localytics 2017; Needleman and Loten 2012). We are able to track each user within the treatment groups and observe their subscription decision. Subscribers who made purchase decisions within the 80 days were tracked for two extra monthly billing cycles. We use this period to look at their possible unsubscription behavior.

OPEN IN VIEWER

Table 1.

Descriptive Statistics for Field Experiment 1

Variables	Mean	SD	Min	Max	Description
Decision Variable
Subscribe to paid content	.010	—	0	1	1 if user i pays to subscribe; 0 otherwise
Treatments
Soft landing	.50	—	0	1	1 if user i receives free content in the paid version; 0 otherwise
Exclusive secondary offerings (ESO)	.50	—	0	1	1 if user i is offered ESO as a paying user; 0 otherwise
Soft landing × ESO	.25	—	0	1	1 if user i receives free content and is offered ESO; 0 otherwise
Demographic Info
Female	.48	—	0	1	1 if user i is female; 0 otherwise
Age (years)	8.07	1.64	5	13	Age of user i
Reading abilitya	372.22	327.08	0	1,250	Reading ability score of user i
Apple products	.18	—	0	1	1 if user i's device is an iOS system, including iPhone or iPad; 0 otherwise
Pretreatment Behavior
Customer tenure	18,601	3,002.13	3,846	33,068	How long user i has been registered on the app before the experiment, in minutes
Usageb	614	768.28	1	12,080	How much time user i has spent on this app before the experiment, in minutes
City Info
Disposable incomec	22,505.54	6,954.33	16,011	58,987	Average disposable income of residents in the province where user i lives
City locationd	.50	—	0	1	1 if the city where user i lives is categorized as a southern region; 0 otherwise

^a

Latest preexperiment score attained by user i.

^b

Records the total time user i spent on the app before the treatment was applied.

^c

Reported in Chinese RMB (CNY); 1 USD ≈ 6.8 Chinese CNY at time of experiment.

^d

Climatologically, China is typically divided into southern and northern regions along the Qinling–Huaihe line.

Notes: N = 1,907,228.

The switch from “free” to “paid” may lead users to perceive the change as unfair, which also hinders the conversion rate. In the app industry, typically only 1%–2% of users will upgrade to the premium plan (Localytics 2017; Needleman and Loten 2012). ¹⁰ As the conversion rate is generally low in the industry, even a small conversion rate means a big revenue difference for a firm. Figure 3 provides model-free evidence on user subscriptions. The conversion rate of each group is shown, with a clear distinction in subscription likelihood visible across different groups. Group B (hard landing, no ESO) yielded the highest conversion rate (8,540 subscribers, or 1.79%): though its sample size is only slightly higher than the other three groups, it generated over 103% more subscribers than Group C (soft landing with ESO). The situation in Group C corresponds to conventional wisdom in the industry in terms of being the preferred option. It is also the strategy that this app's manager was originally planning to implement. Overall, hard-landing groups (Groups A and B) generated over 124% more subscriptions than did soft-landing groups (13,643 vs. 6,080). In hard-landing groups, we further observe that providing ESO (Group A) led to a lower conversion rate than not doing so (Group B). This pattern was reversed in soft-landing groups: ESO seems to drive a higher conversion rate in the soft-landing conditions (Group C vs. Group D). Our finding suggests an interaction effect between ESO and soft landing, that is, the existence of a boundary effect of ESO.

OPEN IN VIEWER

Figure 3.

Subscribers Across Groups: Field Experiment 1.

Of the three pricing options presented, across conditions, 95% of subscribers chose the first (1 CNY for the first month, with 30 CNY/month autorenewal), 4% chose the second (40 CNY for a one-month subscription), and only 1% chose the third (298 CNY annual subscription with yearly autorenewal). We did not find any significant differences in the ratio of pricing options chosen across the four treatment groups. Figure 4 sheds some light on the speed of order generation across groups. We see a relatively consistent pattern: the highest number of orders occurred on the second day postlaunch; the order rate decayed rapidly thereafter and approached a “steady state.” In the following analyses, we aim not only to test the statistical significance of each treatment but also to quantify the causal effect while controlling for individual and city characteristics.

OPEN IN VIEWER

Figure 4.

Daily Orders After Launch: Field Experiment 1.

Model and Analysis

We want to estimate an individual user's likelihood of subscribing to the paid version of the app over the 80 days. User i's utility of choosing to subscribe to the paid service package is

U 1 i = V 1 i + ε i = α 0 + α 1 Soft Landin g i + α 2 ES O i + α 3 Soft Landin g i × ES O i + β Control Var s i + λ Cit y R ( i ) + ε i .

(1)

Pr ( S 1 i = 1 ) = exp ( V 1 i ) 1 + exp ( V 1 i ) .

(2)

We include the vector Control Vars_i, which includes variables to control for individual characteristics that could potentially affect users’ subscription behavior. The first set included consists of user demographic variables: (1) Female_i equals 1 if the user is female and 0 otherwise, (2) Age_i indicates the user's age, and (3) Reading Ability_i measures the user's reading-ability score, which indicates their understanding level of the content. Users with different reading abilities may have different goals in using the app, which may affect their purchase decisions. (4) Apple Products_i indicates the device type. It equals 1 if the user uses an iOS device, such as iPhone or iPad, and 0 otherwise. Evidence has shown that Android and iOS users exhibit different spending habits in some settings (Zarkov 2018). The second set of variables reflects users’ preexperiment behavior on the app. It includes Customer Tenure_i, which measures the time since a user registered on the app (in minutes), and Usage_i, which indicates the total time spent by the user since registration and serves as a gauge of usage behavior before the subscription decision. These variables may proxy for a user's loyalty to the app. The last set of variables, City_R(i), controls for regional location effects and incorporates two other variables. (1) Disposable_R(i) measures the average disposable income of the province in which user i's city is located. This variable captures the spending power of the region where user i lives. (2) City Location_R(i) equals 1 if the city is located in the southern region, and 0 otherwise. Climatologically speaking, China is typically divided into southern and northern regions along the Qinling–Huaihe line (see Li et al. 2017). This variable provides another control for location effects. We conduct multicollinearity tests by computing the variance inflation factors and find that multicollinearity is not an issue, with an average variance inflation factor of 1.45.

The results for the proposed model and other nested models are presented in Table 2. We see that, across all models, soft landing had a consistent, significant negative effect on the subscription likelihood. Specifically, we find that soft landing lowered the subscription likelihood to .25 (= e^−1.38) in terms of the odds ratio, meaning an economically significant loss of revenue for an app with a user base of 4 million. If all 4 million users were in the soft-landing condition, we estimate 26,875 fewer subscriptions relative to when all customers are in the hard-landing condition. Such a difference in revenue, in the long term, could easily affect the survival of a small firm. For ESO, we find, overall, a decrease in the subscription likelihood by a factor of .63 (= e^−.47), again in terms of the odds ratio. We estimate 95,911 fewer subscriptions if the app provides ESO to the full existing user base. This finding suggests users favor “less is better” regarding these supplementary games.

OPEN IN VIEWER

Table 2.

Estimation Results for Field Experiment 1

	Models
Variable	(1.1)	(1.2)	(1.3)	(1.4)
Soft landing	−1.54*** (.14)	−1.45*** (.15)	−1.39*** (.15)	−1.38*** (.15)
ESO	−.51** (.15)	−.48** (.15)	−.46** (.16)	−.47** (.16)
Soft landing × ESO	1.34*** (.25)	1.25*** (.25)	1.15*** (.25)	1.15*** (.25)
Female		−.18*** (.020)	−.19*** (.019)	−.19*** (.019)
Age		−.26*** (.033)	−.31*** (.034)	−.31*** (.035)
Reading ability		1.63*** (.10)	2.24*** (.14)	2.24*** (.14)
Apple products		−.45*** (.024)	−.72*** (.084)	−.72*** (.074)
Customer tenure			−.0045 (.030)	−.0046 (.030)
Usage			6.96*** (.84)	6.99*** (.86)
Disposable income				−.0019 (.0083)
City location				.00018 (.013)
Intercept	−4.01*** (.12)	−2.55*** (.28)	−2.86*** (.26)	−2.82*** (.27)
Demographic	No	Yes	Yes	Yes
Pretreatment behavior	No	No	Yes	Yes
City characteristics	No	No	No	Yes
N	1,907,228	1,907,228	1,907,228	1,907,228
Log-likelihood	−107,348.9	−104,686.8	−102,094.5	−102,092.7
AIC	214,705.8	209,389.5	204,209	204,209.5
BIC	214,755.6	209,489.2	204,333.6	204,359
Hit rate	50.58%	58.25%	63.38%	63.36%
Hit rate for purchase	69.17%	72.06%	71.56%	71.41%
Hit rate for nonpurchase	50.39%	58.10%	63.30%	63.28%

*p < .05, **p < .01, ***p < .001.

Notes: AIC = Akaike information criterion; BIC = Bayesian information criterion. All standard errors are robust estimated clustered sampling at the city level. This table reports the logistic regression coefficient estimations. We scale all the continuous variables (reading ability, customer tenure, usage, and disposable income) down to the range of [0, 10] for ease of estimation and explanation. Column 1.1 reports the logistic regression results with only the main and interaction effects of soft landing and ESO. Column 1.2 adds demographic information. Column 1.3 includes users’ pretreatment behavior. Column 1.4 is the full specification.

Additional analysis on user churn

Apart from initial subscriptions, we also look at the decision to unsubscribe from the service. Users received their first bill after subscribing and could then choose to cancel any time before the next bill. We tracked all the subscribers for two additional monthly billing cycles after their first payment. Figure 5 illustrates the unsubscribe rates for each group; overall, the rate is 19.84%. Furthermore, it shows that soft-landing groups had a higher unsubscribe rate than hard-landing groups. At the same time, the differences in churn rates across treatments are not large. We also plot the speed of churn once a user subscribed to the paid service (see Figure WB4 in Web Appendix B1). It presents the number of unsubscribed orders plotted against the time between subscription and churn (unsubscription). We observe three peaks of churns after subscription, which, unsurprisingly, come near the 30-day, 60-day, and 90-day marks—that is, around the payment times. This finding provides some guidance for firms regarding the time window for customer relationship management.

OPEN IN VIEWER

Figure 5.

Unsubscribe Rate Across Groups: Field Experiment 1.

Exploring Heterogeneity Effects

We adopted a machine-learning approach, generalized random forests (GRF; Wager and Athey 2018), in the framework of double machine learning (Chernozhukov et al. 2017; Cong, Liu, and Manchanda 2021; Oprescu, Syrgkanis, and Wu 2019) to explore heterogeneity effects with observed variables (for detailed results, see Web Appendix C, Figure WC1). The results indicate that the younger the user, the stronger the negative effect of soft landing. In other words, holding all other variables constant, the negative effect of soft landing on the subscription likelihood is stronger for an 8-year-old user than for a 12-year-old user. In the soft-landing condition, limited free content was provided uniformly (in quantity and quality) for existing users. For young users, they may have relatively “more” reading materials to enjoy subjectively than older users. Therefore, this “outside” option in soft landing (reading free materials without paying) is more attractive for young users than older users, and it further decreases the perceived value of the paying option for young users. Thus, for a younger user, the negative effect of soft landing on subscription likelihood is exacerbated. Next, we find that a higher reading ability alleviates the negative effect of soft landing on users’ subscription likelihood. The lower a user's reading ability, the more “subjective” free reading materials there are for them to read, which lowers the attractiveness of subscribing to the service. We did not find significant heterogeneous treatment effect in terms of customer tenure.

We also find that a higher reading ability and higher pretreatment usage mitigates the negative effect of ESO. It appears that, in our context, users with a higher reading ability seem more capable of evaluating the value of the package based on its primary feature (reading service), so we see that the negative impact of ESO is attenuated for these users. Once again, there were no heterogeneous treatment effects of customer tenure. We also conducted similar GRF analysis as part of the robustness check when considering subscribers’ choices of a price option to explore heterogeneity effects of soft landing and ESO associated with different price options. The results show consistent findings with our previous analyses (see Web Appendix C2 for more details).

Model and Analysis

Figure WD2 in the Web Appendix D presents the users’ conversion rates across conditions. Table 3 reports the summary statistics of the data set for Experiment 2. We want to estimate an individual user's likelihood of subscribing. App user i's utility of choosing to subscribe is

U 2 i = V 2 i + ω i = θ 0 + θ 1 Soft Landin g i + θ 2 Extended Offe r i + θ 3 Soft Landin g i × Extended Offe r i + ρ Control Var s i + Cit y R ( i ) + ω i .

(3)

OPEN IN VIEWER

Table 3.

Descriptive Statistics for Field Experiment 2

Variables	Mean	SD	Min	Max	Description
Decision Variable
Subscribe to paid content	.0057	—	0	1	1 if user i pays to subscribe; 0 otherwise
Treatments
Soft landing	.50	—	0	1	1 if user i receives the 3-day free voucher; 0 otherwise
Extended offer	.50	—	0	1	1 if user i receives extended 14-day pricing offer (1 CNY/14 days); 0 otherwise
Soft landing × Extended offer	.25	—	0	1	1 if user i receives both extended 14-day pricing offer and 3-day free voucher; 0 otherwise
Demographic Info
Female	.48	—	0	1	1 if user i is female; 0 otherwise
Age (years)	8.07	1.64	5	13	Age of user i
Reading abilitya	382.6	327.07	0	1,264	Reading ability score of user i
Pretreatment Behavior
Customer tenure	21,082	6,100.43	1,852	45,810	How long user i has been registered on the app before the experiment, in minutes
Usageb	664	768.55	1	11,383	How much time user i has spent on the app before the experiment, in minutes
City Info
Disposable incomec	22,460.97	6,964.77	15,011	59,454	Average disposable income of residents in the province where user i lives
City locationd	.45	—	0	1	1 if the city where user i lives is categorized as a southern region; 0 otherwise

^a

Latest preexperiment score reported for user i.

^b

Total time user i spent on the app before the treatment was applied.

^c

Reported in Chinese RMB (CNY); 1 CNY ≈ .15 USD at time of experiment.

^d

Climatologically, China is typically divided into southern and northern regions along the Qinling–Huaihe line.

Notes: N = 1,140,182.

Again, assuming a Type I extreme value distribution for the errors, we have

Pr ( S 2 i = 1 ) = exp ( V 2 i ) 1 + exp ( V 2 i ) .

(4)

OPEN IN VIEWER

Table 4.

Estimation Results for Field Experiment 2

	Models
Variable	(2.1)	(2.2)	(2.3)	(2.4)
Soft landing	−.73*** (.20)	−.74*** (.21)	−.76*** (.19)	−.74*** (.19)
Extended offer	.40* (.16)	.34* (.17)	.34* (.17)	.36* (.16)
Soft landing  × Extended offer	−.10** (.26)	−.014 (.26)	.092 (.25)	.056 (.24)
Female		−.17*** (.027)	−.18*** (.028)	−.18*** (.027)
Age		−.25*** (.040)	−.31*** (.041)	−.31*** (.042)
Reading ability		1.63*** (.18)	2.32*** (.21)	2.32*** (.21)
Customer tenure			−.028 (.025)	−.028 (.025)
Usage			6.76*** (.81)	6.79*** (.82)
Disposable income				−.015 (.082)
City location				.051 (.12)
Intercept	−5.05*** (.13)	−3.68*** (.28)	−4.00*** (.27)	−4.01*** (.26)
Demographic	No	Yes	Yes	Yes
Pretreatment behavior	No	No	Yes	Yes
City characteristics	No	No	No	Yes
N	1,140,182	1,140,182	1,140,182	1,140,182
Log-likelihood	−39,594.66	−38,831.12	−37,963.32	−37,961.02
AIC	79,197.32	77,676.25	75,944.63	75,944.04
BIC	79,245.11	77,759.87	76,052.15	76,075.45
Hit rate	50.59%	60.35%	62.83%	62.85%
Hit rate for purchase	71.55%	71.14%	75.06%	75.30%
Hit rate for nonpurchase	50.49%	60.30%	62.77%	62.83%

*p < .05, **p < .01, ***p < .001.

Notes: AIC = Akaike information criterion; BIC = Bayesian information criterion. All standard errors are robust estimated clustered sampling at the city level. This table reports the logistic regression coefficients. Column 2.1 reports the logistic regression results with only the main and interaction effects of soft landing and extended offer. Column 2.2 adds demographic information. Column 2.3 includes users’ pretreatment behavior. Column 2.4 is the full specification.

Discussion

Experiments 1 and 2 show consistent findings regarding the causal effect of soft landing; that is, in either form, the soft-landing strategy yields a lower subscription rate than the hard-landing strategy. This finding suggests that consumer learning was unlikely to be at play here, because we gave the users unwilling to subscribe right away (e.g., due to insufficient experience with the app) time to learn about the app's quality by interacting with the free content. Consumer learning is more likely to play a role when users have high uncertainty about the service (or have used the service very little) or the overall diffusion of the app is low in the target markets before the switch. From the diffusion patterns in Figure 1, the market appears to have been approaching saturation (given the slowdown in growth rates). Further, the average customer tenure was 13 days, and users spent about 600 minutes on the app on average before the experiment. This finding suggests that learning may have already occurred for most of these users. Furthermore, only 2.4% of users accessed the free content in the soft-landing condition. At the same time, only 16.5% of users subscribing in the soft-landing condition used the free content during the experiment. Together, these observations suggest the free content did not serve as a vehicle for learning about the content for existing users. Thus, our findings appear to be in line with our proposed conceptual framework based on perceived value—that is, that the presence of the outside option of free content lowers the perceived value of subscribing to the service, which leads to fewer users’ subscriptions.

To quantify the perceived value of the various options to users, we also calculate an average user's utilities for the content and the ESO using the data from Experiment 1 (see Web Appendix B2 for details). In our context, the provision of reading content is the key value provided by the app, where the utility is estimated as V^Books = 26 CNY/month (relative to 0 for the quitting option). We find that limited free content provided in the soft-landing case will provide an option value with positive utility of 1.54 CNY/month to existing users. We estimate the user's average utility from ESO to be negative (V^ESO = −.52 CNY/month). When an app's service switches from free to paid, users are charged for previously free content and have difficulty evaluating the dollar value of the primary feature. However, the ESO is easy to evaluate as being of low value. Thus, consumers use the ESO as a cue to evaluate the value of the package and are therefore less willing to pay. This negative effect is the high evaluability of the low value of the ESO. Moreover, the results from Experiment 1 indicate that though harmful in the hard-landing condition, providing some ESO is beneficial in increasing users’ willingness to subscribe in the soft-landing condition (V^{ESO × Free} = 1.35 CNY/month). The positive interaction effect is in line with our conceptual framework that suggests the ESO provides exclusivity value to paying users in the soft-landing condition because of the uniqueness and status it creates. This increases the perceived value of the package to existing users. Next, we describe findings from an ex post customer survey conducted by the firm and a randomized experiment we conducted on Prolific that provide further support for the theoretical bases of our findings.

Robustness Checks

In this section, we conduct a series of robustness checks and rule out or account for potential alternative explanations for our results.

Implications

This research provides important actionable managerial implications to mobile app managers and to online content providers, with some of the implications running counter to conventional business wisdom on app monetization. The conversion-rate difference among different monetization strategies is sizable and can lead to a substantial revenue difference. For example, by applying the Group B monetization strategy (hard landing, no ESO) in Experiment 1 instead of the strategy planned by the app manager (soft landing with ESO), the focal firm stood to collect about 102% more revenue: about 11.9 million CNY a year, or just shy of 1.79 million USD. ¹³ From Experiment 2, we find that another form of soft landing can hurt the users’ subscription likelihood as well; applying the Group 2 monetization strategy (hard landing) would yield about 128% more revenue than the Group 4 strategy (soft landing with same price option). Overall, the findings underscore the significant cost of providing limited free service when converting an app from free to paid. More broadly, app firms should also be cautious about adding ESO to a paid package. For the app firm under Experiment 1, we estimate that an appropriate choice concerning ESO could yield 65.5% more subscriptions (9.37 million CNY ≈ 1.40 million USD) under the hard-landing scenario and 128% more subscriptions (6.6 million CNY ≈ .98 million USD) in the soft-landing case. App firms face considerable difficulties when attempting to monetize their services for two main reasons. In light of low willingness to pay and high competition, even a .01% increase in the conversion rate could be a life-or-death matter for a start-up app firm.

Implementation

After the 80-day field-experiment period, the focal firm began to implement our recommended pricing and app-product-design strategies gained from Experiment 1. By the middle of 2019, all users on the app were paying users. For existing free users, free service was no longer provided with the recommended hard-landing strategy. A short free trial was initially provided to new users who joined after the experiment (consistent with the implication discussed previously). After the free trial ended, no limited free service was provided; users needed to pay to subscribe to the full service.

The firm's total reported revenue in November 2019 approached about $1.8 million, with subscriptions from converted existing users constituting the bulk of it. ¹⁴ Compared with the zero-revenue level before monetization, the firm viewed the revenue figure as making a big difference to its survival and growth in a competitive market.