Tinkering With the Plumbing of Sustainable Enterprises: The Case for Field Experimental Research in Corporate Sustainability

Introduction

Although field experiments have become the norm in many scientific domains, they have only recently become more prominent in the study of organizations (Chatterji et al., 2016; Eden, 2017). Yet, to date they are rarely used in the study of corporate sustainability (Delmas & Aragon-Correa, 2016; Zollo et al., 2013). To encourage methodological development in organizational sustainability research, we invited contributions for this special issue. In line with the domain statement of the journal, we called for papers that applied field-based experimental designs to study organizational-level interventions aimed to enhance sustainability-related outcomes.

In this introduction to the special issue, our objective is not to provide a technical overview of the methodological and substantive issues involved in designing and implementing effective field experiments in organizational settings (for that purpose, see Chatterji et al., 2016; Eden, 2017). Instead, we aim to further conversations around two interrelated questions: how would corporate sustainability scholarship benefit from the establishment of a field-based experimental approach as a norm, rather than an exceptional mode of research design? And, what could be done to realize the full benefits from field experimentation for the advancement of science and of societal well-being through sustainable business activity?

To answer these questions, we build on similar debates and discussions that have accompanied the rise of field experimentation in the context of developmental economics, leading three of it pioneers, Abhijit Banerjee, Esther Duflo, and Michael Kremer, to win the 2019 Nobel Prize in economics. The Nobel Prize lectures by these researchers provide a broad discussion of the lessons learned about both the benefits and barriers to randomized controlled trials in policy evaluation (Banerjee, 2020; Duflo, 2020; Kremer, 2020). While these reviews are extensive in scope, we focus here on the advantages of field experiments to push beyond aspirational beliefs or localized politics to examine a specific set of issues that Duflo (2020) describes as “plumbing problems” in policy implementation (see also Duflo, 2017). In contrast to engineering problems that relate to the design and choice of new policies,

[i]n plumbing problems, the government is not asking itself whether it should invest in . . . any particular intervention. Rather, it is asking a question of the form: “We are running this particular program and there are issues with it. What can we do to address these issues and achieve our objectives?” (Duflo, 2020, p. 1965)

In this introduction, we build on Duflo’s (2017, 2020) insights to extend the concept of “plumbing problems” to the study of sustainable enterprises. Her work illustrates through decades of evidence that plumbing problems frequently define the long-term effectiveness of any new policy and therefore require independent study in their own right. Despite these benefits, she further observes that many academics nonetheless forego this methodological approach because the humbler objectives and time-intensive work of plumbing-based scholarship does not always conform to existing conceptions of the role of academic research in informing grander policy designs. Building from these lessons, we suggest that barriers to the adoption of field-based methodologies in the study of sustainable enterprises likely rest in similar desires to focus on engineering broad blueprints rather than on the more detailed work required when rewiring the plumbing. Yet, we suggest that the next stage of organizational sustainability research that moves beyond motivational questions of why leaders should seek new objectives to practical questions of how to achieve them in practice can benefit greatly from business–academic cooperation in designing and learning from field experiments. We hope that this special issue inspires future scholars to consider adopting this forward-looking methodology to better understand the plumbing of sustainable enterprises.

Scientists as Plumbers: Lessons From Development Economics

The economist-plumber . . . is more concerned about “how” to do things than about “what” to do. In the pursuit of good implementation of public policy, she is willing to tinker. Field experimentation is her tool of choice. Duflo (2017, p. 3)

In her discussion of the role of academics in the policy-making process, Duflo (2017, 2020) distinguishes between a vision of scientists as engineers and as plumbers. Building on an extended metaphor of building a house, she observes that engineers design the general blueprint of new constructions, whereas plumbers figure out where to put in the pipes and ensures that they function as desired. Thus, while economist–engineers work with policy makers to design new policies—which she refers to as “what to do” in the opening quotation—the economist–plumber starts with general policy objectives as given, and instead seeks to enter into the details of its implementation—which she refers to as “how to do things.”

She further observes that economists generally prefer to advice on general policy design rather than getting deeply involved in implementing a policy in practice. To an academic researcher, the “details of policy implementation . . . may appear far below her pay grade (e.g., the font size on posters) or far beyond her expertise level (e.g., the intricacy of government budgeting in a federal system)” (Duflo, 2020, p. 1967). Scientists therefore tend to act cautiously before stepping beyond “a bounded set of assumptions” found in the broader discipline, concerned that they have little to add in fixing the complex details involved in policy implementation (Duflo, 2020, p. 1996).

Despite these limiting forces, Duflo (2017, 2020) identifies multiple reasons why policy makers need the type of advice that only plumbers can provide. First, policy designers—whether government officials or academic advisors—often grow frustrated in tracing the complex causal chain running from policy design to implementation and final outcomes. Therefore, they are more likely to attribute the failure of a policy to its general design rather than engage in the challenging task of experimenting with the menu of implementation choices that might be capable of achieving better results through more nuanced interventions.

At the same time, and through multiple case studies, Duflo (2020) identifies a second, less visible mechanism that explains the dangers of attending only to broader policy design: in the absence of a clear understanding of plumbing-based implementation issues, policy makers may come to design blueprints that match the aspirational goals of the designers themselves rather than those that fit the reality of local conditions. For example, Duflo (2020) describes the lessons learned during the implementation of a program designed to improve reading capabilities among school children in India (see also Banerjee et al., 2017). In her review of the many years spent tweaking this program, she observes that a primary constraint to implementation rested in the aspirational goals of the original designers. The initial curriculum was designed to educate a small elite group of the most successful children, such as the children of the politicians that designed the program, and therefore “scaling back the ambitious curriculum might have appeared to shortchange children, which is difficult to justify politically” (Duflo, 2020, p. 1960). Given these political realities, policy makers did not wish to revisit or discuss the details of whether such goals could be met at scale. Over time, the academic–plumber research team successfully provided evidence through many randomized controlled trials that tailoring programs to children’s actual skill levels produced better results than the original one-size-fits-all approach. The reworked initiative—now titled “teaching at the right level”—arose because of the power of experimental based data, which led policy designers to adapt to plumbing-based concerns even if they had not at first wanted to attend to these types of implementation issues.

Duflo (2017, p. 21) similarly illustrates the power of field experimentation to put the “spotlight on new problems” through an example of the mismatch between the espoused ambition of microfinance entrepreneurs and actual results in the field (see also Banerjee & Duflo, 2011). Anecdotal stories of low-income clients able to start new businesses and climb out of poverty because of access to new sources of microcredit supported broader efforts to raise money and attention in the new microfinance industry. In contrast to these aspirational stories, field experiments uncovered much more limited results. For instance, in one experiment 52 neighborhoods were randomly selected out of a total of 104 for a microcredit lending organization to enter (Banerjee et al., 2015). Researchers then compared the households in these sets of neighborhoods with those that were not selected 15 to 18 months later. Overall, they found that the fraction of families that started a business went up about 5% to 7% in the neighborhoods in which microfinance programs were introduced over the control neighborhoods.

Despite these positive results, leading microfinance leaders at the time pushed back because they contradicted the aspirational goals of global scale and impact to which they had publicly committed. But, as in the case of “teaching at the right level,” the evidence of field-based experimentation eventually led industry leaders to acknowledge that policies could not be designed based on goodwill and aspirational goals alone but also required close attention to the details of policy implementation.

The Plumbing of Sustainable Enterprises

The development economics experience illustrates that randomized controlled trials represent a strong evidence-based methodology that forces both decision makers and their academic advisors to recognize and attest to the details of policy implementation that might have been originally unrecognized. We propose here that these insights provide a strong rationale for the continued extension of field-based methodology to the study of sustainable enterprises. The efforts of several international institutions like the UN Global Compact and the Organisation for Economic Co-operation and Development Guidelines for Multinational Enterprises, and even self-governed initiatives like Global Reporting Initiative, World Business Council for Sustainable Development, and International Organization for Standardization have encountered significant difficulties in moving from statements of principle and rules of conduct to the activation of the internal organizational change processes required to accomplish desired objectives. Whereas organizational actors and stakeholders may be increasingly motivated to introduce sustainability-oriented change initiatives based on generalized blueprints, the actual change processes appear to be slow (at best) and often never reach original ambitions. In other words, the challenge before both managers and management scholars is to figure out the appropriate plumbing of the house as well as its overall design. This task also remains in light of high-level design efforts such as the UN sustainable development goals. Grand visions and novel architectures still need to be implemented operationally in individual firms—a question of plumbing.

To extend the lessons learned from field-based experimentation in developmental economic to the corporate sustainability literature, we begin with Zollo’s et al. (2013, p. 242) observation of a shift in the “sustainability debate from its historical focus on definitional (‘what’) and motivational (‘why’) questions to the understanding of change and learning process questions (‘how’).” The traditional focus on “what” and “why” questions sits at the center of what Duflo (2017, 2020) identifies as an “engineering”-based approach to policy advice and design. In this logic, the relevant questions relate to the appropriate goals firms should pursue rather than to the details of how to achieve them.

In contrast, a plumbing perspective raises different types of research questions. For instance, Zollo et al. (2013) identify a wide range of plumbing-based infrastructure issues tied to effective transitions to sustainable enterprises. One set of plumbing issues relates to organizing processes, such as structures and systems of governance, coordination, control, and capabilities, that provide the underlying foundations required for any organizational change initiative to succeed. A second set relates to the types and quality of a firm’s stakeholder relationships, found in a firm’s social capital with both internal and external constituencies and grounded in collaborations, dialogues, and trust. These organizational and relational capabilities factor into the ability of the firm to develop the systems that allow it to sense specific issues, ideate and experiment with related solutions, and finally adapt and scale the most effective initiatives over time.

Spicer and Hyatt’s (2017) history of the long-term implementation of Walmart’s sustainability journey provides an example of these types of plumbing issues. Their narrative does not attempt to identify the full set of reasons why Walmart announced large-scale sustainability objectives but instead looks at the details of implementation of these ambitious goals. The challenges facing managers charged with implementation in practice were related to the plumbing problems often skipped over in bold designs, such as the challenges of figuring out the details of how best to identify, measure, and credibly communicate what exactly constitutes a sustainable product in a low-cost retailer like Walmart. The wide breadth of initiatives that the firm undertook showed the firm itself was engaged in its own experiments, some of which fit with its business model, while others, such as delivering a credible system of customer-focused communication about sustainable products, did not. Walmart learned that announcing an ambitious blueprint does not provide the level of detail necessary to discern and guide implementation and learning processes over time (Spicer & Hyatt, 2017).

The Walmart case illustrates the wide range of decisions that require coordination in implementing any new sustainability program. However, few large-N studies are capable of evaluating the effects of any single plumbing-based decision on long-term outcomes, given the difficulty in gathering detailed and commensurate implementation metrics across multiple organizations. In contrast, as illustrated in the decades of experience in development economics, a field-based experimental approach provides a means to systematically examine the often-forgotten plumbing of sustainable enterprises. As we illustrate in our subsequent review of the issues published in this special issue, field experiments provide a means to enter into the black box of organizational implementation to directly isolate the causal effects of implementation choices on long-term outcomes.

Papers in the Special Issue

All the papers in this special issue clearly fit the types of “plumbing” logic that Duflo (2017, 2020) identifies as a central focus of field experimentation. None of the papers are focused on the motivations of the leaders who choose to implement a specific policy but instead on the choices made in their implementation efforts. The studies therefore isolate a small set of organizational levers in an effort to improve implementation of an initiative based on lessons learned through randomized controlled trials of alternative organizational actions.

Salmivaara and Lankoski (2019) examine the possibility of an initiative to increase awareness and sales of sustainable food in a workplace context. To focus on the specific design features in the choice to achieve this objective, they focus on the type of norms that various marketing messages may evoke. In particular, they study the activation of injunctive (as opposed to descriptive) social norms in food service workplaces (restaurants). Analysing experimentally manipulated messages relating to Baltic Sea protection and local food on the choice of a novel sustainable dish, they find that injunctive norm activation selectively impacts on subgroups with specific sociodemographic characteristics and prior consumption habits. They also report evidence of a mutually reinforcing interaction effect from activating multiple injunctive norms simultaneously.

Burbano (2021) studies the issue of communicating firm-level charitable giving to gig workers. As she notes, this broader question of extending corporate social responsibility to gig workers is questionable, given the temporary and malleable relationship between workers and hiring firms in these environments. Specifically, she evaluates the impact of charitable giving on temporary worker behaviour through randomized controlled trials carried out on two major online labour market platforms. She finds that information about charitable giving led to gig workers completing unpaid extra work, with prosocially oriented workers responding in a particularly strong way. Moreover, via a post hoc vignette-based process experiment on one platform, she identified increased perception of closeness to the virtual employer from charitable giving as a potentially important mechanism explaining the observed effects.

Delmas and Kohli (2021) examine potential solutions to urban air pollution issues by focusing on the relative effectiveness of different approaches to information provision as tools for behavioural change for public health protection. Experimentally manipulating smartphone app messaging strategies, they evaluate participants’ intent and actual engagement with air quality information. Distinguishing content and framing of messages, they find that user demographics represent the strongest predictor of engagement. Via a comparison with survey data, the study corroborates that participants with health conditions, as well as women, were more likely to react to the relevant messages.

Finally, while the first three papers of our special issue were carried out in industrialised countries, Wohlgezogen et al. (2021) provide a complementary perspective on more recently developing countries by looking at ways of structuring sustainability-related administrative interventions vis-à-vis supply chains in South Africa. In a randomized field experiment with the suppliers of a large insurance company, they compare treatment effects of compliance-focused and enabling interventions and study how they comparatively affect attitude change related to environmentally sustainable behaviour. Their findings reveal that enabling interventions are indeed associated with administrative systems becoming perceived as supportive for learning, but also found unexpected negative impacts of enabling interventions on perceptions of more specific problem-solving capacities. They suggest that an implicit trade-off between the longer term benefits of learning processes and the shorter term effects of coercive procedures on structuring day-to-day operations might explain these results.

Taken together, the articles examine the impact of alternative ways to implement sustainability-related policies that influence multiple internal and external stakeholders: customers (Salmivaara & Lankoski, 2019), employees (Burbano, 2021), local communities (Delmas & Kohli, 2021) and suppliers (Wohlgezogen et al., 2021). They also examine the organizational processes that shape sustainability implementation, which can be leveraged not only to enhance the effectiveness of sustainability strategy deployment but also to revise and improve the high-level theoretical constructions that contributed to the policy design in the first place.

Discussion

In the previous sections, we have proposed that Duflo’s (2017, 2020) emphasis on plumbing-based issues in developmental economics can inform our understanding of the foundations of sustainable enterprises as well. We have also shown that the articles published in this special issue point to the benefits of field-based experimentation to isolate and analyse implementation choices in sustainability initiatives. Yet, given the benefits for the corporate sustainability academic field from the adoption of field experimental designs, the natural question that subsequently arises is: why has this research design not diffused earlier and more broadly in our work? Why has it not become the modal way to study business transition toward sustainable logics of enterprise (Crilly & Sloan, 2012)?

The experience of development economics suggests that one barrier to the wider adoption of field-based experimental research methodology is that academics and policy makers may not wish to engage in its longer term and humbler research agenda. An aspiration to engineer grand solutions may lead to disinterest and impatience in exploring the details of plumbing-based implementation decisions and processes. However, learning from incremental experimentation and trial is already a familiar strategy for business. For instance, Google continuously and comprehensively experiments with the appearance of their search engine with users in real time in order to gather evidence on its effectiveness for advertising performance (Chen & Konstan, 2015).

One reason for limited interest in controlled experimentation in corporate sustainability initiatives, as opposed to other types of change programs, might be found in the sensitivity of the subject matter for corporate reputation, particularly the potential damage from the publication, or even internal communication, of “failed” experiments. In fact, the resistance could come from the managers responsible for sustainability practices and policies, whose (relative) effectiveness in reaching specific impact targets might be exposed with quantitative evidence through field-based experimental designs. Likely, this issue gets compounded further when attempting to harmonize experiments beyond individual organizations in efforts to derive the fullest benefit from their execution. Therefore, the perceived challenges for firms to publicly explore the failure of sustainability initiatives—no matter the general benefits of learning from such experiments—acts against firm participation in randomized, controlled field experiments. In turn, the relative difficulties of recruiting firm participation, may lead academics to opt against this time-intensive methodology in favour of alternative designs that offer greater statistical power. Business cooperation with academics to systemize experimental designs in the implementation of sustainable initiatives, therefore, represents a logical next step in pursuing a shared agenda of moving beyond grand designs to more closely tinker with the wiring of sustainability initiatives, thereby improving the quality of the change efforts selected to be scaled across organizations.

Another challenge to implementing field experiments in organizational research may relate to the historical conditions that have led to the dominance of other preferred research designs. For example, March (2007) describes how after World War II the field of sociology, a discipline where experiments are less prominent, influenced advanced training in business schools. Imprinting and compounding path-dependency may therefore have led to experiments being crowded out of the methodological repertoire in organizational research in favour of other dominating paradigms, such as the historical case focus of business schools or econometric approaches transferred from economics (where a field experimental tradition developed only in the last two decades). Ultimately, this may have led to a situation where self-limiting beliefs of individual academics from their own PhD training have cumulated in a paradox situation where traditional standards are reproduced.

Moreover, in a classic application of institutional theory, business academics, perhaps with the sole exception of marketing scholars, have strengthened their self-limiting beliefs through mimetic isomorphism and, even more interestingly, by leveraging the fact that there were few precedents of successful cases of randomized, controlled trials being carried out within organizations. In part, this is a self-fulfilling prophecy produced by the combination of self-limiting beliefs (we cannot do it, firms will not allow us to do it) and the missed development of generally accepted (by both companies and researchers) norms and routines in business/academia research-related interactions. The recent Nobel prize to pioneers in development economics provides one point of evidence that such barriers may nonetheless be overcome. If economists can engage governments to run randomized controlled experiments with entire populations, there does not seem to be obvious reasons why business scholars cannot do the same with companies, regardless of their size, as long as they are committed to find the solution to a strategically relevant issue by tinkering with features of their organization.

Yet, challenging these limiting beliefs likely requires a cognitive shift in our community of scholars away from the taken-for-granted idea that research is about studying “what has happened to firms” and toward the complementary notion of contributing to the future actions of firms, especially those aimed at innovative organizational change initiatives. While this change of perspective may seem relatively trivial and innocuous at first glance, it represents, as Duflo (2017, 2020) frequently observes, a completely different way to see our own identity as social scientists. Rather than viewing scientists as distant and passive observers, field experiments require a proactive mind-set that constantly identifies and evaluates the next implementation challenge that needs to be overcome. Note that we do not argue for substitution of a backward-looking research logic with anew, forward-looking, one, but to accept the value of expected positive synergies and complementarities between the two.

Field experimentation also forces academics and business counterparts to recognize the potential gains from collaboration at the frontiers of knowledge, and learn to seize such gains through the establishment of new routines for forward-looking, collaborative research. The notions of engaged scholarship (Van de Ven, 2007) and knowledge co-creation have now become part of our regular vocabulary, especially when we talk about the future developments of our fields. Field-based experimental designs constitute an obvious, powerful, and yet underutilized, way to build such collaboration with our main stakeholders. They also produce a powerful form of evidence in support of causal claims, since unobserved heterogeneity and endogeneity are handled by the randomization and control within a homogeneous population. Control groups operate as the relevant comparative cases to test whether any specific intervention produces desired effects. At the same time, this form of research provides companies the opportunity to actually enact innovative processes and learn from them. These outcomes are clearly out of question in the case of backward-looking research that studies what has already happened to the company.

One might argue that this last feature of field experimentation represents the most powerful reason for scholars of corporate sustainability to consider adopting this methodology. It permits the historic novelty of being able to contribute, as pro-active knowledge (co)producers, to the evolution of a new species of business organization: the company that integrates the voice and the interests of employees, customers, suppliers, and local communities within its structures, processes (especially strategic decision-making), and culture. Tinkering with the “plumbing” of what constitutes a sustainable enterprise is a unique endeavor that future generations of scholars will hopefully take on themselves to develop, master, and enjoy, for the benefit of science, business, and humanity. This will require not only a significant learning effort, but (perhaps most meaningfully) the humility to yield our social role as sole actors in the knowledge development process in favour of shared ownership with the organizations we study as new knowledge is produced and shared.