Establishing and implementing data collaborations for public good: A critical factor analysis to scale up the practice

To conduct our study of success factors we used the Critical Success Factors approach (Borman & Janssen, 2012; Remus & Wiener, 2010; Rockart, 1981). The concept of success factors was first proposed in 1961 by Ronald Daniel of McKinsey & Company. The approach was formulated in 1979–1981 by John Rockart, co-founder of the MIT Centre of Information Systems research (Rockart, 1979, 1981). Rockart’s approach was supposed to help executives who suffer from information overload to define their information needs. In its original interpretation, critical success factors (CSFs) are areas on which executives should focus their attention and seek data and reporting. Although this approach is decades old, it is still relevant for the present time of data explosion. Next to the original approach of Rockart, subsequent approaches were proposed which focus more on the universality of factors (rather than them being context specific), on the implementation process (rather than the outcome), and on the project as a level of analysis (rather than individual manager) (Borman & Janssen, 2012).

Overall, CSF is a widely used approach which proved useful in previous research on data sharing and digital government in general. There are many examples of research studies using this approach in this field, including on the topics of e-government adoption (Rana et al., 2013), national e-ID (Melin et al., 2013), emergency management (Chen, 2011), e-participation (Panopoulou et al., 2014), shared services in public sector (Borman & Janssen, 2012), adoption of electronic document management systems in government (Alshibly et al., 2016), open data publication and use (Susha et al., 2015), to name a few. The strengths of the CSF approach are that it can be helpful for improving project implementation and for making sense of complex problems where many factors occur, it offers an opportunity for learning and transferability of project lessons, and it is a widely tested method with a rigorous procedure. The weaknesses of the CSF approach, however, are that there is a lack of theoretical base (especially in the digital government domain), there is a discrepancy between the original and subsequent approaches, it is difficult to balance specificity with generality, and it may be challenging to define what is meant with ‘success’.

There is a variety of research methods which can and have been used to identify success factors, such as case studies, literature reviews, focus groups, scenario analysis, structured interviewing, group interviewing, Delphi technique, action research and others (De Sousa, 2004). Overall, the CSF approach includes four stages (De Sousa, 2004): state of the art, identification of CSFs, relevance of CSFs, and management of CSFs. We chose to follow this approach because it is informed by previous literature, it is straightforward and easy to apply, and it is comprehensive as it combines theory with practice in its different stages.

Figure 2 shows how we applied this approach in our research design. We have omitted the last phase concerning the management of CSFs as our study is exploratory.

OPEN IN VIEWER

Stage 1. State of the art. The first stage of the research – a literature review to systematize what factors are known – was described in the previous section.

Step 2. Identification of CSFs. To ensure that we did not miss any factors, we used the results of a practitioner workshop held at the International Data Responsibility Conference in The Hague (The Netherlands) in February 2016. The workshop was organized by The Gov Lab and was attended by representatives of companies, research institutions, non-profit sector, government (29 participants in total). The participants were asked to formulate factors important for data collaboratives from their experience. We did not provide them with our framework or categories beforehand, this ensured they were not biased towards any particular category or factor and could think about CSFs with an open mind. The factors proposed during the workshop were included in our framework developed from the literature.

Stage 3. Relevance. Once the framework was complete, we conducted an expert workshop to discuss the criticality of the identified CSFs and to prioritize them. The workshop took place during the IFIP WG 8.5 Electronic Government (EGOV) conference in September 2016. It was attended by 17 experts/researchers in the fields of open and big data, e-government, public private partnerships, and information sharing. The majority of the participants had over 5 years of experience. The experts were asked to select from our framework and rank the factors which (1) influence the process of data collaboratives the most, (2) influence the outcome of data collaboratives the most, and (3) without which a data collaborative would fail. This resulted in a list of top most critical CSFs.

To validate the results of the ranking by the EGOV community experts, we conducted two case studies. To select suitable cases, we used the Data Collaboratives Explorer (see DataCollaboratives.org) which is a repository of cases compiled by The Gov Lab. We chose to limit our inquiry to a certain type of collaborative – the trusted intermediary model – to have a better ground for generalizing our insights. The trusted intermediary model, when a neutral actor mediates the transfer of corporate data into publicly useful knowledge, has seen an uptake recently. The repository of cases we used listed 29 of such partnerships at the time. The following were our selection criteria:

Represents a trusted intermediary model of a data collaborative

We included cases from different domains, as long as they conformed to our selection criteria (selection by diversity). The following are the selected cases for our analysis:

To conduct the case studies (Table 2), we used document studies and interviews to collect the data. The interview protocol is presented in the Annex and was based on the guidelines of Rockart (1981) and Borman and Janssen (2012). The interviews were explorative and took place in November 2017. We were particularly interested in success factors from the perspective of the user of the data, therefore in both cases we interviewed organizations who were on the receiving end of the data chain, i.e. accessing and processing data to provide insights. In both cases we interviewed one top-level manager who played a leading role in conceiving and overseeing the collaboration with a private sector partner(s). We did not provide the interviewees with our framework beforehand to avoid steering the conversation towards any particular factor or category.

To identify more CSF (stage 2 in our CSF approach), in addition to the list we compiled from the literature, we used the results of an expert workshop as explained in the Method section. The participating experts did not suggest any factor that was not already present in our list, however they highlighted the importance of several. 10 factors in total were discussed during the workshop. The results of the workshop are available online.2 In particular, the following factors were emphasized: matching supply and demand and data with problems (CSF7), identifying approaching business models (CSF1), long term strategy and sustainability plans (CSF8), building trust relationships (CSF13).

5. Discussion

Based on the ranking by experts, we proposed a list of Top 15 critical factors for data collaboratives. This list includes factors from most categories of our initial framework of 32 factors: Organizational, Technological, and Environmental. To further validate these factors, we conducted two case studies.

Overall, we find an overlap between the prioritization of factors by experts and the findings of our case studies. Most factors mentioned by the interviewees were also top ranked by the experts, although the exact order of priority varies. The main discrepancy is that data quality was not explicitly mentioned as a critical factor in either of the cases. This can be explained by the fact that in both cases the receiving organization was confident about the quality of the data shared (call detail records, supply chain data). Furthermore, in both cases the research nonprofits were aware of the limitations of the data. In the Flowminder case, the data was only from one, although the largest, telecom operator. And in the case of GFW, the granularity and frequency of data could be improved. This however did not impede the course of the collaboration in either cases.

In both cases which we investigated a research non-profit organization approached a private sector partner(s) with an offer to collaborate in order to address a pressing societal problem (aftermath of a disaster and deforestation, respectively). The cases differ in many aspects, such as the purpose, the organizational arrangement, the type of data shared, duration etc. In both cases one of the main challenges mentioned was persuading the private sector to collaborate and addressing their concerns about potential risks to their data and implications thereof. Therefore, both interviewees tended to focus on the formation phase of the collaboration. Both Flowminder and GFW are organizations with resources and skills which are required to provide state of the art technical solutions in this respect. This may explain why fewer of the factors found as important in the case studies have to do with technology. Overall, we find that the factors which were considered to be most critical for the formation of these partnerships are similar. The common factors important in both cases are: value proposition, gaining trust (through reputation, personal relationships, or prior collaboration), and effect of public pressure. Relative to our integrative framework proposed in Section 2, the case studies emphasized the importance of starting conditions for the formation of data collaboratives. Value proposition, public pressure, and trusting relationship between partners are seen as key elements of a successful collaborative. We discuss them in more detail below. Although in our framework trust was included as part of collaboration experience, our case studies showed that it is equally (if not more) important at the outset of collaboration.

The first broad factor common between the case studies was the value proposition. In the recent CCSP literature a model of success of business – NGO partnerships was proposed (Barroso-Mendez et al., 2016) according to which shared values affect trust and commitment which in turn affect relationship learning and have a direct influence on the partnership success. According to this model, it is essential for businesses to share similar values and beliefs with their potential partners in the partnership formation phase. Shared values can be understood as “the degree to which the partners have beliefs in common about what behaviors, goals, and policies are important or unimportant, appropriate or inappropriate, and right or wrong’ (Morgan & Hunt, 1994, p. 25). Our Top 15 list of critical factors includes several factors which are related to and add to this construct: shared understanding (of objectives, values, and expected outcomes), value proposition, incentives, and business model. Thus, our study conveys a view common to the data collaboratives literature which we reviewed that data collaboratives should not only generate social value but also value to the private sector engaging in such partnerships. In this respect, we concur with the underlying proposition of the ‘shared value approach’ of Kramer and Porter (2011) that engagements of the private sector with social issues should have an explicit connection to the company’s profitability and competitive position instead of simply focusing on reputation (Ibid.). Besides positive reputation, incentives for companies to share data for a societal purpose include: new insights, recruitment opportunities, reciprocity, new revenue models, regulatory compliance, and “an ecosystem-supporting responsibility” (OECD & The Gov Lab, 2017). New studies (GSMA, 2018) described a variety of business models that are emerging around data partnerships for social good, although more systematic research is needed. Articulating these benefits in the formation stage of the data collaborative is of utmost criticality, as our case studies suggest. Current research on cross boundary information sharing does not explain the importance of value proposition sufficiently because until lately it has predominantly focused on data sharing between public sector entities. In the public sector, data is considered as a public good and there is often a moral imperative to open up information. The contrary applies to the private sector where data is treated as a resource and ‘the new currency’, thus, giving it away for free is counterintuitive to the business logics of companies. Therefore, we argue that this factor is specific to data collaboratives.

The second broad factor which was common in both cases is trust. Trust was also highly ranked by the experts in our Top 15 list of factors. Trust is a well described topic in the current CCSP and CBI literature. Therefore, our findings concur with these research streams. Trust can be defined as the expected collaboration in which the expectation is that partners will do what is best for the entire system, even when that may lead to a partner’s disadvantage (Capaldo & Giannoccaro, 2015). The model of Barroso-Mendez et al. (2016) puts trust at the core of partnership success. In fact, the better the relationship between partners in terms of trust and commitment, the greater is the extent to which they exchange information and knowledge (Sanzo et al., 2015). CBI literature concurs that trust is one of the most important factors in making cross boundary information sharing successful (Sutherland et al., 2018). Dawes et al. (2009) distinguish between three types of trust: information-based trust (rational decisions about who to trust), identity-based trust (familiarity between stakeholders), and institution-based trust (norms and social structures that define acceptable behavior). As evidenced by our case studies, all three types of trust are relevant for the formation of data collaboratives, as the interviewees mentioned the importance of personal relationships, credibility, reputation of their organization, and prior collaboration. Moreover, in the context of data collaboratives, there are a number of risks associated with the sharing of data, especially when it concerns personal (customer) data. Data can represent a “competitive advantage within the market” and companies need assurance that their data will not be leaked to unauthorized parties (Praditya & Janssen, 2015). Besides security and potential data leaks, there is a risk that the data may be misinterpreted or used inappropriately (The Gov Lab, 2018). Therefore, gaining trust of private sector partners in a data collaborative involves assuring them that the data will be handled responsibly. It is thus not coincidental that, according to the interviewee in the Palm Oil case, ensuring data confidentiality was one of the critical factors. This means that, besides reputation and credibility, the recipients of data are often expected to have data analytics capabilities which can be trusted. In fact, in an information exchange relationship competence-based trust is often found as more influential than affection-based trust (Sayogo & Pardo, 2011). Therefore, it is not incidental that technical and analytical skills are included by the expects in our Top 15 list of factors.

The third broad factor concerns the effect of public pressure. Unlike the two aforesaid ones, this is an external factor characterizing the environment in which the collaboration is formed. Some authors view such factors as ‘antecedent conditions’ (Bryson et al., 2015) or ‘starting conditions’ (Ansell & Gash, 2008) but few studies actually shed light on the relationship between public pressure and success of partnerships. In situations when public and private organizations collaborate, literature identifies political support, policy support, and/or community support as important for the progress of partnerships (Osei-Kyei & Chan, 2015; Marlier et al., 2015). However, we argue that public support is not equivalent to public pressure. Both of the cases we surveyed highlighted the ‘network effect’ of the public and community pressure to collaborate and share information. The specifics of data collaboratives are that these are partnerships often driven by the need to address a pressing and often urgent societal problem. Many of these problems, like climate change or poverty, are ‘wicked’ (Manning & Reinecke, 2016) and pose ‘grand challenges’ (George et al., 2017) because their magnitude is difficult to assess, and the cause-effect relationship is highly complex. At the same time, there is mounting pressure in the public sphere to address these problems. This explains why public pressure is seen as a particularly influential factor for the formation and implementation of data collaboratives. For instance, in the Flowminder case it was seen as critical that other actors in the ecosystem approached the telecom on a regular basis with requests to share data. And in the Palm Oil case responding to the need to facilitate supply chain transparency was seen as influential. We thus argue that public pressure is particularly influential factor for data collaboratives.

In our study we were interested to find out whether data collaboratives as a novel form of partnership should be implemented in any different way. What we found is that many of the factors known from more ‘traditional’ collaborations are still valid for data collaboratives, such as trust, public support, shared goals, motivations etc. Our Top 15 list of factors captures many factors described in previous research. However, we also found that some factors that received marginal attention or were not at all covered in established literatures (only in emergent data collaboratives research) were included in our Top 15 list. They are business models and value proposition and matching of data to problems and thereby of demand with supply. This is the niche where new research is needed to develop frameworks and tools for business model innovation and for problem formulation in the context of data-driven decision making.

6. Conclusion

In our study, we proposed a theoretical framework of critical factors relevant to data collaboratives based on the review of literature and discussions of an expert workshop (Table 1). We further prioritized the factors from the framework in an expert workshop at an academic conference and reduced the number of factors from 32 to 15. The Top 15 factors (Fig. 2) are expected to be of utmost criticality for data collaboratives. To test whether this is the case, we conducted two exploratory studies of data collaboratives in different domains. Our conclusion is that we found support for the identified factors from the interviews. Thus, the full framework is suggested to be used as a guideline for the implementation of data collaboratives, while the reduced list of 15 factors highlights the most important areas. In Section 5 we further discussed three broad factors which were found critical in both cases: value proposition, trust, and public pressure. We argue that in relation to each of these factors there are certain nuances when it comes to data collaboratives as an emerging form of partnership.

Our study makes a research contribution to the field as it integrates previously fragmented knowledge about what contributes to a successful data collaborative. This is an emerging topic which exists at the intersection of more established research fields. Our study is one of the first to situate this phenomenon in existing academic research and thereby contribute to its conceptual maturation. We offer our initial framework of factors, as well as our Top 15 list, to be tested in future empirical studies. We also contribute to the literatures on cross sector social partnerships and cross boundary information sharing by putting a spotlight on an innovative form of partnership catalyzed by the data revolution.

In terms of practice, our initial list of factors can be used as a general guideline by practitioners contemplating to engage in a data collaborative. It provides a holistic view of elements which come into play when a data collaborative is formed. The value of the Top 15 list is that it highlights elements which typically have a greater influence over the success of the partnership. This list thus can be used to help organizations prioritize and distribute resources accordingly when engaging in a data collaborative. This list also highlights factors which capture the specificity of data collaboratives compared to other types of partnerships. The value of the Top 15 list is not in the rankings of factors (which was exploratory and only provides an indication of relative importance) but in the fact that it offers a concise view of 15 select factors which deserve immediate attention.

To acknowledge the limitations of our study, the case studies primarily focused on the formation phase and did not capture long term impacts of these partnerships. Furthermore, both case studies are limited to the trusted intermediary model of data collaboratives; other forms of collaboratives may require a different mix of critical factors. Future research can test and elaborate our findings using more cases from other contexts (other types of data collaboratives, from other domains, involving other kinds of participants etc.).

The phenomenon of data collaborative is quite recent, so future research can include longitudinal studies of data collaboratives. This may generate a more comprehensive assessment of success of a data collaborative, also taking into account the level of satisfaction of the partners with the outcomes and the actual influence on the policy issue in question.