Abstract
This article provides an overview of the key values that we argue should underpin an index of the legitimacy of the governance of online intermediaries. The aim is ultimately to allow scholars to rank the policies and practices of intermediaries against core human rights values and principles of legitimate governance in a way that enables comparison across different intermediaries and over time. This work builds on the efforts of a broad range of researchers already working to systematically investigate the governance of social media platforms and telecommunications intermediaries. In this article, we present our review and analysis of the work that has been carried out to date, using the digital constitutionalism literature to identify opportunities for further research and collaboration.
Keywords
Introduction
The theme of this special issue is the deeply contested political question of how human rights should be protected in a digital environment. ‘“Digital constitutionalism” encompasses a constellation of initiatives that have sought to articulate a set of political rights, governance norms, and limitations on the exercise of power on the Internet’ (Gill et al., 2015: 2). One of the key challenges of the digital constitutionalism project is to articulate appropriate limits on the private exercise of power by online intermediaries, how those limits may be imposed, and by whom.
This is a complex problem, and requires a new understanding of constitutionalism in an era where regulation is not only, or even not primarily, done by the state (Black, 2001). We take a broad view of ‘governance’ that encompasses all ‘organized efforts to manage the course of events in a social system’ (Burris et al., 2008: 3). This broad definition focuses attention particularly on the policies and practices of online intermediaries as key actors in the governance of online behavior.
Intermediaries play a critical role in governing the internet by developing and managing its infrastructure. By ‘intermediaries’, we mean the broad range of entities that ‘bring together or facilitate transactions between third parties on the Internet’ (Organization for Economic Co-operation and Development [OECD], 2010: 9). Intermediaries of all types—the owners of physical pipes, the providers of core routing services, the search engines that make content visible, the content hosts, and the social media platforms—all shape how people communicate in important but different ways. They are the focal points of control, where pressure can be most effectively deployed to influence user behavior (Goldsmith and Wu, 2006), and the decisions that they make have a real impact on public culture and the social and political lives of their users (DeNardis, 2014). In order to progress the political project of digital constitutionalism, more needs to be known about how intermediaries govern their networks and how their decisions impact the basic rights of individuals in different contexts. In this article, we present an outline of the values we suggest ought to be measured, a guide to the work that has already been completed, and a sketch for future projects to build upon in their own research design.
Our approach: Evaluating governance and measuring change
Global interest in how digital intermediaries are governed is growing rapidly, and both governance practices and the intermediaries themselves are continuously evolving (Burgess, 2015). We suggest that an index is a promising approach to structure thinking about future research. Indices can help to identify trends over time and across different intermediaries in a way that furthers discussions about the role of private firms in promoting human rights, and when states have a responsibility to intervene (Hyden and Samuel, 2011; Langford and Fukuda-Parr, 2012). Well-designed indices can also help encourage firms to develop better governance practices (MacKinnon et al., 2016; Maréchal, 2015), as well as empower users to make more informed choices (at least in markets with effective competition). Ultimately, this evidence base can help lawmakers and advocates consider specific opportunities to improve laws, policies, and practices (Cooley and Snyder, 2015). We are particularly hopeful that an index may help to enable collaboration and comparison between future independently designed research projects that address different aspects of governance and different sites of study.
In this article, we seek to identify the human rights values that might inform an index of legitimacy of the governance of online intermediaries. To generate this list of values, we rely on the very useful analysis of Gill et al. (2015), which extracted a list of values from 30 declarations of internet rights and similar documents. We supplement this list with human rights values drawn from the major international human rights instruments. We also include any relevant procedural governance values from major international rankings of state governance and from existing projects that have begun the work of ranking intermediaries on these values.
From this list of values, we focus on those that will be relevant to assessing the legitimacy of governance by online intermediaries. We understand legitimacy in the normative sense here: the contested political question of how the exercise of power can be morally justified. In this sense, legitimacy is a ‘meta-concept’ that enables comparisons between competing conceptions of morality and ‘opens up space for imagining and adopting alternative visions of legitimacy’ (Thomas, 2014: 738). While human rights are universal, the extent to which firms owe duties to protect or promote these rights is contested and will differ according to context and jurisdiction (Green, 2001). Any theory of the responsibility of online intermediaries must accommodate the substantial differences between different providers, each with different degrees of visibility and control over the communications that pass over their networks. By including a broad range of values, we seek to help ground disparate political discussions about what societies expect from online intermediaries—which rights we expect them to actively protect, in what contexts, and how.
We exclude measures that can only effectively be implemented at a social level. Specifically, we exclude two categories: access and education (including, e.g., the ‘right to access the internet’ and ‘media and digital literacy’), and openness and stability of networks (including network neutrality, open standards, and interoperability). We similarly exclude human rights obligations that have only tangential application to online intermediaries—for example, rights related to human trafficking and literacy, and broader categories of economic rights that focus primarily on state obligations. While telecommunications intermediaries have some role to play in promoting each of these values, we see them as primarily an arena for state legislation and policy, at least in the immediate future.
Our analysis approximates a comprehensive identification of the components of an index of the legitimacy of online intermediaries. For each of the values we identified from human rights instruments and digital constitutionalism declarations, we identify existing methodologies and data sets from academic studies and civil society projects that could be used to inform the content of an index. Specifically, we cross-reference the methodology of each project to our set of procedural and substantive governance values and human rights, in order to identify with some particularity what research exists and where further work is required.
Summary extract of proposed list of fundamental rights and procedural governance values.
Fundamental rights and freedoms
The first set of values to include in our index are the universal rights embodied in international human rights instruments. International human rights are generally binding against nation states, not private actors. In recent years, however, there has been increasing recognition that private actors have a key role to play in protecting human rights. The United Nations Guiding Principles on Business and Human Rights sets out a responsibility that private firms ‘should avoid infringing on the human rights of others and should address adverse human rights impacts with which they are involved’ (2011: 13). This obligation comes from a ‘tripartite’ model (Ruggie, 2008) developed to understand the responsibility of non-state actors to respect human rights.
Internet and telecommunications intermediaries are deeply involved in a range of issues with relevance to substantive human rights (Kaye, 2016; United Nations Educational, Scientific and Cultural Organization [UNESCO], 2014). The requirements that the Guiding Principles imply for intermediaries are set out in a report by the European Commission (2013). This framework has six basic components: companies must make a firm commitment to respect human rights and embed that commitment into their culture; continuously assess their impact; prevent or mitigate harms in which they are involved; track how well harms are addressed; communicate how harms are addressed; and remediate harm that they have caused or contributed to. These principles have been adopted by groups of state and non-state actors in relation to freedom of speech, rights of individual privacy, rights to be free from harassment and abuse, and obligations to resist pressure to unduly limit freedom of speech (e.g., Global Network Initiative, 2012; Internet Rights and Principles Coalition (IRPC), 2014; Manila Principles on Intermediary Liability, 2015; UNESCO, 2014).
Both Ranking Digital Rights (2015) and the Terms of Service and Human Rights (Venturini et al., 2016) projects provide extensive evaluations of how freedom of expression and privacy rights are dealt with by intermediaries in formal documents. Together, these projects examine in great detail when and for what purpose intermediaries collect, retain, and/or share user data, how much control users have over that process and the data itself, how the company responds to third-party requests for information, and whether information is encrypted. They also examine whether rules about acceptable content are clear, whether moderation processes are disclosed, and whether there are adequate systems for resolving disputes. Importantly, however, neither project addresses the more contested substantive scope of restrictions on freedom of expression. Article 19 of the International Covenant on Civil and Political Rights (ICCPR) (United Nations, 1966), for example, not only requires that limits on speech are clear and accessible, but also that they are designed to serve a legitimate purpose, and are both necessary and proportionate to achieving that goal. While restrictions on speech imposed by online intermediaries will often be acceptable where similar restrictions imposed by states would not be, this may not always be the case. Users of particular services often consent to more restrictive rules about acceptable speech than would be legitimate if imposed by a nation-state, so any measurement of this value should always be viewed in context (Suzor and Woodford, 2013). Nevertheless, the breadth of speech permitted by intermediaries is fundamentally important to the user experience. We suggest that this gap should be filled by evaluating the relative scope of restrictions on speech and the adequacy of reasons set out by intermediaries in justification.
There remain several substantive human rights based on international law and digital constitutionalism documents that are not considered in rankings undertaken to date. These represent the most immediate opportunities to extend the work of existing indices. The right to non-discrimination is a key component of a number of digital constitutionalism documents but not specifically used to evaluate intermediaries in existing studies. This right, particularly when conceptualized as freedom from harassment and abuse, is increasingly relevant in controversies about how online networks are governed. Recent work has highlighted the pervasive and massive extents of harmful abuse and hatred online (Citron, 2014; Woodlock, 2016). Intermediaries are increasingly being challenged on their role in facilitating the spread of abuse and their perceived failures to limit it (Citron, 2014; Langlois and Slane, 2017; Massanari, 2015; Shepherd et al., 2015). To drive positive change on this issue, we need to begin to systematically measure their performance in this regard.
The set of human rights against discrimination also deserve special inclusion in evaluations of intermediary governance. The design of technology may itself be discriminatory, and any index of human rights should certainly rank intermediaries on the extent to which their systems are accessible, particularly to people with disabilities. Intermediaries should also be evaluated for their role in enabling or supporting discrimination perpetuated through their systems, whether as a direct consequence of design or a failure to adequately address harmful patterns of user behavior. Now that digital media platforms play such a large role in organizing labor and markets for goods and services, it will become increasingly important to examine how they might enable or prevent discrimination (Edelman and Luca, 2014; Witt et al., 2015; Zukin et al., 2015).
The impact of online intermediaries on freedom of association is also important to measure now that social media in particular plays such an important role in coordinating collective action (Castells, 2012; Tufekci and Wilson, 2012). The recent experience of the Arab Spring, for example, highlighted how states may seek to interfere with telecommunications services to disrupt protests, and it may be possible to rank intermediaries on their willingness to resist such pressure unless validly compelled by law (Brown and Korff, 2012; DeNardis, 2012).
There are a number of other rights that are not well covered and should ideally be incorporated into future research. Digital constitutionalism documents commonly reference the right to cultural and linguistic diversity, and this has been a significant, ongoing challenge for internet infrastructure. Recent work has sought to ensure that the technical standards that underpin the internet are more inclusive (Milan and Ten Oever, 2016), and social media platforms too are often critiqued on their design, policies, and practices (Panigrahi, 2015). A number of major digital constitutionalism documents also specifically reference the protection of children (United Nations General Assembly, Convention on the Rights of the Child (UNCRC), 1989), and intermediaries have long faced a great deal of pressure to protect and empower children using their networks. Other rights may be lower priorities for evaluation, although this may change in future. Freedom of religious belief, for example, tends to fall under the umbrella of freedom of expression in the digital context, and may not warrant specific inclusion at this time.
Procedural values
The second component of our proposed index focuses on procedural limits to the exercise of governance power. Generally, these fall into two broad categories within the digital constitutionalism literature: first, concerns about the rule of law, due process, and transparency; and second, governance and participation concern about multi-stakeholderism and participation in decision making. Rule of law values, as applied to online intermediaries, requires evaluating the legitimacy of private governance against the extent to which it is consensual, transparent, equally applied, relatively stable, and fairly enforced (Suzor, 2011, 2012). Due process includes the right to access to justice, the right to be heard, and a meaningful ability to challenge decisions (Citron and Pasquale, 2014). Transparency incorporates the right of users to know when, why, and by whom decisions that affect them are being made. While many of these values are articulated in the major international human rights instruments, many of these documents were written before the digital age. This is where the digital constitutionalism documents are most relevant, in recognizing the need for legitimate governance of intermediaries in an era where so much of the public sphere has moved online.
All of these values are relatively well covered by the combination of the Ranking Digital Rights (RDR) index and the Terms of Service and Human Rights (ToSHR) study. Between them, these studies examine whether there are clear and accessible rules which are applied fairly, whether users are notified of changes, whether there are internal procedures to challenge decisions (including a presumption of innocence), and whether users can access independent adjudication by a court or tribunal. The most immediate limitation of existing studies is that these procedural concerns are at times examined primarily in relation to rules about content (freedom of expression) or privacy, and not always the full range of substantive issues detailed above. There are a number of additional values within the broad definition of the rule of law and due process that could also be included in future work, including measures of stability and consent to the rules, and equality in the application of rules.
Questions of participatory governance are somewhat less well incorporated into work that has been carried out to date. Many digital constitutionalism documents consider multi-stakeholder and participatory governance to be important, but there remains a significant lack of clarity about what a right to participation might look like in the context of private online governance (Stein, 2013; Suzor and Woodford, 2013). RDR has one indicator that examines whether intermediaries are involved in multi-stakeholder initiatives, but does not rank firms further on the extent to which they incorporate input from relevant stakeholders on decisions that impact users. The efforts of intermediaries toward participatory governance practices are worth exploring further and tracking in future work.
Potential approaches for studying governance-in-practice
In this section, we turn to the next major opportunity for future research: studying and evaluating governance-in-practice. We suggest that more research is required to understand how the design, policies, and processes of intermediaries impact human rights in practice, and how users experience these processes. Our review of existing work to date shows that this is a relatively under-explored component of the digital constitutionalism project. So far, the large systematic examinations of governance across multiple intermediaries have focused on legal rules, contractual terms, and policy documents. Meanwhile, many scholars have contributed excellent detailed studies of the practices of particular intermediaries. We suggest that there is an opportunity to begin to systematize this work. This is where an index can be a particularly useful in providing the structure needed to enable comparisons across intermediaries and evaluate changes over time. In this section, we review some of the more prominent efforts to systematically understand the governance of online intermediaries in practice and provide a review of methodological challenges and opportunities to expand this work.
Transparency reporting
The starting point of these efforts has been the growth of transparency reporting initiatives from intermediaries themselves, a welcome practice used by intermediaries to cultivate legitimacy with users and civil society organizations. Transparency reports typically include quantitative information on legal requests received by intermediaries, breaking down this data on a country by country basis, and often providing information on what proportion of requests have been complied with. This information sheds light on various issues regarding third-party interference, such as government requests for data and copyright takedown requests. Increasingly, standardization of reporting procedures enables comparison among intermediaries, though there is still a long way to go (Woolery et al., 2016). Access Now has also undertaken important work synthesizing transparency reports and legal complaints across companies (Access Now, 2016). Lumen, a project at the Berkman-Klein Center for Internet and Society, acts as a clearinghouse for legal complaints and requests for removal of online materials, and is particularly robust for information such as Digital Millennium Copyright Act (DMCA, 1998) takedown requests received by a small number of participating intermediaries. Researchers have been able to make good use of this data to analyze how the notice and takedown system works in practice, and how it is changing over time (Urban et al., 2016; Urban and Quilter, 2006).
Given the large volumes of information that are now being made available by major telecommunications providers, there is ample room for more academic analysis of governance practices that make use of transparency data. Unfortunately, transparency efforts offer only a partial picture of the legitimacy of companies' practices. In particular, transparency reports often fail to capture the context within which decisions are made. Moreover, legal requests make up only part of the broader landscape of decisions that implicate legitimacy: policies set by intermediaries and design-level choices also have an impact on the legitimacy of intermediary practices, but are not reflected in transparency reports. The next steps in expanding this research are to (a) increase the number and range of intermediaries issuing transparency reports and (b) encourage intermediaries to provide more transparency about their own terms of service enforcement and extralegal requests from third parties.
Qualitative analyses of governance
There is now a wealth of qualitative research that examines governance practices of internet intermediaries. These studies provide insight into context and practice that transparency data cannot convey. There are excellent in-depth studies that examine the institutions involved in internet governance, particularly the major multi-stakeholder groups responsible for technical infrastructure (e.g., Mueller, 2002) and the role of nation-states in influencing telecommunications technology and processes (e.g., DeNardis, 2009; Mueller, 2010). Researchers have also recently been able to develop rich understandings of governance from the perspectives of the digital laborers tasked with enforcing policies (e.g., Roberts, 2014). Investigative journalists have also played a critical role in examining particular controversies, such as discriminatory content moderation policies, abuse of platforms by bad actors, or the need for algorithmic transparency and accountability (e.g., Buni and Chemal, 2016; Dewey, 2016; Rosen, 2013). This growing body of work is too large to adequately represent here; it suffices to note that this burgeoning field promises to greatly improve our understanding of governance, particularly as it pivots to more comprehensively examine the role of infrastructure and of private actors in internet governance (DeNardis, 2013; DeNardis and Hackl, 2015; Hofmann et al., 2016; Musiani et al., 2016).
In general, however, there are still gaps for rigorous studies of governance-in-practice, particularly studies that take into account governance outcomes of private intermediaries. For the majority of larger private intermediaries, gaining access to examine governance processes is exceedingly difficult. This lack of transparency about outcomes makes it challenging to assess the impact of policies at scale. Research in this area often proceeds in a way that is somewhat ad hoc: at present, we lack general methodologies for evaluating how well intermediaries are performing on human rights criteria in a manner that is comparable over time and across intermediaries.
User-oriented studies can help assess the outcomes of intermediary policies. For example, OnlineCensorship.org utilizes a survey approach to crowdsource reports from users who have had their content removed or accounts shut down on social media platforms in order to gain a fuller view of how content moderation impacts users. This method may also prove useful for evaluating the effectiveness of intermediary due process by documenting and comparing the experiences of users who have sought redress through the systems intermediaries make available to them.
We suggest that a possibly fruitful avenue for future work would be to crowdsource the collective work of different groups of scholars and civil society advocates. Given the growing number of studies of different controversies, it seems that it may be useful to combine the efforts of many in order to obtain a more systematic view of the legitimacy of governance practices of online intermediaries. There is a significant methodological challenge here that presents an opportunity for much more extensive analysis: how might a metareview of existing qualitative analyses, all with different cases and methods, seek to extract a set of evaluations that can be used to compare governance practices across controversies, across sites, and over time? We hope that our preliminary work in developing a shared analytical framework can help as a first step to leveraging these existing studies. At any rate, a shared framework can at least help to form the basis of future collaboration.
Quantitative approaches and network measurement
An additional body of research has begun to use network measurement to assess levels of censorship, prodding, and probing intermediaries in order to measure outcomes. The major challenge in studying governance by private intermediaries and the technical and social systems they create is that so often, decisions are opaque, and gaining access to observe internal processes is extremely difficult. Some of the most promising avenues for understanding internet governance in practice require the development of new methods to assess the inputs and outputs of these ‘black box’ systems (Pasquale, 2015; Rosenblat et al., 2014; Sandvig et al., 2014).
There are interesting initial examples of this type of research in recent years. Scholars examining Sina Weibo in China, for example, have collected large numbers of microblog posts shortly after they are posted, and tracked them over time in order to see how certain forms of political speech are removed due to government pressure (Zeng et al., 2016). Diakopoulos (2014, 2015) provides a good overview of how computational journalists are using reverse engineering techniques to make algorithms more accountable, focusing on many of the values outlined above. Perel and Elkin-Koren (2016) have outlined a methodology for systematically experimenting with the content moderation processes of online intermediaries by uploading prohibited and permitted content and observing the results. Much of this work is still new, and often requires interdisciplinary collaborations, but it is part of a range of exciting opportunities to collect the digital traces of governance decisions in order to systematically measure the performance of these systems against human rights principles.
Conclusion
We have presented a high-level sketch of the landscape of studies that examine the legitimacy of governance by online intermediaries. We situate these studies within an explicit framework of digital constitutionalism, in order to identify the potential to extend existing work and opportunities to address gaps in the research. Our analysis suggests several important avenues for future research and collaboration to develop the evidence base that will be required to progress the project of digital constitutionalism. We note first that formal analyses of governance documents of online intermediaries have well developed methodologies that can be extended to examine the policies of a larger number of internet and telecommunications firms. We suggest that these studies can be enhanced by turning to more explicitly address human rights other than freedom of expression and privacy, including personal security and dignity, freedom from discrimination, and freedom of association.
The major research gap that we have identified in this work is the lack of systematic analysis of governance-in-practice in a form that enables comparison across contexts and over time. While there is a great deal of interesting work that has started on this challenge, there are many more opportunities to develop new projects and methods to better understand how the decisions of private intermediaries impact the human rights of their users (or third parties). We call on intermediaries themselves to follow best practices in releasing transparency data to enable researchers to carry on this work. Since comprehensive transparency data are still difficult to find, however, we suggest that new experimental methods and reverse engineering techniques are likely needed to better illuminate the governance practices of intermediaries. Because the challenge facing scholars here is so great, and the social issues so pressing, we also call for more collaboration on these research questions. In developing the outline of an index of intermediary governance that addresses the core human rights challenges of digital constitutionalism, we hope to help guide efforts to develop interoperable research methodologies in the near future. Finally, we encourage the many diverse scholars working on these issues to adopt the open data practices of groups like Ranking Digital Rights, to enable, as far as possible, the development of new analyses that seek to understand governance in rapidly changing environments.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Suzor is the recipient of an Australian Research Council DECRA Fellowship (project number DE160101542).
