Facebook Rules: Structures of Governance in Digital Capitalism and the Control of Generalized Social Capital

Abstract

The article explores Facebook governance – its mechanisms, motivations and sources of power – while identifying wider patterns and logics that apply to other internet corporations. I suggest that (1) when digital capitalism turns mundane human interactions into biopolitical production, corporations gain interest in governing these interactions to maximize profit, and make decisions on core political issues; (2) Facebook can effectively govern and discipline users since it remolds various field-specific forms of capital into a single form, generalized social capital, and since it can threaten to confiscate generalized social capital accumulated by users; (3) digital platforms do not simply epitomize a shift from discipline toward neoliberal decentralized governance. Instead, they engage in intensive legislation, administration of justice and punishment, and develop eclectic governing and legitimation apparatuses consisting of algorithms, proletarian judicial labor and quasi-constitutional governing documents.

Keywords

Facebook governance immaterial labor mass private property social capital social media surveillance capitalism

Should an association aimed at inciting a Palestinian uprising against Israel be dissolved as a terrorist unlawful association? And what about an organization fighting interfaith marriage? Where is the line between distasteful jokes and racist hate speech? Which behaviors in public amount to obscenity? (Are homosexual kisses more obscene than heterosexual ones? Does exposing female breasts cease to be obscene in the context of nursing?) Should organizations such as political parties be held collectively accountable for actions and statements of individual rank-and-file members? How can serious violent threats be distinguished from merely tasteless jokes? And how should offenders be punished to eradicate socially undesirable conduct? Questions of this sort – regarding the boundaries of political legitimacy, the freedoms of association and speech, and regulation of conduct in public space – are core political issues. Their regulation through legislation, law enforcement and formal punishment was traditionally a state prerogative. However, all the questions listed above refer to actual decisions taken by Facebook Inc., the operator of the world’s most popular social network site, with 2.1 billion reported monthly users.

In 2002 Anthony Giddens – making a case that left-wing globalization critique is exaggerated and multinational corporations do not threaten nation-states’ privileged position – claimed that while ‘(n)ations establish frameworks of law, corporations do not’ (Giddens, 2002: xxv). This no longer applies to internet corporations such as Facebook (founded 2004). These corporations engage intensively and systematically in regulating social life in multiple ways, including legislation of ‘governing documents’ and law enforcement through quasi-judicial procedures. Facebook’s apparatus alone makes a gigantic number of judicial decisions, claiming to handle a million reports a day (O’Brien, 2016). Facebook does not govern solely through law, yet its selective borrowing from the state’s governance and legitimation toolkit is consequential. As social interactions shift online, governance of online interactions by internet corporations shape (power)-relationships, with social, political, cultural and economic consequences beyond the platform’s boundaries. Facebook claims the monopoly of legitimate violence within its territory, Weber’s very definition of the modern state (Weber, 1994: 310–11); and while its power over individuals is inferior to that of state sovereignty, it represents a new emerging model of governance with unique logics and toolkit. This model is shared by other online platforms, with accumulated influence on users’ lives, and deserves scholarly attention.

Yet despite their sheer volume and wide social significance, governance projects of social network sites remain undertheorized, and the literature rarely thematizes their policies, legislation and punishment. This article offers a systematic account of Facebook governance – a notion used below in a broad sense to refer to all techniques, policies and design decisions that may be interpreted as directed at influencing user conduct¹ – while addressing three questions: why Facebook is interested in extensive governance; what gives Facebook the power to effectively govern; and how Facebook governs (what its toolkit is for shaping the conduct of users).

While replying to these questions I make four arguments: (1) the relations between operators of online platforms and users are not merely economic relations of exploitation as post-Marxist scholars assume, but also political relations of governance; (2) online platform operators have interest in governing all digitally mediated human interactions because, as these interactions turn into a source of profit, governing them might maximize profit. Thus, governing and exploitation are interrelated; (3) digital platforms do not simply epitomize a shift toward neoliberal governance. Facebook developed an eclectic apparatus of governing and legitimation that borrows heavily (although selectively) from the democratic-statist repertoire of sovereignty and discipline alongside new tools such as algorithms and proletarian judicial labor. The logic of this eclectic apparatus becomes evident once political economy is accounted for; and (4) Facebook’s power to govern relies on its capacity to remold various field-specific forms of power into a single form, generalized social capital, which it strives to monopolize and control. The digital field is dynamic, with digital platforms emerging, decaying and redesigned regularly, but while my account focuses on Facebook, at least the first three arguments apply more generally to multiple other digital platforms, regardless of these transformations.

While much relevant information is private property and hence inaccessible, this analysis and theorization rely on public historical documents (Facebook’s different versions of ‘governing documents’, public statements, referenda and online debates among users), ‘reverse engineering’ (Gehl, 2014) and press reports. Space precludes exhausting all aspects of Facebook governance, yet even an outline of a theory in broad brushstrokes may improve our understanding of governance in digital capitalism.

Literature

While the literature on social network sites thrives, two common tendencies shift it away from a thorough analysis of social network sites governance.

First, post-Marxist scholars conceptualize the relationships between social network sites and their users exclusively in economic terms (Cohen, 2008; Coté and Pybus, 2007; Fisher, 2015; Fuchs, 2011; Gehl, 2011; for a critique: Skeggs and Yuill, 2016). These scholars study the ‘exploitation’ of unremunerated user labor, as users create surplus value without even noticing as they turn their relationships, emotions, judgments and everyday into both valuable contents that seduce other users to view ads and profitable data (sold to advertisers in the form of audience-derivatives: Arvidsson, 2016). This literature offers important insights on the political economy of social network sites on which I rely, yet it often fails to recognize that the relationships between social network sites and users are not merely of exploitation; both parties often frame their relationship in terms of governance. As shown below, social network site politics and political economy, although studied separately (the latter is absent from the most thorough reviews of internet governance, e.g. Ziewitz and Pentzol, 2014), are closely interrelated.

Secondly, scholars who do study social network site power and governance (Arvidsson, 2016; Bucher, 2012; Karppi, 2011; Ritzer and Jurgenson, 2010; Terranova, 2015; Zwick et al., 2008) tend to view the internet as epitomizing a more general shift from sovereignty and discipline towards neoliberal decentralized governance through freedom and seduction (Bauman, 1998; Jessop, 1996). For Cheney-Lippold (2017), digital governance through algorithms exercises control through rarely felt ‘power at a distance’; Van Dijck (2013) showed how Facebook governs users and transforms social norms and practices through platform design. Disciplinary panopticism bounded within institutional boundaries is viewed as a thing of the past (Gane, 2012). These studies offer highly original insights on Facebook’s ‘security dispositive’ (Terranova, 2015) and ‘derivative logic’ (Arvidsson, 2016). Yet, early accounts of the internet’s resistance to centralized governance (Boyle, 1997) lose validity in today’s web of centrally-governed online ‘closed gardens’ and ‘gated communities’. While acknowledging that social network sites must be governed to exploit users’ free labor, this literature neglects important aspects of governance that do not fit within its macro-narrative.²

Some important work has been done nevertheless. A few law scholars conceptualized site operators as sovereigns: Lessig (2006) criticized non-democratic merchant-sovereignty online. Chander (2012) explored Facebook’s non-statist governance. Laidlaw (2015) conceptualized Facebook as mere gatekeeper, yet one that must be accountable for violating human rights of the gated. Media scholars started exploring mechanisms of governmentality (Bucher, 2012, showed how Facebook’s algorithm encourages user productive participation through threats of invisibility) and sovereignty (Crawford and Gillespie, 2016, on flagging; Roberts, 2016, on content moderators; Massanari, 2017, on Reddit’s governance structure and its political consequences). Gehl (2014) criticized the division of labor between social network site operators and users and the incapacity of the latter to shape site structures. Yet a thorough systematic account of Facebook governance, its logic and mechanisms, is still missing.

Why Facebook Governs

To understand why social network site operators engage in governance much more extensively than corporations used to, we must cease to study their politics and political economy in isolation, and explore their interconnections. Historically, corporations exercised governance (in Foucault’s broad sense, the conduct of conduct), yet their governance targeted individuals in particular, well-bounded roles – mainly as employees. Corporations attempted to increase productivity through rational governance of work processes and workplace social and emotional relationships, utilizing management theories, industrial psychology, and surveillance technologies to govern employees (Illouz, 2008; Zuboff, 1988). As consumerism developed, Taylorist management was expanded from workers to consumers (Lyon, 1994: 138ff.). However, Facebook governs users in multiple other roles: as friends, lovers, politically involved citizens, freelancers, small business owners, etc.

I suggest the emerging interest of capitalist corporations in regulating multiple spheres of social life derives from a shift in forms of capitalist accumulation: the rise of immaterial biopolitical production (Hardt and Negri, 2000), which is ‘the production of social life itself, in which the economic, the political, and the cultural increasingly overlap’ (p. xiii) and production is no longer a bounded social sphere, as value is produced outside the factory by capturing the totality of social life. Facebook monetizes the very fabric of social interaction, turning social, emotional, political, familial and phatic interactions, affinities and relationships into data, metadata, attention, affect and engagement, which are all captured and sold to advertisers. Of its 2017 revenues, 98.3 percent came from targeted advertising, selling advertisers the attention of highly-specific clusters of users. This attention is collected by offering users social ‘connectedness’ (Van Dijck, 2013) and free access to content produced by other users, and through algorithmic surveillance and analysis of users’ activity (Cohen, 2008). In this business model users are not customers (the customers are advertisers) but ‘value creators’ (Van Dijck, 2013: 63). While each user makes only a modest contribution to Facebook’s profits (in 2017 it reported net income of $15.9 billion for 2.13 billion users), governing the conduct of users as a population becomes crucial for retaining Facebook’s profitability. Thus, the new logic of accumulation that Zuboff (2015) associated with ‘surveillance capitalism’ begets new interests.

When human interactions becomes sources of profit, they can also be maximized through their rational governance – for example, by preventing users from saying or doing things that might offend other users and discourage them from using the site intensely as productive unremunerated workers. Restrictions on political and sexual content serve this cause. Hence Facebook struggles to formulate universal rules of conduct and enforce them on its multicultural usership. By demonstrating good governance Facebook may also fend off external regulatory pressures that might restrict its control over biopolitical production. Facebook prohibits users from producing lower-quality data by opening multiple accounts (Skeggs and Yuill, 2016). It also retains user populations biopolitically by blocking programs that turned ‘digital suicide’ (account deletion) automated and viral through legal and technological tools (Karppi, 2011), and by sending users willing to quit automated ‘miss you’ messages from friends (Van Dijck, 2013).

Regulating social interaction is thus inseparable from its exploitation. Biopolitical production draws capitalist corporations closer toward the core of the political sphere, motivating them to design governance apparatuses to regulate social interactions. Zuboff (2015) suggests that in this new accumulation regime users turn from workers or consumers into raw materials that may be used without their consent. If so, disciplinary power becomes redundant (in line with Lash’s (2007) account of post-hegemonic power). However, in social network sites users are not passive: their conduct may influence and threaten profits, hence they must be governed.

Facebook’s interest in governance results from the shift of social and political interaction into private spaces where their conduct influences profitability. This dynamic has offline precedents, even if offline political and social interaction is subjected to private merchant-sovereignty (Lessig, 2006) to a much smaller degree. Already in the 1980s, Shearing and Stenning studied the rise of ‘mass private property’ like malls, privately-owned spaces for public functions where a growing part of public life is governed by property owners, and where private policing exerts quasi-juridical authority to serve owners’ contingent commercial interests. Unlike public policing, private policing is not even nominally impartial, and often ignores suspect rights, due process, rules of evidence and equality before the law. It prioritizes minor rule-breaking which threatens commercial interests over severe crimes, and unlike state policing, the main punishment it inflicts is exclusion (Shearing and Stenning, 1983).

As socializing and political activity shifted from town-squares to malls, mall managements started making political decisions, such as banning political activity on specific sensitive issues while allowing it on other issues; or banning entry from specific people (unaccompanied teenagers, or those appearing to be poor) to maximize profits. They enforce quasi-laws (codes of conduct) through constant surveillance, defending civility at the expense of civil liberties, and leaving citizens deprived of truly public space (Staeheli and Mitchell, 2006).

Visitors subject themselves to this private governance since malls are experienced as safer than town squares. Bauman (1999) suggested that as citizens are transmuted into consumers they substitute protection for civil participation. Similarly, keeping digital spaces safe from pornography, violence, bullying, spam, and anonymity motivates and legitimates private policing online. Unlike Myspace, Facebook created a ‘safe’ middle-class white space in times of moral panic around fake identities, cyberbullying and sex predators (Gehl, 2014: 71–91). As MacKinnon (2012) suggests, users are offered a Hobbesian contract to avoid a Hobbesian ‘state of nature’. Safety and security are highlighted in Facebook’s legal code (‘Community Standards’) as main legitimations for governance and for particular controversial policies.

While malls are isolated neo-feudal estates (Shearing and Stenning, 1983), Facebook is an empire striving to monitor all human relationships, interactions and emotional reactions worldwide and map them into its ‘social graph’. Expanding its jurisdiction to be coextensive with social life, it takes the ‘mass private property’ model to the extreme. Furthermore, while the legitimacy of merchant-sovereignty relies on the easiness of leaving them and the competition between them (Post, 1995, cited in Lessig, 2006), this hardly applies when users are invested in accumulating social capital within social network sites. Like malls, Facebook governs through threats of exclusion, yet its threats are much more consequential, due to its status as a central bank of social capital.

The Sources of Facebook’s Power: A Central Bank of Social Capital?

To understand what enables Facebook to effectively discipline its users through threats of expulsion, we must understand the benefits it offers users and deprives them from at will. Facebook turned into a main medium of the social, through which individuals and organizations create, represent, maintain and enact social relationships. Individuals use Facebook to keep in touch with friends and family, meet romantic partners, get to know new acquaintances better, express identity, and engage in community life, public debate and political activism. Some also use Facebook for advertising, marketing, career promotion, group formation, political campaigning and mobilization. All these activities take place within this privately-owned, centrally-governed public space.

Most of these uses may be conceptualized as efficient ways to accumulate and maintain various forms of social capital. Social capital is the capacity to effectively mobilize resources dispersed in one’s social network (Bourdieu, 1986). Connections, group membership and network position may grant access to valuable information, economic resources, status and political power. Despite substantial theoretical differences, scholars such as Bourdieu (1986), Burt (1992) and Lin (1999) share a common understanding of social ties as convertible resources.

Facebook facilitates the accumulation, maintenance and mobilization of social capital in various ways: it objectifies egocentric social networks in a stable form, thus making ephemeral connections persistent; it protects ties from natural decay (once parties have left the social institutions they had shared), keeping communication paths open; and it allows low-cost maintenance of larger networks, as a single message may keep thousands of ‘friends’ updated (Ellison et al., 2007, 2010). It thus operates as a bank, where users invest their social capital to maintain and increase its value, but cannot withdraw it against the bank’s policy.

This applies not only to individual users. Unlike specialized social network sites (e.g. LinkedIn), Facebook wishes to capture and monetize on social relationships in various social spheres. It allows actors in different commercial, political and cultural fields to use it to accumulate social capital and exchange it for economic capital or privileged position in the respective field. In an isomorphic process, politicians, firms, artists, brands, social movements and charities create communities, accumulating Facebook fans like-by-like. Through the ‘like’ button they turn momentary sympathy into sustainable ‘relationships’, which are accumulated as a form of capital. They invest resources in maintaining these relationships in order to mobilize economic and political resources within these networks.

My argument is that Facebook Inc. uses the material affordances of digital communication to remold various field-specific forms of power into a single form, generalized social capital. Monopolistic control over generalized social capital grants it power to govern its users. This is evident in multiple spheres, including the following:

Interpersonal ties. for individual users, Facebook may help preserve and strengthen weak ties and gain access to resources like job information, advice, accommodation while travelling (Ellison et al., 2007), information about social events or houses for sale, alongside emotional support and a sense of community. Some users gather a relatively large audience for their posts, beyond their offline acquaintances, acquiring a micro-celebrity status (Marwick, 2013), which may prove hard to smuggle out of the platform. Social ties are not merely capital (exchange-value), they have use-value too, as sources of emotional support and intimacy. Once social life (from everyday conversations to event invitations) goes online, users ostracized from Facebook may be deprived of both.

Politics. For Bourdieu, capital is always a field-specific currency. Facebook’s role in the political field deserves special attention. Facebook allows users to communicate political emotions of outrage and hope and to initiate protest and social movements by mobilizing weak ties and friends-of-friends. By reducing organization costs Facebook frees political entrepreneurs from dependence on institutional resources, thus becoming a nearly obligatory point-of-passage for grassroots activism. Political entrepreneurs and social movements use Facebook to mobilize activists, diffuse information and propaganda, and coordinate offline mass activism, and as a site for online activism, broadening existing repertoires of contention (Castells, 2012; Earl and Kimport, 2011). They accumulate ties with sympathizers as political capital, turning ephemeral sympathy into online group membership. Established politicians and political parties similarly accumulate fans, interacting with them in an informal style similar to interaction among friends. Campaigners use this informal communication path to disseminate messages, assess their effectiveness by analyzing user comments, raise funds, and engage supporters in campaigns (Bor, 2014). Facebook views politicians as ‘partners’, who should succeed in accumulating and mobilizing ties as capital, and hence keep giving Facebook free high-quality ‘content’ (Hoffmann et al., 2018: 210). Since fan numbers are interpreted as indices of public support, accumulating fans becomes necessary in order to be taken seriously in the political field (Dalsgaard, 2008). By controlling this political infrastructure, Facebook controls who can use it, how, and for which political aims (e.g. by tolerating groups that aimed to topple the Egyptian President Mubarak’s regime while dissolving those aimed at toppling Israel’s occupation of the West Bank).

Marketing. Facebook ties are also a capital in the commerce sphere, as marketing scholarship on ‘brand communities’ demonstrates (Laroche et al., 2012). By ‘liking’ brands, users volunteer to receive commercial contents. Facebook thus remolds advertising and public relations as interactive long-term ‘relationships’ between brands and ‘fans’ or ‘frustomers’ (Van Dijck, 2013: 64–5). These ties have economic value, as Facebook brand communities may increase brand trust and brand loyalty. Some fans also distribute marketing contents through sharing and commenting (thus raising brand awareness), or even produce brand-related content as they preach the brand (‘brand advocacy’), share new ways to use products, and initiate new users (Laroche et al., 2012). Business investment in brand communities is investment in brand social capital, as these ties drive users to provide unremunerated labor for the brand and feel committed to keep consuming it – thus brand social capital yields economic capital. Even if brand communities rarely exist in the idealized way marketers imagine them (Zwick and Bradshaw, 2016), what matters is the huge importance marketers ascribe to brand communities and biopolitical marketing (Zwick and Bradshaw, 2016), and hence the power Facebook holds by controlling them.

Measure of value. In both the economic and the political fields, the number of fans is a rough measure of social capital, the number of people an actor (be it a brand, a celebrity, a social movement or a politician) may be able to mobilize (into buying, unpaid promotional labor, voting or volunteering). However, accumulated ties also have a semiotic sign-value (Dalsgaard, 2008): their number is widely interpreted as indicating worth and power, thus it performatively grants power. ‘Likes’ serve as a universal token of worth, value and affect, the currency of general sentiment (Arvidsson, 2012; Gerlitz and Helmond, 2013). With one million fans, a protest, a startup or a politician cannot be ignored.

Much of this social capital does not exist independently of Facebook: if not objectified as ‘Facebook friendships’, many of these ties will immediately cease to exist, while others will gradually fade away, along with the opportunity to mobilize resources. Social network analysis suggests brokers gain power from controlling bridges over network structural holes (Burt, 1992). In Facebook this applies not only to central nodes (popular profiles) but also to network operators, who exercise ‘networking power’, denying access to the resources accumulated within the network from those they believe do not add value (Castells, 2009).

Facebook’s power relies on its status as an emergent central bank of generalized social capital. Central banks issue currencies used by economic actors while competing with one another. While actors accumulate banknotes, their value remains dependent on the issuing bank, which may remove them from circulation or devaluate them at will. Bourdieu (1994, 1996) used the central bank metaphor for state control over other forms of capital such as academic certificates, for which actors compete, and the value of which depends on state recognition. While Facebook’s status relies on technological power rather than on symbolic power, this metaphor remains instructive.

Since Facebook streamlines the accumulation and maintenance of social capital, those who avoid using it are disadvantaged. Yet those who deposit their social capital in Facebook become increasingly dependent on Facebook Inc. and cannot withdraw their capital against its policy. They may invite ‘friends’ to parties or demonstrations, but not to participate in an uprising against the Israeli occupation. Facebook effectively restricts the mobilization of ties (e.g. by censoring posts); charges commissions (by algorithmically reducing fans’ exposure to page content and charging firms and social movements for ‘sponsored posts’ to avoid devaluation of their social capital); blocks currency transfers by emigrants (blocking applications that allow users to export their friend lists and thus retain the social capital they accumulated while immigrating to its competitor Google+); and confiscates social capital by deleting accounts, groups and pages. Facebook’s disciplinary power thus relies on its emergent status as a central bank of generalized social capital which it can devaluate, freeze or confiscate.

This status is comparable to Bourdieu’s famous account of the state as ‘the culmination of a process of concentration of different species of capital’ (Bourdieu, 1994: 4). Bourdieu claimed that as a central bank of multiple sorts of capital, the state has the privilege of setting their exchange rates, regulating social games in different fields, and thus influencing capital redistribution within its jurisdiction.

Facebook strives to gain a comparable status by becoming a necessary point-of-passage in multiple social spheres (including consumerism, culture and politics), a universal mediator and regulator of social life. Its privileged position allows Facebook Inc. to exert economic, political and socio-cultural influence by enforcing norms of conduct on its users, that is, to govern them.

How Facebook Governs

The next section explores Facebook’s governance toolkit – the set of mechanisms, technologies and strategies it employs to govern users. Indeed, any design decision may be framed as ‘governance’ in the broad sense. Online as offline, architecture shapes subjectivities, knowledge and power relations. For example, Facebook’s public like-counters quantify the value of user-generated contents, encouraging users to compete over attention, produce more contents and adapt content to their audience’s preferences (Coté and Pybus, 2007); rewarding intensive participation with increased visibility disciplines users (Bucher, 2012), while emoticons normalize emotional expression. Power should not be reduced to sovereignty and negative repression. Yet neither should we neglect their crucial role. Alongside design and seduction, digital capitalism governs through discipline, surveillance, legislation and punishment.

As suggested above, Facebook has an interest in governing its usership to maximize biopolitical production, yet governing billions of users may require high costs. While struggling to streamline governance, internet corporations have developed an apparatus consisting of algorithmic automated governance, free peer-produced policing labor, and proletarian judicial labor. Below I briefly review these components, while also addressing other characteristics of social network site governance: ambiguity; punishment by isolation; and constitutional legitimation.

Algorithmic Governance

Algorithms play an increasingly important role in governance, especially online, where social interactions are fully captured as digital data, thus rendered simultaneously economically exploitable, knowable and governable by algorithms. Algorithms are a highly economic governance tool, as they reduce reliance on human deliberation and hence governance costs. Like laws, algorithms are rules, but of a very different kind: they are ‘generative rules’, virtual principles that mechanistically generate sundry actuals (Lash, 2007), enforcing themselves like the laws of nature (Zuboff, 2015). As Cheney-Lippold (2017) suggested, algorithmic regulation exercises exceptionally intimate and efficient power, since it checks up on every datafied step users make.

Algorithmic rules often remain hidden as trade secrets: unlike laws, their efficacy does not require the governed to know and obey them (Beer, 2009; Zuboff, 2015), and they are often too complex to be publicly known. For example, Facebook cannot tell users how many messages they may send without being qualified as spammers, since no such universal upper limit exists. Algorithms classify users as spammers based on multiple risk assessment criteria weighed simultaneously. Furthermore, ‘learning algorithms’ constantly revise the rules they enforce (Cheney-Lippold, 2017).

Algorithms do not simply replace laws as a governance tool; they also enforce traditional laws, which remain central in Facebook’s governance and its legitimation. Algorithms are used to identify suspected law-violators, and to prospectively (and hyperpanoptically) thwart suspected violations. When algorithms classify photos as nudity they may either report them to human content moderators or automatically prevent their posting. In 2016 Facebook claimed to ‘have more offensive photos being reported by AI algorithms than by people’ (Constine, 2016). Users algorithmically classified as possible spammers or sexual predators based on their activity patterns (e.g. high rate of declined friend requests, gender-unbalanced networks, or using certain phrases) may be automatically warned or punished. While all algorithms have politics (Gillespie, 2014), these algorithms are explicitly designed to carry governance law-enforcing tasks.

Algorithmic rules are black boxes, yet users develop assumptions about them as they discipline themselves or try to manipulate algorithms (e.g. when the common assumption that multiple reports automatically produce punishment motivates mass organized flagging of ideological rivals; Crawford and Gillespie, 2016).

In an open Manifesto, Zuckerberg (2017) promised to increase the use of algorithms to both fight sensationalist ‘fake news’ and filter offensive content. Zuckerberg suggested artificial intelligence would allow Facebook to shift from censorship to personalization, lifting restrictions on published content while filtering contents based on individual and culturally-specific preferences, thus moving away from the statist model of universal law. However, currently the complex work of governance cannot be fully delegated to algorithms, which are hence complemented by other governance tools.

Proletarian Judicial Labor

While some policing work is delegated to algorithms, Facebook still considerably depends on civil enforcement. It encourages users to flag offensive contents whenever encountering them, and frames this free peer-produced policing labor as acts of good citizenship within a ‘community’. In social network sites the many watch the many (Gane, 2012: 623), and this lateral surveillance is encouraged to support top-down surveillance (Andrejevic, 2005).

The complainants should only choose the alleged offense from a list, and cannot add details, context or explanations. This ‘limited vocabulary’ (Crawford and Gillespie, 2016) streamlines the administration of justice and reduces governance costs. Since reporting takes only a few seconds, users submit millions of reports weekly, translating political, personal, ethical and national disputes into legal issues.

These disputes are adjudicated by low-status and low-income content moderators (Roberts, 2016) with no formal legal education. Partial outsourcing to subcontractors in poor countries further reduces costs. The administration of justice, traditionally a high-status middle-class occupation, is proletarianized, as moderators are expected to make decisions within seconds (Bazelon, 2013). A premium-bonus system with low hourly wage creates strong pressure to further speed-up judicial labor, whereas exposure to graphic violent and sexual contents turns moderation into a ‘dirty work’ (Roberts, 2016).

To increase productivity, decisions are not explained (defendants are merely told they have violated Facebook’s Community Standards, without reference to particular sections or violations), and appeals cannot include any arguments. Instead of legal education and knowledge of precedents, moderators rely on guidelines that translate Facebook’s rules into simple technical criteria, including procedures to distinguish between credible and empty threats, and extremely detailed rules on sex and violence in conversations and photos (allowing, for example, deep wounds and gay kisses, while banning ‘cameltoes’; oDesk, 2012). Yet this routine proletarian cognitive labor involves interpretive challenges: without testimonies and statements of claims and defense, intentions and context must be inferred from the reported content alone.

Moderators also make politically sensitive decisions in gray areas. Since judgments are not supported by reasons, Facebook can easily overturn them as ‘mistakes’. This has happened in multiple politically sensitive cases where criticism of Facebook’s decisions and policies gained media attention. Facebook overturned both acquittals (e.g. decisions to tolerate hunting photos with carcasses, insensitive remarks on Down syndrome, and Palestinian uprising pages) and convictions (decisions not to tolerate racist-shaming in Chicago, information on Syrian torture facilities, or an Arab feminist campaign). Even dramatic policy changes (such as when Facebook eventually conceded to public pressure and loosened its real identity policy to allow drag queens to use stage names) were framed as corrections of enforcement mistakes. In these cases, mass media operates as an informal appeal mechanism. The low status of moderators, the lack of separation of powers, and the lack of detailed and binding laws and precedents allow Facebook flexibility in responding to public criticism, making local concessions to avoid alienating users. It also allows making exceptions for commercial reasons such as traffic.

Facebook explicitly frames moderation work as judicial (using words such as ‘rulings’ and ‘appeals’). While judicial framing grants legitimation, heavily borrowing on common legal practice while deviating from legitimated legal norms results in much criticism over inconsistency, opacity, collective punishment and injustice.³ The flagging system is indeed opaque and non-democratic, as claimed by Crawford and Gillespie (2016), yet its efficiency and flexibility serve the commercial interests of site operators.⁴

While this mechanism is also used for ‘delegated governance’ (Denardis, 2014), as states urge Facebook to enforce their local laws (e.g. against Holocaust denial in Germany), its main use and the logic that originally shaped it are governing cheaply to provide perceived safety and streamline immaterial production. My contention is not that any individual moderation decision is necessarily rational or shaped by economic interests alone, but rather that this apparatus as a whole allows Facebook to streamline biopolitical production.

Proletarianized administration of justice is common across firms in digital capitalism, since it derives from shared commercial interest in governing cheaply, and from the increased governability of online interaction. When anything said or done is documented and leaves digital traces, higher levels of accountability for everyday talk emerge (Schwarz, 2011; Schwarz and Shani, 2016), and volumes of governance and judicial labor rise respectively, encouraging proletarianization and automatization.

Ambiguity

Facebook’s governance is characterized by high levels of ambiguity. Its public rules are often vague, whereas moderator guidelines that concretize them are kept secret and may change frequently. Decisions are efficient, yet unpredictable and inconsistent since they are not documented as binding precedents – the main mechanism in the Common Law tradition to gradually reduce the margin for interpretation of a law throughout its career. While a decision on any particular item automatically applies to future complaints on the same item,⁵ it is not generalized to other similar items. Without written reasons for judgment, users cannot predict how future cases will be decided. The complexity and unfixity of algorithmic rules similarly restrict predictability.

This ambiguity and unpredictability obviously divert from Max Weber’s (1978) liberal model of the law in the West, where independent professional judges and universal, abstract laws foster freedom and rationality by producing predictability, freeing citizens from the ruler’s mercy and allowing them to make rational calculations. However, this ambiguity is strategic, and while lacking in formal rationality, its disciplinary effects are rational indeed. As one content moderator explains, in digital governance ‘internal policies are not made public because then it becomes very easy to skirt them to essentially the point of breaking them’ (Roberts, 2016: 152). Exactly because ambiguity thwarts rational prediction and calculation, it encourages users to self-censorship of ‘gray’ materials if they wish to avoid risking their accumulated social capital. Legal ambiguity thus encourages the sociability Facebook desires, serving its governmental goals.

Isolation

The punishments Facebook imposes include content removal; temporarily blocking users from certain activities (e.g. messaging); temporary suspension (ranging between hours and months); and permanent expulsion (deletion of accounts, groups and pages). Users are first given a warning or mild punishment, and the threat of severe punishment should deter them from recidivism. While content removal merely fixes the violation, more severe punishment aims to discipline users through social capital freezing or confiscation, social isolation, and denial of attention and visibility, which are desirable goods online (Bucher, 2012).

This isolation reminds us of incarceration, and users often refer to suspension as ‘Facebook jail’. Indeed, Facebook isolates accounts, not bodies, yet accumulated social capital and visibility are similarly attached to accounts: while punished subjects may sometimes open new accounts, even partial restoration of their social capital and visibility would require great efforts. For political networked associations, which often lack any offline existence, group deletion might well be detrimental.

Facebook’s visibility regime is thus not simply ‘a reversal of the regime instantiated by the Panopticon’, as suggested by Bucher (2012: 1166). While the invisibility inflicted algorithmically on passive users encourages participation, visibility – the fact that digital communication leaves digital traces subjected to panoptic surveillance – allows disciplining subjects to conform under threats of invisibility and social capital confiscation. This has proved quite effective: in 2013 a Facebook executive reported low recidivism rates, as only 6 percent of reports referred to users who had already been reported before (Bazelon, 2013).

Constitutional Legitimation

Although vague and subjected to interpretation by secret guidelines, Facebook does have a legal code (‘Community Standards’). It also has quasi-constitutional ‘governing documents’: the Principles document explicitly frames users as governed by Facebook for the common good, whereas the more detailed Statement of Rights and Responsibilities (initially referred to as the ‘Bill of Rights’) officially guarantees users rights and freedoms such as free speech (‘the freedom to share’); transparent laws, policies and processes; ‘fundamental equality’ of all users before the law; and even property rights over the social capital they have accumulated (users are declared free ‘to build trust and reputation through their identity and connections, and should not have their presence on the Facebook Service removed’ for any reason not mentioned in the Statement). While no constitutional procedural mechanisms, checks and balances guarantee these rights (the only mechanism that granted users some power, the referendum, was abolished in 2012), these constitutional documents are not meaningless. They grant Facebook’s governing apparatus legitimation while retorting to common public criticisms of Facebook.

Facebook turned to democratic idioms for legitimation in reaction to a major legitimation and trust crisis. Revisions Facebook surreptitiously introduced in its terms-of-use in February 2009 provoked public uproar. Facebook was perceived as threatening user privacy and demanding unrestricted irrevocable rights over users’ data. Having failed to soothe the uproar with mere explanations, Facebook replaced the contractual terms-of-use document with the quasi-constitutional ‘governing documents’ that frame it as governing for the common good while protecting users’ rights. Users were invited to comment on the constitutional documents’ drafts in the ‘Facebook Town Hall’ group, a public participation procedure described by Zuckerberg as ‘similar to how U.S. federal agencies create regulations’. Dissidents who had led the anti-Facebook protest were given official roles in the process, and Zuckerberg avowed transparent governance. The governing documents also introduced a democratic referendum mechanism (this concession to users was restricted to constitutional changes only, as Facebook claimed that subjecting design changes to voting would hamper progress) and were themselves approved in a referendum with 74 percent support.

This was not the only case where user protest influenced Facebook governance policies: nursing mothers succeeded in revising the Community Standards and lifting the ban over breastfeeding photos after a five-year campaign that included organized mass civil disobedience. Protests over drag artists’ right to use their stage names rather than their legal names and over homosexual kisses similarly succeeded in influencing Facebook’s policies. These protests were unusual: whereas historical anti-corporate protests usually addressed the immorality of firms as actors, whose products and production processes damage the environment, workers, education or morality (Soule, 2009), protests against Facebook addressed Facebook as a regulator, criticizing the way it governs its users-subjects’ social life. Whereas internet scholarship conceptualizes Facebook’s relationship with its users mainly as exploitation of unremunerated labor, actual user resistance to Facebook’s policies is mainly focused on the political dimension of this relationship (for an exception see Fisher, 2015), borrowing repertoires of resistance from the political sphere (civil disobedience) rather than from labor relations (strikes). Facebook itself has contributed to this ‘political’ framing by making policy decisions on politically sensitive issues and by borrowing governance tools and legitimation rhetoric from the political sphere (developing its own constitution, laws, referenda, courts and appeals while recognizing rights, community and the common good as sources of legitimation).

When the referendum turned from legitimation tool into a tool of resistance, Facebook cancelled the mechanism.⁶ Zuckerberg’s later public statements shifted towards a different legitimation strategy, employing a rhetoric of digital ‘enlightened despotism’: portraying Facebook as ‘listening’, ‘attentive’ to users-citizens’ criticism and even grateful for it, but going beyond democratic tools to decipher users’ will, ‘surveilling users in order to understand what they really want versus what they say they want’ (Hoffmann et al., 2018: 213). Indeed, Facebook permanently collects data on usage patterns and conducts A/B testing (Zuboff, 2015), yet its aim is optimizing design to maximize profit extracted from users, which can easily be in conflict with users’ notions of the common good.

In digital capitalism, where governance becomes crucial to the extraction of surplus value, legitimation similarly becomes increasingly central to the business operation of internet corporations. User resistance resulted in both concrete policy revisions and constitutional reforms. While the latter had little influence on Facebook’s actual operation and governance, it reframed Facebook’s relationship with its users as governance rather than contractual-commercial relations. Facebook’s selective borrowing from the cultural repertoire of democratic state governance without subjecting itself to liberal principles that have legitimated state power since John Locke – including popular sovereignty, fair judgment, predictable laws, and separation of powers – might open the door for further resistance.

A General Model

The heterogeneous governance apparatus reviewed above is not unique to Facebook but typical of digital capitalism. All its components have been used by other internet corporations. YouTube, for example, also combines algorithmic governance and proletarian judicial labor (its enforcement workforce should reach 10,000 employees in 2018; Perez, 2017). Flickr, Weibo and Instagram similarly use human content moderators. Google exercises algorithmic governance by algorithmically denying access from IP addresses suspected of violating terms of use, and moving emails classified as likely spam or phishing to the spam folder. Ambiguity is similarly common on other platforms where algorithmic governance and proletarian judicial labor are applied. Punishment by isolation is practiced in platforms such as Second Life and WhatsApp, where suspension or account deletion might isolate business owners from customers, as diverse platforms use their control of accumulated social capital to discipline users, although on a much smaller scale than Facebook.

This cross-platform similarity was shaped by institutional isomorphism (DiMaggio and Powell, 1983), by similar economic and technological challenges, but also by shared motivations for governance discussed above – streamlining biopolitical production at the lowest possible costs. This also explains why the main components of this apparatus remain relatively stable despite constant changes in platform design and operation. Facebook is unique in the degree to which it has engaged in constitutional legitimation and in its dominance across the life sphere which has rendered its governance more powerful and consequential, but its structures of governance are typical of digital capitalism.

Conclusion

As shown above, the political economy of social network sites and their politics are intimately interrelated. Without addressing the former, we can hardly explain why social network sites became interested in governing interpersonal, romantic, cultural and political interactions between users – human interactions in spheres not governed traditionally by corporations. This interest derives, I suggest, from their new capacity to extract surplus value from all these social interactions by translating them into content and data.

Facebook thus teaches us a general lesson. In the contemporary digital economy of ‘surveillance capitalism’ (Zuboff, 2015) – where capitalism reorganizes around immaterial goods such as data and emotions and where surplus value can be extracted from all human interactions – capitalism extends its interest in governance from ‘management’ (governing employees) to the regulation of social interactions and emotions across life-spheres. Internet corporations govern in order to secure ‘safe’, unintimidating social spaces; protect enclosure; and intensify and streamline biopolitical production. This is a modernist project of gardening society by subjecting all digital interactions among users to rational governance. By regulating social life corporations like Facebook influence power-relations and encroach upon the political sphere. In digital capitalism there is thus a strong linkage between exploitation (the capture, enclosure and valorization of social relationships) and governance (attempts to regulate them).

Hence, Facebook simultaneously employs two complementing forms of surveillance, commodifying surveillance and panoptic surveillance, which are similarly interrelated. The first, widely discussed in the literature, turns human actions and affect into commodified data. But there is a second form, which is no less crucial for surveillance capitalism: Facebook monitors and disciplines users to improve biopolitical production and thwart sabotage. Users are more than raw materials: their actions influence production, and as environmental intervention fails to produce docility, surveillance capitalism deploys the heterogeneous governance apparatus portrayed above.

Bauman’s mistake in prematurely announcing the ‘collapse of the “panoptic” model of securing and perpetuating social order’ (Bauman, 1998: 22) in favor of consumer seduction (Bauman, 1998, 1999) derived from his assumption that postmodern capitalism targets us merely as consumers. Internet corporations actually target us mostly as biopolitical producers whose immaterial labor can be captured, and they integrate panopticism into their apparatus of governance-cum-accumulation.

Given the account of Facebook governance offered above, Facebook can hardly be described as demonstrating ‘corporate power that is no longer aimed at disciplining consumers and at giving orders or shaping actions according to a given norm’, or as replacing technologies of surveillance and discipline with ‘provision of ambiences’ and governance through freedom (Zwick et al., 2008: 184–6), an ‘environmental type of intervention instead of the internal subjugation of individuals’ (Foucault, 2010, cited in Terranova, 2015: 124). The vast number of Facebook’s quasi-juridical decisions can hardly be reconciled with claims that ‘Facebook does little to dictate how its users use the site, but rather lets them use it the way they wish’ (Ritzer and Jurgenson, 2010: 31).

The governance toolkit used by Facebook and other digital corporations is highly eclectic, bringing together sovereign legislation and design choices, seduction and disciplinary surveillance, constitutionalism and A/B testing, algorithmic governance and proletarianized administration of justice. It does not easily fit into epochal accounts such as the shift from sovereignty to governmentality, from government into governance, or from discipline to consumer seduction. It is surely too centralized to be subsumed under an account of the internet as a diagram of post-Fordist decentralized governance. The coherence of this eclectic toolkit is only revealed once we shift our attention to its political economy. Online all interactions are documented, which increases accountability for casual interactions (Schwarz and Shani, 2016) and renders them more governable. The resulting increase in the volume of governance work has encouraged rationalization and a shift towards algorithmic governance and proletarian judicial labor. Whereas Facebook might well lose its monopolistic status in the future, other internet corporations employ similar governance apparatuses, shaped by a similar political economy.

What makes Facebook’s governance consequential and effective is its emerging status as a central bank for social capital. This is the result of an isomorphic process, in which power and status in various fields increasingly take the form of objectified generalized social capital accumulated in social network sites. As digital services are increasingly intertwined in our everyday practices, denial of service in general is an effective punishment, yet confiscation of social capital is particularly powerful, as it affects a central social mechanism of hierarchization and accumulation of power. Confiscating social capital for punishment is practiced by several social network sites, virtual worlds and messaging platforms. However, Facebook uses it much more effectively and powerfully, as it controls generalized social capital to unprecedented levels. Viewed from a social capital perspective, multiple changes in Facebook’s design and features over the years have moved in the same direction, making it into a central bank of ever more diverse kinds of social capital.

Facebook’s power to engage in sovereign governance relies on its power to confiscate at will capital accumulated by users, be they ordinary individuals, politicians, social movements, artists or firms. By allowing all these actors to translate momentary sympathy into sustainable ties, Facebook empowers them, while also gaining the capacity to disempower them at will, depriving teenagers of access to chitchat, gossip and party invitations; social movements and rebels of the capacity to mobilize sympathizers to coordinated collective action; and firms of capitalizing on past investment in brand communities.

Liberal sociologists and philosophers have repeatedly suggested that the separation between partly-autonomous life spheres (and multiple institutional authorities) is a prerequisite for freedom (Goffman, 1961; Walzer, 1983; Weber, 1978). Facebook’s endeavor to become a central bank of various forms of social capital which are in circulation in the political, social, cultural and economic spheres, and to become a necessary point-of-passage in all these spheres, may thus be viewed as an alarming concentration of power.

While Facebook has long drawn public criticism, it has usually been formulated in terms of privacy and data control (as in the recent outrage against Cambridge Analytica’s use of Facebook to mine data on millions of Americans for targeted political advertising). Less often public criticism addresses particular politically sensitive governance decisions Facebook made or failed to make (as in the cases listed in this article’s opening paragraph). More recently, criticism has addressed the more systematic level of algorithmic curation, when Facebook was accused of failing to block ‘fake news’ during the 2016 US presidential campaign. However, if Shearing and Stenning were right, the problem lies elsewhere, at the very shift of social interaction and public life to privately-owned digital mass private property and its subjection to rules made by platform operators.

Recent major scandals have posed a clear threat to Facebook and its business model of biopolitical production, and it has reacted to them by introducing changes; however, these changes remain within the framework of governance portrayed above. Thus, the use of algorithms to identify suspicious foreign election meddling is yet another case of algorithmic governance, which results in confiscation of social capital (as in the case of closing down protest event pages; American activists who – not knowing who had opened them – invested time and effort in co-hosting these events had to ‘start over in gathering followers and building momentum to stage an effective protest’; Conger and Savage, 2018).⁷

The scandals of recent years have raised consciousness about algorithmic curation, and few now consider Facebook to be a neutral medium, but neither is it a ‘publisher’, as some suggest; saying so would amount to a textualist reduction of the richness of mediated social life.⁸ It is a quasi-sovereign power ruling its subjects. Since its governance apparatus was designed as a business investment in cultivating user populations and increasing their productivity, Facebook refuses to allow users to shape the laws that govern them. It remains open whether the governed will accept it in the long run.

Footnotes

Acknowledgements

I am grateful to Eran Fisher and Eva Illouz for their comments on earlier versions of this paper, and to Anat Ben David for allowing me access to her personal notes.

Notes

Ori Schwarz is a senior lecturer of sociology at Bar-Ilan University. His research interests include digital sociology, cultural sociology and sociological theory. His work on SNSs explores political unfriending, homophily, governance, identity work and boundary work in social network sites. He has also studied extensively the influence of digital data-objects on attention, memory and notions of value.

References

Andrejevic, Mark (2005) The work of watching one another: Lateral surveillance, risk and governance. Surveillance & Society 2(4): 479–497.

Arvidsson, Adam (2012) General sentiment: How value and affect converge in the information economy. The Sociological Review 59(2): 39–60.

Arvidsson, Adam (2016) Facebook and finance: On the social logic of the derivative. Theory, Culture & Society 33(6): 3–23.

Bauman, Zygmunt (1998) On postmodern uses of sex. Theory, Culture & Society 15(3–4): 19–33.

Bauman, Zygmunt (1999) In Search of Politics. Cambridge: Polity.

Bazelon, Emily (2013) How to stop the bullies. The Atlantic. Available at: https://www.theatlantic.com/magazine/archive/2013/03/how-to-stop-bullies/309217/ (accessed 18 January 2019).

Beer, David (2009) Power through the algorithm? Participatory web cultures and the technological unconscious. New Media & Society 11(6): 985–1002.

Bor, Stephanie (2014) Using social network sites to improve communication between political campaigns and citizens in the 2012 election. American Behavioral Scientist 58(9): 1195–1213.

Bourdieu, Pierre (1986) The forms of capital. In: Richardson, John (ed.) Handbook of Theory and Research for the Sociology of Education. New York: Greenwood.

10.

Bourdieu, Pierre (1994) Rethinking the state: Genesis and structure of the bureaucratic field. Sociological Theory l2(1): 1–18.

11.

Bourdieu, Pierre (1996) The State Nobility. Cambridge: Polity.

12.

Boyle, James (1997) Foucault in cyberspace: Surveillance, sovereignty, and hardwired censors. University of Cincinnati Law Review 66: 177–205.

13.

Bucher, Tanja (2012) Want to be on the top? Algorithmic power and the threat of invisibility on Facebook. New Media & Society 14(7): 1164–1180.

14.

Burt, Ronald (1992) Structural Holes. Cambridge, MA: Harvard University Press.

15.

Castells, Manuel (2009) Communication Power. Oxford: Oxford University Press.

16.

Castells, Manuel (2012) Networks of Outrage and Hope. Cambridge: Polity.

17.

Chander, Anupam (2012) Facebookistan. North Carolina Law Review 90(5): 1807–1844.

18.

Cheney-Lippold, John (2017) We are Data: Algorithms and the Making of our Digital Selves. New York: New York University Press.

19.

Cohen, Nicole (2008) The valorization of surveillance: Towards a political economy of Facebook. Democratic Communiqué 22(1): 5–22.

20.

Conger, Kate and Savage, Charlie (2018) How fake influence campaigns on Facebook lured real people. New York Times, 2 August. Available at: https://www.nytimes.com/2018/08/02/technology/facebook-fake-accounts.html (accessed 18 January 2019).

21.

Constine, Josh (2016) Facebook spares humans by fighting offensive photos with AI. TechCrunch, 31 May. Available at: https://techcrunch.com/2016/05/31/terminating-abuse/ (accessed 18 January 2019).

22.

Coté, Mark and Pybus, Jennifer (2007) Learning to Immaterial Labour 2.0: MySpace and social networks. Ephemera 7(1): 88–106.

23.

Crawford, Kate and Gillespie, Tarleton (2016) What is a flag for? Social media reporting tools and the vocabulary of complaint. New Media & Society 18(3): 410–428.

24.

Dalsgaard, Steffen (2008) Facework on Facebook: The presentation of self in virtual life and its role in the US elections. Anthropology Today 24(6): 8–12.

25.

DeNardis, Laura (2014) The Global War for Internet Governance. New Haven: Yale University Press.

26.

DiMaggio, Paul and Powell, Walter (1983) The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review 48(2): 147–160.

27.

Earl, Jennifer and Kimport, Katrina (2011) Digitally Enabled Social Change. Cambridge, MA: MIT Press.

28.

Ellison, Nicole B., Steinfield, Charles and Lampe, Cliff (2007) The benefits of Facebook ‘friends’: Social capital and college students’ use of online social network sites. Journal of Computer-Mediated Communication 12(4): 1143–1168.

29.

Ellison, Nicole, Lampe, Cliff, Steinfield, Charles and Vitak, Jennifer (2010) With a little help from my friends: How social network sites affect social capital processes. In: Papacharissi, Zizi (ed.) A Networked Self. New York: Routledge.

30.

Fisher, Eran (2015) Class struggles in the digital frontier: Audience labour theory and social media users. Information, Communication & Society 18(9): 1108–1122.

31.

Fuchs, Christian (2011) New media, Web 2.0 and surveillance. Sociology Compass 5(2): 134–147.

32.

Gane, Nicholas (2012) The governmentalities of neoliberalism: Panopticism, post-panopticism and beyond. Sociological Review 60: 611–634.

33.

Gehl, Robert (2011) The archive and the processor: The internal logic of Web 2.0. New Media & Society 13(8): 1228–1244.

34.

Gehl, Robert (2014) Reverse Engineering Social Media. Philadelphia: Temple University Press.

35.

Gerlitz, Carolin and Helmond, Anne (2013) The Like economy: Social buttons and the data-intensive web. New Media & Society 15(8): 1348–1365.

36.

Giddens, Anthony (2002) Runaway World. London: Profile.

37.

Gillespie, Tarleton (2014) The relevance of algorithms. In: Gillespie, Tarleton, Boczkowski, Pablo and Foot, Kirsten (eds) Media Technologies. Cambridge, MA: MIT Press.

38.

Goffman, Erving (1961) Asylums. Harmondsworth: Penguin.

39.

Hardt, Michael and Negri, Antonio (2000) Empire. Cambridge, MA: Harvard University Press.

40.

Hoffmann, Anna Lauren, Proferes, Nicholas and Zimmer, Michael (2018) ‘Making the world more open and connected’: Mark Zuckerberg and the discursive construction of Facebook and its users. New Media & Society 20(1): 199–218.

41.

Illouz, Eva (2008) Saving the Modern Soul. Berkeley: University of California Press.

42.

Jessop, Bob (1996) Post-Fordism and the state. In: Greve, Bent (ed.) Comparative Welfare Systems. London: Macmillan.

43.

Karppi, Tero (2011) Digital suicide and the biopolitics of leaving Facebook. Transformations 20: 1–28.

44.

Laidlaw, Emily (2015) Regulating Speech in Cyberspace. Cambridge: Cambridge University Press.

45.

Laroche, Michel et al. (2012) The effects of social media based brand communities on brand community markers, value creation practices, brand trust and brand loyalty. Computers in Human Behavior 28: 1755–1767.

46.

Lash, Scott (2007) Power after hegemony: Cultural studies in mutation. Theory, Culture & Society 24(3): 55–78.

47.

Lessig, Lawrence (2006) Code: Version 2.0. New York: Basic Books.

48.

Lin, Nan (1999) Social networks and status attainment. Annual Review of Sociology 25: 467–487.

49.

Lyon, David (1994) The Electronic Eye. Cambridge: Polity.

50.

MacKinnon, Rebecca (2012) Consent of the Networked. New York: Basic Books.

51.

Marwick, Alice (2013) Status Update. New Haven: Yale University Press.

52.

Massanari, Adrienne (2017) #Gamergate and the fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society 19(3): 329–346.

53.

O’Brien, Sara (2016) Facebook gets 1 million user violation reports a day. CNN, 12 March. Available at: http://money.cnn.com/2016/03/12/technology/sxsw-2016-facebook-online-harassment/ (accessed 18 January 2019).

54.

oDesk (2012) Abuse Standards 6.2. Scribd. Available at: https://www.scribd.com/embeds/81877124/content?start_page=1&view_mode=list&access_key=key-2ava6s4chjmfzil6rfhg (accessed 18 January 2019).

55.

Perez, Sarah (2017) YouTube promises to increase content moderation and other enforcement staff to 10K in 2018. TechCrunch, 5 December. Available at: https://techcrunch.com/2017/12/05/youtube-promises-to-increase-content-moderation-staff-to-over-10k-in-2018/ (accessed 18 January 2019).

56.

Ritzer, George and Jurgenson, Nathan (2010) Production, consumption, prosumption. Journal of Consumer Culture 10(1): 13–36.

57.

Roberts, Sarah (2016) Commercial content moderation: Digital laborers’ dirty work. In: Noble, Safiya and Tynes, Brendesha (eds) The Intersectional Internet. New York: Peter Lang.

58.

Schwarz, Ori (2011) Who moved my conversation? Instant messaging, intertextuality, and new regimes of intimacy and truth, Media, Culture & Society 33(1): 71–87.

59.

Schwarz, Ori and Shani, Guy (2016) Culture in mediated interaction: Political defriending on Facebook and the limits of networked individualism, American Journal of Cultural Sociology 4(3): 385–421.

60.

Shearing, Clifford and Stenning, Phillip (1983) Private security: Implications for social control. Social Problems 30(5): 493–506.

61.

Skeggs, Beverley and Yuill, Simon (2016) Capital experimentation with person/a formation: How Facebook’s monetization refigures the relationship between property, personhood and protest. Information, Communication & Society 19(3): 380–396.

62.

Soule, Sarah (2009) Contention and Corporate Social Responsibility. Cambridge: Cambridge University Press.

63.

Staeheli, Lynn and Mitchell, Don (2006) USA’s destiny? Regulating space and creating community in American shopping malls. Urban Studies 43: 977–992.

64.

Terranova, Tiziana (2015) Securing the social: Foucault and social networks. In: Fuggle, Sophie, Lanci, Yari and Tazzioli, Martina (eds) Foucault and the History of Our Present. Houndmills: Palgrave.

65.

van Dijck, José (2013) The Culture of Connectivity: A Critical History of Social Media. Oxford: Oxford University Press.

66.

Walzer, Michael (1983) Spheres of Justice. New York: Basic Books.

67.

Weber, Max (1978) Economy and Society. Berkeley: University of California Press.

68.

Weber, Max (1994) Political Writings. Cambridge: Cambridge University Press.

69.

Ziewitz, Malte and Pentzol, Christian (2014) In search of internet governance: Performing order in digitally networked environments. New Media & Society 16(1): 306–322.

70.

Zuboff, Shoshana (1988) In the Age of the Smart Machine. New York: Basic Books.

71.

Zuboff, Shoshana (2015) Big other: Surveillance capitalism and the prospects of an information civilization. Journal of Information Technology 30: 75–89.

72.

Zuckerberg, Mark (2017) Building global community. Available at: https://www.facebook.com/notes/mark-zuckerberg/building-global-community/10154544292806634/ (accessed 18 January 2019).

73.

Zwick, Detlev and Bradshaw, Alan (2016) Biopolitical marketing and social media brand communities. Theory, Culture & Society 33(5): 91–115.

74.

Zwick, Detlev, Bonsu, Samuel and Darmody, Aron (2008) Putting consumers to work: ‘Co-creation’ and new marketing govern-mentality. Journal of Consumer Culture 8(2): 163–196.