Who Gets to Choose? On the Socio-algorithmic Construction of Choice

Abstract

This article deals with choice-inducing algorithms––algorithms that are explicitly designed to affect people’s choices. Based on an ethnographic account of three Israeli data analytics companies, I explore how algorithms are being designed to drive people into choice-making and examine their co-constitution by an assemblage of specifically positioned human and nonhuman agents. I show that the functioning, logic, and even ethics of choice-inducing algorithms are deeply influenced by the epistemologies, meaning systems, and practices of the individuals who devise and use them and that such algorithms are similarly affected by interorganizational relationships, various nonhuman agents, and changing geopolitical contexts. I conclude by discussing the flexibility of choice-inducing algorithms and by arguing that such algorithms are not programmed to induce specific choices but to more generally convert people into choosers, and thus, to algorithmically (re)create the modern need to choose. This article contributes to the growing literature on algorithms and culture and to our understanding of choice-making in contemporary life. At the same time, it provides a new vocabulary that offers to critically engage with algorithms and their power without losing sight of the often very specific contexts from which they arise.

Keywords

algorithms choice data culture agency AI

Introduction

Our choices are fast becoming algorithmic. The ubiquity of recommender systems, personalization engines, and user analytics services has made algorithms almost inseparable from our everyday choice-making. Algorithms play a growing role in how we choose the music we listen to (Morris 2015), the movies we watch (Hallinan and Striphas 2016), the books we read (Striphas 2009), and even our romantic and sexual partners (Illouz 2007). Algorithms also direct our financial investments (Pasquale 2013) and track our weight (Crawford, Lingel, and Karppi 2015), health (Swan 2013), and even sleep to nudge us (Yeung 2017; Thaler and Sunstein 2008) toward “healthier” ways of living. In other words, algorithms are playing the role of editors (Lewis 2015), curators (Morris 2015), moderators (Gillespie 2018), matchmakers (Illouz 2007), and censors (Gillespie 2015). As “virtuals that generate a variety of actuals” (Lash 2007, 71), algorithms are fast becoming central to almost any choice that we make (Graham 2018; Cohn 2019).

This article focuses on what I term choice-inducing algorithms––recommender systems, personalization engines, profiling algorithms, and data analytics algorithms that are explicitly designed to affect people’s choices. Based on an ethnographic study of the Israeli data analytics scene and on the case studies of three user analytics companies, this article aims to, first, examine how choice-inducing algorithms are being used to convert people into choosers and, second, to uncover the cultural practices, worldviews, and narratives behind the formation of algorithmic choices. Seeing algorithms as culture-bound phenomena (Seaver 2017), I will show that choice-inducing algorithms are co-constituted by people’s intuitions, epistemologies, and even fears and that interorganizational relationships, as well as different nonhuman agents, take part in the algorithmic construction of choice. I will accordingly argue that agency does not solely reside in the humans who eventually “choose,” nor only in their interaction with the algorithms that “assist” or “guide” their choices, but in a diverse agentic swarm (Bennett 2010) of human and nonhuman agents who operate within specific social and cultural contexts. Last, I will discuss the flexibility of choice-inducing algorithms and argue that such algorithms are becoming key architects in (re)constructing the modern need to choose.

The Choosing Subject

Choice-making is one of the basic characteristics of modernity. As Giddens (1994) famously argued, in post-traditional societies “people have no choice but to choose” (p. 74). Modern people are constantly expected to reflexively choose between houses, workplaces, and romantic partners and make perpetual and conscious choices as to where to eat, what to wear, and what to watch. People construct their sense of self through an endless string of choices (Rose 1990) and live life as if it were a direct outcome of the choices that they make (Rose and Miller 2008, 18). In other words, in late modernity, choice-making is as free as it is obligatory; it is as quotidian as it is fateful.

With the rise of neoliberalism, the role of the “homo eligens”––the choosing person (Bauman 2007, 61)––was emphasized even further. Neoliberalism has brought to the fore a distinct entrepreneurial logic, according to which people see themselves as businesses, and enact a “corporate form of agency” by trying to actively balance their alliances, responsibilities, and risks (Gershon 2011, 540). This type of “privatized actuarialism” (O’Malley 1996, 198) requires people to rationally decide on how to connect with other people, institutions, and contexts and to constantly choose between and negotiate for different goods and services (Gershon 2011). With consumer capitalism, people’s shopping choices are becoming a means of identity construction as their “inner-truths” supposedly become manifested in their shopping lists (Bauman 2007, 15). Thus, late-modern subjects are choosing subjects––autonomous agents who strive to make responsible choices in a conscious effort to take command of their lives.

But choices are also a medium of power (Giddens 1994, 76). People are governed through their choices (Rose and Miller 2008, 25), and their choices are similarly constrained by, and often stem from, different power relations, social structures, and cultural norms. That is, while choices are often described as stemming from rational, individual agents, sociologists have long shown that choices in fact emanate from norms, structural constraints, and predetermined cultural blueprints. As Bourdieu (2005) has argued, modern people are “collective individuals” rather than “individual choosers” (p. 211).

Illouz (2012) similarly argued that modern choice-making is deeply affected by an ecology of choice––the social environments that compel one to make certain choices as well as by an architecture of choice––culturally shaped internal mechanisms that assist people in reaching a choice (pp. 20-21). Schwarz (2018) has recently expanded this idea, by offering to see choices as “culturally specific ways of doing” (p. 5). According to Schwarz (2018), culture offers actors not only repertoires of options to choose from but also repertoires of “ways of choosing” as well as culturally specific “techniques of choice” (p. 5). Hence, our choices stem from a multitude of cultural, social, and psychological factors that are internal to choosing subjects as much as they are external to them, and they originate from distributed cognitions shared between multiple human and nonhuman agents (p. 6). Indeed, the role of nonhumans in choice-making is by no means new––people have long used papers, pencils, and lists to make choices, and notepads, spreadsheets, and abacuses have been assisting humans in their choices for centuries (Schwarz 2018; Overmann 2017). At the same time, disciplines like psychology (Thaler and Sunstein 2008), marketing (Rose 1991), and advertising (Turow 2012) have been explicitly preoccupied with making people choose for more than a century. But the last decades have introduced a new entity, one whose influence on people’s choices cannot be denied––algorithms. As I will demonstrate below, algorithms are being used to alter people’s “ecologies of choice” (Illouz 2012) and (re)construct their “architectures of choice” (Illouz 2012), in a continuous effort to “convert people into choosers.”

Algorithmic Choices

Algorithms are defined as “encoded procedures for transforming input data into a desired output, based on specified calculations” (Gillespie 2014, 167). Although they are not necessarily computational, it is through computers that algorithms have their most dramatic effects. With computers (or “algorithm machines” [Gillespie 2014, 167]), algorithms run automatically, recursively, and in ever-growing speeds, making the social world increasingly commensurable (Callon and Law 2005), calculable (Totaro and Ninno 2014), and hence “algorithm ready” (Gillespie 2014, 168).¹ With the entrance of algorithmic systems into almost every aspect of human life, algorithms are becoming involved in people’s choices in an overwhelming pace. As Yeung (2017) has shown, big data algorithms often act as “hyper nudges”––“subtle, unobtrusive, yet extraordinarily powerful” techniques that recursively refine users’ choices according to their ever-changing data profiles (p. 2, 5). Graham (2018) similarly argued that the design and architecture of online platforms expand the scope, scale, and speed of choice-making. According to him, platforms are “engines” that drive, expand, and intensify the production of choice by structuring users’ fields of action in particular ways (p. 6-7). Graham and Henman (2019) have similarly shown that the design of platforms offers strategic affordances for structuring choice, as users are being drawn into “a political economy of choice-making.” Cohn (2019) has more recently described how online recommendation systems create “structured choices” and has argued that while they allegedly enhance personal decision-making, they in fact lead individuals toward conformity (p. 187). Thus, while the “choosing subject” has become the basic, almost taken-for-granted building block of neoliberal thought (Gershon 2011; Rose 1991), with algorithms, it is almost impossible to talk about choices as something that is done by a sole, autonomous, human agent: algorithms guide, restrict, and shape our choices. They alter our “ecologies of choice” (Illouz 2012) and (re)construct our “architectures of choice” (Illouz 2012). As Graham and Henman (2019) have argued, contemporary choices stem from constant interactions between users, algorithms, and platforms (p. 14). But a closer look at algorithms and their agency will reveal that our choices in fact stem from an earlier, much longer string of choices made by multiple agents in multiple contexts and from the complex interrelations between them.

Algorithmic Agency

Scholars have been reluctant to describe algorithms as agentic beings––Reichertz (2013) described algorithms as “nothing but instruments of human agency” (Reichertz [2013], cited by Seyfert and Roberge [2016]); Neff and colleagues (2012) have referred to algorithms’ power as a “not-quite, but ‘technically’, agency” (p. 305); and Klinger and Svensson (2018) recently argued that algorithms do not (yet) have the evaluative and reflexive elements of human agency. Others have highlighted algorithms’ opacity and orderliness (Lessig 1999), or their seeming objectivity, accuracy, and impartiality (Gillespie 2014, 180), to describe them as institutional factors (Napoli 2014), and yet others have used the term “computational agency” to describe algorithms’ automation and stealth (Tufekci 2015). Algorithms were also described as demonstrating a “logistical power” (Morris 2015, 452)––a tendency and ability to organize, orient, and rearrange people and property, and as such, they seem to exhibit what Bennett (2010) described as “thing-power”––“the curious ability of inanimate things to animate, to act, to produce effects, dramatic and subtle” (p. 6).²

In accordance with this view, and in opposition to more phenomenological approaches to agency (Archer 2000; Klinger and Svensson 2018), this article sees agency as the ability to actively change the social or material surroundings. An agent is someone or something “that acts” (Latour 1996, 373) and that “makes things happen” (Bennett 2010, 9), whether it is human, nonhuman, material, or immaterial (Sayes 2014; Floridi and Sanders 2004). But at the same time, drawing on the work of actor-network theorists (Latour 2011; Callon and Law 1997), agency is also something that is almost necessarily distributed “across a range of ontological types” (Bennet 2010, 10)—human and nonhuman alike. As Callon (2004) has pointed out: “change the collective, change the socio-technical arrangement, and you change the agency” (p. 8).

Thus, algorithms may be agents––they “do things” (Goffey 2008, 18), they act as engines (rather than cameras; MacKenzie 2008), and their coded instructions can have serious consequences (Gillespie 2014, 192). But at the same time, algorithmic agency necessarily stems from the “socio-material assemblages within which they are embedded” (Introna 2016, 20; see also Seaver 2018, 378) as well as from the heterogeneous assemblages from which they arise. After all, algorithms are human creations that constitute a “complex intermingling of human and non-human actors” (Napoli and McGannon 2013, 8), and the term “algorithm” itself stands for a complex socio-technical assemblage (Gillespie 2016, 24). Thus, algorithms may have “the capacity to shape social and cultural formations and impact directly on individual lives” (Beer 2009, 994), but they are far from autonomous, and their power is far from absolute. As I will argue below, the agency of algorithms is closely intertwined with that of the people who write them, the people who buy and run them, and the users who interact with them.

The last decade has seen a proliferation of studies on algorithmic powers and their consequences (Beer 2009; Neyland 2015; Gillespie 2016; Bucher 2018; Eubanks 2018; Noble 2018; O’Neal 2016), and more recently, scholars have shed considerable light on the ties between algorithmic systems and human choices (Cohn 2019; Graham and Henman 2019; Graham 2018; Yeung 2017). However, these works tend to overlook the socio-technical ways in which algorithmically induced choices are being formed. Accordingly, such works tend to be theoretical in nature, and hence, a detailed, empirical account of the socio-technical drama behind the formation of algorithmic choices, and of the agents involved in such choices, is still missing. Last, while previous research has shed considerable light on algorithmic agency (Neff and Nagy 2016; Tufekci 2015; Klinger and Svensson 2018), it has predominantly focused on how algorithms constrain users’ agency or on users’ changing perceptions of algorithmic agency (Van Dijck 2009; Peacock 2014; Willson 2014; Friedman and Nissenbaum 1997) rather than on the socio-technical constitution of algorithmic agents.

This article aims to fill these gaps by examining the “culturally specific ways of doing” (Schwarz 2018, 5) behind algorithmic choices and the ways in which different human and nonhuman agents write, rewrite, and influence algorithms, which in turn aim to design, restrict, and affect people’s choices. I identify the different agents behind choice-inducing algorithms and highlight the specific mechanisms, practices, and meaning systems behind the construction and implementation of such algorithms. More specifically, I ask: which agents take part in the “agentic swarm” (Bennett 2010, 32) behind algorithmic choices? What beliefs, epistemologies, and practices guide the construction of algorithmic “ecologies of choice” (Illouz 2012)? How are they affected by tensions in, and between, data analytics companies? And how does data (or their lack) inscribe the logic as well as the ethics of choice?

Methodology

This article is part of a larger ethnographic study of the Israeli data analytics scene conducted between 2013 and 2018. I have conducted forty semi-structured interviews with Israeli data scientists, programmers, marketers, CEOs, and investors from companies that produce personalization engines, recommender systems, and user analytics services. I have also conducted participant observations in various data analytics events (conferences, hackathons, “meetups,” etc.) as well as in the offices of three high-tech companies. I have collected and analyzed the promotional videos, white papers, and other publicly available data from such companies and have participated in four data mining courses.

The different data were logged into MaxQDA version 18 for thematic clustering and analysis. Using the thematic analysis method (Braun and Clarke 2006), I read and reread the data, identified recurrent themes and major concepts, and clustered similar segments together. I later highlighted and analyzed illustrative quotations from each category according to an interpretive-constructivist approach (Heikkinen, Huttunen, and Kakkori 2000).

For the sake of brevity, my discussion focuses on three Israeli companies that produce choice-inducing algorithms––FingerPrint, AI Solutions, and Identica.³ The Findings section is accordingly divided into three parts. The first section discusses how algorithms are being used to try and convert people into choosers. The second section focuses on an allegedly autonomous, artificial intelligence (AI)–powered algorithmic agent that gets “overwritten” to meet the expectations, needs, and even fears of the individuals who use the algorithm. And the third and last section revolves around the power of data (and its lack) and the complex power relations between local data analytics companies and international corporations. I will argue that this power structure affects not only the contents and functionality of choice-inducing algorithms but also their logic and ethics.

Findings

Converting the Choosing Subject: From Mouse Movements to Ecologies of Choice

“FingerPrint” is a successful Israeli user analytics company that captures users’ mouse movements to create behavioral analytics. Like many other companies in this field, FingerPrint is a “B2B” or business to business company––they offer their services to other companies that can easily add FingerPrint’s service to their website and start tracking their users. FingerPrint specializes in, but is in no way limited to, e-commerce websites, and their own website is full of “success stories” from the work that they did with very well-known international brands.

After adding FingerPrint’s service to a website, users’ mouse movements get automatically quantified, amassed, and algorithmically analyzed. They are then visualized and automatically processed into a psychological and behavioral “understanding” of users. As explained in one of the company’s promotional videos: “Our product provides a digital body language for enterprise clients to understand their users.” The company’s website similarly states that FingerPrint “transform[s] digital body language into experience wins.”

This “user understanding” is presented to FingerPrint’s clients as detailed reports that include “path analytics,” quite literally––visual “heat maps” of the paths users take with their mice across the client’s webpage as well as psychological and behavioral insights algorithmically derived from these maps. The reports are presented to clients by FingerPrint’s human analysts who help them restructure their websites to “better fit” users’ newly discovered traits and change their “online experiences” (or gain “experience wins,” as they put it) in an attempt to ultimately affect their choices. As Uri, an executive at FingerPrint, explained:

We try to understand what motivates users in their choices […]. The idea is that a better understanding [of users] leads to better [user] experience. And, yes, that better experience leads to better conversion rates. […] Our clients aren’t interested in making their users buy what they intended to buy in the first place. They want their users to expand their purchases, to buy things they didn’t mean to buy. And that’s exactly what we give them.⁴

Uri offers a clear depiction of FingerPrint’s business model and hints at the guiding principles behind the algorithmic construction of users’ choices. FingerPrint’s algorithmic analysis creates a renewed “understanding” of users that is used to change their online experience and hence try to affect their choices and actions. In other words, FingerPrint’s algorithmic insights help their clients tweak their websites, personalize them, and shape users’ “ecologies of choice” in order to nudge them (Thaler and Sunstein 2008; Yeung 2017) toward specific choices.

In describing this process, Uri uses a key trope in the data analytics industry––“conversion rate.” Stemming from online marketing, conversion rate is a quantified measure that is usually defined as “the percentage of users who take a desired action.” The desired action is determined by the website owners or, in this case, by FingerPrint’s clients. Accordingly, this action can be any measurable activity in the website––clicking on a download button, clicking on an ad, completing a purchase, and so on. That is, as the allusion to the religious change of faith suggests, the conversion rate measures algorithms’ ability to change users’ behavior from one state to another and their ability to make users choose what the website owners want them to choose.

The conversion rate is what I term a quantified choice index––a way of assessing whether users have behaved as someone “desired” them to and whether the algorithmic nudge has worked. In this case, as Uri explains, FingerPrint’s algorithms are aimed at converting people into high-spending consumers––making them buy things they never intended to buy. But this index, much like the algorithms it measures, is highly flexible––users’ “desired choices” change from one client to the next. Hence, FingerPrint’s algorithms are programmed to “convert people into choosers,” but it is their clients who decide which choice counts.

Thus, FingerPrint’s algorithms seem to act as “intermediary agents”––standing between high-tech companies and their users. They can be described as “acting at a distance” (Latour 1992, 151), on behalf of FingerPrint’s clients, or as “proxy agents” (Neff and Nagy 2016, 4917) that fulfill clients’ wants and needs. At the same time, the conversion rate acts as a “calculative agent” (Callon and Muniesa 2005, 1237) that not only measures the success of choice-inducing algorithms but that also embodies and affects the relationship between the companies who devise the algorithms and those who use them.

However, algorithms are themselves socio-technical assemblages (Gillespie 2016, 22) with often very local socio-technical histories. I asked FingerPrint’s lead analyst, Keren, how they created their user insights algorithm. She explained:

We just sat there and started watching recordings [of mouse movements] from different devices, and from different “verticals”––different industries. I started noticing the recurring patterns when I realized that in a couple of seconds, I can tell exactly what they [the users] are going to do. And the patterns just kept reoccurring––always five distinct patterns––so I called them “behavioral patterns.” That’s the users’ way to communicate meaning to us […], that’s your body language in the digital world. I mean, your thoughts, your cognitive orientations, are being translated into movements. And then I said: “if I can identify these five models […] maybe we can write an algorithm that would do the same thing!,” and we did! We just let the algorithm run, and saw that at an 85% confidence level, we can put a bullseye on every…once a user goes into the site, we can identify his mindset.

Keren’s words shed light on the construction of one of FingerPrint’s key algorithms. While, as we have seen above, FingerPrint provides automatic user analytics, Keren describes what I term the construction of automation––the ways in which her intuitive insights, after watching hours of mouse movements, were translated (Latour 1992, 154-55) into a rule-based algorithm. Seeing mouse movements as “digital body language” is key to this act of translation. According to this epistemic standpoint, mouse movements mean much more than cursors randomly moving across the screen––it is how users communicate their thoughts and “cognitive orientations” to Keren and her company. This view extends the conventional view of the body––which now expands to the computer screen and beyond––and it particularly expands the conventional view of bodily communication. Users’ prosaic and often unconscious mouse movements are reinterpreted as meaningful and almost deliberate communicative gestures. This reconceptualization of the human body enables to commensurate mouse movements with people’s alleged thoughts and feelings, categorize them into clear and distinct “behavioral patterns,” and, eventually, make these patterns algorithmic. That is, Keren’s unorthodox interpretations were “displaced” (Latour 1992, 154-55) into FingerPrint’s algorithms, transforming a major human effort (Latour 1988, 299) into a minor, automated, and scalable one. Thus, FingerPrint’s algorithm may be an agent in a swarm of agents––playing a role in redirecting people’s choices. But in this case, Keren’s intuitions, and in a sense, her personality and epistemology, have been “folded” into this algorithm and are now automatically and recursively run in an attempt to affect users’ choices. In other words, choice-inducing algorithms stem from the specific people who devise them, and as such, they are the coded manifestation of specifically positioned human subjects.

Keren’s words also highlight the ties between user modeling and choice modeling. She describes how her qualitative, intuitive episteme gets amalgamated with an algorithmic, quantitative one in the creation of five distinct user categories (or “models”). These categories add stability and understandability to the ever-flowing user data, and they hence render it more useful in attempting to nudge users toward different choices. After all, choice is a basic building block for identity construction (Rose and Miller 2008)––what we choose to do, eat, or buy affects who we are. In the case of data analytics however, it is the other way around––users are constantly being categorized into semi-recognizable identity categories (Cheney-Lippold 2017), only to get algorithmically nudged toward one choice or the other. In the case of FingerPrint, the categorization of users’ mouse movements into a limited number of describable “user models” is key to the company’s ability to sell their choice-inducing services to their customers.

In sum, people’s choices may indeed be influenced by data analytics algorithms, but these algorithms are surrounded by other, specifically positioned agents––the humans who devise the algorithms, those who write them, the analysts who communicate the algorithmic outputs, and the web designers who change users’ online experiences. This agentic swarm (Bennett 2010) includes the high-tech companies that write the algorithms and also their clients who, relying on quantified choice indexes, decide on the choices users would make.⁵ Last, users get nudged toward different choices according to their categorization, which often stems from the automated intuitions of the people who devise the algorithms as well as from a complex amalgamation of people’s epistemologies with algorithmic ones. Nevertheless, as the next example will show, companies are diverse entities, and algorithmic choices are also influenced by actors inside these companies’ clients.

“Overwriting Aileen”––On the Need to Control the Algorithmic Agent

FingerPrint’s case study points at some of the central agents involved in the algorithmic induction of choice. Among the different human and nonhuman agents, it seems that the companies that use the algorithms are as influential as the ones that devise them: they set the conversion goals––decide on the desired user choices, (re)shape the ecologies of choice based on the algorithmic outputs, and effectively dictate users’ choice repertoires as well as their specific ways of choosing (Schwarz 2018). But as Callon and Law (1997) remind us, actors are simultaneously points in a network and networks themselves, both individuals and collectives (p. 174-75). Thus, companies may indeed be actors, but they are also networks comprised of various actors themselves. The example of AI Solutions will point at the ways in which different agents inside the companies who use choice-inducing algorithms can actively direct their functioning.

AI Solutions is a Tel-Avivian start-up company that develops an “AI-powered” personal assistant named Aileen.⁶ As the narrator in their promotional video states:

Aileen brings the power of machine-based decisions to your website. […] You just choose your business goals, and Aileen will know how to get there by deciding, in real time, which content to place at each designated area, for each individual visitor. That’s right! A million users will see a million variations of the same website, […] and it’s based on hard facts, not emotions, nor gut feelings.

Like FingerPrint, AI Solutions is a B2B company that works for different clients. Their AI-powered personal assistant automatically “decides” for their clients how to approach each user and how to personalize their website to better affect users’ choices. As Dudi, the company’s cofounder, explained:

Our clients’ sales managers just take Aileen, put her on their website, give her their banners and say: “Go get me sales! Go rack your brains and bring me transactions.” And she really does! She collects the data, studies it, understands it, and changes [their customers’] choices. She then corrects herself until she manages to up the sales.

I mean, when you add her to your website you don’t tell her “my clients are such and such––young, old, from the center, from the periphery, poor, rich, etc.” You don’t have to sit there and tell her who your clients are, you just let her run. We only have to make sure we feed her with enough data, and then she goes ahead and finds patterns in users’ behavior, all by herself.

AI Solutions refer to their personal assistant as a female––Aileen. This type of chauvinistic personification is common in the personal assistants market (Apple’s Siri, Amazon’s Alexa, and Microsoft’s Cortana are merely the most prominent examples of such AI-powered chauvinism).⁷ But beyond their chauvinism, we see that one of the key creators of this algorithmic system takes its personification quite seriously, consistently referring to it as if it was human––“she” does things, you give “her” things, she “racks her brains,” and so on. This personification is particularly intriguing due to the technological determinism in Dudi’s account. According to him, their algorithms can affect users’ choices almost autonomously with little to no human intervention. Moreover, much like the previous example, Dudi refers to the ties between user categorization and the ability to affect them (or “change their choices”), but in contrast to FingerPrint, Dudi refers to a postdemographic categorization (Rogers 2009), one that leaves the traditional categories (“young, old, from the center, from the periphery, poor, rich”) behind. Instead, his autonomous system allegedly creates much more flexible and much more numerous user categories (as the narrator in AI Solutions’ video states: “a million users will see a million variations of the same website”). This ability offers a much more specific way of user “understanding,” one that promises to autonomously, and significantly, “up the sales.” And indeed, Dudi sees great importance in the autonomous capabilities of his system. When asked about his company’s competitive edge in the already crowded field of personalization engines, he explained:

It’s in the autonomy. She is the only true autonomous machine. FingerPrint produces reports; TechnoSphere enables you to [manually] build rules; other companies let you run tests. You, as a human, can take these abilities, run them, make an effort to understand their results, and eventually do something about them. Then you may increase your sales. But I mean, they all leave the execution to humans. Aileen is the only machine that takes this complex process and promises an end-to-end automation. That’s why she has AI in her name.

Much like his previous quote, Dudi refers to their algorithmic system as one that can automatically and autonomously categorize users, offer them personalized suggestions, and hence increase sales. He contrasts his company with their competitors to signal that this choice-inducing system is almost human-free and can operate with little to no human interference. In this case, then, the algorithmic system is portrayed as an autonomous and powerful agent that does much more than merely repeat human-made predetermined rules (as FingerPrint’s algorithm does). This algorithm allegedly studies, understands, acts, and corrects itself if necessary. It seems, then, that the use of AI encourages an especially technologically deterministic discourse and, accordingly, a particularly simple depiction of the agents involved in users’ choices. According to Dudi’s narrative, this agentic swarm contains the user, an AI-powered algorithm, and the data fed to the algorithm, not much else.

But a closer examination of “Aileen” tells a different story. I asked Dudi about their clients’ responses to Aileen. He explained that the people who actively decide to use his company’s services, the ones who eventually use them, are usually from their clients’ marketing departments and that, he claims, makes a difference. He elaborated on this point in an interview:

Our clients are often control freaks––they don’t really care about results. For them, it’s all about them, and their position in the organization. And they’re just threatened by our tool. They worry that now that there’s a robot that works far better than they do, they’ll get fired, that it will make them seem useless. […] So we gave them an ability to overwrite Aileen, so that they could feel in control. If, for example, they want to present some special content during the holidays, they just tell her: “Aileen, during the holidays show this and that item.” We gave them the ability to engage, to intervene, in Aileen’s decisions, to ask her to show specific promotions, and to tell her: “Aileen, you have to do this, you cannot do this”; to control her a little bit and tell her: “these are the goals, that’s the destination, go!”

Beyond highlighting the extent of this AI-powered chauvinism, this account stresses once again the role data analytics companies’ clients play in affecting the structure and functioning of their algorithms and hence in trying to actively affect users’ choices. Dudi explains that the people who work with Aileen were reluctant to work with an autonomous algorithmic system or “robot.” According to him, their reluctance stemmed from their personal and occupational situation––from their fear of losing their job to an algorithm. That is, their specific social position, in light of the threat of being made redundant by algorithmic agents, had made them ask for more power over this algorithmic system and, respectively, for more direct power over their users’ choices. With the change in the algorithm, they could now choose what the users would see and manually construct their ecologies of choice.

As Bijker (1995) argued, machines “work” only when, and because, they have been accepted by relevant social groups (p. 270). In this case, people in specific sociocultural contexts, the specific meanings they ascribed to the algorithmic agent, and, particularly, their fears in light of their looming occupational future have dictated some crucial additions to this algorithmic system, ones that would make it more controllable, more tamed, and, ironically for a personified personal assistant, less human. While AI Solutions’ algorithms are said to be autonomous agents that offer “machine-based decisions” based on “hard facts,” the people who work with this algorithmic system demanded to “de-personify” it, to gain control over it––to actively and personally participate in the construction of users’ choices, and, in effect, to join this agentic swarm.

Thus, choice-inducing algorithms stem from specific organizational contexts and are affected by interorganizational relationships. The ties between data analytics companies and their clients as well as the ties between individual people inside those companies––with their meaning systems, world views, and even emotions––can have considerable effects on the design and functioning of choice-inducing algorithms, on users’ ecologies of choice, and, hence, on the choices such algorithms are programmed to yield. But interorganizational relationships are not limited to the ties between high-tech companies and their clients. In fact, high-tech companies and their algorithms are often crucially dependent on much bigger companies and, specifically, on their data.

Data Giants, Data Ethics

In a Big Data Conference held in Jerusalem in the summer of 2017, Dov, a senior investor in one of Israel’s top venture capital funds, talked about the current state of Israeli start-ups. He said:

Israeli start-ups’ edge is no longer nested in their algorithms. Algorithms are now publicly available on GitHub.⁸ They can no longer be seen as a real advantage. Today’s edge is in the data. To make a product that works, to see that it really works, to show others that it works, to show us [investors] that it works, you have to have data, and lots of it.

Dov alludes to a fundamental myth about the Israeli start-up culture according to which young Israelis come up with brilliant ideas, form innovative start-ups, and make a fortune by selling them to global corporations (Senor and Singer 2009). These ideas are traditionally embodied in, and their intellectual property rests in, these companies’ algorithms. But according to Dov, writing a brilliant algorithm is no longer enough. Instead, start-ups need to put their hands on data––usually user data––to be able to develop, test, and ultimately sell their ideas. Indeed, my interviewees often talk about their anxiety-ridden quests for more user data, the commercial failures they had had for not having enough “training” or “testing” data, and the peculiar tactics they have used to try to get more data. But data is a flexible term (Crawford, Miltner, and Gray 2014) that describes information from various sources, and hence, two almost identical algorithms can run on very different data sets and drive people into making very different kinds of choices. For example, people’s choices in an e-commerce website are morally, culturally, and practically different from those made in an online gambling site, but they may stem from the same recommendation engine. That is, data can affect the shape, structure, and functionality of choice-inducing algorithms, which can in turn affect people’s choices. In that, data is an important and often crucial factor for the functioning of choice-inducing algorithms—it can deeply affect companies, and its lack can make algorithms, as well as the companies who write them, almost meaningless. But data can also embody the relationships between agents and, particularly, between global technological corporations and smaller, local data analytics companies. The case of Identica offers an illuminating illustration.

Founded in 2011, Identica had initially specialized in profiling users based on Facebook data. People would log on to Identica’s website, or to one of its clients’ websites, using their Facebook login and their data streamed right into Identica’s hands through the Facebook application programming interface (API).⁹ In fact, at the time, Facebook gave access not only to users’ data (including their “likes,” pages, groups, locations) but also to their friends’ data. And so, a single login would potentially have yielded data about hundreds of users. That is, while the companies I discussed earlier primarily use data from their clients’ databases (their mouse movement, previous purchases, etc.), or from publicly available data, Identica had based their algorithmic analysis on one of the richest and most detailed data sources available––Facebook.

After its collection and aggregation, the data were used to profile users into different social, psychological, stylistic, and behavioral profiles, using a mix of structured and unstructured machine learning algorithms, and those profiles were then automatically presented to Identica’s clients. As their cofounder explained in an interview, instead of serving one set of clients, from one industry, Identica aspired to create a “generic personalization engine” that could be sold to any company, from any industry. These so-called universal profiles were supposed to help change people’s choices wherever they went.

Identica’s first few years looked promising. They received a generous investment from a renowned venture capital fund and a considerable amount of interest from clients. But then, Facebook changed the rules. As Lior, Identica’s lead data scientist, recounted in a 2015 interview:

Relying on Facebook data, on their API, posed a big problem for us. By the end of May [2014], Facebook updated their API version to ban any use of their data unless you demonstrated that you use it to your “users’ benefit.” We had to prove that it works in favor of our users! And they [Facebook] actually started to review all the applications, checking how we use the data, when, in what ways…and authorizing them one by one! […] This dependence on Facebook data was very problematic, Facebook could just wake up one day, decide they don’t want to give us any more data, and that’s it. We’re done.

Lior describes the complex power structure behind Identica’s activities as well as the challenges that stem from such a structure. Echoing Dov’s words from the beginning of this section, Identica was not only dependent on their algorithms or their clients but also on a tech giant––Facebook––which challenged Identica’s functioning to a point of threatening their existence. This asymmetrical relationship, between a small Israeli start-up company and a global technological giant, is embodied in the once uninterrupted flow of user data. This flow was, and still is, routinely regulated through the Facebook API, and its nature is determined by Facebook’s changing, and often vague, policies: Facebook’s dynamic perception of “users’ good,” or of what “benefits users,” is often changing according to shifting cultural, political, geopolitical, or economic circumstances (Gillespie 2018), and it accordingly changes the velocity, variety, and volume of the data that flow through their API.

That is, Facebook’s API works as a power juncture that establishes the connections between companies but that also regulates, and often threatens the very base of, this relationship. With a change to the API, companies find themselves in an urgent need to readjust––to change the ways their algorithms operate; the way they approach, profile, and eventually nudge their users; or the clients they work with altogether. As Giora, one of Identica’s cofounders, explained:

Facebook started giving us less and less data. They changed their API and now they don’t even give us [data about users’] friends. In the past we used to know everything about them, and now we don’t. For example, they changed the way you see [users’] location. Previously, [users] used to “check in” to places and now people add their “location” when they write a post, but we can’t see it anymore! And so, we can no longer identify the “Travelers” [one of Identica’s lifestyle profiles]. So, Facebook really narrowed us down […] and at a certain stage you have to be mature enough and say: “oops, the data is not there anymore, what do we do?” In this case, one of our investors suddenly said: “I can put you in touch with a telecommunication company!”

As Giora recounts, the change to Facebook’s API changed the types and amounts of data they could get and, hence, their ability to access users, profile them, and try to affect their choices. Under the circumstances, and with their investor actively looking for a way out, Identica had to find a new identity. And they did. Following their investor’s advice, Identica partnered up with a big telecommunication company in Southeast Asia, which offered them a new and somewhat controversial data type––DPI or deep packet inspection.¹⁰ With DPI, Identica gave up on its plans to create “universal user profiles,” and instead, it started to try and induce choices based on much more specific behavioral analytics. As Giora explained:

DPI…that data is really scandalous data. […] That’s just spying! And the things you get out of it, the things I see, that’s a big deal! […] The cellular antennas tell us where you are, the servers tell us what you do with your phone. […] And you can get very high-res with it––you can see text [messages], social [apps], voice [calls], videos, how much time people spend on each app! […] Companies usually use this kind of data for network optimization, not for spying, but now, this data can connect to…think about it––location, social [media]…everything! That’s just crazy.

Giora describes how, following a change to Facebook’s API and with the recommendation of their investor, Identica had shifted its course and went out to find new clients, work with new types of data, and produce different types of user analytics. DPI allowed them to follow users more closely, monitor their interests and whereabouts with greater precision, and, hence, tailor recommendations that would allegedly “better suit them.” With this “hyperindividualist” view of users (Lake 2017, 3-4), Identica’s algorithms were expected to much more easily convert people into choosers.

But Facebook’s infrastructural change had not only led Identica to choose a new technological direction, it had also nudged it toward a new ethical path. Giora conveys how, after getting cutoff from Facebook, they started to work on “scandalous” data, and instead of creating generic profiles that would affect users through “universal” psychological and social attributes, Identica moved to offer a much more specific and a much more invasive view on users, described by Giora himself as “spying.” That is, following Facebook’s request to use their data in a way that “benefits” users, and in an attempt to stay in the business of changing users’ choices, Identica moved into a deeper, and much more controversial, type of dataveillance (Clarke 1988).

Thus, data are crucial for the development, testing, and sustainment of choice-inducing algorithms and hence for the survival of companies that develop such algorithms. But data also flow between agents, its flow embodying and shaping their relationships, their ethics, the intrusiveness of their algorithms, and the tactics they use to affect people’s choices.

Conclusion: The Socio-algorithmic Construction of Choice

Algorithms play a key role in contemporary choice-making. As the burgeoning literature on algorithms has shown, their automaticity, opacity, and stealth make them powerful agents in designing, restricting, and reconstructing people’s choices (Graham 2018; Cohn 2019; Yeung 2017). But algorithms’ power also stems from their flexibility––from their ability to induce almost any kind of choice. As the examples above have shown, choice-inducing algorithms are not programmed to induce people into making specific choices––make them choose a specific item (e.g., a specific pair of pants), or an item from a specific selection (e.g., the 2020 Gap collection), or even select from a specific market (e.g., men’s clothing). Instead, these algorithms are often used by various clients from a variety of fields. The same algorithm can be used to make people buy shoes or to make them gamble their last penny. That is, rather than aiming to induce specific choices, such algorithms are programmed to more generally convert people into choosers––make them choosing subjects. That is, the flexibility of choice-inducing algorithms, their ubiquity, and the fact that they are perpetually being used to nudge people toward choice-making make them incessant generators of choice that not only aim to produce different kinds of choosers but that also recursively corroborate choice’s role in contemporary life.

Late capitalism, and particularly the move from a society of producers to a society of consumers, has made choice a central pillar of modernity (Bauman 2007; Giddens 1994). But rather than naturally stemming from certain socioeconomic circumstances, choosers had to be actively created. And hence, capitalism’s insatiable need for expansion has afforded the creation of various disciplines, methodologies, and techniques dedicated to the induction of choice––from consumer research (Ward 2009; Cohen 2003) to market research (Schwarzkopf 2008) and from the supermarket aisle to the shopping cart (Cochoy 2007). These factors, in line with the neoliberal logic, have actively made people more preoccupied with choosing, have made them choose more, and have primarily made them buy more (Bauman 2007). And so, while choice-inducing algorithms are not unprecedented in their relation to choice and are not the first factors that take part in constructing the “homo eligens” (Bauman 2007, 61), they are quite certainly the fastest, most ubiquitous, and the most invasive “technology of choice” (Illouz 2012, 177). In that, choice-inducing algorithms are joining more traditional choice-inducing mechanisms in actively (re)making choice a central element in contemporary life. With their speed, flexibility, and ever-growing ubiquity, algorithms are becoming key architects in (re)constructing the modern need to choose (Giddens 1994).¹¹

Nevertheless, while the literature on algorithmic choices tends to highlight the interactions between users and algorithmic systems (Graham and Henman 2019, 14; Cohn 2019; Yeung 2017), this article has shown that algorithmically induced choices in fact stem from the complex interrelations between a variety of human and nonhuman agents in a variety of cultural settings. As the case studies of FingerPrint, AI Solutions, and Identica have shown, the people who devise the algorithms (who take part in the construction of automation), the people who run and test them, the people who buy and use them (and decide on users’ modes of conversion), the people who communicate the algorithmic outputs, and of course, the users who eventually choose are all active agents in the incessant attempt to induce choices. At the same time, choice-inducing algorithms are also affected by the flow of data between companies, the types of data that flow, the APIs that act as power junctures and regulate this flow, and the quantified choice indexes that quantify and commensurate algorithms’ ability to make people choose. Thus, the move toward algorithmically induced choices does not downplay their sociocultural origins. On the contrary, just because these choices are algorithmic, they encapsulate the epistemologies, meaning systems, practices, and choices of many different individuals; they are affected by diverse interorganizational relationships as well as by wide social, political, and geopolitical contexts.

While such a distributive understanding of agency may seem to dilute or even disregard the accountability of specific agents, I argue that it in fact offers a more nuanced understanding of algorithmic powers. Thus, rather than being apolitical, this article shows that algorithms indeed have politics (Winner 1986) but that their politics stems from often very specific sociocultural contexts as well as from specific practices, beliefs, and epistemologies. That is, much like people in their choices, algorithms are never autonomous, and their power is never deterministic. Instead, they are dynamically co-constituted by different agents in different social settings. Identifying these agents, as well as the settings they operate in, will help us better conceptualize, and better react to, algorithms’ rising power.

This article contributes to the growing literature on algorithms and culture (Christin 2018; Ribak 2019; Seaver 2018), and to our understanding of choice-making in contemporary life, by offering a detailed, empirically based account of the socio-algorithmic construction of choice. At the same time, this article provides a new, sociologically informed vocabulary that offers to critically engage with algorithms and their power without losing sight of the often very specific contexts, and often very mundane circumstances, from which they arise.

Future research should focus on how different contexts and different sociocultural circumstances affect the creation of choice-inducing algorithms. Research should also delve into users’ role in this agentic swarm––how users see, understand, and react to such algorithms and into the interplay between users’ agency and that of algorithms. Such research could also shed light on the efficacy of choice-inducing algorithms as well as on the ways they operate in different sociocultural fields.

Footnotes

Acknowledgments

I am deeply grateful to the editors and anonymous reviewers for their useful and constructive comments on this article. I would also like to thank Chen Bar-Itzhak, Eva Illouz, Ori Schwarz, Nicholas John, Rivka Ribak, Noortje Marres, Guy Hoffman, and the participants of the 2018 New Agents and Agencies workshop at the Van Leer Jerusalem Institute for their insightful comments on earlier versions of this article. Finally, I thank my interlocutors for providing a fascinating glimpse into their work.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Dan M. Kotliar

Notes

References

Adam

Alison

. 1998. Artificial Knowing: Gender and the Thinking Machine. New York: Routledge.

Anscombe

Gertrude E.M.

1957. Intention. Cambridge, MA: Harvard University Press.

Archer

Margaret S.

2000. Being Human: The Problem of Agency. Cambridge, UK: Cambridge University Press.

Barandiaran

Xabier E.

Paolo

Ezequiel Di

Rohde

Marieke

. 2009. “Defining Agency: Individuality, Normativity, Asymmetry, and Spatio-temporality in Action.” Adaptive Behavior 17 (5): 367–86.

Bauman

Zygmunt

. 2007. Consuming Life. Cambridge, UK: Polity.

Beer

David

. 2009. “Power through the Algorithm? Participatory Web Cultures and the Technological Unconscious.” New Media & Society 11 (6): 985–1002. doi: 10.1177/1461444809336551.

Bennett

Jane

. 2010. Vibrant Matter: A Political Ecology of Things. Durham, NC: Duke University Press.

Bijker

Wiebe E.

1995. Of Bicycles, Bakeliters and Bulbs. Cambridge, MA: The MIT Press.

Bourdieu

Pierre

. 2005. The Social Structures of the Economy. Cambridge, UK: Polity.

10.

Braun

Virginia

Clarke

Victoria

. 2006. “Using Thematic Analysis in Psychology.” Qualitative Research in Psychology 3 (2): 77–101. doi: 10.1191/1478088706qp063oa.

11.

Bucher

Taina

. 2013. “Objects of Intense Feeling: The Case of the Twitter API.” Computational Culture 3. Accessed May 5, 2020. http://computationalculture.net/objects-of-intense-feeling-the-case-of-the-twitter-api/.

12.

Bucher

Taina

. 2018. If…Then: Algorithmic Power and Politics. Oxford, UK: Oxford University Press.

13.

Callon

Michel

. 2004. “The Role of Hybrid Communities and Socio-technical Arrengements in the Participatory Design.” Journal of the Center for Information Studies 5 (3): 3–10.

14.

Callon

Michel

Law

John

. 1997. “After the Individual in Society: Lessons on Collectivity from Science, Technology and Society.” The Canadian Journal of Sociology 22 (2): 165–82.

15.

Callon

Michel

Law

John

. 2005. “On Qualculation, Agency and Otherness.” Environment and Planning 23 (5): 717–34. doi: 10.1068/d343t.

16.

Callon

Michel

Muniesa

Fabian

. 2005. “Peripheral Vision: Economic Markets as Calculative Collective Devices.” Organization Studies 26 (8): 1229–50. doi: 10.1177/0170840605056393.

17.

Cheney-Lippold

John

. 2017. We Are Data: Algorithms and the Making of Our Digital Selves. New York: NYU Press.

18.

Christin

Angèle

. 2018. “Counting Clicks: Quantification and Variation in Web Journalism in the United States and France.” American Journal of Sociology 123 (5): 1382–415. doi: 10.1086/696137.

19.

Clarke

Roger

. 1988. “Information Technology and Dataveillance.” Communications of the ACM 31 (5): 498–512. doi: 10.1145/42411.42413.

20.

Cochoy

Franck

. 2007. “A Sociology of Market-things: On Tending the Garden of Choices in Mass Retailing.” Sociological Review 55 (2): 109–29. doi: 10.1111/j.1467-954X.2007.00732.x.

21.

Cohen

Lizbeth

. 2003. A Consumers’ Republic: The Politics of Mass Consumption in Postwar America. New York: Random House.

22.

Cohn

Jonathan

. 2019. The Burden of Choice: Recommendations, Subversion, and Algorithmic Culture. New Brunswick, NJ: Rutgers University Press.

23.

Crawford

Kate

Lingel

Jessa

Karppi

Tero

. 2015. “Our Metrics, Ourselves: A Hundred Years of Self-tracking from the Weight Scale to the Wrist Wearable Device.” European Journal of Cultural Studies 18 (4–5): 479–96. doi: 10.1177/1367549415584857.

24.

Crawford

Kate

Miltner

Kate

Gray

Mary L.

. 2014. “Critiquing Big Data: Politics, Ethics, Epistemology.” International Journal of Communication 8 (2014): 1663–72.

25.

Davidson

Donald

. 1980. Essays on Actions and Events. Oxford, UK: Clarendon Press.

26.

Eubanks

Virginia

. 2018. Automating Inequality: How High-tech Tools Profile, Police, and Punish the Poor. New York: St. Martin’s Press.

27.

Floridi

Luciano

Sanders

Jeff W.

. 2004. “On the Morality of Artificial Agents.” Minds and Machine 14 (3): 349–79. doi: 10.1023/B: MIND.0000035461.63578.9d.

28.

Friedman

Batya

Nissenbaum

Helen

. 1997. “Software Agents and User Autonomy.” Agents ‘97 Conference Proceedings 466–69. doi: 10.1145/267658.267772.

29.

Fuchs

Christian

. 2013. “Societal and Ideological Impacts of Deep Packet Inspection Internet Surveillance.” Information, Communication & Society 16 (8): 1328–59. doi: 10.1080/1369118X.2013.770544.

30.

Gershon

Ilana

. 2011. “Neoliberal Agency.” Current Anthropology 52 (4): 537–55.

31.

Giddens

Anthony

. 1982. “Power, the Dialectic of Control and Class Structuration.” In Profiles and Critiques in Social Theory, 197–214. London, UK: Palgrave Macmillan.

32.

Giddens

Anthony

. 1994. “Living in a Post-traditional Society.” In Reflexive Modernization: Politics, Tradition and Aesthetics in the Modern Social Order, edited by Beck

Ulrich

Giddens

Anthony

Lash

Scott

, 56–109. Stanford, CA: Stanford University Press.

33.

Gillespie

Tarleton

. 2014. “The Relevance of Algorithms.” In Media Technologies: Essays on Communication, Materiality, and Society, edited by Gillespie

Tarleton

Boczkowski

Pablo J.

Foot

Kirsten

, 167–93. Cambridge, MA: The MIT Press.

34.

Gillespie

Tarleton

. 2015. “Platforms Intervene.” Social Media + Society 1 (1): 1–2. doi: 10.1177/2056305115580479.

35.

Gillespie

Tarleton

. 2016. “Algorithm.” In Digital Keywords: A Vocabulary of Information Society and Culture, edited by Peters

Benjamin

, 18–30. Princeton, NJ: Princeton University Press.

36.

Gillespie

Tarleton

. 2018. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. New Haven, CT: Yale University Press.

37.

Goffey

Andrew

. 2008. “Algorithm.” In Software Studies: A Lexicon, edited by Fuller

Matthew

, 15–20. Cambridge, MA: The MIT Press.

38.

Graham

Timothy

. 2018. “Platforms and Hyper-choice on the World Wide Web.” Big Data & Society 5 (1): 1–14. doi: 10.1177/2053951718765878.

39.

Graham

Timothy

Henman

Paul

. 2019. “Affording Choice: How Website Designs Create and Constrain ‘Choice’.” Information, Communication & Society 22 (13): 2007–23. doi: 10.1080/1369118X.2018.1476570.

40.

Gunkel

David J.

2014. “A Vindication of the Rights of Machines.” Philosophy & Technology 27 (1): 113–32. doi: 10.1007/s13347-013-0121-z.

41.

Gustavsson

Eva

. 2005. “Virtual Servants: Stereotyping Female Front-office Employees on the Internet.” Gender, Work and Organization 12 (5): 400–19. doi: 10.1111/j.1468-0432.2005.00281.x.

42.

Gustavsson

Eva

Czarniawska

Barbara

. 2004. “Web Woman: The On-line Construction of Corporate and Gender Images.” Organization 11 (5): 651–70. doi: 10.1177/1350508404046455.

43.

Hallinan

Blake

Striphas

Ted

. 2016. “Recommended for You: The Netflix Prize and the Production of Algorithmic Culture.” New Media & Society 18 (1): 117–37. doi: 10.1177/1461444814538646.

44.

Heikkinen

Hannu L. T.

Huttunen

Rauno

Kakkori

Leena

. 2000. “‘And This Story Is True…’ On the Problem of Narrative Truth.” In European Conference on Educational Research, 1–15. Edinburgh.

45.

Hester

Helen

. 2017. “Technology Becomes Her.” New Vistas 3 (1): 46–50.

46.

Illouz

Eva

. 2007. Cold Intimacies: The Making of Emotional Capitalism. Cambridge, UK: Polity.

47.

Illouz

Eva

. 2012. Why Love Hurts: A Sociological Explanation. Cambridge, UK: Polity.

48.

Introna

Lucas D.

2016. “Algorithms, Governance, and Governmentality: On Governing Academic Writing.” Science, Technology & Human Values 41 (1): 17–49. doi: 10.1177/0162243915587360.

49.

John

Nicholas A.

Nissenbaum

Asaf

. 2019. “An Agnotological Analysis of APIs: Or, Disconnectivity and the Ideological Limits of Our Knowledge of Social Media.” The Information Society 35 (1): 1–12.

50.

Klinger

Ulrike

Svensson

Jakob

. 2018. “The End of Media Logics? On Algorithms and Agency.” New Media & Society 20 (12): 4653–70. doi: 10.1177/1461444818779750.

51.

Lake

Robert W.

2017. “Big Data, Urban Governance, and the Ontological Politics of Hyperindividualism.” Big Data & Society 4 (1): 1–10. doi: 10.1136/jnnp.50.1.66.

52.

Lash

Scott

. 2007. “Power after Hegemony: Cultural Studies in Mutation?” Theory, Culture & Society 24 (3): 55–78. doi: 10.1177/0263276407075956.

53.

Latour

Bruno

. 1988. “Mixing Humans and Nonhumans Together: The Sociology of a Door-closer.” Social Problems 35 (3): 298–310. doi: 10.3366/ajicl.2011.0005.

54.

Latour

Bruno

. 1992. “Where Are the Missing Masses? The Sociology of a Few Mundane Artifacts.” In Shaping Technology/Building Society: Studies in Sociotechnical Change, edited by Bijker

Wiebe E.

Law

John

, 225–58. Cambridge, MA: The MIT Press. doi: 10.2307/2074370.

55.

Latour

Bruno

. 1996. “On Actor-network Theory: A Few Clarifications.” Soziale Welt 47 (4): 369–81.

56.

Latour

Bruno

. 2011. “Networks, Societies, Spheres: Reflections of an Actor-network Theorist.” International Journal of Communication 5 (2011): 796–810.

57.

Lessig

1999. Code and Other Laws of Cyberspace 2.0. New York: Basic Books.

58.

Lewis

Seth C.

2015. “Journalism in an Era of Big Data: Cases, Concepts, and Critiques.” Digital Journalism 3 (3): 321–30. doi: 10.1080/21670811.2014.976399.

59.

MacKenzie

Donald

. 2008. An Engine, Not a Camera: How Financial Models Shape Markets. Cambridge, MA: The MIT Press.

60.

Morris

Jeremy W.

2015. “Curation by Code: Infomediaries and the Data Mining of Taste.” European Journal of Cultural Studies 18 (4–5): 446–63. doi: 10.1177/1367549415577387.

61.

Napoli

Philip M.

2014. “Automated Media: An Institutional Theory Perspective on Algorithmic Media Production and Consumption.” Communication Theory 24 (2014): 340–60. doi: 10.1111/comt.12039.

62.

Napoli

Philip M.

McGannon

Donald

. 2013. “The Algorithm as Institution: Toward a Theoretical Framework for Automated Media Production and Consumption.” In Media in Transition Conference. Cambridge, MA: Massachusetts Institute of Technology.

63.

Neff

Gina

. 2018. “Does AI Have Gender?” Oxford Internet Institute Lecture. Accessed May 4, 2020. https://www.oii.ox.ac.uk/videos/does-ai-have-gender/.

64.

Neff

Gina

Jordan

Tim

McVeigh-Schultz

Joshua

Gillespie

Tarleton

. 2012. “Affordances, Technical Agency, and the Politics of Technologies of Cultural Production.” Journal of Broadcasting & Electronic Media 56 (2): 299–313. doi: 10.1080/08838151.2012.678520.

65.

Neff

Gina

Nagy

Peter

. 2016. “Talking to Bots: Symbiotic Agency and the Case of Tay.” International Journal of Communication 10 (2016): 17.

66.

Neyland

Daniel

. 2015. “On Organizing Algorithms.” Theory, Culture & Society 32 (1): 119–32. doi: 10.1177/0263276414530477.

67.

Noble

Safiya U.

2018. Algorithms of Oppression: How Search Engines Reinforce Racism. New York: NYU Press.

68.

O’Malley

Pat

. 1996. “Risk and Responsibility.” In Foucault and Political Reason: Liberalism, Neo-liberalism, and Rationalities of Government, edited by Barr

Andrew

Osborne

Thomas

Rose

Nikolas

, 189–207. Chicago: The University of Chicago Press.

69.

O’Neal

Cathey

. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. New York: Broadway Books.

70.

Overmann

Karenleigh A.

2017. “Thinking Materially: Cognition as Extended and Enacted.” Journal of Cognition and Culture 17 (3–4): 354–73. doi: 10.1163/15685373-12340012.

71.

Pasquale

Frank

. 2013. “The Emperor’s New Codes: Reputation and Search Algorithms in the Finance Sector.” Governing Algorithms Conference, New York, USA.

72.

Peacock

Sylvia E.

2014. “How Web Tracking Changes User Agency in the Age of Big Data: The Used User.” Big Data & Society 1 (2): 1–11. doi: 10.1177/2053951714564228.

73.

Reichertz

. 2013. “Algorithmen Als Autonome Akteure?” SozBlog. Accessed May 4, 2020. http://blog.soziologie.de/2013/02/algorithmen-als-autonome-akteure/.

74.

Ribak

Rivka

. 2019. “Translating Privacy: Developer Cultures in the Global World of Practice.” Information, Communication & Society 22 (6): 838–53. doi: 10.1080/1369118X.2019.1577475.

75.

Rogers

Richard

. 2009. “Post-demographic Machines.” Walled Garden 38 (2009): 29–39.

76.

Rose

Nikolas

. 1990. Governing the Soul. London, UK: Routledge.

77.

Rose

Nikolas

. 1991. “Governing by Numbers: Figuring out Democracy.” Accounting, Organizations and Society 16 (7): 673–92. doi: 10.1016/0361-3682(91)90019-B.

78.

Rose

Nikolas

Miller

Peter

. 2008. Governing the Present: Administering Economic, Social and Personal Life. Cambridge, UK: Polity.

79.

Sayes

Edwin

. 2014. “Actor–Network Theory and Methodology: Just What Does It Mean to Say That Nonhumans Have Agency?” Social Studies of Science 44 (1): 134–49. doi: 10.1177/0306312713511867.

80.

Schwarz

Ori

. 2018. “Cultures of Choice: Towards a Sociology of Choice as a Cultural Phenomenon.” The British Journal of Sociology 69 (3): 845–64. doi: 10.1111/1468-4446.12305.

81.

Schwarzkopf

2008. “Discovering the Consumer: Market Research, Product Innovation, and the Creation of Brand Loyalty in Britain and the United States in the Interwar Years.” Journal of Macromarketing 29 (1): 8–20. doi: 10.1177/0276146708327615.

82.

Seaver

Nick

. 2017. “Algorithms as Culture: Some Tactics for the Ethnography of Algorithmic Systems.” Big Data & Society 4 (2): 1–12. doi: 10.1177/2053951717738104.

83.

Seaver

Nick

. 2018. “What Should an Anthropology of Algorithms Do?” Cultural Anthropology 33 (3): 375–85. doi: 10.14506/ca33.3.04.

84.

Senor

Dan

Singer

Saul

. 2009. Start-up Nation: The Story of Israel’s Economic Miracle. New York: Grand Central.

85.

Seyfert

Robert

Roberge

Jonathan

. 2016. “What Are Algorithmic Cultures?” In Algorithmic Cultures: Essays on Meaning, Performance and New Technologies, edited by Seyfert

Robert

Roberge

Jonathan

, 1–26. New York: Routledge.

86.

Striphas

Ted

. 2009. The Late Age of Print: Everyday Book Culture From Consumerism to Control. New York: Columbia University Press.

87.

Sullins

John P.

2006. “When Is a Robot a Moral Agent?” Machine Ethics 6 (2006): 23–30. doi: 10.1017/cbo9780511978036.013.

88.

Swan

Melanie

. 2013. “The Quantified Self: Fundamental Disruption in Big Data Science and Biological Discovery.” Big Data 1 (2): 85–99. doi: 10.1089/big.2012.0002.

89.

Thaler

Richard H.

Sunstein

Cass R.

. 2008. Nudge—Improving Decisions about Health, Wealth, and Happiness. New Haven, CT: Yale University Press.

90.

Totaro

Ninno

. 2014. “The Concept of Algorithm as an Interpretative Key of Modern Rationality.” Theory, Culture & Society 31 (4): 29–49. doi: 10.1177/0263276413510051.

91.

Tufekci

Zeynep

. 2015. “Algorithmic Harms beyond Facebook and Google: Emergent Challenges of Computational Agency.” Journal of Telecommunication & High Technology 13 (2015): 203–18. doi: 10.1525/sp.2007.54.1.23.

92.

Turow

Joseph

. 2012. The Daily You: How the New Advertising Industry Is Defining Your Identity and Your Worth. New Haven, CT: Yale University Press.

93.

Van Dijck

José

. 2009. “Users Like You? Theorizing Agency in User-generated Content.” Media, Culture & Society 31 (1): 41–58. doi: 10.1177/0163443708098245.

94.

Ward

Douglas B.

2009. “Capitalism, Early Market Research, and the Creation of the American Consumer.” Journal of Historical Research in Marketing 1 (2): 200–23. doi: 10.1108/17557500910974587.

95.

Willson

Michele

. 2014. “The Politics of Social Filtering.” Convergence: The International Journal of Research into New Media Technologies 20 (2): 218–32. doi: 10.1177/1354856513479761.

96.

Winner

Langdon

. 1986. “Do Artifacts Have Politics?” In The Whale and the Reactor: A Search for Limits in an Age of High Technology, edited by Winner

Langdon

, 19–39. Chicago: University of Chicago Press.

97.

Yeung

Karen

. 2017. “‘Hypernudge’: Big Data as a Mode of Regulation by Design.” Information, Communication & Society 20 (1): 118–36. doi: 10.1080/1369118X.2016.1186713.