On Organizing Algorithms

This short paper acts as a comment on Totaro and Ninno's ‘The Concept of Algorithm as an Interpretative Key of Modern Rationality’ (2014) and also introduces some new avenues for exploring the organization of algorithms.

Totaro and Ninno's article explores a common feature of contemporary organizing: algorithms. The authors suggest that algorithms can become central to an organizational logic of category and classification, bureaucracy and formalism. Making sense of the algorithm, they suggest, offers an opportunity to shed light on (or provide an interpretative key for understanding) modern rationality. The mathematical beginnings of algorithms become the starting point for Totaro and Ninno's analysis. Drawing on Cassirer's (1953) work, they argue that mathematics, a logic of numbers, pervades forms of organizing. However, algorithms are not restricted to operating on numbers, but even non-numerical objects such as words and workers become formally and numerically defined and then put into action. The key to the operation of algorithms is, according to the authors, recursive function. An intuitive definition of the recursive function ‘is that of an operation that operates on itself’ (p. 3). Algorithms are recursive in the sense that the previous result is always and already the basis for the next result, with former results becoming nested within later results.

Organizationally, the algorithm has a ‘certain duality’ (p. 21). On the one hand, algorithms play a role in enabling communication at a distance and may even encourage the distancing of human interaction. On the other hand, algorithms become a feature of bureaucratization, hindering social interaction by formalizing and mechanizing organizational members. The mechanization of members of organizations enables a recursive logic – similar to that found in mathematics – to pervade organizations. Detail workers, the article suggests, become subject to the algorithmic logic of recursive functions, ordered into a system which has the central aim of repetition. Bureaucracy then plays the role of transforming production processes into algorithmic logics of order. The move toward algorithmic logics of order then generates contradictions for bureaucratic forms of organizing; the more organizations attempt to apply to humans an organizational/bureaucratic algorithmic logic, the more problems are generated with clashes between the formalism required of the recursive function and the disparities of human activity. In sum, ‘Recursive functions are logically equivalent to … algorithmic processes’ (p. 24), the recursive function is characterized by a logic of operating upon itself, and ‘In modern civilization, the logic of recursive functions is, often unconsciously, the main organizational and cognitive inspiration of production processes, and also … of consumption practices’ (p. 24).

The order of this argument appears initially compelling. The algorithm is noted as central to organizations, algorithms are characterized by the recursive function and so the recursive function becomes a means to make sense of (provide an interpretative key for understanding) contemporary organizing. However, after reading the article I was left with a question: if we accept that algorithms are frequently characterized by a recursive function, should we also accept that recursion provides the most appropriate metaphor for making sense of the organizations employing algorithms or, more broadly, a rationale that underpins contemporary organizing? In order to address this question I will begin with an analysis of algorithms at work, explore their recursive functions and in what ways the organizations that employ algorithms might be said to be recursive. I will then offer a discussion of alternative metaphors that might sit alongside recursion as a means to make sense of organizing algorithms.

Algorithms at Work

In recent discussion of algorithms, concerns have been expressed regarding the apparent power, agential capacity and control that algorithms command of our lives (Beer, 2009; Lash, 2007; Slavin, 2011; Spring, 2011; Stalder and Mayer, 2009). The logic of order, if there is one within these discussions, appears somewhat distinct from that suggested by Totaro and Ninno. The latters' focus on bureaucratic formalism appears at odds with the drama through which algorithms are said to shape our lives (Beer, 2009), with power invested in the algorithm (Lash, 2007), and acquiring the status of a troubling truth – a truth that we are only partially able to read (Slavin, 2011). The algorithms in these discussions, whether recursive or not, are situated within organizational settings and sets of interactions which are anything but recursive. And unlike the organizations considered by Totaro and Ninno which seemed to operate upon themselves, these organizations appear characterized by a rapacious drive to accumulate ever more data and to feed that data into digital infrastructures to be read by algorithms, with new ways constantly sought to create relations between data and create the means to financially exploit these relations.

These non-recursive organizational forms are made most explicit in contemporary organizations such as online media agencies ¹ which develop and utilize algorithms to assess an individual's online activity and choose to buy (or not) advertising space for clients in real-time auctions occurring at speeds beyond human calculation. Alongside considering metaphors of recursion, these contemporary forms of organizing appear to offer opportunities to consider the algorithm as outward looking and involved in creating new means of monetization. Unlike Totaro and Ninno's organization in which the bureaucratic formalism of algorithms failed to accurately model human activity, we find an organization not centrally concerned with understanding human activity; the data is sufficient beyond any concern with an explanatory account of why links between sites have been made, what those links might represent, where the individual might link next or even, they claim, who an individual is beyond their aggregate pattern of activity. The algorithmic politics in this instance would not appear to be of a mismatch between bureaucratic formalism and human action but of monetization – who owns what data and what users might be subjected to as a result of decisions made by a less than accountable agency.

Perhaps the work of Beer (2009), Lash (2007), Slavin (2011), Spring (2011) and Stalder and Mayer (2009) tends toward the dramatic, seeking out broad societal trends at the expense of detailed consideration of specific algorithmic examples. Perhaps the example of online media agencies is also at an extreme of algorithmic organizing. However, a number of recent studies of algorithms at work within and beyond organizational settings have raised questions regarding the role, use and consequence of algorithms. These studies would also appear to call for a broader array of metaphors to sit alongside recursion as a means to understand organizational activity. First, the work of Gillespie on Google's algorithms suggests that important questions involve ‘the choices behind what makes it into an index in the first place, what is excluded … the implications of algorithm providers’ attempts to thoroughly know and predict their users, and how the conclusions they draw can matter … [and] how the algorithmic presentation of publics back to themselves shape a public's sense of itself, and who is best positioned to benefit from that knowledge' (2013: 2). Alongside considering the algorithmic organization as operating on itself, in this example, we can note an algorithmic-organizational politics of inclusion and exclusion stemming from modelling preferred relations with users, designing these models into organizations’ algorithmic practices and the active response made by users.

Second, Introna's research on Turn-It-In plagiarism software algorithms suggests that the central point to understanding the need for plagiarism software lies in the commodification of education. ‘What is at stake for the institutions is not education and learning, in the first instance, but rather the risk to the brand value of their product – likewise for the students that participate legitimately in the exchange' (2013: 9). Except, Introna argues, algorithms cannot detect plagiarism – if by plagiarism we mean presenting another's work as our own. And this is known to the designers and clients of these systems. It is not a failure of the system, nor does it create a tension between the difficulties of modelling human activity and the limitations of mathematical models. Instead, the system highlights copied text – strings of characters that appear in the same order elsewhere. The purpose of doing this is to provide a potential basis for bringing to light educational exchanges that divert and risk reducing the quality of economic exchanges. The resultant politics according to Introna, is not unveiled through the mathematical details of the algorithm. These details are neither relevant nor is their revelation particularly useful. Instead, the political question for Introna is what underpins the move to introduce plagiarism algorithms into educational settings and what consequences follow?

The work of Gillespie (2013) and Introna (2013) suggest that alongside a recursive metaphor we could explore the following: configuration through which users and/or clients are modelled and then encouraged to take up various positions in relation to the algorithm at work; commodification whereby the algorithm sits centrally in sales pitches which aim to inscribe external audiences into a future in which the algorithm and user happily coincide; staging through which users and clients are called upon to witness and corroborate an account of the near future (for more on staging the future of technologies see Brown, 2003; Brown and Michael, 2003; Coopmans, 2010; Smith, 2009; Suchman, 2011); continuous searching for a new stream of data to be collected and codified and made ready for the algorithm to act upon; and linking, generating value by creating connections between, for example, the data prepared by an organization and published data available in the world or between searches, where the more searches that take place, the more data the algorithm has to work on.

Alongside the inward turn of the recursive algorithm, configuration, commodification, staging, searching and linking provide alternative metaphors to account for the practices of organizing algorithms. Furthermore, the organizations introduced above draw together multiple algorithms for multiple practices; there is no single logic of algorithmic use. The organizations are also continually developing, abandoning and instituting new algorithms. Once again there is no single logic of order within the organization, but a constant move between more inward and more external facing modes of operation. And these organizations are dedicated to developing and either providing a service based on their algorithm (such as real-time auctioning of online advertising spaces) or directly selling the algorithm produced (Poon, 2013).

Turning attention to a detailed ethnographic study of organizing algorithms, ² one in which direct attempts were made to model human action through mathematical logics of order, can provide us with a further resource for considering the relevance of a recursive metaphor to account for algorithms and the organizations through which they are put to work. This study involved three years of ethnographic work, engaged with the design, development and testing of a new ‘privacy-sensitive’ algorithmic video-surveillance system. The project was set up in response to growing claims that video-surveillance systems compile ever more data about individuals' lives, creating digital data stores to be searched, tracked, mined and scraped by algorithms oriented toward the creation of ever more associations between data (for more on these surveillance concerns see Lyon, 2001; Davies, 1996; Norris and Armstrong, 1999; Bennett, 2005; Taylor, 2010; McCahill, 2002; Haggerty and Ericson, 2000; Introna and Wood, 2004). The project formed an experimental space in which a large international IT firm, two teams of computer scientists, a technology consultancy, an airport and train operator (as putative end users) and a team of social scientists could explore the possibilities of privacy-sensitive algorithmic video-based surveillance. The project proposed to create a privacy-sensitive system based on a three-part premise: that algorithms could be used to reduce the amount of video-based data made available to surveillance operatives by selecting suspicious activity and only showing that activity to operatives (reducing by 95 to 99% the amount of data made visible), that algorithms could be used to automatically delete all data deemed irrelevant (reducing data stored by 95 to 99%, withdrawing opportunities for data to be mined or scraped) and that no new algorithms would need to be developed for this purpose. A central tenet of this three-part premise was that it was insufficient to achieve these goals in an isolated manner within, for example, a computer lab or design studio. Instead the project sought to explore the possibility of producing an accountable surveillance system. The role of the social scientists in the project was to sceptically assess the three-part premise and claims to accountability, the algorithms in use and the activities of the project team.

The three-part premise required a particular organization of algorithms. First, and in order to reduce the amount of surveillance video data seen by operatives, the computer scientists explored ways to select ‘relevant’ video excerpts to show to surveillance operatives. The form of this ‘showing’ was not inconsequential. The computer scientists sought to reverse the conventions of video-based surveillance and switch from a bank of monitors purporting to show ‘everything’ occurring in a particular locale to using a single monitor on which a series of text-based alerts would appear. These alerts would let operatives know if a relevant event had occurred within the space in which the surveillance system operated and could be clicked on by operatives to reveal a video excerpt which had triggered the alert. Given the participation of an airport and train operator, the alerts initially selected by computer scientists included such actions as crossing train tracks, abandoning luggage or entering private spaces. Switching from comprehensive to narrowly-selected visibility required that the algorithmic system could in some way select relevant video excerpts and provide operatives with an alert. Relevance selection involved combining an object detection system (using background subtraction to decide what a fixed attribute of a setting was and what was new), an object classification system (using 200 factors such as various forms of size and shape measure to assess whether an object that was not a fixed attribute of a setting, was human-shaped or luggage-shaped for example) and an object tracking system (which placed bounding boxes around classified objects and then traced their speed and direction across a screen).

This combination of detection (is there something?), classification (what is it likely to be?) and tracking (how is it moving?) underpinned algorithmic IF … THEN rules which produced alerts for surveillance operatives. The IF … THEN rules for abandoned luggage were as follows: IF a human-shaped object and a luggage-shaped object separated, IF the luggage-shaped object remained stationary, IF it remained more than x metres from its human-shaped object, IF it remained separate from its human-shaped object for more than y seconds, THEN send an alert to operatives (with the x and y as variables to be set and adjusted by the computer scientists in conjunction with train and airport operatives).

The imagined future of organizing this first area of algorithmic work was not inward looking, nor was it designed such that the organization could operate on itself. Instead, the future imagined algorithmic work was organized around a logic of bringing together multiple agencies (airport owners, workers, unions, security operatives, police and data protection agencies), instituting a form of accountability for the privacy-sensitive advances proposed by the algorithmic system and continually searching for more data to create a more effective algorithmic system. The politics of algorithms in this instance appeared to involve questions of the form, outputs, reliability and consequences of the algorithmic system and its attempts at transparency and accountability.

Alongside illuminating the organization of algorithms through the recursive function, a treatment of algorithms as metaphorical account opens up a distinct entangling of production and organizational politics. Counter to the tension suggested by a clash between the recursive bureaucratic formalism and the messiness of human action (Totaro and Ninno, 2014), we might instead look toward an opening up of the unintended consequence (Strathern, 2000, 2002), the accidental productivity (Power, 1997) or the constitution of audience (Neyland and Woolgar, 2002) of the algorithmic account. Alternatively, the account as metaphor might be used to develop a focus on the enabling and constraining agency (Law, 1996) of the organizational algorithm, posing questions not of the relationship between algorithmic abilities to see the truth of any particular matter but instead to interrogate what works gets done (Mouritsen et al, 2001), who and what gets hailed to account (Munro, 2001), the timing and spacing of such accounts (Munro, 2004) and their consequences.

Second, the computer scientists experimented with broadening the selection of ‘relevant’ footage made available to surveillance operatives. Although a stated aim of the project was to reduce the amount of video data made visible to operatives, a concern had been raised by the airport and train operators that simply showing a video excerpt of a stationary item of luggage that had set off an alert would make it difficult for operators to judge the potential relevance of the luggage or give any idea where the luggage had come from, who had carried it and so on. The computer scientists created a probabilistic system to provide further video excerpts of where an item of luggage had come from, the human-shaped object which had most likely last been attached to that object and where the human-shaped object went next. This second area of activity involved combining a learning algorithm, a topological database (of the most popular routes that people took through a space and the average times that it took humans to move from one camera location to another) and object tracking bounding boxes (which generated meta-data on the size of, for example, human-shaped objects, their location, speed and direction). Starting with an image of abandoned luggage, the system would sift backwards through the video stream on that particular camera until the object detection system could pair the luggage with a human-shaped object. From there, the system could then establish a bounding box, its size, location, speed and direction for that human-shaped object and luggage pair. The system could then search for matching bounding boxes on other cameras, using size, speed, direction and location as clues to track the pair as they moved through the space covered by cameras. The system could also take the human-shaped object bounding box and track this forward through time – where did the human-shaped object go after leaving the luggage?

The system was designed to produce continual probabilistic calculations of likely bounding boxes to give to operatives as possible matches for the human-shaped objects they sought. Operatives could then look at images they were offered and accept them as a match (thus building a package of video excerpts, tracing the movement of an object through space) or reject them (in which case the clips would be added to the store of data to be deleted – see below). If the system failed too often to find the correct human-shaped object, adjustments to the topology database or other areas of the system could be made.

This probabilistic system appears to pose distinct questions of the algorithm in contrast with the recursive bureaucratic system which seeks to formally model human activity in order to mechanize human action. Instead, the algorithms here are reproductive, spawning many children (children is the technical term for probabilistic returns produced by the system). Furthermore, the children produced by the algorithms are probabilistic and require further human intervention to be made actionable. The results that the system produced are also subject to continual review, which can even lead to components within the system being re-configured (such as the topology database). When the human operative decides on the irrelevance of an algorithmic child, it is put forward for deletion (see below).

Unlike the recursive metaphor, the focus on probability suggests the need for something less certain, obdurate or fixed. Perhaps alongside recursion it would be possible to conceive of algorithms as uncertain and messy (Law, 2004), characterized by an ongoing fluidity (Law and Mol, 1998) which, rather than undermining the algorithmic system, is key to its endurance. That is, the algorithmic system endures through its continual adaptations, through its results being positioned as probable rather than definitive, through the opportunity the system provides for delegating responsibility for decision-making back onto the human operative. We might then move beyond a singular metaphor of recursive function to what, in Mol's (2009) terms, would amount to an exploration of the ways in which worlds get fluidly added together.

A third step in the development of the algorithmic system involved gathering all the data not seen by operatives along with those clips deemed irrelevant by operatives, and deleting that data. ³ A conventional approach to deleting data would be to simply change the route through which data was retrieved, making it less visible rather than expunging it from a system. However, in this project's experimental set-up, the computer scientists wanted to go further and explore the possibility of deletion as expunging (i.e. completely removing the data from the system and preventing resurrection of the data). Hence the computer scientists decided that the deletion component would work through an auto-overwrite deletion algorithm and associated code that would operate every 24 hours. This would also include deletion of metadata (such as bounding boxes, timestamps for objects detected, and so on) and would produce reports of its own operation.

The success of the overwriting system could be checked manually against the files stored. Were files successfully removed every 24 hours? Were any traces left behind? Was metadata successfully removed? Were any files deleted that should have been stored? In contrast to the recursive metaphor and the reproductive focus of the second system component (the probabilistic algorithm and its children), we have a further metaphor in this third system component: algorithmic deletion. Unlike the inward-looking work of the recursive organization, the work to organize this algorithmic deletion system is a feature of the work to make the organization outward looking; to institute a form of accountability and transparency, as we noted in the first system component.

Further, deletion operates to instantiate a kind of absent-presence (Hetherington and Lee, 2000); a need to be accountable by proving the non-existence (or no-longer existing status) of data. In place of two different forms of presence, was a hybridisation of presence-absence. That is, data that was no longer in the system had to be presented as a kind of blank figure (Hetherington and Lee, 2000), a zero which simultaneously signified nothing (data that no longer existed) and something (the ability of the system to delete and maybe even to prove the marketability of this system – that on behalf of prospective client organizations, the system can take on the tasks of accountably deleting and hence rendering a surveillance system privacy-sensitive). Alongside the seeming finality of recursion, to be judged according to its limitations in modelling action, comes an algorithmic blank figure, a motile disruption and interjection into social orders, generating new provocations and uncertainties, connecting entities previously unconnected. Importantly, through interjections and disruptions, the algorithmic blank of deletion both accomplishes a new kind of order and a new kind of uncertainty; it becomes possible for us to think of the new connections generated through the interjection of the blank figure as an emerging kind of order, even if the blank figure of deletion itself remains ontologically unstable or only partially knowable.

However, the complexity of the system did not end here. The computer scientists also had to figure out a means to get the distinct components of the algorithmic system – most of which preceded system development – to work together and talk to each other. System communication was more complex than the apparent singular logic of the inward-facing recursive algorithm. Within the social world of the interior of the surveillance machine, these algorithmic components could not simply operate on themselves but needed to actively talk to each other.

An Extensible Messaging and Presence (XMPP) protocol was used for communication between the different components of the system in order that, for example, the probabilistic system could send requests and receive responses from the event detection system. These requests took the form of XML messages. Data interchanges would rely on JSON (JavaScript Object Notification) as this was deemed relatively easy for the computer scientists to write and for the system to parse and then generate a response. The different components were linked together using standards developed by ONVIF (Open Network Video Interface Forum) for video streaming. These were particularly attuned to camera networks. Even with this range of technical means for communication decided, further decisions were required on such features as the type of lossy compression desired for streaming video (and those decisions followed previous decisions against downloading whole packets of data). The choice between MJPEG and H.264 compression was a choice between much lower and much higher compression rates for video streams, meaning that the amount of original data stored in each frame was lower or higher and the speed of streamed data was faster or slower.

In this sense, a world of standards, exchange protocols and formats had to be built into the algorithmic system. The system was not inward looking at its conception but rather had to be open to the world (of standards and protocols) that would be built into it. In a similar manner to the city streets of Paris that had to be designed into the electric car (Callon, 1989), or the seafaring conditions that required integrating into Portuguese vessels (Law, 1986) or the contingencies, conditions and anomalies that had to be built into and smoothed out through the speed bump (Latour, 1992), the algorithmic system had to accommodate the world. The socializing algorithmic system had to be drawn together in such a way that a world imagined outside of the system, of conversation and exchange, of sociality and communication, could become the world in here, the social world of the algorithmic system. This world did not end at the limits of standards and protocols, but also had to incorporate the world of the space beyond the cameras. Space had to be configured in such ways that it made sense to the system and could indeed occupy the system. Furthermore, each algorithmic component of the system (event detection, probabilistic children and deletion) could not be limited to operating on themselves; these algorithms required social action – they had to learn to talk to each other, send each other requests, share data and work on joint tasks. The most appropriate languages, forms of communication and means of delivery had to be worked out and implemented. Alongside the inner and singular logic of recursion explored by Totaro and Ninno, here we are presented with an opportunity to explore algorithms as characterized by a social logic of algorithmic interaction, and instead of a singular inward trajectory we have a continual oscillation between constituting the world out there in here, within the algorithmic surveillance system.

On Metaphors and Their Consequence

Totaro and Ninno offer the metaphor of the recursive function as a means to understand not just the algorithm but the organizational work through which the algorithm is put to use and how the algorithm can act as an interpretive key for understanding modern rationality. In this short piece and through drawing on studies of algorithms in action, I have suggested a number of alternatives to the recursive metaphor. These can help to broaden opportunities for engaging with algorithms and the organizations through which algorithms are put to work. Drawing on existing studies, I suggested configuration, commodification, staging, searching and linking might provide compelling metaphors for engaging algorithms. Through the ethnographic study of a video-based surveillance system I offered four further metaphors to broaden our engagement with algorithms. First, I suggested the account might act as a focus for considering the detailed work of algorithmic products, their unintended and uncertain consequence and also perhaps their distributions of responsibility, beyond the formal sense of merely counting an activity to accounting for an activity. Second, fluidity was introduced as a means to shift attention away from the suggestion that algorithms might be assessed solely in terms of their abilities to produce the truth of any matter toward considering the means through which worlds might be probabilistically constituted or added together through the algorithm. Third, absent-presence became a means to consider the productive disruptions of algorithms, the joining and disjoining work accomplished through new kinds of order and uncertainties. Fourth, a notional sociality of the algorithmic system could be utilized to explore the ways in which algorithms, their code, protocols and standards could be conceived as a means to build a world out there into a world in here, in the algorithmic machine.

These are just a few metaphorical possibilities for engaging with the algorithm. We could also explore the algorithm as creative, destructive, corrective and reproductive. But what consequence might we draw from this metaphorical consideration? Drawing inspiration from Totaro and Ninno's efforts to articulate the organizational politics of algorithms, there seems to be an important set of emerging questions regarding organizing algorithms. Who and what is included or excluded, on what terms and to what ends? Algorithmic metaphors provide us with one basis for making associations. These might be associations between aspects of organizational work, organizational work and the worlds to which they aspire, between algorithms, and between algorithms and longer-standing efforts at theorizing the politics of technology. These associations can provide a number of ways to engage with algorithmic politics. However, given the rapidly changing, emerging and disappearing world of algorithms, it appears too early to commit to a singular algorithmic metaphor as an adequate means of association. Hence the multiplicity of metaphors suggested here might provide a number of ways forward.