Artificial intelligence,illegalised mobility and lucrative alchemy of border utopia

Abstract

Artificial intelligence (AI) has captured the interest of academia and a range of industries. Much of this appeal is driven by the obsession with military and political supremacy, and a desire to control people and their movements. This article looks at the expansion and impact of AI systems deployed along physical borders on the mobility of illegalised non-citizens and border security in the Global North. Using the case study of Silicon Valley’s Anduril, the article focuses on the US–Mexico border and assesses AI’s role in hindering illegalised mobility and reconfiguring border control. Two key contributions are made in the paper. First, AI advances must become the focus of border criminologies, examined within the milieu from which the technology emerged. Second, virtual walls are commercial, political and anti-humanitarian. They are opaque, resembling alchemy, flawed but with profound consequences. The article adds to the debate on whether AI experiments should be permitted in border control.

Keywords

AI start-ups alchemy artificial intelligence border illegalised non-citizens virtual wall

Introduction

There seems to be a growing conviction in public discourse that technological advances could ‘completely modify the future course of the humanity’ (Ghimire, 2018: 6). As the Fourth Industrial Revolution unfolds, computers and devices based on automation and artificial intelligence (AI) have captured the interest of academia¹ and a range of industries. Indeed, the technology seems to ignite both utopian and dystopian visions of the future, from advances in medicine and tackling climate change, to inequality-promoting algorithms or even ‘killing machines’ that could threaten the entire human race.

Whether simple or complex in physical design (viewing suggestions vs social learning robots that use natural language and facial expressions), we use smart devices every day. Examples of narrow or weak artificial intelligence such as Amazon’s Alexa are so pervasive and convenient that, even when we are irritated by technology we continue to rely on it, captured by devices’ ease, accuracy and speed. Technology’s capacity to advance from inputs/learning sets, own experience and the environment, as well as its ability to process, analyse and act on a vast amount of data seem unprecedented. But is it? How much of it is hype, powered by profit-hungry venture capitalists (Horgan, 2020; Katz, 2020; Larson, 2021)? Recently, the director of the US Customs and Border Protection (CBP) Innovation Team suggested that ‘there’s [no] organization on the planet that doesn’t want to do something more efficiently using AI’ (Ghaffary, 2020). While such assessments are exaggerations, commentators seem to agree that life in the mid-21st century will fit the formula ‘take X and add AI’. Yet, much of this contemporary interest, as Paul Virilio (cited in Wall and Monahan, 2011) conveys, is driven by the obsession for military and political supremacy, and a growing desire to control people and their movements. Should we, then, continue to develop and deploy AI in all areas of human engagement and governance?

In times of accelerated global flows of people, only temporarily hampered by the COVID-19 pandemic, the promise that decision making by semi or fully autonomous algorithms could expedite authorised and prevent unauthorised mobility remains unabated (Beduschi, 2020). Building virtual instead of physical walls is gaining bipartisan support across the Global North as a cheaper and more humane way to deal with the illegal migration ‘problem’ (Ghaffary, 2020; see also Vukov and Sheller, 2013). The emphasis is on the promise of algorithms in preventing migrants’ arrivals and reinforcing non-entrée policies at border crossings, as well as land and sea borders. By approaching the border, people become targets of a range of systems that perform surveillance and reconnaissance, assess risk, and make a preliminary decision on their lawfulness. However, the assurance of hi-tech borders that ‘enable the fantasy of total security’ (Bourne et al., 2015: 313; or security continuum – Vaughan-Williams, 2010) comes with many caveats: overpromise of technology, discrimination, privacy violations, black-boxing (non-transparency and incoherence), privatisation of border control and security, and ultimately many harms caused to people with no legal means to cross borders.

Method

In this article, I explore the expansion and possible impact of AI systems and devices deployed along physical borders (which I also call border lines) on the mobility of illegalised non-citizens² and border security in the Global North. While externalisation of border control and the expansion of places of bordering beyond the geographical lines of demarcation have been the focus of academic debate in the last couple of decades,³ I bring the attention back to borders as ‘continuous line[s] demarcating the territory and sovereign authority of the state, enclosing its domain’ (Walters, 2006: 193). Using the single-case study of one of the youngest yet leading AI companies in border policing⁴ – Silicon Valley-based Anduril – I zoom in on the US–Mexico border and analyse AI systems’ role in countering illegalised mobility and reconfiguring border control. I selected the US–Mexico border because it is one of the biggest testing grounds for the application of autonomous algorithmic systems in countering unauthorised movement of people, with the private sector at its helm. Given the scarcity of empirical research and contributions on AI enterprises in border security, the paper draws on publicly available news articles, institutional and government reports, policy documents on Anduril from the company’s foundation in 2017 to the time of writing in February 2022, as well as material provided by the company and its co-founder Palmer Luckey on social media.⁵ While artificial intelligence powers a range of frontier technologies, in this article I focus on code, the Internet of things (IoT) and unmanned aerial vehicles (UAVs; drones) given their current and predicted application in border control.

I make two key theoretical contributions to border criminologies. First, I suggest that AI advances must become the focus of border studies, given technology’s likely harmful impact on managing illegalised migration and despite the overpromise of current innovations in this space. Scholarly analysis ought to include the milieu from which the technology emerged: a new order of young, white, conservative, utopian, and often geeky Silicon Valley men, tycoons with commercial ambitions and strong ideology. While privatisation of social control and criminal justice is certainly not new, and while the technology they offer is not (yet) radically different from previous attempts in border control, the prominence of ideologically clad AI start-ups, whose main motivation is not only profit, is novel. New is also the process of knowledge production (Martins and Jumbert, 2020) and the decisive solution for illegalised mobility: AI-powered, smart borders.

Traditionally, Silicon Valley avoided partnerships with the defence and border security sector.⁶ Contrary to their predecessors, new leaders in the field do not shy away from such partnerships or pretend to/simulate partisan neutrality. Quite the opposite: they seek such alliances and are open about their political affiliations.⁷ These new business leaders are also clear about the mission and purpose of technology they develop and deploy at the border: the political and military supremacy of ‘forces of good’ in the new technological world. They secure billion-dollar contracts regardless of who is in the office, by approaching the border as an engineering problem that can be solved (Weston Phippen, 2021), but with an ambiguous definition of what constitutes a ‘success’ in border control (for more discussion on the failure of border technology, see Lisle, 2018). The article offers a critique of this unfolding AI insurrection and tech-solutionism in mobility management in the United States and beyond.

The second key point I make in this paper is that the process of assembling virtual walls is not just commercial and political, but anti-humanitarian. It is also opaque, just like alchemy. Alchemy in this article is understood as a practice of transmuting base metals into gold. Similar to the promise of medieval alchemists, contemporary entrepreneurs claim that, by using AI to process big data at the border, they can achieve a total visibility and increased automation in border control, reaching an ideal of smart and secure borders (Heyman, 2008) at low cost. The final product – an AI-generated ‘full picture’ of the border – is sold to us as gold, transmuted from base metals of border control (border paraphernalia, surveillance devices, border guards). Yet, as I demonstrate below, we have no idea what is actually delivered, or what lies under the bonnet. What we do know, however, is the likely impact of such developments on people on the move and ultimately, on Western democracies. As such, we ought to ask important ethical questions, such as whether AI experiments should be permitted in border control. Critical advances in facial recognition technology and biometrics deployed at border crossings are intentionally omitted from this article, as they are beyond the scope of this inquiry and require a more comprehensive focus than I can offer here. I begin with a brief overview of the significance of AI in the 21st century.

The AI society

Artificial intelligence refers to many things. Experts advise that, when you hear someone talking about AI, you should always ask what exactly they mean by it (Broad, 2018). Computer scientist and mathematician John McCarthy coined the term artificial intelligence in 1956 as ‘the science and engineering of making intelligent machines’ (cited in Goodman, 2016: 469). Physicist Max Tegmark (2017) described AI as a non-biological intelligence, while Russel and Norvig (2016) call it the process of planning and building intelligent agents that receive precepts from the environment and through taking actions change the environment. Yet, some recent attempts reveal just how hard it is to define AI. An European Union White Paper, for example, suggests that AI ‘is a collection of technologies that combine data, algorithms and computing power’ (European Commission, 2020: 2). To avoid confusion, I define existing (weak) AI as a computer system that receives data from the environment and, with a degree of autonomy, acts to achieve complex goals that would otherwise require human intelligence to be completed successfully. Artificial intelligence, thus, emulates a specific human ability, skill or sense (such as calculation, optical or audio recognition) and uses heuristic models of learning to solve the problem with accuracy that is good enough for our current use (Osiński, 2020). In AI, we tell the machine what to do, but not how to do it – or at least not every step of the way. Nevertheless, AI always has a ‘human in the loop’ (Ugwudike, 2021).

Big data enables a mode of artificial intelligence known as machine learning, in which algorithms detect and learn from patterns in a large quantity of data and act automatically and autonomously to find the best solution for the given problem. Given that data created by us and about us doubles every year, and having in mind that AI funding doubles every 2 years (O’Neill, 2020), the machine learning revolution might be just around the corner. Although we are yet (if ever) to see the birth of artificial general intelligence (AGI – strong AI) that understands, learns and performs any intelligent task a human being can with no human input, smart devices powered by weak AI are everywhere. Virtual assistants, facial recognition software, and smart home devices are just some examples of our overwhelming AI reality. They identify objects and faces, learn from experience and data/input, use natural language, and make decisions with a varying degree of human interference.

Similar to other technological advancements, the question of whether specific technology will be beneficial or detrimental to humankind is difficult to answer, until we see it in action (Matthewman, 2011). Artificial intelligence is no exception, although there are warning signs about its current and future use (O’Neill, 2016). The push for a ‘good AI society’ or beneficial AI has been heavily promoted in government, business and policy circles (see Cath et al., 2018). However, contemporary AI is much like a black box, or even alchemy (see Katz, 2020; O’Neill, 2016): a mysterious practice of purifying, transforming and perfecting matter. Just like alchemists in the Middle Ages strived to transmute lead to noble metals, AI’s ‘scientific’ processes and decisions are predicted to solve many of our problems. Similar to alchemy, AI mechanics are obscure and impenetrable to the outsiders’ gaze and understanding (Broad, 2018; O’Neill, 2016) and the outcomes not quite what it was initially promised. While the underperformance of technology has been flagged in previous instalments of border security (Heyman, 2008), the quest for omnipotent yet explainable AI continues. Yet, the more intelligent AI becomes, the more opaque its design turns out to be (Larson, 2021). Critically, as ‘we make [AI systems] more and more complex, we will be less and less able to understand how they arrived at a decision and we’ll be trusting those decisions’ (Kevin Kelly, cited in Danaylov, 2016: Section Kevin Kelly, Subsection The Dangers of AI). Experts and industry leaders do little to dispel this techno-fog: only about 15% of AI studies share their code with experts and the wider community (Horgan, 2020).

Another concern in AI development is the limitation of learning sets and coders.⁸ The foundation of every AI system is data sets from which code learns and develops. If this learning set is skewed or limited (intentionally or unintentionally), or if the people developing technology embed bias into it (again, deliberately, or not), the learning process will likely be skewed. Although we do not know what learning and testing sets developers use in developing AI, we are told to trust the outcome. Finally, the development of AI has been happening in a regulatory void, in which technology, it is argued, serves to advance neo-colonial and capitalist projects by furthering consumerism, sanctioning dispossession and land accumulation and fostering mass incarceration and surveillance (Katz, 2020; see also Joh, 2017; Lemberg-Pedersen, 2018; O’Neill, 2016). The risk that the private sector will fill this regulatory, legal, strategic and research void, push AI development towards their vision of the ‘good AI society’, and take over the sovereign decisions of the border is of utmost importance for border studies (Amoore, 2013; Cath et al., 2018). It is also one of the key concerns I raise vis-à-vis border infrastructures of the future.

From virtual reality to virtual walls: Technology and illegalised mobility in border control

The contemporary state’s preoccupation is to know your identity and movements. A range of techno-social networks established to regulate mobility have been deployed at border lines and beyond to ascertain people’s identity and assist in assessing the lawfulness of mobility projects (see Adey, 2012; Broeders and Hampshire, 2013; Csernatoni, 2018; Jumbert, 2016; Milivojevic, 2019). Sophisticated, increasingly autonomous and profitable devices seek to render people visible and legible, ascertain risk and regulate the flow of citizens, non-citizens, goods and trade (Broeders and Hampshire, 2013; Ceyhan, 2008; Cote-Boucher, 2008).

Given the importance of borders in contemporary political and social contexts, it is hardly surprising that the issue of finding a technological solution for illegalised mobility ‘problem’ is a holy grail of contemporary politics. Countries of origin and transit are the principal sites for the detection, surveillance, and classification of border crossers (Singler, 2021). When such efforts fail, the focus returns to physical borders equipped with scanners, security cameras, sensors and guards who monitor the border and respond to human and non-human threats.

The United States has been at the forefront of this process for over two decades. Surveillance from the sky emerged as a preferred method for strengthening risky borders in the 1990s (for a comprehensive historical overview, see Andreas, 2003; Kanstroom, 2022; Nieto-Gomez, 2014). The first drones patrolled the 2000 miles long US–Mexico border early in that decade (Jumbert, 2016; Koslowski and Schulzke, 2018). After 9/11, the US legislators and some nongovernmental organisations called for an expansion of technology in border control (Koslowski and Schulzke, 2018; Wall and Monahan, 2011).

In the 2000s, the management of the US–Mexico border has seen the quest for ‘system of systems’ – a range of integrated devices such as radars, cameras, mobile robots, and sensing towers (Ceyhan, 2008; Vukov and Sheller, 2013). The first operational system of systems – the Secure Borders Initiative Network (SBInet) – was ‘one of the most ambitious deployments of surveillance technology ever tried in borderlands’ (Nieto-Gomez, 2014: 199), costing the American taxpayer US$3.6 billion in 3 years (2006–2009). It comprised integrated fixed towers with sensors and communication equipment, command centres, aerial surveillance, border patrols and other paraphernalia deployed to thwart the movement of illegalised non-citizens. Yet, the system was riddled with false alarms, ever-rising costs and technological malfunctions. In 2011, the project was shut down as a management failure (Nieto-Gomez, 2014). Nevertheless, development of virtual walls continued, under the pretence of combating crime and saving lives.

Today, the ever-growing border security assemblage includes AI-powered technologies such as digital command platforms, the IoT and mobile robots. They enable a new ‘operative vision’ (Dijstelbloem et al., 2017) at the border that is strategic, schematic, and systematic, but go a step further by producing a ‘composite image’ (Amoore, 2013) of the border, with pre-approved subjects and transgressors clearly outlined on the digital map. The promise of AI in border control is the one of total visibility and increasing autonomy: AI pledges to distinguish humans from animals and objects, prompting humans to observe and act, when needed. In the future, algorithms may act autonomously to remove the threat, with potentially disastrous consequences for people on the move. We can expect a lot more alchemy in the future; transparency, fairness and humanitarianism – although nominally upheld – are not, and are unlikely to be the backbone of virtual walls. These new, supposedly seamless borders have been fashioned by small technology start-ups from the Silicon Valley, led by pioneers of virtual reality.

‘The flame of the West’: Total visibility, automation and privatisation of border infrastructure

Tech-business expansions in areas of social control and corrections have been dogged by controversy. Academia and social commentators questioned whether such a role traditionally reserved for state agencies should be privatised at all, given the morality of profiting from punishment, the quality of services provided in private prisons and the link between privatisation and growing imprisonment rates (for an overview of key issues and recent debates, see, for example, Bean, 2020). The rise of private entities in border control has also been scrutinised in academia (see Andersson, 2016). Early developments saw big businesses such as Boeing, Lockheed Martin, or Israel’s Elbit Systems win multimillion-dollar ‘borderscaping contracts’ (Lemberg-Pedersen, 2013, 2018) to create SBInet at the US–Mexico border (Brustein, 2020; Chambers et al., 2019; Ghaffary, 2020; Vukov and Sheller, 2013).

Recently, there has been a notable change on the market, as small defence start-ups put their hats in the ring. Led by young, white men of conservative affiliations, these new powerbrokers claim to be cheaper, more efficient, and, given AI applications, less reliant on human participation (border guards). Their workforce is also different: there are no unhappy workers, like at Amazon and Google, who rebel against partnerships that enable ‘immoral’ government policies (Ghaffary, 2020).⁹ Below, I investigate a meteoric rise of one such start-ups, and a new leader in the field of border control – Anduril.

Anduril¹⁰ is a defence technology company founded in 2017 by Palmer Luckey, an American entrepreneur, virtual reality pioneer and an enthusiastic Donald Trump supporter (Harley, 2020). His new venture, dubbed ‘tech’s most controversial start up’ (Brustein, 2020), is backed by Peter Thiel,¹¹ a billionaire tech investor, former Trump advisor and co-founder of Palantir Technologies, infamous for its controversial big data analytics software Palantir Gotham, used for predictive policing. Anduril develops and builds hardware and software for military and border security applications. Its tools for detection and ‘target classification’ are autonomous surveillance drones, battering-ram UAVs, sensor surveillance towers, VR headsets, and IoT network. These AI-powered structures are set to transmute data into gold, via a system of systems called Lattice (see Photo 1).

As suggested on the company’s website,¹² Lattice as a ‘seamless networked system’ can help ‘understand the world’ (detect, classify, and track vehicles, persons, drones, or other threats) and enable ‘fast response and save lives’ by identifying risks in real time. Lattice gives us ‘a single autonomous picture, alerting [border] agents in real-time and providing intelligence for rapid and accurate response – whether administrative, medical or tactical’ (Anduril, 2021: my emphasis). The company’s virtual border wall, thus, has three key levels:

Continuous surveillance from sentry towers or drones.

When targets (people, vehicles, objects) are detected, AI systems autonomously process data and identify whether a target is a threat (human).

When humans are identified, border guards receive a push message on Lattice (on mobile phone or a computer), with rendered image and its location.

Altering base metals (the vast amount of unfiltered data, ‘noise’ at the border) into gold (an accurate and instant image of the border, when needed) is critical as ‘every reasonable person (including every politician I have ever spoken with on both sides of aisle) can agree that we need to know what happens on our borders so we can stop the bad guys’ (Palmer Luckey’s Twitter feed, 4 July 2020).

Unlike the previous systems, AI-based interventions slowly push humans out of border surveillance. This was the pitch of Anduril: providing border security for low cost. Technology observes 24/7. It needs no holiday, or a bathroom break. It is cheaper than human workforce and could potentially reduce the size one of the biggest law enforcement agency in the country (Heyman, 2008; Weston Phippen, 2021). Anduril’s towers are solar and mobile. They don’t melt in the harsh desert sun (Photo 1). And critically, there yield a few false positives as AI systems learn from data sets. During a trial, Anduril systems ‘helped customs agents catch 55 unauthorised border crossers, a notable figure for a system still in development’ (Levy, 2018). Three years after it was founded, in July 2020, the company’s worth was estimated at US$1.9 billion, with over 30 contracts in military and border security sector in the United States and the United Kingdom (Brustein, 2020).

Photo 1.

Sentry tower (top) and Lattice.

Anduril’s (2021) Lattice became, as one of the CBP agents suggested, ‘the eyes in the back of my head’. Elimination of drug trade and human trafficking, as Luckey suggested on Twitter, is at the heart of Anduril’s efforts (Palmer Luckey Twitter feed, 4 July 2020). Yet, the technology’s key performance indicators are unknown. Moreover, as one reporter noted after witnessing new AI borders,

[i]t struck me . . . that, aside from the drug smugglers they helped intercept on the border, I had not heard the founders mention the people who might get caught in their omniscient zone. What is the right way to treat those individuals? What of the children and parents who are now being torn apart while crossing? . . . [I]t is increasingly the case that the people who build new technologies trigger political consequences. . . . It now seems obvious that tech was never going to make us better human beings; we are still our flawed selves. Instead, those same technologies that once seemed full of promise are finding their way into all-too-human clashes – led by a company named after an avenging sword. (Levy, 2018: my emphasis)

Muted in the debate are harms experienced by people caught in this and similar political and commercial games, as well as harnessing technology for personal and partisan gains. Stifled is also the vision of future society AI alchemists propagate, under the banner of progress.

Virtual wall’s black-boxed harm: Theorising AI in border security

Artificial intelligence systems’ effect on people on the move, the way we conceptualise borders and our societies is certain, yet unknown. In this section, I first flag two areas of potential modification. What separates AI from other technological advancements in border control (such as CCTV, ground-level radars or thermal camera imaging) is a promise of total visibility at the border and increased automation. Such promise, relational to circumstances in which it arises and situations it addresses (O’Grady, 2021), is increasingly moulded by ideology. The cloak of humanitarianism,¹³ as seen in the case study of Anduril, is the lingo companies invoke, when needed. AI borders, conversely, are deeply non-humanitarian and unlawful, as they prevent illegalised non-citizens from reaching safe countries and expose them to danger and in some cases, death at the border.

Humanitarian technology that could prevent if not eradicate crime and suffering inflicted by traffickers and transnational criminal networks validates the development of security technologies in border policing (Aas and Gundhus, 2014; Franko, 2020; Jumbert, 2016; Lemberg-Pedersen, 2018; Milivojevic, 2019; Molnar, 2020; Sandvik and Lohne, 2014). AI start-ups nominally uphold this narrative. Granted, drones can and do save lives at the border and beyond (Koslowski and Schulzke, 2018). However, the concern identified by scholars and social commentators (Csernatoni, 2018; Jumbert, 2016) is that humanitarian narrative here, similar to other border security technologies, obscures the main objective of hi-tech advances: social sorting and control of non-citizens. The supposed search and rescue element of drone surveillance, or Anduril’s promise to stop traffickers and smugglers, fails to disguise the reality of profit and ideology-driven techno-alchemy.

Experiments in border control are performed by private companies, thus limiting nation states’ responsibility for border deaths or additional collateral damage along border lines (for other contexts, see Lemberg-Pedersen, 2018). Businesses are becoming integral players in developing and upholding migration control agenda (Molnar, 2020), ‘experts’ with very little to no independent oversight. New IT players breed non-transparency. Workers’ opposition or public scrutiny is absent. Alike alchemists of mediaeval times, they speak the ‘expert language’, so that the uninitiated would not understand the ‘powerful knowledge’ they possess. Their technology is largely a business secret, known to a limited few. At Anduril (2021), people who create and modify technology are sought based on their passion for ‘our mission’; ‘to transform United States and allied military capabilities with advanced technology’. Code and systems they design define risk as anything and anyone jeopardising the integrity of the US border, and its military and border security might. In doing so, these new power brokers redefine security at the border and ultimately, the border itself.

Borders have long been a site of visibility (Adey, 2012). The essence of contemporary border interventions is to see and verify people and their movements, via technology. The key question, as Dijstelbloem et al. (2017) suggest, is what kind of seeing is in place in border control, and what are consequences to border crossers. Artificial intelligence systems might appear to have superior, uninterrupted, 24/7 vision of the border. But what they see does not translate to an understanding of data and contexts (for examples see Larson, 2021; Osiński, 2020). In border control, such understanding relates to the bigger picture and the impact of technology on ‘targets’, the border itself and our societies.

Artificial intelligence alters borders to its own logic. The assessment of risk at the border is, while still in the hands of humans, increasingly based on data captured, processed and analysed by a range of non-human technological advancements. This leads us to a likely shift within AI-based border security initiatives – a (yet another) revival of the panopticon as a political tool for mobility governance. While I acknowledge invitations to demolish or reinterpret the panopticon,¹⁴ I would like to revisit it here.

AI-based border security technologies, just like in Bentham’s design of prison (although not so much in Foucault’s famous interpretation of it), focus on the architectural design (not the panoptic subject) of the border. The process of seeing and observing is certain and ongoing (not simply likely or probable). The scaled-down eye of God as ‘an ever present technologically-augmented gaze’ (Wilson, 2018: 59) monitors, simplifies, analyses and streamlines real-time border. Digital eyes of surveillant assemblage (Haggerty and Ericson, 2000) scan the border, communicate, analyse what they ‘see’ and make automated decisions along the way (whether the object is human or not). They render digitised human bodies into what Haggerty and Ericson (2000) call a data double – an entity entirely composed of information, streamlined to Lattice as a part of the ‘total picture’ of the border, and scrutinised for an intervention. Transmuting data into ‘knowledge’ and ‘solution’ for border security ‘problem’ appears complete.

The above process of ingestion – bringing something or someone to attention from the vast amount of data, as opposed to data collection that brings everything to attention – is a critical new moment in border management. Amoore and Piotukh (2015) warned about this phenomenon almost a decade ago. In border control, this means that performative surveillance is accomplished by a single guard in the watch tower based on ingestion of data, done by AI. Guards do not have to monitor the border 24/7, like in the original design of the panopticon. They are prompted to watch, when the process of knowledge production is complete and risk identified as such. There will be, the technology promises, no false positives or omissions. Long gone will be the days when many humans – officers and their supervisors – were needed to determine which event at the border is relevant, and which one is not. Just like in Bentham’s vision of a prison, a single guard will be sufficient to observe and preserve the integrity of borders, seen and streamlined by AI. The technology will, we are told, provide an unobstructed view for the decision making and, in the near future, a decision itself.

What triggers automated interventions? Is every human the same? The one in the uniform and the one with a bottle of water in their hand? Is a colour of their skin relevant at all? How will AI systems of the future identify risk? We do not know. Critically, ‘[n]ot fully understanding how something works has proven to be a fundamental, enduring factor in instances of technological failure’ (Broad, 2018: Section 5: Alchemy). I would like to suggest that a fusion of ignorance, absence of public and expert oversight, and alchemy, as observed in the case study of Anduril, is a recipe for disaster. Yet, apparently, one glance at a glossy system of systems such as Lattice, whether on smartphone or a computer, provides a snapshot of threats and devices in the ‘near real-time border’ (Wilson, 2018). We ought to trust the system – the very one we are not quite sure how it works. And, as Amoore suggests, when the system makes a mistake, said mistake is turned into data that will help rewrite code (Johnson et al., 2011).

Just as Anduril drones have the capability to attack and take another drone off the sky, AI systems of the future are likely to autonomously intercept and push back people they judge as unauthorised. In fact, another AI start-up backed by Peter Thiel, Brinc, had long advertised drones that interrogate and taser migrants at the US–Mexico border (Biddle, 2021). These processes have political and ideological objectives. Palmer Lackey identified some of them in a recent interview: the preservation of Western values and ensuring the supremacy of the American (and US allies) military and border security might (Stebbings, 2020; see also Harley, 2020). Just as its name suggests, Anduril’s aspiration is to uphold the Flame of the West. The border lines are its testing grounds, with other applications to follow.

The visual spectacle of AI borders, with incipient landscapes of sharp, tall surveillance towers and menacing drones, implies violence. Virtual wall’s intimidating and stark façade against the blue desert sky along US–Mexico border is the essence of the new border infrastructure. The very image of such a mighty opponent might be enough, the designers of virtual walls hope, to deter future border transgressions (for deterrence in earlier iterations of border control, see Heyman, 2008). The novel system of systems sends the message to unwanted Others: do not even try to cross the border. In doing so, technology changes the very essence of what border is, not just the border landscape.

Yet, people will keep pushing. The displacement of cross-border movement that leads to border deaths on land and at sea is well documented in border criminology.¹⁵ Virtual borders are no exception. Anduril executives acknowledge that traffic of people across borders move ‘to the east and west of the systems’ (Weston Phippen, 2021). AI borders are ‘a 10-million scarecrow’ (Weston Phippen, 2021) likely to produce a ‘funnel effect’ and more border deaths (Chambers et al., 2019). Illegalised crossers and their allies may siege the towers and beat the system through creativity (such as ‘blinding’ the sentry towers with lasers) and other acts of resistance (de Leon, 2015; Nieto-Gomez, 2014; Vukov and Sheller, 2013; Weston Phippen, 2021). Ultimately, the question of the effectiveness or ‘success’ of AI borders remains unknown, even for start-up executives who are quick to suggest that such questions should be put to border agencies, not technology providers (Weston Phippen, 2021). The question whether the outputs match the promise remains blank for the public and government funders.

What is known, however, is that AI borders dehumanise men, women and children on the move and violate their rights, freedom and privacy. They frame illegalised non-citizens as mathematical problems that must be dealt with using smart technology. By creating mechanical distance (Grossman, cited in Sandvik and Lohne, 2014) between non-humans (code, devices) and humans (guards, demos of the Global North), people are reduced to targets, numbers, collateral damage, transferees, lawbreakers or illegals that need to be detected, restrained and removed. AI systems become political swords for thwarting unwanted mobility. Human cost, so clearly visible in the context of military AI, is obscured in border control, hidden behind the smokescreen of search and rescue, the fight against drug lords, smugglers and human traffickers, and the preservation of the nation.

Conclusion

Artificial intelligence’s promise to revolutionise the humankind is commonly compared to electricity, as the impact of AI-enabled systems could unleash modernising forces. Yet, as Broad (2018) reminds us, electricity also gave us the electric chair and electroconvulsive therapy. I am not suggesting here that the development of algorithmic governance as currently predicted is certain, or always bad.¹⁶ AI could help us become better humans in a better world. But its usefulness might not be the same in every aspect of our lives.

Given the technology’s likely impact on fundamental human rights such as the right to liberty and security, and freedom from torture and inhuman or degrading treatment, we need clear regulatory and legal guidance towards the ‘good AI society’. We ought to steer AI towards the public good, and flag if not prevent any violation of human dignity that requires care and respect for people, individuals or groups, regardless of who they are (Cath et al., 2018). We also need audits (Ugwudike, 2021) and multidisciplinary research ventures that will uncover the black box, compare the outputs with a promise of AI, and reduce alchemy. As this article demonstrates, our ability to study technological innovations in border control is currently constrained, as methods we use only scratch the surface. A clear limitation of this article is a reliance on the company-released material and publications. The need for a more robust, empirical research is evident. Experts such as computer scientists, roboticists, data science and intelligent system engineers, machine learning experts, computer vision and image processing specialists, intelligence reasoning designers and others have to be included in future research on the matter, and new methodology ought to be devised to remove the techno-fog in border control. One possible direction we could pursue is to devise community-led, interdisciplinary methodologies by utilising innovative research collaboration spaces such as Bristol Digital Futures Institute’s Reality Emulator and Neutral Lab.¹⁷

Unfortunately, as my analysis suggests, none of these strategies have been in place in border control. This begs the question: do we, academics and public commentators, argue for cease-and-desist policy in this area, or do we proactively engage in the development of technology to mitigate if not eliminate its negative consequences? While in many other sites of engagement I tend to advocate the latter approach, border control might be a whole different beast.

AI promises new ways of seeing and controlling the border. The role of technology and AI in border control remains unchallenged, if not desired, and the harm they create largely uncharted (Molnar, 2020). As Pamela Ugwudike (2021) suggests, even when harmful consequences arise, AI systems, considered neutral and objective, are not to blame. This narrative should be resisted. I go even a step further by suggesting that automation and algorithmic governance in border control are ideological instruments, developed and deployed to serve specific interests. Promoted and refuelled by ‘the imperial and capitalist projects of its patrons’ (Katz, 2020: 4), AI is currently used in border control to identify and apprehend the Other in places where physical walls are not present. Experts-lobbyists from the Silicon Valley are both knowledge and resolution producers. Private companies offering ‘security solutions’ are no longer neutral capitalists; they are ideology-driven warriors, backed by equally radical venture capitalists. Anduril’s Peter Luckey is simply the first among an army of followers who advertise the idea of seamless AI borders, while enforcing the doctrine of exclusion, profit, and Western military/border security supremacy. Artificial intelligence systems, thus, benefit few –corporate bosses and exclusionary and technology-powerful states of the Global North – while significantly disadvantaging or harming many, in particular illegalised non-citizens and asylum seekers. There is a big reason for pessimism, especially if we aim to stop the development of virtual walls: there is simply too much money to be made, as the case study of Anduril demonstrates.

Seamless AI borders of the future, built by Anduril and other defence start-ups, are akin to alchemy. As the devices (for now) largely fail to transmute base metals into gold and protect the demos, the outcomes are nevertheless grim. As scientist Ali Rahimi (cited in Broad, 2018: Section 5: Alchemy) pointed out, alchemy is fine if you are building photo-sharing service. It is not fine if you are devising systems that govern border control or punishment. Between October 2000 and September 2014, more than 2700 bodies were recovered in the Sonoran Desert alone (de Leon, 2015). How many more will die trying, while circumventing the virtual wall of Anduril?

AI systems in border management are opaque and dangerous. Computers, like humans, make mistakes (see Joh, 2017; Katz, 2020). Even their ‘right’ decisions can be harmful. At first, a humanitarian purpose of the technology might hide it: via technology we could rescue some people in distress and stop a handful of traffickers and smugglers. Ultimately, however, AI borders represent an extension of the imperial ambitions of the West, the desire to preserve our (Western) way of life, and the right to exclude. The slippery slope is obvious. Just as healing was the initial justification for alchemy, and similar to other innovations in the border security assemblage, virtual walls validated by the supposed humanitarian narrative or the need to protect demos are brutal.

We need more of normative evaluation and less alchemy in border control. Normalising black-boxing, obscurity, and non-transparency that goes along with privatisation and automation of border control and ignoring growing influence of politics and ideology that underpins its expansion is set to have devastating consequences. How long do we wait before harm at the border is not only in diverting traffic towards more inhospitable terrains and border deaths? And how long before this target is not just the Other at the border?

Technology is likely to advance ‘futuristic and high-tech security fantasies’ (Adey, 2012: 193) of pervasive, seamless borders that segregate wanted from unwanted mobility. While I limited my analysis here to the US–Mexico border, the developments in Europe, Canada and Australia are also concerning (Aas and Gundhus, 2014; Broeders and Hampshire, 2013; Cote-Boucher, 2008; Csernatoni, 2018; Franko, 2020; Jumbert, 2016; Lemberg-Pedersen, 2013; Martins and Jumbert, 2020; Molnar, 2020). We are beholding ‘a dark step into morally dangerous territory’, and ‘it is only a matter of time before a drone will be able to take action to stop people’ (Noel Sharkey, emeritus professor of robotics and artificial intelligence at Sheffield University, cited in Campbell, 2019).

At the helm of these interventions are venture capitalists and businessmen of a particular political affiliation and with little to no accountability for the consequences of their inventions. As Louise Amoore (2013: 17) noted almost a decade ago, in what is seen as a constant state of emergency, a call to arms has been sent to a range of private actors who now sell the same dream to the state ‘in their promise to make manageable the risky mobile bodies of the global economy’. Privatisation also means that we often do not know how the technology operates or who owns the data (Milivojevic, 2021; Molnar, 2020; Ugwudike, 2021). While businesses have little incentive to address black- boxing or collateral damage of AI borders, they do have a moral (if not yet legal) imperative to act (Beduschi, 2020; Broad, 2018).

Finally, it is not only our attitude towards technology that begs critique. It is our insolence towards the most vulnerable and precarious among us, who are used as guinea pigs and against whom the effectiveness of these and similar technological advancements is assessed. Deconstructing the narrative of migration as a problem and undoing non-entrée policies are critical starting points in the debate, as is debunking the supposed humanitarian nature of virtual walls. We also must be clear about who is benefitting from (or being harmed by) the technology: demos, the majority, or a privileged few. Only then we can look at a promise of technology, including AI, in delivering healthier, fairer, and a just society we strive for.

Footnotes

Acknowledgements

I thank the reviewers for their thoughtful feedback and efforts. This article was much improved thanks to their guidance and suggestions. I greatly appreciate their time and expertise. I thank Professor Mary Bosworth and Samuel Singler from Oxford University for their comments and ideas in the early stages of the development of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Sanja Milivojevic

Notes

Author biography

Sanja Milivojevic is Associate Professor in Digital Futures at the Bristol Digital Futures Institute and School for Policy Studies, University of Bristol. She is also Co-Director of Border Criminologies and Adjunct Associate Professor at La Trobe University, Australia. Her research interests are borders, mobility, security technologies, gender and victimisation, and international criminal justice and human rights. She has published six books and over 60 publications in English and Serbian. Her latest book, Crime and punishment in the future Internet: Digital frontier technologies and criminology in the twenty-first century, was published by Routledge (2021).

References

Aas

Gundhus

HOI

(2014) Policing humanitarian borderlands: Frontex, human rights and the precariousness of life. The British Journal of Criminology 55(1): 1–18.

Adey

(2012) Borders, identification and surveillance: New regimes of border control. In: Ball

Haggerty

Lyon

(eds) Routledge Handbook of Surveillance Studies. London and New York: Routledge, pp. 193–200.

Amoore

(2013) The Politics of Possibility: Risk and Security beyond Probability. Durham, NC and London: Duke University Press.

Amoore

Piotukh

(2015) Algorithmic Life: Calculative Devices in the Age of Big Data. London and New York: Routledge.

Andersson

(2016) Hardwiring the frontier? The politics of security technology in Europe’s ‘fight against illegal migration’. Security Dialogue 47(1): 22–39.

Andreas

(2003) Redrawing the line: Borders and security in the twenty-first century. International Security 28(2): 78–111.

Anduril (2021) Our work. Available at: https://www.anduril.com/work#lattice (accessed 2 March).

Bean

(2020) Criminal Justice and Privatisation: Key Issues and Debates. London and New York: Taylor & Francis.

Beduschi

(2020) International migration management in the age of artificial intelligence. Migration Studies 9: 576–596.

10.

Biddle

(2021) Startup pitched tasing migrants from drones, video reveals. The Intercept, 13 December. Available at: https://theintercept.com/2021/12/13/brinc-startup-taser-drones-migrants/

11.

Bourne

Johnson

Lisle

(2015) Laboratizing the border: The production, translation and anticipation of security technologies. Security Dialogue 46(4): 307–325.

12.

Broad

(2018) Made by Humans: The AI Condition. Melbourne, VIC, Australia: Melbourne University Publishing.

13.

Broeders

Hampshire

(2013) Dreaming of seamless borders: ICTs and the pre-emptive governance of mobility in Europe. Journal of Ethnic and Migration Studies 39(8): 1201–1218.

14.

Brustein

(2020) Peter Thiel-backed surveillance startup Anduril is valued at $1.9 billion. Bloomberg, 1 July. Available at: https://news.bloomberglaw.com/private-equity/thiel-backed-surveillance-startup-anduril-valued-at-1-9-billion

15.

Campbell

(2019) Swarm of drones, piloted by artificial intelligence, may soon patrol Europe’s borders. The Intercept, 11 May. Available at: https://theintercept.com/2019/05/11/drones-artificial-intelligence-europe-roborder/

16.

Cath

Wachter

Mittelstadt

, et al. (2018) Artificial intelligence and the ‘good society’: The US, EU, and UK approach. Science and Engineering Ethics 24(2): 505–528.

17.

Ceyhan

(2008) Technologization of security: Management of uncertainty and risk in the age of biometrics. Surveillance and Society 5(2): 102–123.

18.

Chambers

Boyce

Launius

, et al. (2019) Mortality, surveillance and the tertiary ‘funnel effect’ on the U.S.-Mexico border: A geospatial modeling of the geography of deterrence. Journal of Borderlands Studies 36: 443–468.

19.

Cote-Boucher

(2008) The diffuse border: Intelligence-sharing, control and confinement along Canada’s ‘smart border’. Surveillance and Society 5(2): 142–165.

20.

Csernatoni

(2018) Constructing the EU’s high-tech borders: FRONTEX and dual-use drones for border management. European Security 27(2): 175–200.

21.

Danaylov

(2016) Conversations with the Future: 21 Visions for the 21st Century. Toronto, ON, Canada: Singularity Media.

22.

de Leon

(2015) The Land of Open Graves: Living and Dying on the Migrant Trail. Berkeley, CA: University of California Press.

23.

Dijstelbloem

van Reekum

Schinkel

(2017) Surveillance at sea: The transactional politics of border control in the Aegean. Security Dialogue 48(3): 224–240.

24.

European Commission (2020) White Paper on Artificial Intelligence – A European Approach to Excellence and Trust. Brussels: European Commission.

25.

Franko

(2020) The Crimmigrant Other: Migration and Penal Power. London and New York: Taylor & Francis.

26.

Ghaffary

(2020) The ‘smarter’ wall: How drones, sensors, and AI are patrolling the border. Vox, 7 February. Available at: https://www.vox.com/recode/2019/5/16/18511583/smart-border-wall-drones-sensors-ai

27.

Ghimire

(2018) Future as an object of inquiry: An introduction. In: Ghimire

(ed.) Future Courses of Human Societies: Critical Reflections from the Natural and Social Sciences. Milton: Routledge, pp. 1–25.

28.

Goodman

(2016) Future Crimes: Inside the Digital Underground and the Battle for Our Connected World. New York: Anchor Books.

29.

Gunderman

(2015) Tolkien and the machine. The Conversation, 6 January. Available at: https://theconversation.com/tolkien-and-the-machine-35826

30.

Haggerty

Ericson

(2000) The surveillant assemblage. The British Journal of Sociology 51(4): 605–622.

31.

Harley

(2020) Palmer Luckey and the rise of contemporary virtual reality. Convergence 26(5–6): 1144–1158.

32.

Heyman

(2008) Constructing a virtual wall: Race and citizenship in US–Mexico border policing. Journal of the Southwest 50(3): 305–333.

33.

Horgan

(2020) Will artificial intelligence ever live up to its hype? Scientific American. Available at: https://www.scientificamerican.com/article/will-artificial-intelligence-ever-live-up-to-its-hype/

34.

Joh

(2017) Artificial intelligence and policing: First questions. Seattle University Law Review 41: 1139–1144.

35.

Johnson

Jones

Paasi

, et al. (2011) Interventions on rethinking ‘the border’ in border studies. Political Geography 30(2): 61–69.

36.

Jumbert

(2016) Creating the EU drone: Control, sorting, and search and rescue at sea. In: Sandvik

Jumbert

(eds) The Good Drone. London: Routledge, pp. 89–108.

37.

Kanstroom

(2022) Deportation Nation: Outsiders in American History. Cambridge, MA: Harvard University Press.

38.

Katz

(2020) Artificial Whiteness: Politics and Ideology in Artificial Intelligence. New York: Columbia University Press.

39.

Koslowski

Schulzke

(2018) Drones along borders: Border security UAVs in the United States and the European Union. International Studies Perspectives 19(4): 305–324.

40.

Larson

(2021) The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do. Cambridge, MA: Harvard University Press.

41.

Lemberg-Pedersen

(2013) Private security companies and the European borderscapes. In: Sørensen

Gammeltoft-Hansen

(eds) The Migration Industry and the Commercialization of International Migration. New York: Routledge, pp. 152–172.

42.

Lemberg-Pedersen

(2018) Security, industry and migration in European border control. In: Weinar

Bonjour

Zhyznomirska

(eds) The Routledge Handbook of the Politics of Migration in Europe. London: Routledge, pp. 239–250.

43.

Levy

(2018) Inside Palmer Luckey’s bid to build a border wall: How the Oculus founder, along with ex-Palantir executives, plans to reinvent national security, starting with Trump's agenda. Wired, 6 November. Available at: https://www.wired.com/story/palmer-luckey-anduril-border-wall/

44.

Lisle

(2018) Failing worse? Science, security and the birth of a border technology. European Journal of International Relations 24(4): 887–910.

45.

Martins

Jumbert

(2020) EU Border technologies and the co-production of security ‘problems’ and ‘solutions’. Journal of Ethnic and Migration Studies 48: 1430–1447.

46.

Matthewman

(2011) Technology and Social Theory. Basingstoke: Palgrave Macmillan.

47.

Milivojevic

(2019) Border Policing and Security Technologies. London and New York: Routledge.

48.

Milivojevic

(2021) Crime and Punishment in the Future Internet: Digital Frontier Technologies and Criminology in the Twenty-First Century. London and New York: Routledge.

49.

Molnar

(2020) Technological testing grounds: Migration management experiments and reflections from the ground up. Available at: https://migration-control.info/archive/technological-testing-grounds-migration-management-experiments-and-reflections-from-the-ground-up/

50.

Nieto-Gomez

(2014) Walls, sensors and drones: Technology and surveillance on the US-Mexico border. In: Vallet

(ed.) Borders, Fences and Walls: State of Insecurity? Farnham and Burlington, VA: Ashgate, pp. 191–210.

51.

O’Grady

(2021) Automating security infrastructures: Practices, imaginaries, politics. Security Dialogue 52(3): 231–248.

52.

O’Neill

(2016) Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. London: Penguin Books.

53.

O’Neill

(2020) Explainer: What is artificial intelligence? In: Healey

(ed.) Artificial Intelligence, 450th edn. Thirroul, NSW, Australia: The Spinney Press, pp. 1–3.

54.

Osiński

(2020) Cunning Machines: Your Pocket Guide to the World of Artificial Intelligence. Milton: CRC Press.

55.

Russel

Norvig

(2016) Artificial Intelligence: A Modern Approach. Harlow: Pearson.

56.

Sandvik

Lohne

(2014) The rise of the humanitarian drone: Giving content to an emerging concept. Millennium 43(1): 145–164.

57.

Singler

(2021) Biometric statehood, transnational solutionism and security devices: The performative dimensions of the IOM’s MIDAS. Theoretical Criminology 25(3): 454–473.

58.

Stebbings

(2020) 20VC: Anduril Founder, Palmer Luckey: ‘I am here to build a $50BN Company’ – How Palmer evaluates his relationship to money pre & post Oculus’ $2.3BN exit & why the US DoD needs to be more like China in it’s approach. The Twenty Minutes VC. Available at: https://music.amazon.com/podcasts/35cf4c38-b1cc-4fe5-9256-dd9dbaae8885/episodes/0e0da796-6512-4cc2-8e5f-9455f9171ada/the-twenty-minute-vc-20vc-venture-capital-startup-funding-the-pitch-20vc-anduril-founder-palmer-luckey-i-am-here-to-build-a-50bn-company-how-palmer-evaluates-his-relationship-to-money-pre-post-oculus’-2-3bn-exit-why-the-us-dod-needs-to-be-more-like-china-in-it’s-approach

59.

Tegmark

(2017) Life 3.0: Being Human in the Age of Artificial Intelligence. London: Penguin Books.

60.

Ugwudike

(2021) AI audits for assessing design logics and building ethical systems: The case of predictive policing algorithms. AI and Ethics. Epub ahead of print 13 December. DOI: 10.1007/s43681-021-00117-5.

61.

Vaughan-Williams

(2010) The UK border security continuum: Virtual biopolitics and the simulation of the sovereign ban. Environment and Planning D: Society and Space 28(6): 1071–1083.

62.

Vukov

Sheller

(2013) Border work: Surveillant assemblages, virtual fences, and tactical counter-media. Social Semiotics 23(2): 225–241.

63.

Wall

Monahan

(2011) Surveillance and violence from afar: The politics of drones and liminal security-scapes. Theoretical Criminology 15(3): 239–254.

64.

Walters

(2006) Border/control. European Journal of Social Theory 9(2): 187–203.

65.

Weston Phippen

(2021) ‘A $10-million scarecrow’: The quest for the perfect ‘smart wall’. Politico, 10 December. Available at: https://www.politico.com/news/magazine/2021/12/10/us-mexico-border-smart-wall-politics-artificial-intelligence-523918

66.

Wilson

(2018) Constructing the real-time border: Frontex, risk and dark imagination. Justice, Power and Resistance 2(1): 45–65.