‘Ethical’ artificial intelligence in the welfare state: Discourse and discrepancy in Australian social services

Abstract

In recent years, a discourse of ‘ethical artificial intelligence’ has emerged and gained international traction in response to widely publicised AI failures. In Australia, the discourse around ethical AI does not accord with the reality of AI deployment in the public sector. Drawing on institutional ethnographic approaches, this paper describes the misalignments between how technology is described in government documentation, and how it is deployed in social service delivery. We argue that the propagation of ethical principles legitimates established new public management strategies, and pre-empts questions regarding the efficacy of AI development; instead positioning implementation as inevitable and, provided an ethical framework is adopted, laudable. The ethical AI discourse acknowledges, and ostensibly seeks to move past, widely reported administrative failures involving new technologies. In actuality, this discourse works to make AI implementation a reality, ethical or not.

Keywords

artificial intelligence ethical artificial intelligence digitised welfare delivery public sector technology new public management

Introduction

Global public investment in new digital technologies has undergone a marked increase in the past decade. Summarized variously as artificial intelligence (AI), big data analytics, machine learning, and algorithmic or ‘stochastic governance’ (Sanders and Sheptycki, 2017); this terminology describes the application of computational techniques to state records, and, thereby, the people they represent, in decision processes previously conducted by humans. These technologies, associated with promises of efficiency gains, productivity growth, and the zero-contact administrative encounter, are being deployed to varying degrees across the public sector in the Global North and elsewhere, with extensive research addressing their deployment in welfare administration (see Dubois et al., 2018; Oswald, 2018; Roy, 2013). There is increasing and empirically justified concern that such technologies codify bias, discrimination, and injustice; entrenching existing inequalities and generating new ones (Madden et al., 2017; Mann and Daly, 2018). Technological design encodes the ideologies and agendas of the actors responsible for the digital delivery of state services (Noble, 2018; Timcke, 2020). Well-publicized controversies in Australia, described below, have damaged public trust in the state and corporate actors involved.

As policy positions (sometimes, higher level digital transformation policies, rather than client-oriented policies) come to be reified and black-boxed in digitised systems, traditional ideals of oversight, process and transparency are increasingly supplanted. The discourses rationalising the transformation of public sector delivery thus warrant attention: one such discourse is the global iteration of ‘ethical AI’.

A range of networks and agencies are involved in these developments. Some of these are vocally involved in espousing ethical AI; others are not. Where those agencies are involved, the ethical AI discourse circulates selectively: producing claims about what and how ethical AI will be; less frequently decrying systems which do not meet those claims. Our first objective here is to describe these networks and agencies, the relations between them, and their evidential traces. Our second objective is to draw out some contradictions in policy discourse in Australia.

Focusing on how ‘ethical AI’ is foregrounded alongside the advancement of AI, the following thus presents a necessarily brief overview of the global and Australian institutions, departments, and strategy documents central to the public deployment of AI. Policy documents, white papers, media releases, and news reports provide a basis for tracing the relations between key sites within the state driving welfare automation, and evidencing the themes espoused at these sites.

We identify two significant discursive frames around AI implementation in Australia. There is the widely promoted and recognized discourse around ethics. Then there is the implementation of AI in the public sector, informed by a discrepant ‘government as a platform’ discourse, produced by key departments such as the Digital Transformation Agency and Services Australia (formerly the Department of Human Services, or DHS). The documentation we canvassed evidences the disjuncture between these discourses. Ethical principles are espoused in response to well-publicised unethical outcomes resulting from automation. In the context of the contemporary welfare state, these ethical principles serve to legitimate the established new public management strategies which drive those initial failings.

On this basis, we posit that the citizen experience of welfare state AI is unlikely to be informed by the ethics frameworks being promulgated. This incongruity, and the misalignment across government agencies and between the incompatible discourses espoused by these agencies, is not merely evidence of complex organisations speaking past each other. The incongruity is likely to become more visible, given how those agencies and their technologies are developing in response to changes in welfare delivery and eligibility associated with the Covid-19 crisis. The discussion to follow is relevant to those interested in contemporary Australian social policy and the Australian welfare state, but the Australian case is likely an instructive comparison to those considering the welfare-technology-policy matrix elsewhere. The relations between policy discourses and the institutional sites of their articulation and disarticulation also bears broader sociological interest.

Method

In the effort to understand the discursive terrain of AI in social welfare, we utilised an institutional ethnographic orientation. We started from the ‘ground up’, paying close attention to the sites where levels of automation in the welfare state are being developed, and tracing the circuits by which policy frameworks moved into and then across Australian state agencies.

Institutional ethnography (Griffith and Smith, 2014; Kearney et al., 2018), applied in a whole range of sectors, including notably health, is a project and a body of work devoted to detailing how institutional practices are shaped by and governed through the circulation of documents. These documents or text effects are both material and discursive. Their circulation manifests social hierarchies, social organisation, discourses and norms of good organisational and discursive practice. Paying attention to such documents, both in terms of their discursive content and how they make manifest relations between and within organisations and institutions, institutional ethnography seeks to detail and render navigable the administrative processes by which social power is enacted and enforced.

The literature on policy, the state, and bureaucracy provides valuable perspectives on how processes like digital transformation shape service delivery, and thereby the contours of the administrative state as it is encountered in everyday life (Gupta, 2012; Hoag, 2011; Hull, 2012; Lipsky, 2010; Shore and Wright, 2003). Positions on audit (Strathern, 2000), metrics (Beer, 2016), documents (Riles, 2006), statistics (Stigler, 1986; Espeland and Stevens, 2008) commensuration (Adkins and Lury, 2012; Van der Vlist, 2016) and rank (Davis et al., 2012) also furnish pertinent trajectories. This literature provides an invaluable grounding in terms of methods, epistemologies, and critical stances with which to approach state welfare delivery, administrative logics, and the bureaucratic encounter.

In following the pipelines of information available to ordinary people, we started from the central welfare agency of the Australian state: the DHS. We investigated the material provided by the Department regarding the usage of AI in service provision, the stated departmental goals, aims and frameworks, and the linkages between the DHS and other state agencies and private bodies (including as evidenced in publicly available procurement records).

For current purposes, we focus on the publicly available DHS reports, particularly the annual reports and technology plans from the years 2015 to 2018. In this period the federal government began implementing digital transformation and developing ethics frameworks (as described below). In each instance, the documents selected were thematically coded (Braun and Clarke, 2014; Fairclough, 2001), and summaries were prepared and independently confirmed by both authors of this paper. Key word searches were performed to determine the frequency, clarity and context of the technology discussed; the key words searched were ‘automation’, ‘algorithm’, ‘artificial’, ‘smart’, ‘technology’, ‘data’, ‘ICT’, ‘technology’ and ‘machine’. A linkage database was established as a means to track the stated connections between DHS and other actors; the direction of these connections and any linking documents were recorded to facilitate further investigation. Although we regularly canvassed the DHS website, we excluded it from final analysis given the propensity of online information to be altered and/or removed.

We then repeated this process with those organisations having connections with and informing the DHS. The Digital Transformation Agency (DTA) emerged as a key organisation, providing guidance and support to the DHS. The Digital Transformation Strategy and available annual DTA reports were also subjected to thematic analysis, summarising, key word searching, and linking to further organisations. Key identified connections here included the Australian Digital Council (now the Australian Digital and Data Council). However, somewhat limited reports were produced by the ADC for analysis – with annual reports unavailable for the ADC itself. Rather, annual reports were only provided for the Department of Prime Minister and Cabinet, the portfolio under which the ADC is located. As such, this trail terminates at the apex point of the clearly delineated pipelines of connection and information.

The key discursive strategies identified in the DHS-trail reports were contrasted with those produced within the ‘ethical AI’ documents described in the following section. This process involved the canvassing and recording of news reports and media releases pertaining to AI in Australia, concentrating particularly on 2018 because of its coincidence with significant developments in welfare payment transformation. High level reports were identified via a variety of media releases and reporting on government expenditure on AI initiatives. Using the method detailed above, these reports were thematically analysed for the purpose of identifying key themes and discourses.

Subsequent to the investigation undertaken, in May 2019 there was a reorienting and rebranding of government departments, including the Department of Human Services and the Digital Transformation Agency (Hendry, 2019).¹ This change has not fundamentally altered their function: it consolidates the transformation pursued by government. To ensure chronological accuracy and consistency, the terminology used throughout reflects the departments as canvassed at the time, rather than following their renaming.

Tracing the emergence of ‘ethical AI’ discourse

In rapid succession, nation states have individually released AI national strategies and statements. In March 2017, Canada was the first to produce a national strategy; this was soon followed by strategies or budget inclusions by Japan, Singapore, China and Finland (Dutton, 2018). Increased investment in AI spurred a domino effect, where nations partake in the global ‘AI race’, lest they be left behind in what is considered the forthcoming technological revolution (Walch and Cognitive World, 2020; Pecotic, 2019). By the end of 2018, the number of states and state entities releasing AI strategies increased significantly, with the UK, the EU, Germany, France and India producing some of the notable strategy releases (Dutton, 2018).

Several high-level Australian reports and government initiatives were produced in the same timeframe. The 2015–2016 budget indicated an important priority shift toward public sector digital transformation, and attention to national AI adoption and associated strategies escalated in 2018 (Hamilton, 2019). In the 2018 Australian national budget, $29 million was allocated to the development of strategies for AI development and deployment (Reichert, 2018a). This saw several important reports released in the following months, including an Ethics Roadmap for AI adoption (Dawson et al., 2019) produced by Data61 and the Department of Industry, Innovation and Science (DIIS), with a supporting intellectual framework for that document (Walsh et al., 2019). The latter was commissioned by the Chief Scientist and produced by the Australian Council for Learned Academies (ACOLA). In conjunction with this, the Australian Human Rights Commission initiated a 3-year project examining the relationship between human rights and technology, with a focus on emerging technologies (Australian Human Rights Commission, n.d.). These initiatives have culminated in the establishment of an ethical oversight body for AI in Australia, the Gradient Institute, formed by Data61 and the private insurance company IAG (Chelvan, 2018). The Gradient Institute cements the role of Data61 in shaping the direction and implementation of AI strategies in Australia.

A notable feature of the documents described here is their emphasis on ethics in technological development. Ethics provides an alternative marketing platform for home-grown technologies in countries unable to replicate the technological dominance of China and the USA. Ethical AI has been proposed as both the necessary, and perhaps defining, feature of AI in Australia, with the ACOLA report noting that Australia and New Zealand can distinguish and market themselves through ethical AI, ‘as relatively small countries with diverse populations provides advantages that can be exploited’ (Walsh et al., 2019: 187):

We make an ideal test bed for new developments and an ethical AI strategy should enable us to attract significant overseas investment. Our reputation as a forward-looking, open and liberal society also allows us to play an important role in the development of international frameworks for regulating AI. We have the opportunity to ensure that the development of AI does not come at the expense of human rights, either at home or internationally. An AI strategy that places equity at its forefront will strengthen our international reputation in this arena and ensure that we are not left behind by some of the most important developments of the 21st century (Walsh et al., 2019: 187).

This framing locates ethical AI not merely as a proper goal in and of itself, but rather as a strategy for pursuit of positional market share. The ethics discourse enables state agencies to bypass concerns around the implementation of algorithmic and AI techniques, reframing the discussion in terms of how and when, rather than if and why. It remains to be seen whether the development of ethical frameworks will involve meaningful democratic participation, but there is little evidence to suggest that. Corporate actors, with the support of state agencies, have enthusiastically established self-governing codes of conduct, with initiatives, frameworks and, albeit occasionally short-lived (Murgia and Shrikanth, 2019), ethics boards established by leading companies such as Microsoft, IBM, and Google (Finley, 2016). Tracking the processes by which ethical frameworks are produced is critical to directing greater attention to the emergence of algorithmic governance futures.

The optimistic narrative around ethical AI is that it has the potential to benefit everyone. This narrative extends across all the ethical AI documents and strategies we encountered. For example, the opening paragraph of the Data61 report states:

AI has the potential to increase our wellbeing, lift our economy, improve our society by, for instance, making it more inclusive, and help the environment by using the planet’s resources more sustainably. For Australia to realize these benefits however, it will be important for citizens to have trust in the AI applications developed by businesses, governments and academia. One way to achieve this is to align the design and application of AI with ethical and inclusive values. (Dawson et al., 2019: 3)

Here we see a subdued warning. A better world is possible, but only if you will trust ‘businesses, governments and academia’.

Ethical frameworks, it is suggested, ameliorate potential harms resulting from the use of these new technologies, and engender public trust. The extent to which voluntary, nonbinding ethical frameworks can alter the trajectory of emerging technologies is debatable. Such technologies are designed to operate within, implement and expedite prevailing political orientations. This is particularly evident in the context of social welfare.

The DTA and ‘government as a platform’

Alongside the developments described thus far, the government formed a Digital Transformation Agency, to oversee the transformation of the public sector though the use of emerging technologies, including AI. Previously known as the Digital Transformation Office, the DTA was established as an agency of the Prime Minister and Cabinet in 2016, later shifting to the Social Services Portfolio of the Australian government (AusTender, n.d.). The DTA facilitates all-of-government purchases and procurements for digital technologies, provides oversight for digital adoption programs in all government departments and agencies, and develops technologies and systems in-house (Senate Finance and Public Administration References Committee, 2018). In addition, and pertinent to an understanding of the citizen’s experience of AI technologies in social welfare, the DTA is responsible for the development of a roadmap for digital transformation and standards for digital delivery. Despite the leading role of the DTA, it was not involved in any of the ethics framework documents we reviewed. Nor can reference to ethics be found in the DTA documents.

The DTA’s Digital Transformation Strategy (2018), reproducing in its rhetoric the language used in commercial contexts, sets out the aspirations for a digital government, to be fully operational by 2025. This is the guiding document for all government agencies and departments. It is most evidently adopted by the DHS, the central department for welfare delivery in Australia through Centrelink, Medicare and the Child Support Agency. Strong linkages between the DTA and the DHS are evidenced throughout departmental literature. The DHS consistently frame their uptake of digital technologies as a means to support and further the DTA strategy and become ‘the lead agency for Citizen to Government business’ (Department of Human Services, 2016). The two departments work closely together, with the DTA providing guidance on digital transformation, training DHS staff, and, in some instances, providing the technologies themselves.

The DHS has outlined their role as the proving ground for the transformation of government. The intention, evinced by the DHS and the DTA, is that the DHS will be the first department to roll out intelligent technologies and provide new platforms to citizenry, in accordance with the DTA’s roadmap, which will be adopted later by other agencies. In effect, this continues the tradition of prototyping innovations on vulnerable populations. The shift to new technologies is described as a transformation of government itself: repeated mention is made throughout DTA and DHS documents of the redesign of the workings and operation of government (eg. Digital Transformation Agency, 2018: 7). The goal is to transform the government into a platform: citizens, through a single digital identity, will interact with (what will appear to the user as) one single government portal. The Department describes this platform as the primary way citizens in the future will interact with and access government. The relationship between the citizen and government agencies will be ‘transformed’ through the use of AI and automation; the professed goal is to increase efficiency and effectiveness for the department (Department of Human Services, 2016: 7).

The Department here espouses a normative view of the proper, ‘transactional’ relationship between citizens (‘customers’) and the state, derived from the conventional tenets of new public management (NPM) (Johnston, 2000), or in the more contemporary idiom, neoliberal bureaucracy (Hibou, 2015). The DHS discourse is nominally citizen-centric, with technologically literate customers interacting with service providers. This is grounded, however, in strong and repeated emphasis on efficiency, cost savings, and calculated public-private partnerships. Government as a platform also, of course, transfers administrative labour to citizens: empowered, as the DTA puts it, to ‘do things for yourself’ (Digital Transformation Agency, 2018: 9).

In this near-future, oddly approximating to a radical market-libertarian techno-utopia, government has almost entirely evacuated itself, functioning largely as an online broker: an ‘eMarketplace’, rather like the App Store. Government as a platform aspires to replicate private sector user services, particularly as with online banking and shopping (Department of Human Services 2016), without consideration for broader or singular functions of government (Senate Finance and Public Administration References Committee, 2018). As the pronouncements from the DHS and the DTA indicate, at the departmental level, government as a platform is a form of ‘technological sublime’ (Nye, 1994). Information technology is the engine delivering the famous ‘three Es’: economy, efficiency and effectiveness (Bannister, 2017: 38; du Gay, 2000: 104–107).

The foreword to the Digital Transformation Strategy states that ‘Australians expect the same experience interacting with government as they have with innovative, leading private sector organisations’ (Digital Transformation Agency, 2018: 4). The interaction with the state should be modelled on the interaction with a private enterprise, perhaps one like Facebook – people expect it to be the same. This is the discursive context against which to contrast operational outcomes such as robo-debt, described below.

The discursive ethics gap: Botched delivery as ethical malfeasance

There are disjunctures here between how technology is being deployed, and how it is being described. The frictionless welfare state conjured by the DTA and DHS is not congruent with the ‘higher’ level frame of the Australian governmental discourse around ethical AI. The specific ethical principles proposed for the Australian context were put forward in 2019 by Data61 in conjunction with the Department of Industry, Innovation and Science in the AI Ethics Principles. These eight aspirational, nonbinding, voluntary principles for AI design and implementation include: Human, social and environmental wellbeing; Human-centred values; Fairness; Privacy protection and security; Reliability and safety; Transparency and explainability; Contestability; and Accountability (Dawson et al., 2019).

It would be challenging to argue that the agendas and outcomes enacted by the DTA and DHS accord with these principles. The DTA seems to address ‘ethics’ solely as pertaining to the handling of data in accordance with privacy law (Digital Transformation Agency, 2018: 32). The DHS makes no mention of ethics of the sort espoused by Data61. Public service delivery is instead professed as based on ‘applied’ principles of citizen-centred design, a capacity to foster trust, collaboration, innovation, and cost savings (Department of Human Services, 2016). In these accounts, the agential, informed citizen is prioritized; the sort of citizen wishing to access and manage their own governmental interactions at any time (Department of Human Services, 2016; Digital Transformation Agency, 2018).

The transition to algorithmic governance puts ideals of accountability, transparency and equality before the law under pressure. Immediately relevant is the lack of clarity as to how transparency will be achieved, considering the black box working of AI. Ethics frameworks have suggested that transparency can, in such instances, involve notifying users of the deployment and operational principles of AI and algorithms (Walsh et al., 2019). However, a continuing lack of transparency is apparent in the DHS’s publicly accessible annual reports. These are notable for the absence of reference to the use of algorithmic processes, particularly in relation to compliance mechanisms, despite this being a key technology underpinning and driving welfare delivery reform across this period. It is worth emphasising the developing range of principles being applied by the DTA and the DHS: whilst the DHS may be the initial laboratory for the digitisation of government, the DHS and DTA are establishing links with other essential state agencies, including the Department of Veteran’s Affairs, the National Disability Insurance Agency, the Department of Social Security, the Department of Health, State and Federal Police, and Defence.

The ethical principles put forward by Data61 are essentially absent in the discourses espoused by the DTA and DHS. There are also breaches of these principles in the delivery of services, by the DHS and other agencies, before, during, and after the release of the national ethics documents. The ideals of both government as a platform, and ethical AI, are difficult to reconcile with the implementation of large-scale digital transformation by Australian government agencies. Digitisation frequently appears to entail inappropriate encroachment on personal data, botched in execution, to ends contrary to public best interest.

The rolling saga of Australian government PR disasters involving novel information technology arguably began in 2016 with #censusfail, continuing with My Health Record and the Medicare data breach. Held up as privacy and data security failures, these debacles bear scrutiny in terms of the ethical orientations they exhibit. In the instance of #censusfail, the first wholly online Australian census crashed on the night of the census. Originally (and incorrectly) described as the result of hacking, the outage and resulting fallout reportedly cost the government $24 million (Gothe-Snape, 2017). In September 2016 the Medicare data breach occurred: a large dataset of public healthcare claims was placed online, with researchers able shortly thereafter to re-identify every medical practitioner in the set. The encryption methods used by the government were reportedly insufficient to protect personal information (Cowan, 2016). My Health Record (MHR), involving the centralising of medical records, was more insidious in terms of what it revealed about the government’s approach to privacy and consent. MHR was opt-out rather than opt-in: health records were automatically generated unless individuals undertook efforts to opt out (Kemp et al., 2018). The MHR scheme was based on, and almost identical to, an aborted similar system in the UK, care.data, through which data were sold to health insurers, pharmaceutical companies and other interested parties (Zhou, 2018). Perhaps the most egregious feature of the MHR (in terms of ethical competence) was the assumption of consent through the opt-out requirement (Kemp et al., 2018). These examples are ethically indicative, insofar as they evidence incompetence and a form of cavalier disdain for the public. In order to be understood as such, ethical AI would have to be ethical, but also meta-ethical: delivered by trustworthy actors who can competently design and implement, for ethics. My Health Record and the other well publicized blunders of the Australian government in big data systems do not inspire confidence in ethical AI, because they do not inspire confidence in the basic competencies of the agencies involved.

The most spectacular of such blunders thus far was first implemented in July 2016. Through the Online Compliance Intervention (OCI), known colloquially as ‘robo-debt’, the DHS issued automatic debt notifications to current and former welfare recipients (Macleod, 2017). Debts were raised by an algorithm, data-matching reported income to Australian Tax Office (ATO) records in pursuit of overpayments, with the stated aims of recovering undeserved welfare payments and ensuring ‘integrity’ in the social security system.

Overpayments arise usually from reporting errors, although government representatives commonly overstate the extent of welfare fraud (Keenan, 2018; Wilcock, 2019). The robo-debt algorithm used flawed averaging, leading to a high number of false and inflated debts (Carney, 2018; Henman, 2017). Despite receiving many warnings regarding the illegality of the scheme (Henriques-Gomes, 2020), the government continued with it, even expanding its application to vulnerable categories (Reichert, 2018b). The averaging of income from tax records was challenged in court, and suspended in November 2019 when ultimately found unlawful. The precise numbers of people incorrectly targeted for debt recovery is not known. Over 400,000 people were listed in the class action suit, settled in November 2020 for $1.2 billion.

The difficulty in determining how many people paid debts they did not owe is attributable in part to the ‘reverse onus’: the robo-debt algorithm required claimants demonstrate they did not owe a debt, rather than the DHS demonstrating monies were owed. Many people were unable to demonstrate they did not owe a debt, assumed they must owe the debt, or found themselves paying off a debt (through garnished tax returns or reductions in income support) regardless of whether they thought they owed. Debts were recovered even from those who appealed the debt, up until the appeals were successful.

Senior DHS and government officials consistently withheld information about the development and application of robo-debt, and denied there was anything amiss about it (Barbaschow, 2020). The government has been especially committed to avoiding examination in court. Thus the class action was settled out of court: the government accepts no liability and claims no ‘knowledge of unlawfulness’ (Gordon Legal, 2020). Robo-debt constitutes the most pernicious instance of Australian digitised service delivery. Further to its extraordinary design flaws, it was knowingly executed and continued. Insofar as there were intelligible goals for robo-debt, they seem to have involved the coercive extraction of revenue from the un- and under-employed, and rendering life on benefits so bleak as to drive claimants into whatever low wage work is available. Put differently, robo-debt was a form of extortion which functioned to proletarianise welfare recipients by compelling them to pay fictitious debts (Grover, 2019).

The frictionless welfare state conjured by the DTA and DHS is incommensurate with the ethical AI discourse, and also incongruent with the actually existing digital welfare state. A claim to ethical AI on the part of Australian government agencies would have to be demonstrably evidenced, and would also entail remedial work establishing public trust. On its own evidence, the digitised processes enacted by the DHS and described above do not seem to accord with the principles proposed in high level Australian ethical AI discourse. We argue below that there are further, broader grounds on which to consider that discourse with scepticism.

The spectre of Whitehall is haunting Canberra

Ethics is now normalised as something to be vocal about in the tech industry, perhaps unsurprisingly, given some of the recent calamities encountered in that field (Metcalf and Moss, 2019). Five core principles dominate the ethics discourse that has emerged in global policy documents (Jobin et al., 2019). By order of frequency, these are: ‘transparency, justice and fairness, non-maleficence, responsibility, and privacy’ (Jobin et al., 2019: 6). These principles accord with those from the DIIS approximately as follows:

DIIS principles	International principles
1. Human, social and environmental wellbeing 2. Human-centred values 3. Fairness 4. Privacy protection and security 5. Reliability and safety 6. Transparency and explainability 7. Contestability 8. Accountability	Non-maleficence Justice and fairness Privacy Transparency Responsibility

Of course, there is not a word-for-word alignment. Beneficence, for example, stronger than non-maleficence (‘do no harm’ in the original Data61 discussion paper), is further down the list of international principles canvassed by Jobin et al. (2019), coming in sixth by order of frequency, and might be a better match for the DIIS principles 1 and 2.

The principles contain appealing terminology and phrasing, albeit lacking specificity in definition and application. Multiple incompatible definitions could, for example, be put to a term like ‘fairness’. Independently of all this, however, seen from perspectives in the sociology of organisation and historical sociology, the principles elicit a kind of déjà vu. This is due to the positing of a set of ‘classical’ public administration ethics as the solution to risks posed as now apparently inevitable (but also, simultaneously, reassuringly foreseen and managed) by our development of technology.

Principles 1 and 2 are vague and ‘feel good’. Insofar as they can be specified, they seem to be concerned with design philosophy rather than the procedural use of AI technologies. The mundane use of these technologies is more pertinent to the remaining six principles. Those are: fairness, privacy protection and security, reliability and safety, transparency and explainability, contestability, and accountability.

How did these latter principles come to be propagated by the Australian government Department of Industry, Innovation and Science in 2019? According to Artificial Intelligence: Australia’s Ethics Framework, they are derived from and refresh international human rights law:

Philosophers, academics, political leaders and ethicists have spent centuries developing ethical concepts, culminating in the human-rights based framework used in international and Australian law. Australia is a party to seven core human rights agreements which have shaped our laws. An ethics framework for AI is not about rewriting these laws or ethical standards, it is about updating them to ensure that existing laws and ethical principles can be applied in the context of new AI technologies (Dawson et al., 2019: 5–6).

A full account of the emergence of these six principles cannot be provided within the confines of this paper. These are traditional public administration principles. They are antithetical to the neoliberal dynamic of NPM in social welfare systems as instantiated by the DTA and DHS.

These traditional public administration values, sometimes referred to as ‘the Whitehall model’, cannot be definitively and exhaustively codified. Permutations of them, however, are well known across the literature and across the public sector in Australia and other OECD countries (see du Gay, 2000, 2015; du Gay and Vikkelsø, 2016; Goodsell, 2014; Goss, 1996; Greenaway, 1995; Kernaghan, 2000). Traditional public service values have been described in terms of orientations to duty (propriety, integrity, parsimony), to service (respect, effectiveness, transparency), and to the social (fairness, due process, equality of access, impartiality) (Bannister and Connolly, 2014: 123). The latter six DIIS principles (fairness, privacy protection and security, reliability and safety, transparency and explainability, contestability, and accountability) are, in this context, hardly novel.

The roots of the espoused ethical underpinnings of contemporary administrative formations, and their locations in longer histories of public administration, bear scrutiny here. They work to expose contradictions in the statements of the government, particularly when contrasted with its activities in this context. It is worth distinguishing ‘bureaucracy’ (Weberian or otherwise) from the Whitehall model. Analogously, ‘NPM’ represents a terminology still in widespread use, though the specific technologies of new public managerialism are now so ubiquitous that they rarely draw comment as such among critical scholars. Neoliberalism (under which NPM is typically tacitly implied), has taken on a kind of hegemonic dominance as a critical frame. Specificity in terminology and in tracing lineage helps clarify the irreconcilable relationship between algorithmic governance as implemented through the DHS, and the ethics of AI the government apparently supports.

The Whitehall model, as its name implies, is an artefact of British colonial rule. Its emergence is associated with the 1854 Northcote-Trevelyan report, which was in part the outcome of an interest among the British ruling classes in the mandarin system of imperial China (Boyer and Kang, 2001). This set of values evokes, but predates, Weber’s description of the ideal typical features of bureaucracy. Spirited defences of this ethos point out that it is a bulwark against authoritarianism and the foundation of equality before the law:

For Weber, bureaucracy was a historically contingent and variable ‘life-order’ (Lebensführung) constituting a distinctive ethical milieu in its own right, one whose practices of formalistic impersonality gave rise to certain substantive ethical goals. . . . without the ‘art of separation’ that the state bureau effected and continues to effect, many of the qualitative features of government that are regularly taken for granted – for instance, formal equality, reliability, and procedural fairness in the treatment of cases – would not exist. (du Gay, 2013: 280–281)

The latter six principles espoused by the DIIS, and the styles of governance instantiating them, have been subject to interventionist reform for decades, driven by political actors who consider them inefficient and obstructionist (Diamond, 2018). Their disruption and upheaval coincided, incidentally, with widely discussed problems, including increasing economic inequality, exacerbated by austerity following the global financial crisis. It is paradoxical that the long march of neoliberal managerialism should culminate, in search of a rhetorical placebo against fears of socially destructive automation, in the blithe espousal of precisely those traditional public service ethics that neoliberal managerialism sought so strenuously to expunge. The disrespect for norms of privacy and confidentiality, hostility towards welfare service clients and workers, resistance to scrutiny, and absence of accountability characterising robo-debt and the other recent controversies of Australian government digital transformation are signatures of this neoliberal managerialism.

Robo-debt is ethically problematic on several levels. It is objectionable to castigate and intimidate vulnerable members of society (it is contrary to human-centred values and wellbeing), and to treat them as though guilty until proven innocent, as under the reverse onus (it is unfair). There is a problem of ethical competence in implementing an algorithm to ‘catch’ overpayments, for several years, in the knowledge that it has a high rate of error: the technology is unreliable and unsafe, and data security and privacy are compromised where malformed processes are being mass applied with significant consequences. It is anathema to traditional concepts of public service duty (propriety, integrity, parsimony) to knowingly continue in this course of action despite mounting evidence, and there are fundamental problems of accountability and transparency where the government has declined to address questions regarding responsibility throughout. The robo-debt misadventure has been corrosive to public trust in governance. Given the workings of the robo-debt have still not been adequately explained, and those subjected to it at no point gave informed consent to automated data-matching, criteria of explainability and contestability cannot be meaningfully applied. As a case of actually existing AI in public service delivery, the priorities displayed through robo-debt do not accord with the ethical AI principles the government was simultaneously espousing, and do not accord with traditional Whitehall principles either.

It is unsurprising that the government would want to avoid culpability regarding robo-debt. Putting aside the regular opacity surrounding neoliberal public policy processes, additional obscurity underpins the design and implementation of digital governance. There is uncertainty as to whether managerial staff or ICT professionals are the key actors pursuing digital opportunities and developing technical innovations. The parameters which determine these processes remain concealed, given the complexity of digital functionalities and processes, and the commitment to commercial-in-confidence contracts. Processes of fundamental importance to clients of state services are likely to become increasingly opaque to those clients, and the assumed neutrality and scientificity of these computational processes is widely regarded with suspicion.

Conclusion

Developments in policy and delivery are shaped in and through existing power dynamics. Algorithmic governance does not appear, fully formatted, and roll out through the state apparatus. The introduction of novel administrative techniques, or the introduction of ethics as the discursive padding for those techniques, is not a fait accompli. These are heterogeneous processes, running on parallel rails. Even in what is ostensibly one process – the adoption and implementation of advances in information technology and the development of new affordances out of that – multiple social groups, agencies, recipients and discursive frames and technologies are mobilized and reconfigured. Recent shifts in the digital delivery of government services culminated in the rebranding of the DHS to Services Australia. This manoeuvre further cements the link to the DTA, which resides in the same departmental portfolio.

In Australia, the discourse around ethical AI has obscured the reality of AI deployment in the public sector. The critical public dialogue about information technology and social welfare (framed as evidence of callous or incompetent government) is disconnected from the conversation about ethical AI (framed as a future-building affirmation of ethics in system design), despite there being actors (such as Data61) active in both conversations. At both global and local levels, ethics discourses pre-empt questions regarding the rationale of AI development, positioning investment and implementation as inevitable and, provided ethical frameworks are adopted, laudable (indeed, the ‘race’ to ethical AI makes ‘winning’ imperative). Bracketing questions as to whose ethics are installed and by what means, and indeed whether ethical AI is meaningful given the logics within which it is developed, in Australia, the ethics propagated are remote from those displayed in the implementation of AI in social welfare.

While the ethics discourse ostensibly moves us past widely recognised legal and ethical AI failures, in actuality, the ethical AI discourse entrenches AI implementation, ethical or otherwise. Discussions of ethics do not challenge the legitimacy of AI, or the basic power dynamics in which it is developed, but instead reinforce an understanding of AI as imminent and inescapable. Codifying ethical approaches might result in better outcomes, but this still ignores the structural contexts in which AI is implemented. AI inevitably operates within powerful institutional systems, being applied to the ‘problems’ identified by those systems. Digital transformation reinforces and codifies neoliberal agendas, limiting capacities for expression, transparency, negotiation, democratic oversight and contestation (Eubanks, 2018). This can be demonstrated by juxtaposing the AI ethics discourse in Australia with how AI has been implemented in social welfare. Ultimately, ethical AI speaks to the prospect of adapting to a world with AI, rather than questioning whether that is the world we want. Such questioning directs attention to the AI systems we are already entangled in, rather than to lofty ideals espoused for an AI future otherwise absent from the here and now.

Footnotes

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author biographies

Alexandra James is a sociologist specialising in the fields of technology studies, digital intimacies, and gender studies. Her most recent publication was in a special edition of Health Sociology Review. Previously she has published in the Journal of Sociology; Community, Work and Family; and the Journal of Social Inclusion.

Andrew Whelan is a sociologist at the University of Wollongong in Australia. His research interests are in the relations between digital media and cultural and organisational practices. He has published recently on the temporal implications of algorithmic governance in social welfare in Critical Sociology, and, with Katrin Tiidenberg, on the critical affordances of social media platforms in Communication and Critical/Cultural Studies.

References

Adkins

Lury

(eds) (2012) Measure and Value. London: Wiley-Blackwell.

AusTender (n.d.) Annual Procurement Plan View – Digital Transformation Agency. Available at: www.tenders.gov.au/App/Show/cdb00a26-f579-1309-1c0c-b7be0cfdcee9

Australian Human Rights Commission (n.d.) Our Work. Human Rights and Technology. Available at: https://tech.humanrights.gov.au/our-work

Bannister

(2017) In defence of bureaucracy: Governance and public values in a digital age. In: Paulin

Anthopoulos

Reddick

(eds) Beyond Bureaucracy. Public Administration and Information Technology, Vol. 25. Cham: Springer, pp. 27–47.

Bannister

Connolly

(2014) ICT, public values and transformative government: A framework and programme for research. Government Information Quarterly 31(1): 119–128.

Barbaschow

(2020) Services Australia accused of ‘covering up’ robo-debt’s automation mess. ZDNet, March 6. Available at: https://www.zdnet.com/article/services-australia-accused-of-covering-up-robo-debts-automation-mess

Beer

(2016) Metric Power. London: Palgrave Macmillan.

Boyer

Kang

(2001) Asia’s influence on public administration in the West. International Area Review 4(2): 3–20.

Braun

Clarke

(2014) What can ‘thematic analysis’ offer health and wellbeing researchers? International Journal of Qualitative Studies on Health and Well-being 9.

10.

Carney

(2018) The new digital future for welfare: Debts without legal proofs or moral authority. UNSW Law Journal. Available at: http://www.unswlawjournal.unsw.edu.au/wp-content/uploads/2018/12/2018-1-CARNEY.pdf

11.

Chelvan

(2018) Gradient Institute to Create Ethically Aware Artificial Intelligence. CSIRO. Available at: www.csiro.au/en/News/News-releases/2018/Gradient-Institute-to-create-ethically-aware-Artificial-Intelligence

12.

Cowan

(2016) Health pulls Medicare dataset after breach of doctor details. iTnews, 29 September. Available at: https://www.itnews.com.au/news/health-pulls-medicare-dataset-after-breach-of-doctor-details-438463

13.

Davis

Fisher

Kingsbury

Merry

(eds) (2012) Governance by Indicators: Global Power through Classification and Rankings. Oxford: Oxford University Press.

14.

Dawson

, et al. (2019) Artificial Intelligence: Australia’s Ethics Framework. Data61: CSIRO.

15.

Department of Human Services (2016) Technology Plan 2016–2020. Services Australia. Available at: www.servicesaustralia.gov.au/organisations/about-us/publications-and-resources/technology-plan-2016-20

16.

Department of Prime Minister and Cabinet (n.d.) Australian Data and Digital Council. Available at: www.pmc.gov.au/public-data/australian-data-and-digital-council

17.

Diamond

(2018) The End of Whitehall? Government by Permanent Campaign. Cham: Springer.

18.

Digital Transformation Agency (2018) Digital Transformation Strategy. Available at: www.dta.gov.au/digital-transformation-strategy

19.

du Gay

Vikkelsø

(2016) For Formal Organization: The Past in the Present and Future of Organization Theory. Oxford: Oxford University Press.

20.

du Gay

(2000) In Praise of Bureaucracy: Weber, Organization, Ethics. London: Sage.

21.

du Gay

(2013) New spirits of public management. . . ‘post-bureaucracy’. In: du Gay

Morgan

(eds) New Spirits of Capitalism? Crises, Justifications, and Dynamics. Oxford: Oxford University Press, pp. 274–293.

22.

du Gay (2015) ‘Is Office a Vocation in “Post-Bureaucratic” Public Management?’. In Neoliberalism and the Crisis of Public Institutions: Working Papers in the Human Rights and Public Life Program, No 2, ed. Anna Yeatman, Whitlam Institute within Western Sydney University.

23.

Dubois

Paris

Weill

(2018) Targeting by numbers. The uses of statistics for monitoring French welfare benefit recipients. In: Creating Target Publics for Welfare Policies. Cham: Springer, pp. 93–109.

24.

Dutton

(2018) An overview of national AI strategies. Medium, 29 June. Available at: https://medium.com/politics-ai/an-overview-of-national-ai-strategies-2a70ec6edfd

25.

Espeland

Stevens

(2008) A sociology of quantification. European Journal of Sociology/Archives Européennes de Sociologie 49(3): 401–436.

26.

Eubanks

(2018) Automating Inequality: How High-tech Tools Profile, Police, and Punish the Poor. London: St. Martin’s Press.

27.

Fairclough

(2001) Critical discourse analysis as a method in social scientific research. In: Wodak

Meyer

(eds) Methods of Critical Discourse Analysis. London: Sage, pp. 121–138.

28.

Finley

(2016) Tech giants team up to keep AI from getting out of hand. Wired, 28 September. Available at: www.wired.com/2016/09/google-facebook-microsoft-tackle-ethics-ai/

29.

Goodsell

(2014) The New Case for Bureaucracy. Washington: CQ Press.

30.

Gordon Legal (2020) Income compliance program (‘Robodebt’) class action settlement. Gordon Legal, 16 November. Available at: https://gordonlegal.com.au/media/1308/201116-settlement-joint-statement.pdf

31.

Goss

(1996) A distinct public administration ethics? Journal of Public Administration Research and Theory 6(4): 573–597.

32.

Gothe-Snape (2017) #CensusFail year at ABS. ABC News, 19 October. Available at: https://www.abc.net.au/news/2017-10-19/abs-annual-report-censusfail/9064970

33.

Greenaway

(1995) Having the bun and the halfpenny: Can old public service ethics survive in the new Whitehall? Public Administration 73(3): 357–374.

34.

Griffith

Smith

(eds) (2014) Under New Public Management: Institutional Ethnographies of Changing Front-line Work. Toronto: University of Toronto Press.

35.

Grover

(2019) Violent proletarianisation: Social murder, the reserve army of labour and social security ‘austerity’ in Britain. Critical Social Policy 39(3): 335–355.

36.

Gupta

(2012) Red Tape: Bureaucracy, Structural Violence, and Poverty in India. Durham, NC: Duke University Press.

37.

Hamilton

(2019) Public sector digital transformation: A quick guide. Department of Parliamentary Services. Available at: www.aph.gov.au/About_Parliament/Parliamentary_Departments/Parliamentary_Library/pubs/rp/rp1819/Quick_Guides/PSDigitalTransformation

38.

Hendry

(2019) Services Australia assumes govt-wide IT function, replaces DHS. iTnews, 29 May. Available at: www.itnews.com.au/news/morrison-scraps-department-of-human-services-525872

39.

Henman

(2017) The computer says ‘debt’: Towards a critical sociology of algorithms and algorithmic governance. Zenodo. Available at: http://doi.org/10.5281/zenodo.884117

40.

Henriques-Gomes

(2020) Robodebt court documents show government was warned 76 times debts were not legally enforceable. The Guardian, 19 September. Available at: https://www.theguardian.com/australia-news/2020/sep/19/robodebt-court-documents-show-government-was-warned-76-times-debts-were-not-legally-enforceable

41.

Hibou

(2015) The Bureaucratization of the World in the Neoliberal Era: An International and Comparative Perspective. New York: Springer.

42.

Hoag

(2011) Assembling partial perspectives: Thoughts on the anthropology of bureaucracy. PoLAR 34: 81.

43.

Hull

(2012) Government of Paper: The Materiality of Bureaucracy in Urban Pakistan. California: University of California Press.

44.

Jobin

Ienca

Vayena

(2019) The global landscape of AI ethics guidelines. Nature Machine Intelligence 1(9): 389–399.

45.

Johnston

(2000) The new public management in Australia. Administrative Theory and Praxis 22(2): 345–368.

46.

Kearney

Corman

Gormley

Hart

Johnston

Smith

(2018) Institutional ethnography: A sociology of discovery – in conversation with Dorothy Smith. Social Theory and Health 16(3): 292–306.

47.

Keenan

(2018) $1.4 billion saved in welfare fraud crackdown. Minister for Human Services and Digital Transformation, 11 February. Available at: https://www.mhs.gov.au/media-releases/2018-02-11-14-billion-saved-welfare-fraud-crackdown

48.

Kemp

Baer Arnold

Vaile

(2018) My Health Record: The case for opting out. The Conversation, 16 July. Available at: https://theconversation.com/my-health-record-the-case-for-opting-out-99302

49.

Kernaghan

(2000) The post-bureaucratic organization and public service values. International Review of Administrative Sciences 66(1): 91–104.

50.

Lipsky

(2010) Street-level Bureaucracy: Dilemmas of the Individual in Public Service. New York: Russell Sage Foundation.

51.

Macleod

(2017) Lessons learnt about digital transformation and public administration: Centrelink’s online compliance intervention. AIAL Forum 89: 59.

52.

Madden

Gilman

Levy

Marwick

(2017) Privacy, poverty, and big data: A matrix of vulnerabilities for poor Americans. Wash. UL Rev 95: 53.

53.

Mann

Daly

(2018) (Big) data and the North-in-South: Australia’s informational imperialism and digital colonialism. Television and New Media. DOI: https://doi.org/10.1177/1527476418806091

54.

Metcalf

Moss

(2019) Owning ethics: Corporate logics, Silicon Valley, and the institutionalization of ethics. Social Research: An International Quarterly 86(2): 449–476.

55.

Murgia

Shrikanth

(2019) How big tech is struggling with the ethics of AI. Financial Times, 29 April. Available at: www.ft.com/content/a3328ce4-60ef-11e9-b285-3acd5d43599e

56.

Noble

(2018) Algorithms of Oppression: How Search Engines Reinforce Racism. New York: New York University Press.

57.

Nye

(1994) American Technological Sublime (first edition). Cambridge, MA: MIT Press.

58.

Oswald

(2018) Algorithm-assisted decision-making in the public sector: Framing the issues using administrative law rules governing discretionary power. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 376: 2128.

59.

Pecotic

(2019) Whoever predicts the future will win the AI arms race. Foreign Policy, 5 March. Available at: https://foreignpolicy.com/2019/03/05/whoever-predicts-the-future-correctly-will-win-the-ai-arms-race-russia-china-united-states-artificial-intelligence-defense/

60.

Reichert

(2018a) Budget 2018: Funding Australia’s data-sharing framework. ZD Net, 8 May. Available at: www.zdnet.com/article/budget-2018-funding-australias-data-sharing-framework/

61.

Reichert

(2018b) Budget 2018: Government to extend DHS data matching. ZDNet, 8 May. Available at: https://www.zdnet.com/article/budget-2018-government-to-extend-dhs-data-matching/

62.

Riles

(2006) Documents: Artifacts of Modern Knowledge. Ann Arbor, MI: University of Michigan Press.

63.

Roy

(2013) From Machinery to Mobility: Government and Democracy in a Participative Age. Volume 2. New York: Springer-Verlag.

64.

Sanders

Sheptycki

(2017) Policing, crime and ‘big data’: Towards a critique of the moral economy of stochastic governance. Crime, Law and Social Change 68(1/2): 1–15.

65.

Senate Finance and Public Administration References Committee (2018) Digital Delivery of Government Services. Canberra: Parliament House. Available at: www.aph.gov.au/Parliamentary_Business/Committees/Senate/Finance_and_Public_Administration/digitaldelivery/Report

66.

Shore

Wright

(eds) (2003) Anthropology of Policy: Perspectives on Governance and Power. London: Routledge.

67.

Stigler

(1986) The History of Statistics: The Measurement of Uncertainty before 1900. Cambridge, MA: Harvard University Press.

68.

Strathern

(2000) The tyranny of transparency. British Educational Research Journal 26(3): 309–321.

69.

Timcke

(2020) Algorithms and the critical theory of technology. SSRN. DOI: https://dx-doi-org.web.bisu.edu.cn/10.2139/ssrn.3551467

70.

Van der Vlist

(2016) Accounting for the social: Investigating commensuration and Big Data practices at Facebook. Big Data and Society 3(1).

71.

Walch

Cognitive World (2020) Why the race for AI dominance is more global than you think. Forbes, 9 February. Available at: www.forbes.com/sites/cognitiveworld/2020/02/09/why-the-race-for-ai-dominance-is-more-global-than-you-think/#14e5edef121f

72.

Walsh

, et al. (2019) The effective and ethical development of artificial intelligence: An opportunity to improve our wellbeing, The Australian Council of Learned Academies.

73.

Wilcock

(2019) (De-)criminalizing welfare? The rise and fall of social security fraud prosecutions in Australia. The British Journal of Criminology 59(6): 1498–1519.

74.

Zhou

(2018) My Health Record privacy framework ‘identical’ to failed UK scheme, expert says. The Guardian, 22 July. Available at: www.theguardian.com/australia-news/2018/jul/22/my-health-record-identical-to-failed-uk-scheme-privacy-expert-says