Screen-level bureaucrats in the age of algorithms: An ethnographic study of algorithmically supported public service workers in the Netherlands Police

Abstract

Algorithms are rapidly transforming government bureaucracies. The implications of this transformation for the work of public service employees are not yet well understood. So far, the literature has mostly neglected the use of algorithms by these “screen-level bureaucrats”, and this constitutes a major gap in our knowledge about how algorithms affect bureaucracies. To understand the work of screen-level bureaucrats and to explore how they actively engage with algorithmic software to support their assessment of online fraud, we analyzed 45 hours of observations and 12 in-depth interviews at the Netherlands Police. We employ a socio-material perspective to analyze the dynamics between screen-level bureaucrats and algorithms. We conclude that for administrative tasks, algorithms help screen-level bureaucrats to perform their work by providing structured data and allowing them to focus more on assessments which need a nuanced judgement. At the same time, algorithmic advice in a decision-making task is simply ignored by the screen-level bureaucrats as they predominantly rely on their professional judgement in the assessment of online fraud reports. This highlights the need to further investigate how an algorithm should not only provide accurate advice to the screen-level bureaucrats but also convince them to follow it.

Keywords

Algorithmization screen-level bureaucrats discretion socio-materiality

1. Introduction

Governments are increasingly using algorithms, as they bring the promise of efficient service delivery, better informed decisions, and more insight into complex organizational processes (Meijer & Grimmelikhuijsen, 2020). The automation of work in street-level bureaucracies is a recurring and often studied topic in the academic discourse. In their classic article, Bovens and Zouridis (2002) have investigated how new technologies transform street-level bureaucracies via screen-level to finally system-level bureaucracies. The automatization of work processes shifts decision power away from individual street-level bureaucrats towards the developers of (semi-)automated decision systems. The tasks of street-level bureaucrats in turn are transformed towards the maintenance and optimization of information flows. This has given rise to a new type of workers that do not interact with citizens directly, but do make important administrative decisions that affect them through computer-mediated interactions: so-called screen-level bureaucrats. Their work should be understood as a human-machine collaboration inside bureaucracy.

Even though multiple studies have mapped the consequences of automation for street-level bureaucrats (e.g. Alshallaqi, 2022; Peeters, 2020; Young et al., 2019), few studies have paid explicit attention to the work of screen-level bureaucrats. While recent research indicates that the number and importance of screen-level bureaucrats is on the rise (Zouridis et al., 2020), the literature on public sector automation has hardly investigated this group. This group demands more attention since, with the rise of algorithms in their work, their decision-making processes are both becoming increasingly important and are supported by algorithms in a variety of ways, such as the application of child allowance in the Netherlands (Giest & Klievink, 2022). While the ‘extreme’ patterns of algorithmic aversion and algorithmic penetration have been receiving much attention in the literature (Eubanks, 2019), a more nuanced perspective on decision-making by screen-level bureaucrats resulting from the screen-level bureaucrats interaction with algorithms is lacking. We will apply insights from literature on decisions support systems (e.g. Gillingham, 2021; Petersen et al., 2021) to help understand how screen-level bureaucrats interact with algorithmic systems. Our research aims to open this black box by presenting in-depth qualitative research of human-machine collaborations in government bureaucracies or, more precisely, the perception and use of algorithms in public service delivery by screen-level bureaucrats.

The studies in public administration literature that have touched upon the topic of decision support by algorithms, conceptualize the use of algorithms in public service delivery through a dichotomous framework of enabling versus curtailing (Buffat, 2015; de Boer & Raaphorst, 2021). The idea is that automation can curtail the discretion of the civil servant when a decision is automated or when a system automatically provides suggestions based on case information. In contrast, automation may enable decision-making when it improves current decision-making, for instance, by providing new or improved decision options. Interestingly, there seems to be some evidence for seemingly contradictory patterns: as Petersen et al. (2020) reveal that discretion is a cooperative effort based on consultation and skill, which suggest that contextual factors play a role in automating discretionary practices.

Our research seeks to explain these contradictions, by developing a contextualized understanding of how the use of algorithms changes the work of screen-level bureaucrats. To do so, we employ the classic socio-material practice perspective in order to understand this interaction (Orlikowski, 2000). In recent work, Alshallaqi (2022) employs this socio-material lens to analyze how screen-level bureaucrats (social) interact with algorithms (material) in the provision of public service. He finds that street-level discretion is shaped relationally, and that new technologies reconfigure hierarchical relationships. Thus, the theoretical lens of socio-material practice perspective focuses our attention on the mutual shaping of technology (Williams & Edge, 1996) and, therefore, provides deep insights in the dyadic relation between the screen-level bureaucrat and algorithms.

To enhance our theoretical and empirical understanding of the perception and use of algorithms by screen-level bureaucrats, we apply ethnographic research to study how a department with screen-level bureaucrats working for the Dutch Police interact with a newly developed algorithm that is supposed to enhance their administrative operations. This department uses the Intelligent Crime Reporting tool to process reports of online fraud. Ethnographic research has been underutilized in the field of public administration and e-government and is particularly suited to provide in-depth ‘thick’ description (Cappellaro, 2017; Grimmelikhuijsen et al., 2017). We obtained around 45 hours of observation data, conducted twelve in-depth interviews, and held a presentation to validate the research findings at the Operational Information Processing department.

Our ethnographic research contributes to the growing body of literature on algorithms in public service delivery (Burrell, 2016; Seaver, 2017) by explicitly recognizing the unique and context-dependent role of the human decision-maker: the screen-level bureaucrat. We do not focus on algorithms which independently make decisions about citizen requests but instead focus on the human-machine collaboration between algorithm and screen-level bureaucrat. Our results aim to tell the story of how screen-level bureaucrats at the Dutch Police work with an algorithm in their everyday work activities. This paper thus provides an understanding of the perception and use of these algorithms by screen-level bureaucrats and highlights how bureaucracies are being transformed through the increased use of algorithms as decision-support systems.

2. Concepts and perspectives for studying modern screen-level bureaucrats

In this section we present the concepts and perspectives used to study the interactions between screen-level bureaucrats and the Intelligent Crime Reporting tool. The relevant literature discussed here allowed us to provide a nuanced analysis of the empirical data. This section focuses on literature relating to screen-level bureaucrats and decision-support systems and algorithmization and sociomateriality.

2.1 Screen-level bureaucrats and decision-support systems

The task of a street-level bureaucrat is to provide service in complex and ambiguous situations (Lipsky, 1980). Inherent to their job is the conflict between citizens’ and organizational goals. Street-level bureaucrats exercise discretion in their application of general rules in specific situations in direct interactions with citizens. The introduction of information and communication technology (ICT) alters the way civil servants interact with citizens. Bovens and Zouridis (2002, p. 177) define screen-level bureaucrats as civil servants that indirectly interact with citizens via a computer screen, and hence, the provision of public service is mediated by a software system. Both street and screen-level bureaucrats have the same task of applying general rules in specific situations. The key difference is that the screen-level bureaucrats do not directly meet citizens but mostly interact with citizens indirectly through computer systems (i.e. “the screen”). This difference is not trivial since media theory posits that the medium will transform this interaction through the affordances of the medium (Seaver, 2017).

Zouridis and colleagues (2020) claim that the introduction of algorithms to provide public service leads to the reduction of discretionary leeway for human decision-makers. Discretion is the decision-making leeway that both street- and screen-level bureaucrats have in applying governmental policies in specific and uncertain situations (Bullock, 2019; de Boer & Raaphorst, 2021). Terms such as ‘digital’ (Busch & Henriksen, 2018), ‘algorithmic’ (Young et al., 2019) or ‘automated’ (Zouridis et al., 2020) discretion coin the shift of discretionary leeway from the civil servant to the designers and developers of a software system. Therefore, the application of algorithms in work processes can partially or fully replace the civil servant in situations where discretion is necessary (Busch & Henriksen, 2018). This form of discretion is called artificial discretion (Young et al., 2019). A potential advantage of algorithmic discretion is the improvement of public administration at the task level as it provides increasing scalability, decreasing cost, and improving quality (Young et al., 2019).

The promise of improved organizational processes is at the heart of ideas about decision-support systems (DSS). Academic research on DSS has a history of approximately fifty years. The broad historical progress of DSS research can be categorized in five categories; model-driven DSS, data-driven DSS, communications driven DSS, document-driven DSS and knowledge-driven DSS (Power, 2008a). Power (2008) defines DSS as: “an interactive computer-based system or sub-system intended to help decision makers use communications technologies, data, documents, knowledge, and/or models to identify and solve problems, complete decision process tasks, and make decisions. DSS is a general term for any computer application that enhances a person or group’s ability to make decisions.” (p. 149) For example, knowledge-driven DSS (also called intelligent DSS) is typically associated artificial intelligence as this type of DSS can suggest and recommend actions to the user (Power, 2008b).

In a recent study Bannister and Connolly (2020) suggest that managing the risk of algorithms that “makes a decision bases on subjective data supplied by humans and executes a policy decision based on those data” (p. 481) is the most significant aspect in public administration. Interestingly, research into social workers and the use of decision support tools for administrative operations show that these systems create new workload, burdens and standardize aspects of decision making, which leaves social workers with another feeling of autonomy (Dearman, 2005). Further research into this government sector by Gillingham (2021) suggests social workers’ knowledge and expertise should play a crucial role in the use and development of decision support tools. Finally, the underlying mechanism (e.g. a form) of a decision support tool structures how information is presented and the interaction between the social worker and the citizen (Hoybye-Mortensen, 2015). The focus of this research is on a DSS which makes decisions on data supplied by citizens and the implications of the research into social workers’ use of a decision support tool may have resemblance in the law enforcement sector.

The academic literature presents several indications of how algorithms can transform the relations between bureaucracies and citizens (Zouridis et al., 2020). A central point in these analyses is the question whether the use of algorithms enables or curtails the work of bureaucrats (Buffat, 2015). A large number of studies, however, suggest that the algorithms function autonomously and generally fail to analyze the decision-making process as a joint process of screen-level bureaucrats and algorithms. At the same time, an empirical understanding of modern screen-level bureaucrats who are supported by algorithms is lacking (Bullock et al., 2020). To provide an understanding of algorithmic decision-making as the transformation of work of screen-level bureaucrats supported by the introduction of algorithms we build upon insights from studies of algorithmization of public organizations (Meijer & Grimmelikhuijsen, 2020; Meijer et al., 2021) and more specifically the perspective of socio-materiality (Leonardi, 2011; Orlikowski, 2000).

2.2 Algorithmization and socio-materiality

The literature on algorithmization provides nuances to the dominant (often technical or legal) literature on algorithms in the public sector. This by highlighting that we should not understand the changes in the work of bureaucrats as a process of the introduction and use of new artifacts but rather as a set of complex socio-technical organizational changes around the algorithm (Meijer & Grimmelikhuijsen, 2020; Meijer et al., 2021). A narrow focus on the algorithm as an artifact fails to acknowledge that the algorithm only becomes meaningful in the actual use in a specific organizational setting which consists of roles, competencies, policies et cetera, and provides no insight in the interactive process of shaping between artifact and social context. Therefore, a social science perspective on algorithmization as a complex process of transformation is required.

In this paper we seek to understand how screen-level bureaucrats interact with algorithms by employing the theoretical lens of sociomateriality (Alshallaqi, 2022; Leonardi, 2011; Orlikowski, 2000). This lens helps to understand how specific material conditions are not separate from but defining elements of social situations (Orlikowski, 2007). Building on work of Jane Fountain (2001), Gil-Garcia (2012) addresses these complex interactions with the term ‘enacted technology’. This term refers to the perception and use of an ICT system within an organization and therefore not only focuses on the technology, but also on the social aspects of a technological innovation. In this line of argument, studying social media should not only entail analyzing how the technology is widely used to post messages online. The use of social media also has social implications that change how people give meaning to the freedom of speech and the functioning of politics, media and government. These social implications in turn also change the material agency of social media.

The discussion of the literature has resulted in the identification of the sensitizing concepts and theoretical perspectives that we need to analyze the empirical data in order to understand screen-level bureaucrats working with algorithmic software: Bovens and Zouridis (2002) concept of screen-level bureaucrats, Bullock’s (2019) conceptualization of discretion of bureaucrats, Buffat’s (2015) understanding of the enabling of curtailing effect of algorithms on the position of bureaucrats, Power’s (2008) discussion of decision-support systems, Meijer et al.’s (2021) focus on algorithmization as a s socio-technical process and, finally, the concept of sociomateriality (Orlikowski, 2000; Leonardi, 2011; Alshallaqi, 2022) provides a lens for studying processes of algorithmization. Drawing on these concepts and perspectives, this paper will investigate the perception and use of the Intelligent Crime Reporting tool (i.e. material) by screen-level bureaucrats (i.e. socio).

3. Case description and methodology

The focus of ethnography is on producing local observational data – over an extended period of time – to systematically explore how people give meaning to algorithms in their specific situations (Cappellaro, 2017; Christin, 2020). By means of ethnography, this research disentangles the black box of screen-level bureaucrats’ perception and use of algorithms (Burrell, 2016; Seaver, 2017). First, the case description provides an example of an online fraud report and coming about of the Intelligent Crime Reporting tool.

3.1 Case description

We researched the department of Operational Information Processing, a department of the Netherlands Police. This case was chosen as it is a typical example of screen-level bureaucracy. Traditionally, the Operational Information Processing department is an administrative department with the responsibility to process handwritten fines in police IT systems, validate pictures of speeding in traffic and other administrative tasks. Under pressure of new technologies, the administrative handling in this department is quickly changing and administrators who would occasionally be in touch with citizens to take in reports are increasingly becoming screen-level bureaucrats.

A core task of this department, which used to require interaction with citizens, is the assessment of online fraud. Based on our empirical data, the following presents a generic example of a fraud report aimed to clarify what a fraud report entails.

Merijn, a citizen in need of a new lawn mower, searches the internet for a special offer. In his search, he came across the website named ‘randomwebsite.com’. There he finds a lawn mower with a fifty percent discount. The delivery time is five working days and so Merijn orders a lawn mower. After five working days he does not receive his order. Merijn tries to contact the webstore, but they are not responding to his e-mails. Merijn decides to file an online fraud complaint after waiting an additional five working days and still no e-mail response from the webstore. On the website of the Netherlands Police he files an online fraud complaint. The department of Operational Information Processing receives the complaint and has the task to process Merijn’s online fraud complaint about the missing lawn mower. The task of these screen-level bureaucrats is to check if the fraud complaint is valid based on current law and if the necessary information is in the fraud complaint.

Around thirty civil servants are appointed to perform the task of processing online fraud reports from citizens as Merijn. In 2020 this department received more than 60,000 reports (Boere, 2021) which needed to be processed. Incoming reports have to be checked for validity and completeness. The fraud complain of Merijn is a straightforward case as the payment was made and he did not receive his order. In other cases, part of the order could be missing, or the quality of the product could be disappointing. These cases require a nuanced judgement to determine if the fraud complaint meets the requirements of Article 326 of the Dutch Criminal Code. By processing incoming reports, which police officers may need in larger investigations, the department is perceived as the “the backbone” of the organization. In general, the assessment of fraud reports was time-consuming due to the quality of the report and the need to ask additional questions. Over time, some tasks within the department were automated to support their need to process the ever-growing information flow. The assessment of online fraud was recently supported by the Intelligent Crime Reporting (ICR) tool. The goal of this algorithmic tool is to enhance the quality of incoming reports of online fraud by indicating to citizens the expected validation outcome and to reduce administrative operations.

3.2 Data collection and analysis

In order to understand how these screen-level bureaucrats interact with the ICR tool we obtained around 45 hours of observational data, conducted 12 interviews and held a presentation to validate the research findings at the department. In accordance with abductive reasoning (Schwartz-Shea & Yanow, 2012), the results are a combination of empirical data (i.e. observations, interviews, presentation, and discussion with peers and informants) and theoretical considerations (i.e. algorithmization, screen-level bureaucrats, discretion, and socio-materiality). Observations and interviews were used as complementary methods as the observational data was verified in interviews and the interviews gave leads for when the observer was taking fieldnotes at the department.

The data collection was carried out in five iterations. Each iteration consisted of a period of two weeks on-site gathering field data and time period off-site. The first author of this research was the observer and interviewer at the department of Operational Information Processing. On average the first author observed one civil servant for four hours a day. This period corresponded with the time the civil servant usually assessed online fraud reports on a given day. The degree of participation by the first author was passive (Spradley, 1980). This entails that the first author constantly reduced his active participation when in the field.

The time off-site allowed the first author to reflect on the data and discuss it with peers and informants. These discussions were an appropriate and reliable member-checking mechanism (Schwartz-Shea & Yanow, 2012). After three iterations the data were categorized and validated in a presentation at the department. In the fourth iteration six interviews were held to primarily verify the research findings and check if the criterion of data saturation was met. The final iteration interviews were held with new civil servants who started to work for the department within the previous two months.

Figure 1.

Overview of the five iterations.

Figure 1 provides an overview of the five iterations. For the interviews, we used a semi-structured topic list, allowing to ask clarification questions on observations obtained the same day. Building upon our theoretical perspective, the topics were algorithmization, use of the ICR tool, and bureaucratic discretion. The time in the field took place between April and October 2021, and the first author observed sixteen participants.

Prior to the start of the field research, the observer (first author of this article) was an employee of the Dutch Police. As a former colleague, the first author was able to engage in close interactions with observants on a familiar level. With his prior knowledge of the police, civil servants were more open on discussing police specific themes, such as the everyday experience of working in a bureaucracy and working with the new algorithmic system.

However, even with the author’s police background, difficulties had to be overcome. Obtaining observational data during Covid-19 was uncertain, several appointments were adjusted or canceled at a short notice, and several observers were less enthusiastic about being observed. Furthermore, we recognize that the observer’s police background could hinder the independence of this research, and his role as observer and former colleague might cause confusion. To prevent such confusion, the observers’ role as an observer was stated at the start of each observation session or interview.

The interviews were transcribed verbatim and coded using Nvivo 12 Pro software. First, open codes were used to structure the interview data and observations. Following this, the codes were categorized which resulted in three themes. Finally, these themes were discussed with peers and civil servants which lead to the findings presented in this paper. The names of the civil servants in the result section are pseudonyms.

4. Findings

To present the findings in an orderly manner and introduce the reader to the specifics of the in-depth case-study, we first introduce the socio-material context of the department of Operational Information Processing and the design history (4.1). Then we proceed to the analysis of the key themes that emerged from the abductive analysis of empirical analysis and the confrontation with insight from the literature. The three main themes present a layered examination of the task of assessing an online fraud report (4.2) and the advantages (4.3) and disadvantages (4.4) of the ICR tool in these assessments.

4.1 Socio-material context: design history and functioning of the Intelligent Crime Reporting tool

To introduce the reader to the specific empirical context of the case, this section aims to clarify the design history and presents the work environment in which civil servants of the department of Operational Information Processing at the Netherlands Police perform the task of assessing an online fraud report. In addition, the explanation of the design history and functioning of the Intelligent Crime Reporting tool allows us to analyze, in detail, how the civil servant interacts with the Intelligent Crime Reporting tool.

In the many hours that we observed their work, it became clear that the work of screen-level bureaucrats can be demanding: processing data from an information source in a database is time-consuming due to the necessary manual actions, the quality of incoming reports is in need of improvement and correspondence with citizens is done via email. Finally, revising an incorrect report is time consuming due to the necessary manual actions in the software system. Department coordinator Jop explained the time-consuming nature of revising an incorrect report in the software system as follows: “it takes our revise team six to eight minutes per report to complete the revise form […] this consumes a lot of hours each week.” Moreover, each year the police had to revise thousands of incorrect fraud reports. These demanding administrative operations were important reasons for the development of the Intelligent Crime Reporting tool. The goal of the Intelligent Crime Reporting tool is to give citizens an immediate reply whether their report is indeed an online fraud report or not (Van Wijnen, 2019).

The Intelligent Crime Reporting tool is a rule-based algorithm that runs on an argumentation engine and is part of the Internet Report Module. This module is the software system in which civil servants process online fraud reports. Based on a number of questions, the Intelligent Crime Reporting tool produces advice, which gives the citizen insight into whether the fraud report meets the required legal specifications (i.e. Article 326 of the Dutch Criminal Code). If the advice is positive, the Intelligent Crime Reporting tool advises filing an online fraud report and if negative advice is given, the tool gives tips on how to proceed with the complaint. Interestingly, the Intelligent Crime Reporting tool was originally designed to help citizens fill out their fraud reports, thereby reducing the workload for the civil servants and providing algorithmic assistance to citizens. As a result of this design history, the Intelligent Crime Reporting tool has only minor compatibility with the Internet Report Module and, hence, the software program in which they need to process the online fraud report. Department coordinator Jop explains: “On the front-end citizens have an amazing product.” However, at the back end, civil servants are processing the report in the legacy system and therefore they are maintaining their familiar work practices.

For example, the original output of the Intelligent Crime Reporting tool was meant to explain to citizens whether they should or should not file a fraud report with the police. To make the output of the Intelligent Crime Reporting tool fit the interface of the Internet Report module, only one sentence could be adjusted in the interface of this software system. In this interface screen-level bureaucrats can read what the advice (e.g. accept, deny or no conclusion) of the algorithm is, but they do not get insights into the logic behind the advice (i.e. how the Intelligent Crime Reporting tool came to its decision). Developer Steven elaborated on the output of the Intelligent Crime Reporting tool:

“There are two aspects. One is that the Intelligent Crime Reporting tool provides a conclusion [advise to the citizen] and this adds to the report and is visible for the employees […]. What the employees do with it [the advice of the algorithm] was a minor concern.”

While the logic behind the advice is accessible through another program, this type of access through additional manual actions is time-consuming. Due to the time constraints and the necessity to process the report in the Internet Report module we observed that screen-level bureaucrats neglect the other program.

What this section pronounces, is the socio-material context, design history and functioning of the Intelligent Crime Reporting tool. In essence, the output (e.g. the advice) of the Intelligent Crime Reporting tool is a check whether the report of the citizen is valid, based on current legal requirements. In other words, the administrative operation of validating an online fraud report is automated.

4.2 Algorithmization of administrative operations

A core promise about algorithm-use in government is that such systems make administrative processes and service delivery more effective and efficient. However, such promise can only be delivered under the assumption that technologies are easy to use and can be integrated in the work of screen-level bureaucrats. In our empirical evidence, we find that such assumptions are not easily met. In their daily work with the Intelligent Crime Reporting tool, screen-level bureaucrats express irritation and even find it irrelevant to their work. To understand how they come to these assumptions, we specifically analyzed the rich empirical material to find out how they assess an online fraud report. In other words, we need to look at the collaboration between the human and the software system when performing an administrative task.

When screen-level bureaucrats perform the task of judging an online fraud report, they check the report for validity and completeness, and ask citizens additional questions by email if necessary. The software system in which screen-level bureaucrats processes the report is called the Internet Report module. From the top to the bottom of the screen, the interface of this software system shows: 1. the advice of the Intelligent Crime Reporting tool, 2. information of the reporter, 3. description of the event, 4. information about the accused, and 5. the option to accept, decline or to assign the report to a colleague. In general, screen-level bureaucrats first read the ‘description of the event’ to get an impression of what happened, as interviewee Saskia elaborated on this practice:

“You go to the description to see what exactly happened, I read it every time, based on that I know basically a lot, after that I will continue [assessing the report].”

The quote of Saskia indicates that although the description of the event says “a lot” about the validity of the fraud report, she also needs to check the report for the necessary information (e.g. name website, bank accounts). If the online report is valid and complete she can accept the fraud report. In general, we found that all the screen-level bureaucrats reject the online report if it is invalid (e.g. the report is a civil case) or incomplete (e.g. missing bank account) and assign the online report to a colleague if they are in flux. The following field note – and we had many similar field notes since this was a frequent observations – explains the generic work practice of assessing and report of online fraud:

Observant reads the description of the event, checks if the necessary information is provided; the order has been cancelled so this report is a civil case.

The validation of the report is what screen-level bureaucrats describe as the core of their task. In the fieldnote presented above, the reporter has cancelled the order, which means the online fraud report is a civil case. Additionally, screen-level bureaucrats check whether the report has the necessary information to be further processed. The check for completeness may give additional contextual information to accept and decline the report. In these checks for completeness, the we regularly observed that complaints against trustworthy webshops are instantly rejected and likewise, complaints against well-known suspicious webshops are accepted. A nuanced view of the essence of assessing an online fraud report is explained by civil servant Jan:

“In my experience it is nice to answer the question when the payment is done […] In the description of the event the person can say they never received something, however the person files a report after two days. […] the date when the payment is done can change the manner in which I read the report.”

In other words, the explanation of civil servant Jan indicates that the moment when a payment is made can influence if an online fraud report is valid or not. Multiple screen-level bureaucrats expressed that they read the description of the event cautiously to validate the online fraud report. Interpreting the description of the event is therefore the practice of validating an online fraud report. Observant Kevin further nuances this practice:

“When I quickly read the report, I’ll read ’crane’. You would think it’s about a crane, but if you read closely the report is about a model crane. […] You need to check the report because if you look closely the citizen submitted a random bank account of the counter party just to continue with the submission of the fraud report.”

Performing the administrative task of assessing an online fraud report entails interpreting the text provided by citizens and check if the report meets the conditions for further processing. In the next section, screen-level bureaucrats explain their experience with the Intelligent Crime Reporting tool for providing structured data.

4.3 Intelligent Crime Reporting is a helpful assistant in providing structured data

During our observations of screen-level bureaucrats in their assessment of fraud reports, they often ignored the algorithmic advice. Notably, the limited adaptation of the algorithm in the work practice of screen-level bureaucrats does not mean that they do not use it at all. Instead, we observed that they use it for specific tasks. Interestingly, screen-level bureaucrats explain the supportive nature (i.e. the affordance) of the Intelligent Crime Reporting tool by contrasting it with another task with a different work process. The task of assessing online fraud reports is just one of the administrative operations screen-level bureaucrats are responsible for, and it is highly automated compared to the other administrative tasks. In a less automated task we called ‘fine collection’, screen-level bureaucrats have to manually process incoming reports of citizens. As we observed, to process a report implies that screen-level bureaucrats collects information from different data sources and processes this manually in a software system. This constraint in comparison to assessing an online fraud report was regularly expressed by the observants. Elaborating on these two work processes Alex explains:

“The process of assessing an online fraud report is automated on the front side of the process […] Back in the day, the assessment [of an online fraud report] was the same as with fine collection.”

The statement of Alex and our observations indicates that the supportive nature of the Intelligent Crime Reporting tool lies within its ability to provide structured data. In this way, in comparison to fine collection, screen-level bureaucrats are alleviated from the task of manually copying the same data from one system into another. By reducing the time needed for the administrative task by several minutes, civil servants can assess an online fraud report faster compared to fine collection. Eshter compares these two tasks and explains the following:

“And the development is just really beautiful, you just run through the report, check if it is correct, is it the right type of report, do we have all the information we need, accept and the report is processed in the right software system. With fine collection its totally different and that is, if you ask me, a disappointment.”

Furthermore, the shift in administrative operations from the civil servant to the citizen implies that the tasks of manually processing information in a software system lies with the citizen. Screen-level bureaucrat Jan explains that the time of the citizen to make a notification of fraudulent behavior is spent more effectively, and it affords him to focus on another task:

“I think that I spent around five percent of the time assessing an online fraud report compared to fine collection. […] The citizen fills in the online fraud report, I read it and assess it […]. With fine collection it’s such a long chain from report to verdict. […] I need to write the report and it is just like if the reporter sits in front of me and I have to type the report, which takes a lot of time. […] While in Internet Report Module the time it takes to write a report lies with the reporter.”

Jan’s quote indicates the key supportive nature of the Intelligent Crime Reporting tool; as a helpful assistant in its ability to provide structured data and therefore reducing administrative operations. Interestingly, the process of structuring data, either manually or automatic, is a process with no discretion involved. It seems that in cases where discretion is not threatened by an algorithm, screen-level bureaucrats have no issues and even see algorithmic systems as a ‘helpful administrative assistant’. Additionally, the reduction of administrative operations means that part of screen-level bureaucrats’ job characteristics is lost and changed. Now that the Intelligent Crime Reporting tool has automated administrative tasks (i.e. the validation of the report), it allows civil servants to focus on reports which need nuanced judgement. These are reports which require more attention as the context (e.g. one component of the order is delivered, or the product seems fake) determines if it is an attempt at online fraud. The next section focuses on how screen-level bureaucrats use the Intelligent Crime Reporting tool for validating an online fraud report.

4.4 The ‘computer says yes’ problem limits its value for validating an online fraud report

Our final theme focuses on the dynamics between screen-level bureaucrats and the advice of the Intelligent Crime Reporting tool in the validation of an online fraud report. Most notably, our research revealed that for validating an online fraud report the algorithmic advice provides little value. This is what we call “the computer says yes” problem: the advice of the Intelligent Crime Reporting tool provides little added value for screen-level bureaucrats due to the failure to present arguments for recommendations. Although screen-level bureaucrats were positive about the reduction of administrative operations in the task of assessing a report, they found the output (advice) of the Intelligent Crime Reporting tool irritating and irrelevant.

During our observations we noted a crucial reason for the irrelevance of the algorithmic advice. Screen-level bureaucrats need to read the ‘description of the event’ in order to check if the fraud report is valid. René reflects on the irrelevance of the algorithmic advice for validating an online fraud report:

“Crudely said, if it says “accept” it does not help me, because I still have to check if the report is valid and if all information is in it […].”

What René indicates to be a significant part of her job characteristics is her need to validate the online fraud report. We observed that to validate the report of the citizen, screen-level bureaucrats read the explanation in the description of the event. After reading this section, screen-level bureaucrats have the information needed to validate the report. Jan, for example, elaborated on the irrelevance of the Intelligent Crime Reporting tool’s output:

“It [the advice] supports my own conclusion, I cannot blindly follow the advice of the Intelligent Crime Reporting tool. […] If it says accept, it tells me nothing, I cannot say, ‘sure I will accept it’. I still have to read it all. Then it verifies my assumption if I think, yes indeed, this one [fraud report] meets the requirements.”

Jan makes clear that in situations where civil servants have some discretion, they are hesitant to only base their assessment on the advice of the Intelligent Crime Reporting tool when no rationale for the advice is presented. Relying only on the advice of the Intelligent Crime Reporting tool does not allow civil servants to fully grasp the intricacies of an online fraud report. In this way, they always have to read the description of the event to judge for themselves if an online fraud report meets the acceptance specifications.

During our observations one screen-level bureaucrat stressed the irrelevance of the output of the Intelligent Crime Reporting tool “look, it shows possible fraud attempt, but I can’t do anything with it”. After the explanation, the screen-level bureaucrat reads the description of the event, checks if the necessary information is in the report and decides to assign the report to a colleague. Merel explains her experience with the irrelevant advice of the Intelligent Crime Reporting tool:

“For example, if the Intelligent Crime Reporting tool states ‘approve’, advice approved or something else, yeah then I’ve already passed it before I fully comprehend what I saw. And in my experience it regularly states something which I think of, ‘didn’t wait long enough’ as an example. That is correct, but it [the description of the event] states clearly the reasons for why this person filled a complaint. It looked on all kinds of websites and saw that other people where victim of the same person and that an online marketplace had banned him for the attempt of fraud. And sometimes it is just that clear [why the citizen didn’t wait long enough] that I think, yeah nice that the advice is on there, but I still need to read it to know the reason why.”

Merel’s explanation clearly illustrates the computer says yes problem in validating a report; due to a lack of explanation she still needs to read the description of the event. In essence, screen-level bureaucrats relies on their professional judgement to judge for themselves if the report is valid or not and otherwise assign the report to a colleague.

In summary, these empirical findings show that the design of an algorithm for a specific goal (e.g. helping citizens fill an online fraud report) does not automatically entail added value for screen-level bureaucrats. In the performance of an administrative task these screen-level bureaucrats view the algorithm as a helpful assistant in providing structured data. However, in the assessment of a fraud report the added value of the algorithm remains unanswered since the algorithm fails to provide the rationale for its advice.

5. Discussion and conclusion

Our main objective in this research was to investigate how screen-level bureaucrats interact with algorithms in the provision of public service. This is one of the few studies in public administration literature to provide an in-depth empirical understanding - over an extended period of time - of how screen-level bureaucrats work with an algorithm. In doing so we answer to a recurring call for this type of research in the public administration literature (e.g. Bullock et al., 2020; Busch et al., 2018; Peeters et al., 2020; Zouridis et al., 2020). The results of our empirical study reveals three main findings: algorithmization of administrative operations, Intelligent Crime Reporting is a helpful assistant in providing structured data, and the ‘computer says yes’ problem limits its value for validating an online fraud report. Through the lens of affordances and constraints (Orlikowski, 2000; Leonardi, 2011; Alshallaqi, 2022) this research illustrates that the same algorithm can both enable and curtail (Buffat, 2015) screen-level bureaucrats in the provision of public service and, therefore, the impact it has on the discretionary freedom of screen-level bureaucrats. Our findings show that contextual factors such as ‘when the payment of an order is done’ are relevant in determining the usefulness of an algorithmic advice for screen-level bureaucrats. In public administration literature, the notion of contextual factors has thus far received little attention.

As our study reveals, human-computer interaction research in public administration may indeed be a venue for academic investment. Interestingly, research in the social sectors (Gillingham, 2021) shows the necessity to include screen-level bureaucrats in the development and implementation of algorithms. Human-computer interaction research, and specifically Explainable Artificial Intelligence, shows that the role of the user should take central point in “any type of situation in which users exploit the possibilities of decision support agents for their own decision, in the context of their task, and within the framework of their activities and responsibilities” (Chaput et al., 2021, p. 2). Peeters (2020) suggests that professionals (e.g. screen-level bureaucrats) should have sufficient instructions to supervise and work with and algorithmic system. To this end, we argue that future work should focus on how algorithmic explainability may increase the insight in (and usefulness of) algorithms in the work of screen-level bureaucrats. Recent experimental work has shown that explainability increases citizen trust in algorithmic outcomes (e.g. Grimmelikhuijsen, 2022). Likewise, screen-level bureaucrats may be better able to assess the usefulness of an algorithmic outcome when the underlying basis of the assessment is explained to them. When they better understand the reasons or logic underlying an algorithmic outcome, so that they can better assess whether they can use it or not, and for which reasons.

It may seem trivial to describe why the recommendations of algorithms are ignored, however this description reveals that the initial promise of algorithms as a means to improve an administrative operation is not easily met. A seemingly simple technological solution, in this case automating screen-level bureaucrats’ task of validating a report, to enhance public service neglects the role of screen-level bureaucrats in the automated process. Reducing the screen-level bureaucrats into just an approval mechanism may have enhanced efficiency, but this enhancement alters how a public organization is organized and specifically in this case who is responsible of validating an online report. On one hand, if an algorithmic system is responsible for validating a request of public service, then the mechanisms to question the recommendation and accountability may also need reconfiguration. On the other hand however, the collaboration between screen-level bureaucrats and algorithms could be a venue for further academic research.

Our final contribution is that an algorithm is embedded in a complex technological environment. We need to enrich this complex technological environment with a socio-material context to understand the relation between the screen-level bureaucrat and the algorithm. Referring to Giddens (1984), part of the task in which a screen-level bureaucrat mobilizes some kind of resource is replaced by the algorithm. This entails that performing a task is partly automated. However, in a system-level bureaucracy the mobilization of ‘some kind of resource’ to provide public service is diffuse as the algorithm is part of a larger web of software systems. In today’s bureaucracies implementing an algorithm means implementing a software program in a larger web of software systems. The dependency of an algorithm can vary; it may need data from or conform to the legacy system or the other way around. This dependency may alter the development possibilities of the algorithm and hence how screen-level bureaucrats can use the algorithm to provide public service. A seemingly trivial question of whether to process an application though the new algorithm or the legacy system may determine the useability of the algorithm. In public administration, this overlooked notion of a complex technological environment may indeed alter the way a bureaucracy is organized.

Each study has its limitations. The findings presented in this paper are based on qualitative research of one specific bureaucratic organization. The contextual nature (e.g., one department and organization) of this study means that other contextual factors may play a role in other bureaucratic organizations. For this reason, the key outcomes of the study need to be investigated further in different contexts to explore to which extent design histories generate problems for algorithms, to which extent algorithms are indeed capable of carrying out administrative tasks and, most importantly to obtain a theoretical and empirical understanding of how the algorithmic rationale influences whether screen-level bureaucrats also rely on algorithmic advices when carrying out assessments which need a nuanced judgement.

To conclude, this study underlines the urgent need to put interactions between bureaucrats and algorithms on our research agenda. Our study shows that the reliance of the screen-level bureaucrat on a computer system to perform a task means that an algorithm alters the way these civil servants provide public service. In the case of flying an airplane, Bainbridge (1983) revealed that the autopilot function altered the way pilots fly an airplane. In this light, government organizations are at the same point as aviation was almost forty years earlier. We believe that public administration literature and government organizations can further investigate if and why algorithms can be of use in the provision of public service and what is needed in terms of the rationale provided to explain advice to build a trusting human-machine collaboration.

Footnotes

Acknowledgments

This work is part of the ALGOPOL research project and has received financial support by the Netherlands Organization for Scientific Research (NWO, grant number 406.DI.19.011).

Authors' biographies

Carlos Soares is an advisor at the Dutch National Police. Currently, he is doing a parttime PhD student at the Utrecht University School of Governance and the Dutch National Police AI-lab. His research focuses on the impact of algorithms on the work practices of civil servants.

Stephan Grimmelikhuijsen is associate professor at the Utrecht University School of Governance, The Netherlands. His research concerns technology in government, citizen-state interactions, and behavioral public administration.

Professor Albert Jacob Meijer teaches public administration and policy sciences at the bachelor and master level. He does research on technology and governance.

References

Alshallaqi

(2022). The complexities of digitization and street-level discretion: A socio-materiality perspective. Public Management Review, 1-23. doi: 10.1080/14719037.2022.2042726.

Bainbridge

(1983). Ironies of automation. Automatica, 19(6), 775-779. doi: 10.1016/0005-1098(83)90046-8.

Bannister

, & Connolly

(2020). Administration by algorithm: A risk management framework. Information Polity, 25(4), 471-490. doi: 10.3233/IP-200249.

Boere

(2021, 24 November). Het wemelt van de nepwinkels en we tuinen er massaal in. Het Algemeen Dagblad. https://advance-lexis-com.proxy.library.uu.nl/api/document?collection=news&id=urn:contentItem:6451-34X1-JC8X-62XV-00000-00&context=1516831.

Bovens

, & Zouridis

(2002). From street-level to system-level bureaucracies: How information and communication technology is transforming administrative discretion and constitutional control. Public Administration Review, 62(2), 174-184. doi: 10.1111/0033-3352.00168.

Buffat

(2015). Street-level bureaucracy and e-government. Public Management Review, 17(1), 149-161. doi: 10.1080/14719037.2013.771699.

Bullock

J.B.

(2019). Artificial intelligence, discretion, and bureaucracy. The American Review of Public Administration, 49(7), 751-761. doi: 10.1177/0275074019856123.

Bullock

Young

M.M.

, & Wang

Y.-F.

(2020). Artificial intelligence, bureaucratic form, and discretion in public service. Information Polity, 25(4), 491-506. doi: 10.3233/IP-200223.

Burrell

(2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 205395171562251. doi: 10.1177/2053951715622512.

10.

Busch

P.A.

, & Henriksen

H.Z.

(2018). Digital discretion: A systematic literature review of ICT and street-level discretion. Information Polity, 23(1), 3-28. doi: 10.3233/IP-170050.

11.

Cappellaro

(2017). Ethnography in public management research: A systematic review and future directions. International Public Management Journal, 20(1), 14-48. doi: 10.1080/10967494.2016.1143423.

12.

Chaput

Cordie

, & Mile

(2021, 2 February). Explanation for humans, for machines, for Human-Machine interactions? https://hal.science/hal-03106286.

13.

Christin

(2020). The ethnographer and the algorithm: Beyond the black box. Theory and Society, 49(5–6), 897-918. doi: 10.1007/s11186-020-09411-3.

14.

de Boer

, & Raaphorst

(2023a). Automation and discretion: Explaining the effect of automation on how street-level bureaucrats enforce. Public Management Review, 25(1), 42-62. doi: 10.1080/14719037.2021.1937684.

15.

Dearman

(2005). Computerized social casework recording: Autonomy and control in australia’s income support agency. Labor Studies Journal, 30(1), 47-65. doi: 10.1177/0160449X0503000104.

16.

Eubanks

(2019). Automating inequality: How high-tech tools profile, police, and punish the poor (First Picador edition). Picador St. Martin’s Press.

17.

Fountain

J.E.

(2001). Building the virtual state: Information technology and institutional change. Brookings Institution Press.

18.

Giddens

(1984). The constitution of society: Outline of the theory of structuration. Polity Press.

19.

Giest

S.N.

, & Klievink

(2022). More than a digital system: How AI is changing the role of bureaucrats in different organizational contexts. Public Management Review, 1-20. doi: 10.1080/14719037.2022.2095001.

20.

Gil-Garcia

J.R.

(2012). Enacting Electronic Government Success: An Integrative Study of Government-wide Websites, Organizational Capabilities, and Institutions (Vol. 31). Springer US. doi: 10.1007/978-1-4614-2015-6.

21.

Gillingham

(2021). Algorithmically based decision support tools: Skeptical thinking about the inclusion of previous involvement. Practice, 33(1), 37-50. doi: 10.1080/09503153.2020.1749584.

22.

Grimmelikhuijsen

Tummers

, & Pandey

S.K.

(2017). Promoting state-of-the-art methods in public management research. International Public Management Journal, 20(1), 7-13. doi: 10.1080/10967494.2016.1169066.

23.

Hoybye-Mortensen

(2015). Decision-making tools and their influence on caseworkers’ room for discretion. British Journal of Social Work, 45(2), 600-615. doi: 10.1093/bjsw/bct144.

24.

Leonardi. (2011). When flexible routines meet flexible technologies: Affordance, constraint, and the imbrication of human and material agencies. MIS Quarterly, 35(1), 147. doi: 10.2307/23043493.

25.

Lipsky

(1980). Street-level bureaucracy: Dilemmas of the individual in public services. Russell Sage Foundation.

26.

Meijer

, & Grimmelikhuijsen

(2020). Responsible and accountable algorithmization: How to Generate Citizen Trust in Governmental Usage of Algorithms. In: R. Peeters & M. Schuilenburg (eds), The Algorithmic Society. London: Routledge.

27.

Meijer

Lorenz

, & Wessels

(2021). Algorithmization of bureaucratic organizations: Using a practice lens to study how context shapes predictive policing systems. Public Administration Review, 81(5), 837-846. doi: 10.1111/puar.13391.

28.

Orlikowski

W.J.

(2000). Using technology and constituting structures: A practice lens for studying technology in organizations. Organization Science, 11(4), 404-428. doi: 10.1287/orsc.11.4.404.14600.

29.

Orlikowski

W.J.

(2007). Sociomaterial practices: Exploring technology at work. Organization Studies, 28(9), 1435-1448. doi: 10.1177/0170840607081138.

30.

Peeters

(2020). The agency of algorithms: Understanding human-algorithm interaction in administrative decision-making. Information Polity, 25(4), 507-522. doi: 10.3233/IP-200253.

31.

Petersen

A.C.M.

Christensen

L.R.

Harper

, & Hildebrandt

(2021). We would never write that down? Classifications of unemployed and data challenges for AI. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1-26. doi: 10.1145/3449176.

32.

Petersen

A.C.M.

Christensen

L.R.

, & Hildebrandt

T.T.

(2020). The role of discretion in the age of automation. Computer Supported Cooperative Work (CSCW), 29(3), 303-333. doi: 10.1007/s10606-020-09371-3.

33.

Power

D.J.

(2008a). Decision Support Systems: A Historical Overview. In F. Burstein & C.W. Holsapple, Handbook on Decision Support Systems 1, Springer Berlin Heidelberg, pp. 121-140. doi: 10.1007/978-3-540-48713-5_7.

34.

Power

D.J.

(2008b). Understanding data-driven decision support systems. Information Systems Management, 25(2), 149-154. doi: 10.1080/10580530801941124.

35.

Schwartz-Shea

, & Yanow

(2012). Interpretive research design: Concepts and processes. Routledge.

36.

Seaver

(2017). Algorithms as culture: Some tactics for the ethnography of algorithmic systems. Big Data & Society, 4(2), 205395171773810. doi: 10.1177/2053951717738104.

37.

Spradley

J.P.

(1980). Participant observation. Holt, Rinehart and Winston.

38.

Van Wijnen

J.F.

(2019, January). Politie schakelt kunstmatige intelligentie in om aangiftes internetfraude te reduceren. Het Financieele Dagblad. https://advance.lexis.com/document?crid=9a54de3a-22f9-4edf-9043-b25630558372&pddocfullpath=%2Fshared%2Fdocument%2Fnews%2Furn%3AcontentItem%3A5V6V-6SC1-JCD9-2058-00000-00&pdsourcegroupingtype=&pdcontentcomponentid=413081&pdmfid=1516831&pdisurlapi=true.

39.

Williams

, & Edge

(1996). The social shaping of technology. Research Policy, 25(6), 865-899. doi: 10.1016/0048-7333(96)00885-2.

40.

Young

M.M.

Bullock

J.B.

, & Lecy

J.D.

(2019). Artificial Discretion as a Tool of Governance: A Framework for Understanding the Impact of Artificial Intelligence on Public Administration. Perspectives on Public Management and Governance, gvz014. doi: 10.1093/ppmgov/gvz014.

41.

Zouridis

van Eck

, & Bovens

(2020). Automated Discretion. In T. Evans & P. Hupe (Eds.), Discretion and the Quest for Controlled Freedom, Springer International Publishing, pp. 313-329. doi: 10.1007/978-3-030-19566-3_20.