RUDI: An evidence-based police-centric guide for approaching the development of algorithmic models in policing

Multi-disciplinary team oversight

The first step in developing the rationale for why a particular MLA solution is required is bringing together the right team of people to define and oversee the project. Indeed, the role of experts in the development of algorithms is extremely important and the consequences of not having the right people in place on a team can encourage misuse of the model and cause reputational risks for the police (Bland, 2020; Sayer et al., 2024). Assigning responsibility via a multidisciplinary team at the earliest opportunity ensures consistent accountability for the model’s performance at all stages and allows internal auditing of the process to take placethat is, those in charge of developing the model are different from those reviewing the model’s performance. Certainly, responsibility to rigorously test not only the model but also the governance implications such as fairness, bias, and transparency, falls on the team collectively (Busuioc, 2021). Babuta (2017) notes that when police forces seek to develop a model, they often work completely independently from model developers; likely to rarely or never meet. However, collaboration between data scientists and domain experts ensures a shared understanding of diverse business concepts, preventing potential misinterpretations and misrepresentations that could lead to failure (Viaene, 2013). Modelling that adopts a ‘conversation mode’ allows data scientists to tap into the intimate knowledge held by domain experts regarding business ideas and their context. This approach facilitates the identification and challenge of any underlying assumptions or biases in the data. Through constructive and focused conversations, a comprehensive, holistic view of the business context can be attained.

Develop a business case

Once established, the main task of the multi-disciplinary team is to develop a business case. This is because the introduction of modelling represents a substantial commitment, involving both time and resources. Therefore, at the outset, it is important to establish a foundation for modelling through properly conceptualising the project’s aims and developing a shared understanding of key definitions, for example, ‘high harm’, who the initiative will target, and what mechanism will be best for doing this. Additionally, it ensures the modelling is necessary and that both the benefits and long-term costs are considered (Babuta and Oswald, 2020). Without such forethought there is a high chance the deployment of a model will fail (McKay and Richard, 2022).

A clear business case facilitates a shared understanding of how the model works, as inability to fully explain the algorithmic output can lead to a lack of transparency (Mittelstadt et al., 2016), directly affecting accountability for the model and its decisions (Yeung, 2018). Additionally, it allows for the entire process to be auditable, enabling transparency on all decisions, which helps create an institutional memory and documentation so if key personnel leave the project, the learning does not leave with them. It can also be used to establish specific, measurable and achievable evaluation criteria at the outset of the project. Currently it is difficult to ascertain whether algorithmic models achieve what they set out to do, that is, reduce crime – particularly because they are often data-driven rather than theory-driven (Saunders et al., 2016). Flawed or insufficient theoretical underpinning can result in a range of issues and unintended consequences, such as distortions in the analysis that would likely produce patterns that turn out to be spurious correlations (Kaufmann et al., 2019). However, documenting decisions and processes readies the model for transparent validation, evaluation and the comparative studies needed if an evidence base is to be created regarding the costs and benefits of AI applications in policing (EUCPN, 2022). These can then be kept in mind during the development phase, as well as used for more formal evaluations, adding to the evidence-base. It should be noted that projects can take an ‘exploratory’ approach, which involves starting with the data analysis and not establishing a clear purpose for analysis until after the insights have been generated if a business case is then clearly established (Babuta and Oswald, 2020).

A business case also allows forces to think through likely costs and capacity issues. Certainly, there are concerns that police are using MLAs as a solution to bypass the real issue – lack of personnel and resources (Babuta and Oswald, 2020). Therefore, algorithmic tools as opposed to non-technological solutions may be unnecessary and incur many potential unrealised costs when money might be better spent elsewhere. For example, most forces’ storage infrastructures are unlikely to be suitable for real-time dynamic data and data mining applications and cloud technology can be costly, especially on an ongoing basis (Robinson, 2019). Equally, there are staff costs associated with upskilling officers to lead on data projects, as well as potentially teaching staff how to develop and maintain algorithmic models that depend on ML techniques. Additionally, extra staff or resources may be required to ensure there is capacity to properly respond to the algorithm’s output, as well as allow for external evaluations of the model and training of end-users on how to use the model.

Moreover, documenting the aims and suitable and unsuitable use cases ahead of time guards against ‘function creep’. This can also be done through purpose limitation at the development stage (Fantin et al., 2020). Function creep occurs when the use of an algorithm expands beyond its original purpose, resulting in applications that are incongruent with its original purpose, which can create unintended and uncontrolled consequences. For example, a study by Saunders et al. (2016), evaluated the Strategic Subjects List (SSL), which was intended to identify high risk individuals deemed to be suitable for bespoke intervention to reduce the prevalence of gun violence. Instead, they found being identified by the SSL was related to an increase in the likelihood of being arrested for a shooting, suggesting the algorithm drove greater surveillance and enforcement rather than increasing other preventative strategies.

Data quality

The poor data quality of police forces has been recognised (Office for Statistics Regulation, 2024). One of the most straightforward ways to mitigate this is to improve force data recording and quality, ensuring robust oversight and regular data quality assessments (EUCPN, 2022). MLA performance is dependent on the data that it is being trained and tested on. This concerns the quality of input data, which can be mediated programmatically to a degree (Jain et al., 2020), but still affects the predictive outcomes through missingness (Davies et al., 2022). Also, to make accurate predictions MLAs require access to all the knowledge that might be pertinent to decision making. For example, if we are predicting the likelihood of an escalation into serious crime, we need to know of all the previous relevant instances of criminal patterns of escalation and whether they were interrupted by successful interventions, prison, removal from the area, or death. This relevant data is currently spread across disparate databases and, therefore, requires unification.

Development

Before the development of an MLA solution takes place, thought must be given both to what data is pertinent, why it is required and its quality, as well as if there is the necessary expertise within force to develop an algorithm.

Data needs

To fulfil the intended purpose of an algorithm and help to ensure transparency and auditability, Felzmann et al. (2020) advocate a transparency-by-design approach, requiring discussion and agreement of what data specifically is needed. Gebru et al. (2021) proposed using standard documentary procedures for every ‘component’ of the model, no matter how simple or complex, in similar fashion to the electronics industry. Forces are encouraged to document their decisions not only in the business case template, but also using data and model cards. These allow potential costs to be considered at the start of the project and make issues like data quality clear. The templates and cards can be completed iteratively throughout the project to help ensure the model and the process of its development is explainable and transparent. The data cards document an overview of the properties of the dataset used in the model, whilst the model cards document processes so that models can be audited, compared and evaluated. These prompt data scientists to describe any data quality issues and the resulting model limitations, which can feedback into data quality improvement efforts. Additionally, the cards can explain how the model was developed, for instance how data was split into training and validation sets or decisions on cut offs, such as what values would be considered errors for example, a suspect aged 5, so it is clear to internal and external stakeholders. These cards improve model transparency, explicability, and reproducibility. The documentation could also be used to make algorithms more public facing, enabling external scrutiny and alternatives to be tested, especially regarding categories that can serve as proxies for protected characteristics like race and sex (Palmiotto, 2021).

Nevertheless, to some extent ML algorithms are going to be a black box - an explanation as to why a model has generated a particular output or decision (and what combination of input factors contributed to that) is not always possible. In these circumstances, justification of what data is going into the model and why and linking it to theory (Oswald et al., 2018), as well as other explicability measures (e.g. traceability, auditability and transparent communication on system capabilities) can help, provided the system respects fundamental rights.

Data science needs

Currently only a few UK forces have a dedicated data science (DS) team. This leaves them at a disadvantage when it comes to navigating the current data and technology landscape both from an organisational perspective and while pursuing criminal behaviour in a world where crime is increasingly data and machine learning powered. In-house DS teams drive data quality and optimisation of resources, while also providing a ready source of expertise for evaluation of external vendors or understanding of digital crime. As a result, many forces are considering setting up their own teams, but there are several considerations.

Police forces are understandably security conscious when it comes to adopting new technologies. However, this can negatively affect internal data science teams who need access to latest programming tools and specialised computer hardware. It is important that efforts are made to provide adequate computational power on a secure network, so that the team is not limited in their creativity when examining potential solutions.

Likewise, ML skills are in high demand and require extra specialisation and experience. Given the speed of advancements in AI, which can already do much of the preparatory work such as merging datasets and querying databases, future competencies will likely focus on specialist technical skills, the ability to oversee ML tools and integrating data insights into wider decision-making processes (Afzal and Panagiotopoulos, 2024; Muir and O’Connell, 2025). However, hiring and integration into an existing organisation can be difficult. Funding is still scarce, which makes it difficult for public services to invest in salaries that are competitive with the wider market and can also be perceived unfair in comparison with other employees. Alternative solutions can include upskilling current employees with interest and aptitude or setting up regional data science centres that operate as service points for a few forces but are more tightly integrated than a national body would be. It is important that the teams include a depth of scientific experience as well as administrative staff that is versed in project management and communication with stakeholders from frontline to executive levels.

Mitigating bias in development and implementation

During the development process itself, it is crucial to check for and mitigate against bias. Certainly, one of the biggest obstacles in achieving success with algorithmic policing is unrepresentative data which can result in ‘algorithmic bias’, which can creep in during any stage of data modelling projects (Barocas and Selbst, 2016). This is where algorithms systematically and unfairly discriminate against certain individuals or groups of individuals in favour of others (Friedman and Nissenbaum, 1996), as shown when the previously discussed COMPAS algorithm disproportionally flagged African Americans. This is likely due to biased data starting with the fact African Americans are more likely to be arrested and incarcerated in the US due to historical racism and inequalities in the criminal justice system. This would then be reflected in the training data and used to make suggestions about whether a defendant should be detained. This demonstrates that if historical biases are factored into the model, it will make the same kinds of incorrect judgments people do (Lee et al., 2019). Models which create direct or indirect discrimination on the grounds of protected characteristics contravene the Equality Act (UK, 2010) and Article 14 of European Court of Human Rights (2022b). Although it may be impossible to eliminate all biases from police data, every effort should be made to do so (Oswald et al., 2018) and model development should assess, document and outline the steps taken to reduce bias where possible.

Police have statutory obligations to assess all information that pertains to their case and their judgement and expertise takes precedence. Algorithmic policing should be used in an advisory context and as one factor alongside the many other sources that police are statutorily obliged to consider (Bland, 2020; Oswald et al., 2018). By ensuring the algorithm is used in an advisory context only and keeping the human decision-maker in the loop, risks to individuals by means of false positives and false negatives can be mitigated (Oswald et al., 2018). Moreover, training end-users in how to interpret the algorithmic outputs ensures they are aware of police dataset limitations, recognise the incomplete picture they present, and understanding that an algorithm cannot consider perpetrators or crimes for which no data is on the system. The EUCPN (2022) recommends specific training on algorithm limitations and individual/organisational responsibilities. Exposure to failures during training can also help guard against complacency, whereas relying on telling users about the limitations, and warning them to always verify does not sufficiently reduce automation bias (Skitka et al., 2000). Establishing protocols for end-users to challenge algorithmic decisions can empower them to have the confidence to do so.

Over-reliance on algorithmic outputs can leave users unable to understand, explain and justify their decisions and make full use of algorithm-provided information (Klein et al., 2006). Automation bias can become more problematic over time, particularly if police become accustomed to working with algorithmic outputs and view them as highly accurate and reliable (Alon-Barkat and Busuioc, 2022; Prinzel et al., 2001). As a growing number of domain experts begin to use algorithmic outputs when making decisions, training can facilitate them in understanding, overseeing, explaining and justifying algorithmic decision-making, which is fundamental to mitigating negative social impacts or harms (Simkute et al., 2021). Similarly, confirmation bias, where police officers form a hypothesis and interpret information to prove that hypothesis, can be a problem (Selten et al., 2022). Further to assessing model accuracy, evaluations could also consider possible proxy effect on officer deskilling, for example, decision-making atrophy and over reliance on algorithms to the detriment of their own judgement (Bland, 2020).

Model maintenance

Developing and building models in-house rather than commissioning external agencies or paying for software has three major advantages. Firstly, the model will be specific to the aims outlined in the business case and the data of each force; secondly, any model needs to be maintained over time and thirdly, having access to the training data enables biases specific to the that dataset to be properly assessed and mitigated (Babuta and Oswald, 2019). Understandably, many forces may not have the available personnel or capacity to have an internal data science team. However, using an external agency to fulfil some of these roles may lead to problems when maintaining the model over time, and using software that has been developed externally, which is subject to licence fees can prove to be very costly. For example, Durham Constabulary’s HART model was developed in collaboration with statistical experts based at the University of Cambridge (Oswald et al., 2018). However, the force did not then have the resources required to constantly refine and refresh the HART model on an ongoing basis and as a result they stopped using it (Durham Constabulary, 2021).

Data properties can change over time for a variety of reasons, for example, fluctuations in crime, changes in investigative practices, changes in input software. This can lead to model performance changing over time and without proper test procedures in place errors can creep in undetected. Likewise, technology is continuously improving, and better performance might be possible, and not using the best available technology for key functions can be considered unethical.

* An important caveat to note is that with the advancement of MLAs, particularly generative models, there are new and evolving challenges - notably adversarial attacks such as prompt injection, model inversion, and data poisoning, which are well-documented in computational research. The RUDI framework does not offer direct technical prescriptions on cybersecurity, to avoid offering potentially outdated advice in a rapidly changing domain overseen by specialist bodies such as the National Cyber Crime Unit and the Police Digital Service, we recognise the importance of aligning policing practices with these emerging risks.