Predicting burglars’ risk exposure and level of pre-crime preparation using crime scene data

Abstract

OBJECTIVES:

The present study aims to extend current research on how offenders’ modus operandi (MO) can be used in crime linkage, by investigating the possibility to automatically estimate offenders’ risk exposure and level of pre-crime preparation for residential burglaries. Such estimations can assist law enforcement agencies when linking crimes into series and thus provide a more comprehensive understanding of offenders and targets, based on the combined knowledge and evidence collected from different crime scenes.

METHODS

: Two criminal profilers manually rated offenders’ risk exposure and level of pre-crime preparation for 50 burglaries each. In an experiment we then analyzed to what extent 16 machine-learning algorithms could generalize both offenders’ risk exposure and preparation scores from the criminal profilers’ ratings onto 15,598 residential burglaries. All included burglaries contain structured and feature-rich crime descriptions which learning algorithms can use to generalize offenders’ risk and preparation scores from.

RESULTS:

Two models created by Naïve Bayes-based algorithms showed best performance with an AUC of 0.79 and 0.77 for estimating offenders’ risk and preparation scores respectively. These algorithms were significantly better than most, but not all, algorithms. Both scores showed promising distinctiveness between linked series, as well as consistency for crimes within series compared to randomly sampled crimes.

CONCLUSIONS:

Estimating offenders’ risk exposure and pre-crime preparation can complement traditional MO characteristics in the crime linkage process. The estimations are also indicative to function for cross-category crimes that otherwise lack comparable MO. Future work could focus on increasing the number of manually rated offenses as well as fine-tuning the Naïve Bayes algorithm to increase its estimation performance.

Keywords

Predictive models classification crime linkage offender behavior serial crime residential burglary

1. Background

For crime categories that involve serial offenders, i.e. where the same offender commits two or more crimes in the same category, law enforcement agencies strive to link crimes into series [1, 2, 3]. The linking of crimes enable investigators to get a more comprehensive understanding, based on the combined knowledge and evidence collected from the different crime scenes, compared to when investigating each crime in isolation [4]. In addition, such linking also enable more efficient use of the police force’s scarce resources than if investigating each crime individually [1].

The linking of crimes into series can be done based on physical evidence, e.g. DNA or fingerprints. However, such evidence is not available in all crime scenes and is more frequent for certain crime categories than for others [5, 6]. Additionally, the processing of physical evidence is costly and time-consuming. Thus, it is difficult for law enforcement agencies to handle large amounts of physical evidence from high-volume crime categories [7].

For crime categories where physical evidence is not available or simply not practically feasible, “soft evidence” such as the perpetrators’ modus operandi (MO), i.e. habits, techniques and peculiarities of behavior when committing an offense [8], offer a promising avenue for linkage of crime [9, 10]. Such behavioral patterns can be traced from the crime scene and later on interpreted using behavioral and criminal profiling [11, 12]. In behavioral profiling the profiler considers patterns of individuals similarities and differences between offenses (not offenders) by analyzing crime scene behaviors that reflect the offender’s internal workings [13].

Case linkage rests on two key assumptions. First, the offender consistency hypothesis [16], which suggests that offenders display similar behaviors across time and place. Secondly, the offender specificity hypothesis [17] unveil that offenders have an approach that deviate or is distinct from other approaches. These assumptions apply for offender behavior in residential burglary [18], but have also been questioned [19]. However, the accumulated evidence published (in approximately 30 studies) during the last 15–20 years provide a solid base for conducting case linkage based on behavioral consistency and distinctiveness for some offenders [20], some of the time, in various crime categories and types, e.g. commercial and residential burglary [11, 21, 22, 23].

Both of the key assumptions have behavioral underpinnings and focuses on how psychological characteristics generate behavioral patterns [24, 25]. In-depth ethnographic studies suggest burglars to make use of a bounded rational approach. Although not being completely rule following in terms of selecting, breaking, entering and escaping, the offenders deliberation of pros and cons for particular objects unveil systematic behavior [26]. Offenders are further described to develop expertise which ensures a sequencing of behavior which in turn is based on the triggering of certain cues [27]. Thus, offenders typically show some systematic behavior that could be traceable for investigators. Moreover, this line of research is further supported by [21] suggesting individual differences between offender behavior, i.e. behavioral distinctiveness, in terms of their exposure of risk, level of pre-offense preparation, and sensitivity to situational factors. Consequently, the offenders’ risk assessment and level of preparation may be detected as a pattern of either consistency or distinctiveness based on the soft evidence found in detailed reports of residential burglaries.

Case linkage requires crime analysts and offender profilers to manage a substantial amount of information, which can put a heavy cognitive load on the analysts [3, 28]. To counter the fallibility of human cognition, law enforcement agencies have created computerized databases enclosing a large number of reported crimes that analysts can search for similarities [29]. The advantages of computational logic can then be used in combination with such databases in order to calculate similarity scores based on pair-wise comparisons of crimes. Further, learning algorithms from the area of computer science and machine learning can be used for grouping crimes with similar characteristics together, e.g. by using cluster algorithms [15]. Such systems can be packaged as decision-support systems that assist crime analysts in the case linkage process.

1.1 Aim and scope

The present study aims to extend current research on how soft evidence can be used in crime linkage by evaluating the possibility to automatically estimate offenders’ risk exposure and level of pre-crime preparation for residential burglaries. If possible, such an estimation could assist law enforcement agencies when linking crimes into series and enable more efficiently use of scarce resources. As the offender risk and preparation scores are aggregated from detailed crime data, they can be included in crime linkage processes to supplement existing MO characteristics, e.g. search strategy and entrance method and location. Also, the supplementing scores discussed in this work can be calculated and compared between crime categories that more or less lack comparable MO behaviors, e.g. burglaries and diesel thefts.

In the study, two criminal profilers individually rate the offenders risk exposure and level of pre-crime preparation for 50 randomly selected residential Swedish burglaries (other crime categories are omitted). Then, a set of learning algorithms is used to generalize from the profilers’ ratings, using feature-rich structured crime scene descriptions. The top performing classifier is then used to estimate the risk and preparation score for a large subset of burglaries, where it is known whether the cases are linked or not. Finding optimal configurations for the studied algorithms is out of scope for the current study as the aim instead is to test the possibility and potential of the concept rather than finding an optimal combination of an algorithm and configuration. Finally, analysis of the risk and preparation consistency of crimes within linked series compared to randomly sampled crimes is conducted.

2. Related work

Yokota and Watanabe et al. present a computer system for suspect retrieval based on MO similarity in prior offenses [29]. The system includes a database with 107,233 burglaries as well as the MO of 12,468 prior offenders, where each MO is represented as a set of actions collected from crime scenes. Based on the MO calculated similarities, using statistical chance probabilities, it is possible for law enforcement officers to retrieve ranked lists of suspects for crimes under investigation. The performance shows that the hit rate was 20%, i.e. 20% of the offenders were assigned rank 1 for crimes they did commit. The median rank was 29 which can be regarded as successful since a total of 12,468 offenders were included. Offenders with numerous offenses in the database have lower median rank scores, while offenders with a single or a few prior offenses have significantly higher median ranks, i.e. it was hard for the system to correctly retrieve offenders with few prior offenses. The motivation is that offenders develop a preferred MO after achieving a number of successes and failures [8], i.e. inexperienced offenders are associated with significantly larger variance in their MO.

Adderley and Musgrove investigate an approach for analyzing members of criminal networks involvement in burglaries [30]. The approach combines three data mining methods by analyzing spatiotemporal and MO characteristics for 4,159 UK burglaries. The methods included were a multi-layer perceptron, a radial basis function and a Kohonen self organizing map. Although the evaluation lacks comparisons to random samples, the results are promising with improved levels of accuracy of 3–4 times the expected levels from manual analysis by law enforcement officers.

Tonkin and Woodhams empirically study whether spatial, temporal and MO crime scene behavior over different crime categories can be used to support case linkage [20]. Seven hundred and forty-nine solved commercial burglaries and robberies from the UK were grouped into 2,231 linked crime pairs and 273,422 unlinked crime pairs. The results indicate that linked and unlinked crime pairs can be distinguished with a high level of accuracy. A combination of spatial, temporal and MO behavior showed highest accuracy. These results were valid for both offenses in the same crime category as well as for offenses from cross-categories.

In another study, Tonkin et al. tested whether case linkage findings from the UK could be replicated abroad using residential burglaries committed in Finland [31]. Seven measures of similarity (including spatial and temporal) were calculated for both linked and unlinked crime pairs. Logistic regression and receiver operating characteristics (ROC) analysis were used to test the performance of distinguishing between linked and unlinked crimes using the seven measures. The results show that more measures were able to distinguish between linked and unlinked crimes in the Finnish data, compared to the original UK data. The most successful features were spatial proximity, temporal proximity and a combination of target, entry and internal property behaviors.

Wang et al. propose an algorithm for case linkage called Series Finder that grows a pattern of discovered crimes from a seed of a few initial crimes [32]. The algorithm analyzes both spatiotemporal and MO characteristics from crimes. It takes into account both the common characteristics of all patterns and the unique aspects of specific patterns, which result in several positive aspects, e.g. being able to handle dynamic MO that change over time. The study included 4,855 residential burglaries in US, of which two-thirds were used for training the model and the rest for testing. Crime analysts were asked to manually analyze the result for nine series. For these nine series the algorithm correctly suggested 52 crimes that were indeed within the series, together with 6 false hits. Also, 9 additional links were suggested that seemed reasonable for the crime analyst, i.e. interesting crimes that could have been overlooked during the previous crime investigation. It is suggested that Series Finder provides analysts with crime patterns that are previously unknown to them as well as reducing the time spent of manual analysis tasks.

Reich and Porter propose a Bayesian model-based clustering approach based on spatiotemporal and MO characteristics from 11,524 US residential burglary crime scenes [1]. The proposed approach is semi-supervised because the offender is known for a subset of the burglaries. For each crime under investigation the proposed approach calculates the probability of similarity to each cluster. An evaluation using the subset of linked crimes concluded that the worst results were reported when solely relying on spatial data, while improved results were reported for a model including spatial and temporal data. However, a model including MO behaviors together with spatiotemporal data showed best performance and ranked the correct cluster within top 5 60% of the time, and within the top 25 75% of the time. The approach was investigated on residential burglaries but work on other crime types and categories, and possibly also for series containing multiple crime types.

Numerous studies on crime linkage exists, but to the best of our knowledge, no studies that use data-driven machine learning approaches to automatically estimate offenders’ behavioral characteristics has been found. Therefore, in this work we investigate the possibility to estimate offenders’ behavioral characteristics based on crime scene descriptions. In the future it is possible that such estimated characteristics can improve crime analysis and clustering solutions such as the models presented by Reich and Porter above.

3. Method

In this section, we describe the data used, present our experimental approach, the metrics and the statistical tests used to evaluate the algorithms’ performance at estimating offender behavioral characteristics. In addition, we discuss the choice of algorithms, the algorithm configurations and the feature selection processes used.

3.1 Available data

A dataset consisting of feature-rich and structured descriptions for 15,598 burglaries was provided by Swedish law enforcement. Each burglary in the dataset have a detailed crime scene description consisting of 137 features as shown in Table 1. These descriptions include aspects concerning the offender’s location and means of entry, details about the residence and the plaintiff etc. All features were coded in a standardized format according to the coding-scheme described in [14]. This dataset is referred to as the unlabeled dataset.

Table 1
Summary of the 137 binary features collected from crime scenes

Name of subsection	#	Description
Time and place	7	Date and time range as well as residence address
Residential area	7	Rural or urban, number of neighbors, etc.
Type of residency	12	{House, townhouse, apartment, farm}, number of flats, etc.
Burglary alarm	5	If an alarm existed, if enabled, activated or sabotaged
Object description	10	Lights lit in/outside, member in neighborhood watch, etc.
Plaintiff	16	Plaintiff away or home, prior suspicious events, etc.
Break in	33	Method and location of break in
Search strategy	3	How the residence was searched for goods
Stolen goods	21	Categories of stolen goods, e.g. cash, gold, medicine, etc.
Trace evidence	10	Trace evidence secured, e.g. DNA, fingerprint, etc.
Miscellaneous	13	Witness, confidential hints, and searchable goods
Feature count:	137

Two criminal profilers manually estimated both offenders’ risk exposure and level of pre-crime preparation for 50 burglaries each, with an overlap of 25 crimes to allow for calculation of inter-rater agreement between the two profilers. So, the two profilers were given 75 unique burglaries to rate according to both a risk score and a preparation score. However, during the rating process two burglaries were discarded due to suspected fraud and erroneous information. As a result the second dataset contains 73 burglaries referred to as the labeled dataset.

Finally, the Swedish police provided anonymized information about crime links between burglaries carried out by the same offender. Using this information it was possible to link 153 burglaries from the unlabeled dataset into 41 series. As shown in Table 2 the smallest series contain 2 unique burglaries while the largest contain 14. Based on these linked series it was possible to evaluate the consistency of risk exposure and pre-crime preparation scores per series.

Table 2

Burglaries grouped into linked series by law enforcement

Crimes per series	2	3	4	5	6	13	14
Number of series	22	8	5	1	1	2	2
Crime count:	44	24	20	5	6	26	28

3.2 Experimental approach

Because of the limited number of labeled instances, 10-fold cross-validation was used to estimate the performance of the studied learning algorithms on the labeled dataset. The average and standard deviation from 10 different 10-fold cross-validation executions were used to compare each algorithm’s performance.

The experiment was split into two sub-experiments, one investigating the candidate algorithms’ performance in estimating offenders’ risk exposure, and the other for estimating level of pre-crime preparation. In each sub-experiment 10 tasks were executed, of which the first four are listed below.

1.
Evaluate each candidate algorithm’s performance in estimating risk exposure scores and preparation scores on the labeled dataset using 10 times 10-fold cross-validation.
2.
Test if significant differences exist between the candidates using a Kruskal-Wallis test.
3.
If significant differences exist a Nemenyi post-hoc test is used to determine which algorithms that statistically dominate others.
4.
The algorithm that shows the best performance, by means of the metrics in Section 3.5, is chosen to estimate offenders’ risk exposure and level of pre-crime preparation scores for all crimes in the unlabeled dataset.

Next, when all burglaries in the unlabeled dataset have been assigned both a risk exposure score and a pre-crime preparation score, the consistency of both scores within the linked series are evaluated. This evaluation of the scores consistency is investigated for both the crimes in the linked series as well as for pseudo series that consist of randomly sampled crimes. The following six tasks describes the process.

1.
Calculate the pair-wise difference of scores between all unordered pairs of crimes in each series consisting of linked crimes, see Table 3.
2.
Calculate the mean and standard deviation of the score differences over all linked series.
3.
For each series randomly sample the same number of crimes as detailed in Table 2 out of the 15,598 unlabeled crimes available. This creates 41 pseudo series that each contain the same number of crimes as the linked series.
4.
Calculate the pair-wise difference in scores between all unordered pairs of crimes in each one of the pseudo series.
5.
Calculate the mean and standard deviation of the score differences over all randomly assigned pseudo series.
6.
Determine the difference in score distribution between linked crimes and randomly assigned pseudo series using Wilcoxon’s signed rank test and Cohen’s $d$ .

Table 3
Example that shows the calculation of score differences in series, where 0 $=$ low risk/preparation and 3 $=$ high risk/preparation scores

Crime scores Pair-wise score difference Total score difference

Series 1 {0,0,0,0} 0,0,0,0,0,0 0

Series 2 {0,1,0,0} 1,0,0,1,1,0 3

Series 3 {0,1,2,0} 1,2,0,1,1,2 7

Series 4 {0,1,2,3} 1,2,3,1,2,1 10

3.3 Statistical tests

	Crime scores	Pair-wise score difference	Total score difference
Series 1	{0,0,0,0}	0,0,0,0,0,0	0
Series 2	{0,1,0,0}	1,0,0,1,1,0	3
Series 3	{0,1,2,0}	1,2,0,1,1,2	7
Series 4	{0,1,2,3}	1,2,3,1,2,1	10

The multiple statistical tests used in this paper are briefly descibed and motivated below. Cohen’s Kappa is used to determine the agreement between two raters/observers [33]. The test is preferable as the detected agreement is corrected for chance. Further, Cohen’s Kappa works with categorical data [33].

To detect if risk and preparation scores are different between linked and unlinked crimes, Wilcoxon’s signed rank test is used. Wilcoxon’s test is a non-parametric test, equivalent to the t-test [33]. It is used as normality cannot be assumed.

Cohen’s $d$ is used to measure the effect size, or magnitude of treatment effect, e.g. how strongly an independent variable affects a dependent variable (where a larger value indicates a stronger effect) [33].

The Kruskal-Wallis test, also known as the Kruskal-Wallis One-Way Analysis of Variance by Ranks, is a non-parametric test to detect if there exists a significant difference between two or more samples [33]. As the test does not identify between which samples the difference occur, a post-hoc test is necessary. The Nemenyi test is selected as a suitable test for this purpose [34]. It ranks the samples and calculates if the difference in ranks between pairs of samples is greater than the critical difference (which is calculated using a Tukey distribution). If the difference between ranks is greater, a significant difference is found [33, 34].

3.4 Included learning algorithms

In order to investigate the potential of using supervised learning algorithms to estimate offenders’ risk exposure and pre-crime preparation we include a diverse population of the 16 supervised learning algorithms shown in Table 4 in the experiment. These algorithms utilize different learning techniques, such as perceptron and kernel functions, instance-based learners, Bayesian learners, rule learners, decision tree inducers and meta-learners. For an introduction to the different learning techniques covered, as well as the specific algorithms included, we refer to [35] and to [36].

The original implementation of each algorithm in the Weka 3.6.6 machine learning workbench was used in the experiment. Additionally, all algorithms were executed with their default configurations. In Table 4 the configuration of each algorithm is shown. No systematic feature tuning was done since the aim of the study is not to find the best performing algorithms but rather to see if it is possible, in general, to estimate offenders’ behavioral characteristics using learning algorithms.

Table 4
Configuration of each of the investigated 16 learning algorithms

Weka algorithm	Configuration
Naïve Bayes Multinomial	Kernel estimator: false, Supervised discretization: false
Naïve Bayes Ordinal ${}^{*}$	N/A
SimpleLogistic	Heuristic stop: 50
RBFNetwork	Ridge: 1 $\times$ 10 ${}^{-8}$
SMO	Kernel: Polynomial, Complexity: 1.0
IBk	Number of neighbors: 1, Distance weighting: false
KStar	Missing values treatment: average column entropy curves
AdaBoostM1	Classifier: DecisionStump
Bagging	Classifier: REPTree (Pruning: true)
MultilayerPerceptron	LearningRate: 0.3, Momentum: 0.2
JRip	Pruning: true, Number optimizations: 2
PART	Binary splits: false, Pruning: true (confidence factor: 0.25)
Ridor	N/A
DecisionStump	N/A
J48	Pruning: Subtree raising, Pruning confidence factor: 0.25
RandomForest	Number of trees: 10

${}^{*}$ OrdinalClassClassifier with Naïve Bayes Multinomial as classifier.

3.5 Metrics

True positive (TP) is an instance that is correctly classified as belonging to category $y$ . A false positive (FP) is an instance that is incorrectly classified as belonging to category $y$ , i.e. the instance actually belongs to another category.

True negative (TN) is an instance that is correctly classified as not belonging to category $y$ . A false negative (FN) is an instance which should belong to category $y$ , but has been classified as belonging to another category.

Precision is defined as $\frac{\textit{TP}}{\textit{TP}+\textit{FP}}$ . It is a measurement of how dispersed the positives are.

Recall is defined as $\frac{\textit{TP}}{\textit{TP}+\textit{FN}}$ . The recall shows how many of the TP is actually classified as TP.

Accuracy is defined as $\frac{\textit{TP}+\textit{TN}}{\textit{TP}+\textit{FP}+\textit{TN}+\textit{FN}}$ . The accuracy is the same as Recall weighted over all classes per class frequency, i.e. identical to the True Positive Rate (TPR) [35, 36].

False Positive Rate (FPR) is defined as $\frac{\textit{FP}}{\textit{FP}+\textit{TN}}$ .

F-score is defined as $\frac{2*\textit{TP}}{2*\textit{TP}+\textit{FP}+\textit{FN}}$ .

The class distribution is skewed which means that area under the ROC curve (AUC) is a more appropriate metric than traditional accuracy [37]. In this study we use the weighted AUC metric that illustrates the performance of a classifier as its discrimination threshold is varied. AUC is calculated as the area under the curve plotted with the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings.

3.6 Experimental environment

Evaluations of the learning algorithms were executed in the Weka 1

¹
Weka version 3.6.6. URL: http://www.cs.waikato.ac.nz/ml/weka/.

machine learning workbench. A MySQL 2

MySQL version 5.5.42. URL: https://www.mysql.com.

relational database was used to hold the attributes for the burglaries. Statistical tests and plots were handled in R,3

R version 3.0.3. URL: https://www.r-project.org.

a free software environment for statistical computing and graphics.

3.7 Feature selection

An important aspect in developing a machine learning system is to remove redundant and distracting features, also known as attributes, from the dataset as they otherwise can confuse the learning system [35]. Such a process is commonly known as feature selection. Various available feature selection methods in the Weka workbench were evaluated, including CFSubsetEval, ClassifierSubsetEval and WrapperSubsetEval, using 10-fold cross-validation and evaluating on the AUC metric. Out of the attribute selection methods investigated, the CFS algorithm (CFSubsetEval) showed best performance, in terms of AUC, when estimating offenders’ risk exposure. The algorithm uses a correlation-based approach and seeks a subset of features that are highly correlated with the class labels, yet uncorrelated with each other [38]. Out of the 137 features available for each burglary in the dataset, the following eight were recommended by CFS to include in the experiment when estimating offenders risk exposure:

1.
If the residence was located in an urban setting
2.
If there were street lights outside the residence
3.
If a vehicle was parked on the driveway
4.
If kids lived in the residence
5.
If the resident was crippled or elder
6.
If an escape route was prepared during the offense
7.
The location of entrance, either: door, window, using mail slot or unknown
8.
Whether electronics was stolen

It can be concluded that all of the highlighted features are related to the offender’s risk exposure in one way or the other. A discussion of these eight features as well as the ones that were filtered out can be found in Section 6.

When considering the estimation of offenders’ level of pre-crime preparation the WrapperSubsetEval algorithm showed best performance in the feature selection process. The algorithm uses a learning algorithm, in this study we used Naïve Bayes Multinomial with 10-fold cross-validation to estimate the accuracy of the learning algorithm for the subset of features being evaluated [39]. The following eight features were found to best estimate offenders’ level of pre-crime preparations:

1.
If the police had a known suspect directly after the offense
2.
If the residence had more than one floor
3.
If the plaintiff was home during the offense
4.
If the plaintiff received an unknown phone call just prior to the offense
5.
If the plaintiff had a car parked at an airport during the offense
6.
If the plaintiff’s tools were used by the offender at the crime scene
7.
The method of entrance, either: drilling, breaks, illegal key, smash windows in, unlocked, ventilation position or unknown
8.
If alcohol or tobacco was stolen

All highlighted features are related to the offender’s level of pre-crime preparation. A discussion of the features, as well as the ones that were filtered out, is available in Section 6.
4. Estimating burglars’ risk exposure and level of pre-crime preparation

In this section the process used by offender profilers when estimating offenders’ risk exposure and level of pre-crime preparation is described and the inter-rating agreement is analyzed.

4.1 External raters

Two criminal profilers from the National offender profiling group within the Swedish police were involved in this study. The group consists of six profilers, and the two involved in this study have worked as profilers for 10 and 4 years respectively with education from NLECTC in the US and from Jill Dando Institute of Crime Science in the UK. Their contribution in this study was to estimate both offenders’ risk exposure as well as level of pre-crime preparation for residential burglaries. Both the risk exposure and level of pre-crime preparation were estimated on a four level ordinal scale as follows: $\{\textit{high, increased},$ $\textit{decreased, low}\}$ .

The profilers were told to estimate both the risk exposure and level of pre-crime preparation for 50 residential burglaries each, out of which 25 overlapped to allow for inter-rating analysis. A total of 73 unique burglaries were rated in the study since two were excluded by the profilers, in one case due to fraud rather than burglary, and in the other case due to erroneous information. The two omitted burglaries did not affect the overlap of 25 burglaries rated by both profilers. All estimations by the profilers were carried out individually without being able to affect each other’s decisions. The residential burglaries that were included in the study were randomly sampled from crimes committed in Sweden during 2014 and 2015.

Both profilers had access to the official crime reports and they were allowed to use any additional source of information or software system that could aid them in their decisions. During the task they had access to crime scene photographs as well as aerial and satellite photos of the premises. They were also given access to feature-rich and structured descriptions of the burglaries, each consisting of the previously described 137 features, see Table 1.

4.2 Inter-rater agreement

Based on the overlapping 25 burglaries rated by both profilers it was possible to analyze the inter-agreement between the raters. Cohen’s Kappa was used to determine the inter-agreement between the two offender profilers based on the overlapping crimes [40]. Calculating the inter-rater agreement for the estimated offender’s risk exposure resulted in a Kappa value of 0.401 and a p-value of 0.026. Cohen’s Kappa for offender’s level of pre-crime preparation resulted in a Kappa value of 0.595 and a p-value of 0.0025. Landis and Koch provide magnitude guidelines to help interpret Kappa values [41]. According to these guidelines the risk exposure Kappa was moderate, just bordering fair agreement. The level of pre-crime preparation Kappa was moderate, bordering substantial agreement. Thus, we conclude that the raters had sufficient inter-agreement to motivate further work based on the ratings.

5. Results

This section is divided into three parts, first the results concerning the estimation of offenders’ risk exposure is presented, followed by the results concerning estimating of offenders’ level of pre-crime preparation. Finally, the distribution of the scores and their overall distinctiveness/specificity within series are presented.

5.1 Estimating offender’s risk exposure

Table 5
Results for the models estimating burglars’ risk exposure presented using precision, recall (same as the true positive rate), the false positive rate (FPR), the F-measure and the area under the ROC curve (AUC), with best result per metric in bold font

Algorithm	Type	Precision	Recall, TPR	FPR	F-measure	AUC
Naïve bayes multinomial	Bayes	0.708 (0.215)	0.719 (0.168)	0.126 (0.084)	0.687 (0.187)	0.786 (0.147)
Naïve bayes ordinal	Bayes	0.621 (0.221)	0.656 (0.170)	0.168 (0.095)	0.610 (0.192)	0.776 (0.137)
SimpleLogistic	Function	0.615 (0.217)	0.638 (0.168)	0.163 (0.088)	0.597 (0.189)	0.755 (0.142)
RBFNetwork	Function	0.574 (0.217)	0.601 (0.155)	0.179 (0.089)	0.555 (0.173)	0.695 (0.142)
SMO	Function	0.608 (0.211)	0.633 (0.152)	0.171 (0.088)	0.591 (0.173)	0.755 (0.117)
IBk	Instance-based	0.617 (0.206)	0.625 (0.167)	0.167 (0.091)	0.589 (0.181)	0.781 (0.134)
KStar	Instance-based	0.688 (0.208)	0.688 (0.160)	0.142 (0.090)	0.657 (0.182)	0.788 (0.138)
AdaBoostM1	Meta	0.261 (0.084)	0.467 (0.102)	0.316 (0.056)	0.326 (0.092)	0.611 (0.066)
Bagging	Meta	0.547 (0.204)	0.584 (0.164)	0.199 (0.094)	0.534 (0.173)	0.715 (0.140)
Multilayer perceptron	Neural network	0.597 (0.223)	0.608 (0.183)	0.171 (0.095)	0.571 (0.194)	0.756 (0.145)
JRip	Rules	0.575 (0.195)	0.610 (0.148)	0.180 (0.078)	0.564 (0.160)	0.698 (0.120)
PART	Rules	0.602 (0.220)	0.619 (0.147)	0.165 (0.093)	0.580 (0.191)	0.711 (0.160)
Ridor	Rules	0.517 (0.206)	0.552 (0.145)	0.200 (0.092)	0.501 (0.160)	0.676 (0.111)
DecisionStump	Trees	0.261 (0.084)	0.467 (0.102)	0.316 (0.056)	0.326 (0.092)	0.611 (0.066)
J48	Trees	0.582 (0.211)	0.604 (0.158)	0.179 (0.084)	0.562 (0.174)	0.700 (0.134)
RandomForest	Trees	0.617 (0.223)	0.621 (0.172)	0.162 (0.092)	0.586 (0.192)	0.750 (0.133)
ZeroR (baseline)	Rules	0.161 (0.033)	0.398 (0.045)	0.398 (0.045)	0.228 (0.041)	0.500 (0.000)
Average		0.538 (0.187)	0.594 (0.147)	0.200 (0.083)	0.533 (0.162)	0.710 (0.120)

Table 5 shows the results for the 16 learning algorithms evaluated. As can be seen in the table, the Naïve Bayes Multinomial model reached the highest precision with 0.708, highest recall with 0.719,

Table 6

Nemenyi post-hoc test for differences in algorithms ability to classify burglars risk exposure according to the AUC metric

Algorithms

Naive

Multilayer

RBF

Simple

SMO

IBk

KStar

AdaBoost

Bagging

Naive

JRip

PART

Ridor

ZeroR

Decision

J48

Bayes Multi.

Perceptron

Network

Logistic

Bayes Ordinal

Stump

Multilayer Perceptron

-

RBFNetwork

-

Simple Logistic

-

-

-

SMO

-

-

-

-

IBk

-

-

-

-

KStar

-

-

-

-

-

AdaBoost M1

Bagging

-

-

-

-

-

-

-

Naive Bayes Ordinal

-

-

-

-

-

-

-

JRip

-

-

-

-

-

PART

-

-

-

-

-

-

-

-

-

-

Ridor

-

-

-

-

-

ZeroR

Decision Stump

-

J48

-

-

-

-

-

-

-

-

Random Forest

-

-

-

-

-

-

-

-

-

-

-

-

${}^{*}=$ statistical significant difference, $p<$ 0.05, $CD=$ 24.69.

lowest false positive rate with 0.126 and the highest true F-measure score with 0.687. The KStar model achieved marginally higher AUC with 0.788. However, since the Naïve Bayes Multinomial showed best results in four out of the five metrics we conclude that it is the most suitable model for estimating offenders’ risk exposure from crime scene data. Other models that reached an AUC above 0.750 were Naïve Bayes Ordinal, SimpleLogistic, SMO and IBk. The Kruskal-Wallis test showed a significant difference between the different algorithms ( $\chi^{2}=$ 768.78, $df=$ 525, $p<$ 0.05). Since significant differences existed among the candidates a post-hoc Nemenyi test was used to determine which candidates that dominated each other, see Table 6. The post-hoc test shows that all performed significantly better than the baseline (ZeroR). While Naïve Bayes Multinomial had, in general, the best scores, it didn’t perform statistically significant better than Multilayer Perceptron, SimpleLogistic, SOM, KStar, IBk, Bagging, Naïve Bayes Ordinal, PART, or RandomForest. However, Naïve Bayes Multinomial has the most pair-wise comparisons where a statistical significant difference was detected in comparison to the other evaluated algorithms see Table 6. As such, this model is used for estimating offenders degree of risk exposure through-out the rest of this work.

Table 7

Confusion matrix of the estimation of offender’s risk exposure by the Naïve Bayes Multinomial model

Estimated risk
High	Inc	Dec	Low
9	4	1	0	High	Actual risk
3	19	3	4	Inc
1	6	12	1	Dec
0	1	3	6	Low

Table 7 shows a confusion matrix based on the result of the Naïve Bayes Multinomial model. The diagonal in the table shows the instances correctly estimated by the model, e.g. 9 instances were estimated as High and 19 as Increased. In total, 46 out of 73 (63%) instances were correctly estimated, while 27% and 9.6% of the instances were predicted $\pm$ 1 and $\pm$ 2 steps from the correct risk value respectively. This means that the model estimated just over 90% of the instances within $\pm$ 1 step from the correct risk value. The model never estimated any instance $\pm$ 3 steps from its true risk value.

Figure 1.

Histogram of differences in offenders’ risk exposure estimated by the Naïve Bayes Multinomial model using pair-wise comparisons for crimes within either (a) linked series committed by the same offender, or (b) randomly sampled crimes in pseudo series committed by different offenders.

Offenders’ risk exposure were also calculated for crimes included in linked series, based on estimations of the Naïve Bayes Multinomial model. In Fig. 1 all risk exposure differences from pair-wise crime comparisons within series are presented for both linked series and randomly selected crimes. From the histograms it is clear that linked series have considerably more crime-pairs with identical risk exposure (i.e. a difference of 0) within the series compared to the randomly selected crimes, in fact 182 compared to 109. Although linked series have more crime-pairs with a difference of 1, the randomly selected series have considerably more crime-pairs with a difference of both 2 and 3. The total risk difference for linked series was 319 compared to 577 for the randomly selected series. To evaluate if any statistical difference existed between the two types of crime pairs, Wilcoxon’s signed rank test was used. The Wilcoxon’s signed-rank test showed that linked crime pairs are statistically significant different from the random pairs, i.e. linked crime pairs have a increased inter-crime similarity with regards to risk exposure ( $W=$ 65,360, $p<$ 0.05). The effect size, according to Cohen’s $d$ , between the two distributions is of an medium magnitude ( $d=$ 0.71) [42]. Thus, linked series have more consistent risk exposure per series and offender compared to randomly selected crimes. The randomly selected crimes have on average 81% higher variance in risk exposure between crimes.

It can be concluded that the offender’s risk exposure is more consistent for crimes they have committed, i.e. within linked series, compared to randomly sampled crimes committed by various offenders. This suggests that estimated risk exposure for offenders can aid law enforcement in the case linkage process.

5.2 Estimating offender’s level of pre-crime preparation

Table 8
Results for the models estimates of burglars’ pre-crime preparation presented using precision, recall (same as the true positive rate), the false positive rate (FPR), the F-measure and the area under the ROC curve (AUC), with best result per metric in bold font

Algorithm	Type	Precision	Recall, TPR	FPR	F-measure	AUC
Naïve bayes multinomial	Bayes	0.579 (0.190)	0.697 (0.141)	0.384 (0.167)	0.617 (0.165)	0.762 (0.164)
Naïve bayes ordinal	Bayes	0.583 (0.190)	0.691 (0.138)	0.384 (0.167)	0.615 (0.162)	0.766 (0.162)
SimpleLogistic	Function	0.445 (0.153)	0.595 (0.115)	0.491 (0.136)	0.498 (0.127)	0.609 (0.174)
RBFNetwork	Function	0.555 (0.181)	0.592 (0.146)	0.346 (0.171)	0.551 (0.148)	0.690 (0.165)
SMO	Function	0.448 (0.153)	0.616 (0.103)	0.509 (0.113)	0.506 (0.123)	0.558 (0.113)
IBk	Instance-based	0.415 (0.148)	0.544 (0.130)	0.486 (0.145)	0.459 (0.127)	0.688 (0.155)
KStar	Instance-based	0.398 (0.146)	0.552 (0.107)	0.533 (0.118)	0.453 (0.117)	0.668 (0.152)
AdaBoostM1	Meta	0.345 (0.044)	0.552 (0.046)	0.589 (0.036)	0.433 (0.046)	0.609 (0.125)
Bagging	Meta	0.359 (0.085)	0.577 (0.075)	0.579 (0.060)	0.439 (0.074)	0.612 (0.180)
MultilayerPerceptron	Neural network	0.510 (0.183)	0.581 (0.150)	0.395 (0.179)	0.528 (0.156)	0.718 (0.163)
JRip	Rules	0.370 (0.089)	0.573 (0.072)	0.564 (0.077)	0.445 (0.073)	0.504 (0.052)
PART	Rules	0.370 (0.091)	0.551 (0.091)	0.547 (0.100)	0.436 (0.076)	0.548 (0.114)
Ridor	Rules	0.416 (0.136)	0.508 (0.135)	0.417 (0.188)	0.436 (0.117)	0.546 (0.115)
DecisionStump	Trees	0.345 (0.044)	0.581 (0.046)	0.589 (0.036)	0.433 (0.046)	0.609 (0.125)
J48	Trees	0.356 (0.058)	0.576 (0.060)	0.568 (0.069)	0.439 (0.057)	0.507 (0.061)
RandomForest	Trees	0.409 (0.151)	0.531 (0.132)	0.494 (0.142)	0.451 (0.129)	0.691 (0.155)
ZeroR (baseline)	Rules	0.347 (0.044)	0.588 (0.036)	0.588 (0.036)	0.436 (0.044)	0.500 (0.000)
Average		0.426 (0.123)	0.583 (0.101)	0.498 (0.114)	0.481 (0.105)	0.623 (0.128)

$p<$ 0.05, two-tailed, Bonferroni corrected.

As can be seen in Table 8, the Naïve Bayes Ordinal model reached the highest precision with 0.58 and AUC with 0.77 when estimating offenders’ level of pre-crime preparation. RBFNetwork showed best false positive rate with 0.35, while Naïve Bayes Multinomial showed highest recall at 0.70 and best F-measure score with 0.62. Even if both the Naïve Bayes Multinomial model and the ordinal version performed very similar, we choose the Naïve Bayes Multinomial model as the best candidate for simplicity reasons, i.e. the Naïve Bayes model is used to estimate both offenders’ risk exposure and level of pre-crime preparation. No other models achieved an AUC above 0.75, RandomForest was the closest at 0.69. The Kruskal-Wallis test detected a statistical significant difference between the algorithms for the AUC ( $\chi^{2}=$ 661.71, $df=$ 334, $p<$ 0.05). A Nemenyi post-hoc test was used to determine which candidates that dominate each other as can be seen in Table 9. The post-hoc test shows that eleven algorithms performed significantly better than the baseline (ZeroR). The Naïve Bayes Multinomial did not perform statistically significantly better than Multilayer Perceptron, RBFNetwork, IBk, Naïve Bayes Ordinal, or Random Forest. However, Naïve Bayes Multinomial has the most pair-wise comparisons statistical significant differences were detected, in comparison to the other algorithms evaluated, see Table 9. As such, this model is used for estimating offenders offenders level of pre-crime preparation through-out the rest of this work.

Table 10 shows a confusion matrix based on the result of the Naïve Bayes model. The diagonal in the table shows the instances correctly estimated by the model. It is clear that the Increased class dominates the other classes with 43 out of the total 73 instances. In total the Naïve Bayes model correctly estimated 51 out of 73 (70%) instances, while 26% and 4% of the instances were predicted $\pm$ 1 and $\pm$ 2 steps from their correct preparation values respectively. This means that the model estimated 96% of the instances within $\pm$ 1 step from the correct preparation value. However, in large this is due to the model’s performance of the dominating Increased class, since it has problem estimating instances from the other classes, especially the Low and High classes. On the other hand, the model was never so wrong that it estimated an instance $\pm$ 3 steps from its correct preparation value.

Offenders’ level of pre-crime preparation was also calculated for crimes included in linked series by using the same method as for offenders’ risk exposure. In Fig. 2 all pre-crime preparation differences from the pair-wise crime comparisons within series are presented for both linked series and randomly selected crimes in pseudo series. From the histograms it is clear that linked series have considerably more crime-pairs with identical pre-crime preparation values (i.e. a difference of 0) within the series compared to the randomly selected crimes, in fact 344 compared to 211. Randomly selected crimes in pseudo series have considerably more crime-pairs with a difference of both 1 and 2, while linked series have slightly higher number of pairs with differences of 3. However, the difference in numbers between random and linked series for differences of 3 is negligible. The total pre-crime preparation difference for linked series was 128 compared to 357 for the randomly selected series, i.e. difference with a factor of 2.8.

To evaluate if any statistical difference existed between the two types of crime pairs, Wilcoxon’s signed rank test is used. The Wilcoxon’s signed-rank test showed that linked crime pairs suggest statistically

Table 9

Nemenyi post-hoc test for differences in algorithms ability to classify burglars level of pre-crime preparation according to the AUC metric

Algorithms

Naive

Multilayer

RBF

Simple

SMO

IBk

KStar

AdaBoost

Bagging

Naive

JRip

PART

Ridor

ZeroR

Decision

J48

Bayes Multi.

Perceptron

Network

Logistic

Bayes Ordinal

Stump

MultilayerPerceptron

-

RBFNetwork

-

-

Simple Logistic

-

SMO

-

IBk

-

-

-

KStar

-

-

-

-

AdaBoostM1

-

-

-

-

-

Bagging

-

-

-

-

Naive Bayes Ordinal

-

-

-

-

JRip

-

PART

-

-

-

-

-

Ridor

-

-

-

-

-

ZeroR

-

-

-

-

DecisionStump

-

-

-

-

-

-

-

-

J48

-

-

-

-

-

RandomForest

-

-

-

-

-

-

${}^{*}=$ statistical signifcant difference, $p<$ 0.05, $CD=$ 24.69.

significant more similar levels of pre-crime preparations than unlinked crime pairs ( $W=$ 128,470, $p<$ 0.05). The effect size, according to Cohen’s $d$ , between the two distributions is medium ( $d=$ 0.67) [42]. Thus, linked series show more consistent pre-crime preparation values per series and offender compared to randomly selected crimes.

Table 10

Confusion matrix of the estimation of offender’s level of pre-crime preparation by the Naïve Bayes model

Estimated level of preparation
High	Inc	Dec	Low
2	8	0	0	High	Actual prep
1	40	2	0	Inc
0	8	8	0	Dec
0	3	0	1	Low

Figure 2.

Histogram of differences from pair-wise comparisons of offenders’ level of pre-crime preparation for crimes within either (a) linked series committed by the same offender, or (b) randomly sampled crimes in pseudo series committed by different offenders.

Table 11

Distribution of offenders’ risk and preparation scores from the Naïve Bayes model within (a) the labeled dataset, (b) the linked, i.e. within series, dataset and (c) the unlinked dataset that was rated by the profilers

Risk	Prep.	Count	%	Count	%	Count	%
		(a) Labeled		(b) Linked		(c) Unlinked
Low	Low	1	1.37	11	7.19	596	3.82
Low	Dec	7	9.59	18	11.76	2,290	14.68
Low	Inc	2	2.74	4	2.61	308	1.97
Low	High	0	0.00	3	1.96	213	1.36
Dec	Low	2	2.74	9	5.88	589	3.77
Dec	Dec	11	15.07	22	14.38	1,860	11.92
Dec	Inc	6	8.22	1	0.65	222	1.42
Dec	High	1	1.37	3	1.96	217	1.39
Inc	Low	0	0.00	5	3.27	376	2.41
Inc	Dec	19	26.03	56	36.60	5,701	36.54
Inc	Inc	8	10.96	7	4.58	412	2.64
Inc	High	2	2.74	9	5.88	404	2.59
High	Low	5	6.85	4	2.61	187	1.19
High	Dec	7	9.59	1	0.65	1,727	11.07
High	Inc	1	1.37	0	0.00	108	0.69
High	High	1	1.37	0	0.00	388	2.48
Sum:		73	100.00	153	100.00	15,598	100.00

It can be concluded that the offender’s level of pre-crime preparation is more consistent for crimes they have committed, i.e. within linked series, compared to crime series consisting of randomly sampled crimes committed by various offenders. This indicates that estimated pre-crime preparation scores for offenders can aid law enforcement in the case linkage process.

5.3 Score distribution and distinctiveness within series

Predictive models that use crime scene data to estimate offenders’ behavioral characteristics could be useful for law enforcement in the crime linkage process. For a burglary that is investigated such models would allow crime analysts to record the crime scene data, feed the data into a predictive model and get estimates of the offender’s behavioral characteristics. Previous work, as well as the results in the present study, indicate that offenders’ behavioral characteristics show both intra-series consistency as well as inter-series distinctiveness. For the characteristics analyzed in the present study, this mean that offenders will show similar degree of risk exposure and pre-crime preparation for the crimes they perform and that these characteristics differ, to various extent, when compared with other offenders.

To benefit from these behavioral characteristics in the crime linkage process the crime analysts should compare crimes with regards to the combination of characteristics. We have studied two behavioral characteristics that would render pair-wise combinations of characteristics, e.g. low risk exposure and high degree of pre-crime preparation. By considering the pair-wise combination of characteristics for crime(s) being investigated it is possible to find other crimes that show similar pairs of characteristics. However, it is also possible to exclude crimes with different offender characteristics as they not likely have the same offender as the burglary being investigated. In Table 11 the distributions for the different pair-wise combinations of both behavioral characteristic scores estimated by the models are shown for the crimes in the labeled, linked and unlinked datasets.

Table 12
Risk and preparation scores for crimes within a sample of linked series to show the consistency of scores within series and distinction of scores between series

Series	Crime	Risk score	Prep. score
Series 1	Crime 1.1	2	1
	Crime 1.2	2	2
Series 2	Crime 2.1	1	0
	Crime 2.2	2	1
Series 3	Crime 3.1	0	0
	Crime 3.2	1	0
Series 4	Crime 4.1	2	1
	Crime 4.2	2	1
	Crime 4.3	2	1
Series 5	Crime 5.1	0	0
	Crime 5.2	0	0
	Crime 5.3	0	0
	Crime 5.4	0	3
Series 6	Crime 6.1	2	3
	Crime 6.2	2	3

Figure 3.

Distribution of offenders’ risk and preparation scores from the Naïve Bayes Multinomial model within (a) the unlinked dataset, (b) the linked, within series, dataset and (c) the labeled dataset.

The risk exposure scores for the crimes in the labeled dataset is divided over all four levels, but with an increment in the increase level that holds 40% of the crimes. There is also a slight reduction in the high and low levels with 19% and 13% of the crimes respectively. The pre-crime preparation scores for the crimes in the labeled dataset are dominated by the decreased level as it holds 60% of the crimes. Since the crimes in the labeled dataset were randomly sampled it indicates that the decreased level of the preparation score is also dominating in the burglary population as a whole, i.e. a large proportion of the offenders do not engage in much pre-crime preparation. As shown in Table 11 63% of the crimes in the linked dataset as well as 74% of the crimes in the unlinked dataset were estimated with decreased preparation scores. As such, the model seems capable of distinguishing features for the different categories. This shows the potential to distinguish other types of offenders, that do prepare their burglaries to a larger extent, from the large group of more impulsive offenders, allowing law enforcement to reduce the group of potential offenders for the investigates crime(s). An ideal model would have high distinctiveness between categories, and high consistency within categories.

In the linked dataset the increased risk exposure level is estimated for 50% of the crimes while the high level only is estimated for 3% of the crimes. A hypothesis is that law enforcement apprehend more offenders that carry out burglaries with increased risk than for low or decreased risks. On the other hand, not many offenders carrying out high risk crimes are apprehended, but this could be because a clear majority of the offenders reject such crime opportunities in favor for less risky.

For better overview of the different combinations of score-pairs Fig. 3 shows the distribution of score-pairs in a jitter plot. In the plot the preparation score is dominated by the decrease level, as well as the the heightening of crimes with an increased risk score. However, Fig. 3 mainly shows how the 16 possible score-combinations are distributed for each of the three datasets. The over-representation of burglaries with decreased preparation scores can clearly be seen. It is also clear that most of the different combinations are populated by burglaries from all three datasets, which is a prerequisite for having distinctiveness between series.

In Table 12 the risk and preparation scores are shown for the crimes in six linked series. Combinations of scores show consistency for crimes within series, although there are some fluctuations of $\pm$ 1 score. In the Series 5 a larger difference is shown for the Crime 5.4. However, overall there is consistency of scores within series as well as some distinction of scores between series that is interesting to utilize in the crime linkage process. Especially as these scores should be used in combination with other discriminating aspects for the crimes investigated, e.g. spatio-temporal proximity.

6. Discussion

Using the models investigated in this study, law enforcement agencies could benefit from a decision-support system that automatically estimates offenders’ risk exposure and pre-crime preparation scores for all crimes as they are being reported. This may, in turn, help to further profile offenders, perhaps even reduce number of suspects for particular crimes or be other means for operational law enforcement. However, it is necessary for these models to have access to an extensive dataset of features from the crime scene when estimating the scores. In a previous study we describe a method for collecting structured crime scene data using a digital click-based form that is filled in by law enforcement officers at the crime scene [43]. The forms themselves could be translated and implemented in other countries to aid cross-border intelligence. The methodology is currently being implemented by the Swedish police for a number of volume crime categories. In that implementation the content of the forms will momentarily be available for search and analysis as soon as they are filled in by officers on the crime scene. Based on the content of such forms the models investigated in this study could automatically estimate scores for risk exposure and preparation level for the sought offenders.

Since the experiment results indicate that the consistency of both risk exposure scores and preparation scores are significantly higher for crimes in linked series compared to crimes in general, we argue that these scores can benefit crime analysts in their crime linkage tasks. When linking crimes, the analysts could use the scores as one feature, out of many, when identifying possibly related crimes. The general idea is that given a single crime (or a set of several crimes) that are investigated, the analysts should focus on other crimes where the offender has comparable risk and preparation scores. Both scores could be combined to produce score-combinations ranging from offenders with high risk exposure and low level of preparation to combinations with low risk exposure and high level of preparation. Including a spectrum of outcomes in between, which in this initial study was reduced since grading scales with only four levels were used. In addition to the scores, traditional features used when linking crimes should still be used, e.g. spatio-temporal proximity, MO characteristics, search strategy for goods as well as target selection criteria. The scores should be seen as complementary features in the linkage process rather than the only ones.

Listing 1
Ruleset from the PART model estimating offenders’ risk exposure.

If (urban = No) and

(vehicle_on_driveway = No) Then

risk := Low

Elsif (crippled_or_elderly = No) and

(escape_route_prepared = No) and

(entrance_place = Door) and

(kids_at_home = No) and

(street_lighting = Yes) Then

risk := High

Elsif (electronics_stolen = Yes) and

(kids_at_home = No) Then

risk := High

Elsif (escape_route_prepared = No) and

(crippled_or_elderly = Yes)

risk := Decreased

Elsif (escape_route_prepared No) and

(entrance_place = Door) and

(kids_at_home = No) Then

risk := Decreased

Elsif (escape_route_prepared = No)

risk := Increased

Else

risk := Decreased

In the feature selection process, described in Section 3.7, eight features were found to best correlate with the risk exposure scores. Some of these were related to increased risk of being observed or exposed, e.g. whether a residence was located in an urban setting, whether there was a car parked on the driveway, if kids lived in the residence or if there were street lights outside the residence. Other features were associated with decreased risk, e.g. if resident was crippled or elder or if an escape route was prepared. The location of entry mainly associated with high/increased risk were door and window, but they could also indicate decreased risk together with other features. Interestingly, no features regarding burglary alarm or whether the residence were included in neighborhood watch was identified as good indicators of risk.

Two of the models (one for predicting risk-exposure scores and another for pre-crime preparation scores) are presented here in order to visualize how such models actually look like. We hope this could concretize how the proposed approach of predicting offender characteristic scores based on crime scene data work in practice. In Listing 1 the ruleset of the model generated by the PART algorithm is shown. We chose to show a representation of the PART model rather than the Naïve Bayes Multinomial model since the prior is more transparent and can more easily be interpreted by humans, although the decision logic differs from the Naïve Bayes Multinomial model. Also, the model created by the PART algorithm was selected over for instance the model from the tree-based J48 algorithm, as the former reached higher AUC scores.

The feature selection process concerning pre-crime preparation also identified eight features that best correlate with the preparation scores. Some of the features were associated with increased/high levels of preparation, e.g. if there was no suspect to the crime when police officers filed the crime report or whether the location of entry was through a door. Other features were associated with decreased/low levels of pre-crime preparation, e.g. if the plaintiff was home during the offense or if the plaintiff’s own tools were used by the offender during the offense. Methods of entry mainly associated with increase/high preparation levels were drilling the lock, breaking door and the use of an illegal key, while smashing a window were associated with decreased/low levels.
Listing 2
Ruleset from the Ridor model estimating offenders’ level of pre-crime preparation.

preparation = Low

If (entrance_method = Other) Then

preparation = Decreased

Elsif (suspect_exists = No) Then

preparation = Increased

If (suspect_exists = No) Then

preparation = Increased

Elsif (multilvl = No) Then

preparation = Increased

Elsif (entrance_method = Illegal key) Then

preparation = High

If (suspect_exists = N/A) Then

preparation = High

Elsif (multilvl = Yes) Then

preparation = Increased

The PART algorithm produces a trivial model with a single rule predicting all instances according to either the majority category or the 2 ${}^{\rm nd}$ largest category. Since the PART algorithm does not produce any usable model for the preparation scores we instead used the ruleset of the Ridor model, see Listing 2. The model starts by setting low as the default preparation level which then sticks unless any of the preceding if-statements match the offense. Note that the decision logic of this model differs from the Naïve Bayes Multinomial model that showed best result in the experiment.

Based on the results from the experiment it is indicated that Naïve Bayes Multinomial generates models with best classification performance on the problems at hand with AUC measures of 0.79 and 0.77 for estimation of risk and preparation scores respectively. Since the aim of this study was to test the feasibility of estimating risk and preparation scores using predictive models, no structured configuration-tuning was carried out and all evaluated algorithms used default out-of-the-box configurations. Also, no methods for evening out the number of burglaries that are assigned to each level of the risk and preparation scores have been investigated. For instance, using traditional over-sampling or the more advanced Synthetic Minority Over-sampling Technique (SMOTE) for levels with few burglaries, e.g. the low and high preparation scores [44]. Thus, it is probably possible for the models to reach better estimation performance after such opportunities have been investigated further.

The time it takes to train and use the models should not pose any problem in a practical situation. Training the Naïve Bayes Multinomial model on the labeled dataset took a couple of seconds. Using the model to estimate the risk and preparation scores for the 15,598 unlabeled burglaries took about 2 minutes on an ordinary workstation computer.

There is no correlation between the risk exposure scores and pre-crime preparation scores calculated for the unlabeled burglaries since both Spearman’s and Pearson’s correlation coefficients are 0.11. However, in an ongoing parallel study we investigate how these scores are affected by both spatio-temporal and socio-economic factors.

In the present study we set out to test the possibility to estimate offenders’ risk and preparation scores using manual ratings from criminal profilers. However, before knowing whether the estimation of the scores were really possible the profilers could only invest a limited amount of time. In dialog with the profilers it was decided that two profilers should rate the risk and preparation scores for 50 burglaries each. It was also decided that the rating scales should have four levels, i.e. $\{\textit{low, dereased, increased, high}\}$ , as it would take too long to rate the burglaries using more levels. However, future studies should include a more nuanced set of levels of the rating scales and also include more manually labeled crimes that provide the learning algorithms with better opportunities to identify the intrinsic characteristics of each level. A training dataset of 73 crimes is quite low and might affect the generalizability of the results. Further, various means of increasing the inter-agreement in the profilers’ ratings should be investigated, e.g. a preceding workshop where the profilers are presented with a number of real cases for which they discuss how to rate them and what aspects to consider when doing so. Such a workshop could possibly provide a larger shared understanding among the profilers on what features, and their relative importance, to consider in the crime scene data when assessing and rating offenders’ risk exposure and level of pre-crime preparation.
7. Conclusion

This study investigates the possibility to estimate burglars’ risk exposure and level of pre-crime preparation using predictive models generated by machine-learning algorithms. Two criminal profilers manually rated 50 burglaries each with regards to the offender’s risk exposure and level of pre-crime preparation scores. In an experiment 16 machine-learning algorithms were evaluated on their performance in estimating the two scores based on feature-rich structured crime scene descriptions. The Naïve Bayes Multinomial algorithm generated the best performing models with an AUC metric of 0.79 for estimating risk exposure and 0.77 for level of pre-crime preparation. Using the Naïve Bayes Multinomial model the risk and preparation scores were estimated for an additional 15,598 burglaries.

By analyzing the scores for both linked burglaries (performed by the same offender) as well as for unlinked burglaries (performed by different offenders), it was shown that the consistency for both risk and preparation scores were significantly more consistent for crimes within series committed by the same offender. In addition, it was shown that there exists some degree of distinctiveness between crimes in different series. Thus, indicating that the scores add valuable input to the crime linkage process, e.g. when linking large numbers of volume crimes. It is possible that these scores can be used when linking crimes from different crime categories without easily comparable MO, since they rather reflect behavioral characteristics of the offender than specific MO characteristics at offenses.

8. Future work

In future studies we will investigate how the risk and preparation scores are affected by spatio-temporal and socio-economic factors. We also plan to extend the present study by increasing the number of offenses manually labeled by criminal profilers, to provide a larger training dataset, as well as using a more fine-grained rating scale. It would also be interesting to investigate the consistency and distinctiveness in risk and preparation scores for crime series consisting of cross-category crimes, e.g. both burglaries, diesel thefts and crimes against elders. Additionally, as the present study is indicating that Naïve Bayes Multinomial generates suitable models, a future study could target only those models and instead focus resources on finding the optimal configurations and feature selection to reach best performance. Possibly also investigating methods to handle the class-imbalance between the different levels. Finally, it could be interesting to consider the proposed approach in alternative contexts, e.g. various types of fraud.

Footnotes

Acknowledgments

We would like to extend our gratitude to the National offender profiling group within the Swedish police for their assistance. This work was carried out with financial support from Vinnova, the Swedish Innovation Agency, through research grant 2015-05977.

References

Reich

B.J.

and Porter

M.D.

, Partially supervised spatiotemporal clustering for burglary crime series identification, Journal of Royal Statistical Society, Series A (Statistical Society) 178 (2015), 465–480.

Grubin

Kelly

and Brunsdon

, Linking serious sexual assults through behaviour, in Home Office Research Study 215, 2001. http://webarchive.nationalarchives.gov.uk/20110314171826.

Woodhams

Bull

and Hollin

C.R.

, Case linkage: identifying crimes committed by the same offender, in: Criminal Profiling: International Theory, Research, and Practice, 1st edn Kocsis

R.N.

, ed., Humana Press, Riverview Drive, New Jersey, 2007, pp. 117–133.

Rossmo

, Geographic Profiling, CRC Press, Boca Raton, 1999.

Davies

, The use of DNA profiling and behavioural science in the investigation of sexual offences, Medicine, Science and the Law 31 (1991), 95–101.

Home Office, Home Office, DNA Expansion Programme 2000-2005, 2005. http://webarchive.nationalarchives.gov.uk/+/http://www.homeoffice.gov.uk/documents/DNAExpansion.pdf?view=Binary.

Roman

Reid

Chalfin

Adams

and Knight

, The DNA Field Experiment: Cost-effectiveness Analysis of the Use of DNA in the Investigation of High-volume Crimes, 2008. http://www.urban.org/UploadedPDF/411697_dna_field_experiment.pdf.

O’Hara

C.E.

and O’Hara

G.L.

, Fundamentals of Criminal Investigation (7th ed.), Charles C Thomas Publisher Ltd., Springfield, IL, 1956.

Hazelwood

R.R.

and Warren

J.I.

, Linkage analysis: Modus operandi, ritual, and signature in serial sexual crime, Aggression and Violent Behavior 8 (2003), 587–598.

10.

Bennell

and Canter

D.V.

, Linking commercial burglaries by modus operandi: tests using regression and ROC analysis, Science & Justice 42 (2002), 153–164.

11.

Bennell

and Jones

N.J.

, Between a ROC and a hard place: A method for linking serial burglaries by modus operandi, Journal of Investigative Psychology and Offender Profiling 2 (2005), 23–41.

12.

Tonkin

Woodhams

Bull

and Bond

J.W.

, Behavioural case linkage with solved and unsolved crimes, Forensic Science International 212 (2012), 146–153.

13.

Godwin

, Reliability, Validity, and Utility of Criminal Profiling Typologies, Journal of Police and Criminal Psychology 17 (2002), 1–18.

14.

Borg

Boldt

Lavesson

Melander

and Boeva

, Detecting serial residential burglaries using clustering, Expert Systems with Applications 41 (2014), 5252–5266.

15.

Borg

and Boldt

, Clustering Residential Burglaries Using Modus Operandi and Spatiotemporal Information, International Journal of Information Technology & Decision Making 15(1) (2016), 23–42. http://www.worldscientific.com/doi/abs/10.1142/S0219622015500339.

16.

Canter

, Psychology of offender profiling, in: Handbook of Psychology in Legal Contexts, 1st edn Bull

and Carson

, eds, John Wiley and Sons, Chichester, UK, 2000, pp. 343–355.

17.

Pervin

L.A.

, Current Controversies and Issues in Personality, John Wiley and Sons, New York, 2002.

18.

Wright

R.T.

and Decker

S.H.

, Burglars on the job, Northeastern University Press, Boston, MA, 1994.

19.

Nee

and Meenaghan

, Expert decision making in burglars, The British Journal of Criminology 46 (2006), 935–949.

20.

Tonkin

and Woodhams

, The feasibility of using crime scene behaviour to detect versatile serial offenders: An empirical test of behavioural consistency, distinctiveness, and discrimination accuracy, Legal and Criminological Psychology (2015).

21.

Bouhana

Johnson

S.D.

and Porter

, Consistency and specificity in burglars who commit prolific residential burglary, Legal and Criminological Psychology 21 (2014), 77–94.

22.

Woodhams

Hollin

C.R.

and Bull

, The psychology of linking crimes: A review of the evidence Legal and Criminological Psychology 12(2) (2010), 233–249.

23.

Markson

Woodhams

and Bond

J.W.

, Linking serial residential burglary: comparing the utility of modus operandi behaviours, geographical proximity, and temporal proximity, Journal of Investigative Psychology and Offender Profiling (2010).

24.

Mischel

and Shoda

, A cognitive-affective system theory of personality: Reconceptualising situations, dispositions, dynamics, and invariance in personality structure, Psychological Review 102 (1995), 246–268.

25.

Wischel

Shoda

and Smith

R.E.

, Introduction to personality: Toward an integration (7th ed.), John Wiley and Sons, New York, NY, 2004.

26.

Cromwell

P.F.

Olson

J.N.

and Avary

D.W.

, Breaking and entering: An ethnographic analysis of burglary, Sage, Newbury Park, CA, 1991.

27.

Nee

, Understanding expertise in burglars: from pre-conscious scanning to action and beyond, Aggression and Violent Behavior 20 (2015), 53–61, Special issue: Expertise and offending, ISSN 1359-1789.

28.

Santtila

Korpela

and Häkkänen

, Expertise and decision making in the linking of car crime series, Psychology, Crime & Law 10 (2004), 97–112.

29.

Yokota

and Watanabe

, Computer-Based Retrieval of Suspects Using Similarity of Modus Operandi, International Journal of Police Science & Management 4 (2002), 5–15.

30.

Adderley

and Musgrove

, Modus operandi modelling of group offending: a data-mining case study, International Journal of Police Science & Management 5(4) (2003), 265–276.

31.

Tonkin

Santtila

and Bull

, The linking of burglary crimes using offender behaviour: Testing research cross-nationally and exploring methodology, Legal and Criminological Psychology 17 (2012), 276–293.

32.

Wang

Rudin

Wagner

and Sevieri

, Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23–27, 2013, Proceedings, Part III Blockeel

Kersting

Nijssen

and Železný

, eds, Springer Berlin Heidelberg, Berlin, Heidelberg, 2013, pp. 515–530, Chap. Learning to Detect Patterns of Crime.

33.

Sheskin

, Handbook of parametric and nonparametric statistical procedures, Chapman & Hall, 2007.

34.

Demšar

, Statistical Comparisons of Classifiers over Multiple Data Sets, The Journal of Machine Learning Research 7 (2006), 1–30.

35.

Witten

I.H.

Frank

and Hall

M.A.

, Data Mining – Practical Machine Learning Tools and Techniques, (3th ed.), Elsevier, Burlington, MA, 2011.

36.

Flach

, Machine Learning: The Art and Science of Algorithms that Make Sense of Data, Cambridge University Press, Cambridge, UK, 2012.

37.

Provost

Fawcett

and Kohavi

, The case against accuracy estimation for comparing induction algorithms, in: Proceedings of the 15th International Conference on Machine Learning, Morgan Kaufmann Publishers, 1998, pp. 445–453.

38.

Hall

M.A.

, Correlation-based Feature Subset Selection for Machine Learning, PhD thesis, The University of Waikato, Computer Science Department, 1998.

39.

Kohavi

and John

G.H.

, Wrappers for feature subset selection, Artificial Intelligence 97(1–2) (1997), 273–324.

40.

Cohen

, A coefficient of agreement for nominal scales, Educational and Psychological Measurement 20 (1960), 37–46.

41.

Landis

J.R.

and Koch

G.G.

, The measurement of observer agreement for categorical data, Biometrics 33 (1977), 159–174.

42.

Cohen

, Statistical power analysis for the behavioral sciences, (2nd ed.), Lawrence Earlbaum Associates, Hillsdale, NJ, 1988.

43.

Borg

Boldt

Lavesson

Melander

and Boeva

, Detecting serial residential burglaries using clustering, Expert Systems with Applications 41 (2014), 5252–5266.

44.

Chawla

N.V.

Bowyer

K.W.

Hall

L.O.

and Kegelmeyer

W.P.

, SMOTE: Synthetic Minority Over-sampling Technique, Journal of Artificial Intelligence Research 16(1) (2002), 321–357, ISSN 1076-9757.

Predicting burglars’ risk exposure and level of pre-crime preparation using crime scene data

Abstract

OBJECTIVES:

METHODS

RESULTS:

CONCLUSIONS:

Keywords

1. Background

1.1 Aim and scope

2. Related work

3. Method

3.1 Available data

Table 1 Summary of the 137 binary features collected from crime scenes

3.4 Included learning algorithms

Table 4 Configuration of each of the investigated 16 learning algorithms

3.6 Experimental environment

1 Weka version 3.6.6. URL: http://www.cs.waikato.ac.nz/ml/weka/.

4.1 External raters

4.2 Inter-rater agreement

5. Results

5.1 Estimating offender’s risk exposure

Table 5 Results for the models estimating burglars’ risk exposure presented using precision, recall (same as the true positive rate), the false positive rate (FPR), the F-measure and the area under the ROC curve (AUC), with best result per metric in bold font

Table 8 Results for the models estimates of burglars’ pre-crime preparation presented using precision, recall (same as the true positive rate), the false positive rate (FPR), the F-measure and the area under the ROC curve (AUC), with best result per metric in bold font

Table 12 Risk and preparation scores for crimes within a sample of linked series to show the consistency of scores within series and distinction of scores between series

8. Future work

Footnotes

Acknowledgments

References

Table 1
Summary of the 137 binary features collected from crime scenes

Table 4
Configuration of each of the investigated 16 learning algorithms

¹
Weka version 3.6.6. URL: http://www.cs.waikato.ac.nz/ml/weka/.

Table 5
Results for the models estimating burglars’ risk exposure presented using precision, recall (same as the true positive rate), the false positive rate (FPR), the F-measure and the area under the ROC curve (AUC), with best result per metric in bold font

Table 8
Results for the models estimates of burglars’ pre-crime preparation presented using precision, recall (same as the true positive rate), the false positive rate (FPR), the F-measure and the area under the ROC curve (AUC), with best result per metric in bold font

Table 12
Risk and preparation scores for crimes within a sample of linked series to show the consistency of scores within series and distinction of scores between series