Randomized experiments in Scandinavian criminal justice: Reviewing the past and looking to the future

Abstract

Randomized controlled trials are reported on with increasing frequency within the criminological literature. This development, which is commonly seen as being a part of a global shift towards evidence-based policies, relies heavily on reviews of American research. However, other regions face distinct challenges and employ distinct policy solutions, potentially undermining the uniformity of this trend. In particular, the Scandinavian nations (Denmark, Norway and Sweden), with distinct penal philosophies, may offer a counter-narrative. Here, we conduct a multi-lingual systematic review of crime-related experiments in Scandinavia. Findings show that only eight experiments with an offending or delinquency outcome were published before 2015, six of which focused primarily on medical or psychological treatments. We suggest this distribution is driven by unique, regional epistemological traditions and conclude by outlining distinctive opportunities for experimental criminology in Scandinavia.

Keywords

Evidence-based policy (EBP)experiment randomized controlled trial (RCT)Scandinavia systematic review

Introduction

Within criminal justice, researchers and agencies have increasingly directed their attention to the development and implementation of crime prevention programmes that are supported with empirical evidence. This reflects a counter-reaction to the ‘nothing works’ ideology that characterized much of criminological and criminal justice scholarship during the 1970s and 1980s (Cullen and Gendreau, 2001) and an embrace of rigorous evaluation methodologies (Mears, 2007; Sherman et al., 2002).

Crime prevention programmes are often implemented in complex environments and among challenging populations. To establish whether a given criminal justice policy or intervention produces the desired outcomes, it is essential to disentangle the effects of the programme from other factors that also influence criminal behaviour. Accordingly, randomized controlled trials (RCTs), long recognized as a keystone of evidence-based treatment in medicine, have taken on a more prominent role in criminal justice (Boruch et al., 2000a; Campbell, 1969; Mears, 2007). In many situations, both academics and policy-makers now expect evidence of programmatic impact derived from more rigorous scientific methodologies (Sherman, 2009).

Existing reviews show that the rate of randomized experiments in criminal justice settings has increased significantly over the last decades (see Farrington, 1983; Farrington and Welsh, 2005; Petrosino et al., 2003).¹ At the same time, assumptions about the role of RCTs in contemporary criminology rely heavily on American studies and may be affected by an overall Anglo-American bias within the evaluation literature (Nelken, 2009). Past reviews suggest that the increased adoption of RCTs might be less prominent outside the US (see Farrington, 1983; Farrington and Welsh, 2005; Petrosino et al., 2003), but this assumption has not been extensively tested. Here, and through an examination of Scandinavia (that is, the geographical region inclusive of Denmark, Norway and Sweden), we take the first steps in this regard.

Scandinavia is the focus of this empirical enquiry for two main reasons. First, these countries have demonstrated support for evidence-based crime policies driven by scientific knowledge, evaluations and programme accreditations (see, for example, Estrada et al., 2012; Hansen and Rieper, 2010; The Norwegian Ministry of Justice and the Police, 2008; The Swedish Prison and Probation Service, 2016). At the same time, they belong to a distinctively different penal tradition than the US and most of continental Europe (Györy, 2015; Pratt, 2008; Pratt and Eriksson, 2013), with both the low number of crime researchers located in these countries and the critical and qualitative focuses of Scandinavian criminological scholarship setting them apart (Lyngstad and Skardhamar, 2011: 640). How the goals of evidence-based policy (EBP) have been approached, and whether RCTs have been accepted as the ‘gold standard’ (Farrington, 2003b; Sherman et al., 1998; Weisburd, 2003) of evaluation research here too, are open empirical questions. Second, these nations are widely cited within the international literature as having some of the most effective criminal justice systems in the world (Lappi-Seppälä, 2012; Pratt, 2008; Pratt and Eriksson, 2013). Therefore, and in light of the general, potentially global shift towards empirical policy evaluation, claims of Scandinavian penal exceptionalism should be considered in light of both the distinct philosophical goals of those justice systems, as they commonly are, and the ways in which EBPs are developed, implemented and evaluated.

We begin our enquiry by providing a brief overview of the role of RCTs in informing evidence-based policies and outlining the historical trajectory of RCTs in criminal justice. Here we consider trends within the overall literature as well as the conclusions of previous systematic reviews and supplement them with an overview of all experiments published in the Journal of Experimental Criminology (JoEX, Springer) between 2010 and 2015. Although not representative of all publications, this journal is an outlet specifically devoted to experimental research, and a survey of its contents illustrates the most recent development within the sub-discipline. Next, we present a systematic review designed to locate and identify all crime-related RCTs that have been conducted in the Scandinavian countries. Conducted in English, Danish, Norwegian and Swedish, this review represents the first organized effort to identify all Scandinavian experiments in criminal justice. Given the paucity of criminal justice experiments identified in this review, we then propose some potential explanations for why an experimental tradition has not become an integrated part of Scandinavian crime research. Finally, we briefly explore the potential for a Scandinavian experimental tradition in criminology and consider the policy and pragmatic contexts that are a prerequisite for exploiting this latent opportunity.

On the role of RCTs in evidence-based crime prevention

The relationship between criminal justice research and public policy has undergone several significant shifts in recent decades. In the US, and beginning in the 1970s, changes in the availability of federal funding and a renewed academic interest in criminal justice resulted in the proliferation of potentially policy-relevant applied research (Tonry, 2013). Although the actual impact on policy was variable (see, for example, Reuter, 2013), this embrace of rigorous methodologies and support for EBP based on the ‘highest standards of science’ (Welsh and Farrington, 2012: 129) laid the foundation for further increases in the rate, scope and methodological rigour of criminological research over the next decades (Farrington, 2001; Sherman et al., 1998; Visher and Weisburd, 1997). Randomized experiments quickly became the ‘gold standard’ of evaluation research (see Farrington, 2003b; Sherman, 2003; Weisburd, 2003), as well-designed RCTs provide rigorous evidence of between-group equivalence, internal validity and the limitation of biases (Boruch et al., 2000a; Farrington, 2003b; Weisburd, 2000, 2010). Failure to use RCTs, Sherman (2003) argues, may therefore undermine conclusions about the efficacy of programmes and mislead the processes through which policy-makers select programmes for implementation. Thus, some have argued that the decision to conduct an experiment is more than a methodological preference and that criminologists have a ‘moral imperative’ to randomize wherever possible (Weisburd, 2003: 350).

Despite obvious methodological strengths when well implemented, RCTs are not without both practical (Boruch, 1997) and ethical (Boruch et al., 2000b) hurdles. A ‘gold standard’ experiment requires more than simple random assignment; limiting attrition, maintaining between-group independence, delivering consistent treatment and collecting data from the field are essential, yet sometimes hard to achieve in practice. For instance, service providers may object to randomized designs when they believe that a beneficial programme is being withheld (for example, Cook and Payne, 2002) and RCTs can be expensive and time-consuming (Weisburd, 2000).² It is also important to acknowledge that causal estimates may not always be neither necessary or sufficient to test predictions from theories or influence public policy (Sampson, 2010; Weisburd, 2010) and, in in some cases, quasi-experimental designs may provide virtually identical results as field experiments (Berk et al., 2010). The proclaimed superiority of RCTs as compared with other research designs has been criticized as overreaching and overly generalized (Berk, 2005; Sampson, 2010). However, even accounting for these qualifications, randomized experiments are an essential part of the methodological tool kit required for evaluation and assessment for criminal justice policies (Sampson, 2010: 490).

Literature reviews spanning the mid-1940s to early 2000s (for example, Farrington, 1983; Farrington and Welsh, 2005; Petrosino et al., 2003) demonstrate that RCTs have become increasingly more common in criminal justice over time. For example, Farrington (1983) identified 35 global crime-related RCTs in the 25-year period between 1957 and 1981, a rate of 1.4 new RCTs a year. This endeavour was repeated 10 years later, focusing on publications between 1982 and 2003 (Farrington and Welsh, 2005). The 83 studies identified translate into 3.8 new experimental publications per year. The dominant position held by experiments conducted in the US is clear from these reviews.³ For example, Farrington and Welsh (2005) were able to identify only 10 RCTs that were conducted outside of the US, representing only 12 percent of all experiments located. The differences between these two reviews highlight that the increase in RCTs over time has been more substantial in the US, although RCTs have been implemented in an increasingly diverse range of countries.⁴

There have been no subsequent surveys of experimental research at the global level since Farrington and Welsh’s review (2005). An update on how general trends in experimental research have changed since 2003, however, is needed to provide a foundation and contextualization for the current enquiry. A review of the vast criminological literature is beyond the scope of this study. To provide some insight, though lacking the scope of a true systematic review, we surveyed all articles published in the Journal of Experimental Criminology from 2010 to 2015. The JoEX was established in 2005, and is currently the primary outlet specifically devoted to criminological, experimental research. Therefore, the distribution of studies in the journal, found in Table 1, provides insight into the locales in which it is reasonable to assume that experimental research has been recently conducted.

Table 1.

Number of RCTs published in Journal of Experimental Criminology, in total and by location of experiment, 2010–2015 (absolute numbers).

Year	Australia	Canada	Germany	Netherlands	Portugal	Switzerland	UK	USA	Total
2010	0	0	0	0	0	1	0	7	8
2011	0	0	0	0	0	0	0	8	8
2012	1	1	0	2	0	1	0	8	13
2013	0	1	1	1	0	1	1	9	14
2014^a	1	0	0	1	0	0	1	8	11
2015	0	0	0	0	1	0	1	8	10
Total	2	2	1	4	1	3	3	48	64

Note: a. Issue 4 in 2014 was a special issue on meta-analyses, and did not include any new RCTs.

This survey covers any article in which the subject was a randomized experiment. This includes the reporting of outcomes from RCTs, as well as writings about the studies themselves (that is, development, implementation or ‘lessons learned’). Unlike a systematic review, we placed no limitations upon the trials themselves (for example, intervention type, sample size) and a study was eligible for inclusion as long as the publication (though not necessarily the underlying experiment) took place within the timeframe. We did not review each individual RCT included in systematic reviews, and RCTs that generated multiple eligible articles were counted only once and recorded at the first incidence of publication.

As Table 1 shows, 64 experiments were published in JoEX during this six-year period. This finding supports several assumptions about RCTs in criminology. First, experiments in policing remain the most common, as observed in previous, more systematic, studies covering other outlets (for example, Petrosino et al., 2003). Secondly, the majority of RCTs published are macro-level analyses of places or practices targeted by an intervention. These factors are likely related (Braga et al., 2014). Thirdly, and most germane to the current enquiry is the assumption that the overall number of published RCTs has continued to increase over time. The counts for the later years are suppressed by several of the published studies being duplicates; for instance, 2015 saw new research published that was derived from experiments previously reported on from the US, the UK and Australia.

We also observe evidence of a global adoption of experimental research in criminology. An average of 25 percent of the RCTs were conducted outside the US, which means that we find slightly higher rates than prior and more extensive reviews. In terms of the geographical distribution, a total of eight countries are represented in the review. RCTs in JoEX are most commonly implemented in the English-speaking regions of North America, the UK and Australia, but other countries in Western Europe are also represented. We cannot discount the influence of publication, replication and linguistic biases, a limitation that has beleaguered previous reviews as well (Rothstein, 2008). However, even with that caveat, it is clear that awareness of experimental methodologies has not passed by Europe. For instance, roughly 8 percent of the panels at the 2014 Conference of the European Society of Criminology included an experimental design (Kivivuori, 2014: 103). At the same time, it is worth emphasizing that experiments were not evenly distributed geographically – even when setting aside the research activity that took place in the US.

Considering the results of both this study and prior systematic reviews, the Scandinavian nations do not appear to occupy a prominent position in the experimental literature. Kivivuori (2014) noted that only 3 of the 168 articles published in the Scandinavian Journal of Studies in Crime and Crime Prevention (SJSCCP, Taylor & Francis) between 2000 and 2013 employ or advocate for experimental methods. It is possible that the tradition for research dissemination in these countries favours non-peer-reviewed and/or non-English outlets (for example, official reports in local languages), and that previous reviews of RCTs – although thorough – therefore underestimate the contribution of Norwegian, Swedish and Danish scholars. It is also important to note that experimental criminology has enjoyed a period of growth since these earlier reviews were completed, as has the emphasis placed on EBPs more generally. This highlights the importance of conducting an up-to-date literature review that expands the scope and depth of the pervious searches, with a specific focus on those venues in which non-Anglo, Scandinavian scholars may choose to disseminate their research. In particular, a complete search must include non-peer-reviewed literature and publications in native languages, as well as encompass the research activity of the last decade.

Systematic review

The goal of this systematic review is to locate and identify any RCT on a crime-related topic that was conducted in Scandinavia. The inclusion criteria for the present paper parallel those used in earlier systematic studies of experiments in crime and justice (that is, Farrington and Welsh, 2005), though sample size criteria are not imposed. Experiments were included in this study if:

units (of some discrete type) were randomly assigned;

the experiment was carried out in Denmark, Norway or Sweden;

the outcome(s) included some measure of crime, delinquency or deviance;

the study was published in any year up to, and including, 2015 (regardless of when the experiment took place);

the study was published in either peer-reviewed scholarly journals, dissertations or academic publications, publicly available official reports or was otherwise represented in the ‘grey’ literature;

the study was published in English or one of the most common Scandinavian languages (Danish, Norwegian and Swedish⁵).

The search was carried out using social science databases and article abstracting services, as well as Scandinavian and international criminal justice/criminology journals and websites, using Boolean search strings (see the Appendix for the detailed search strategy). We also searched articles and systematic reviews on evaluations generally (for example, Farrington, 1983; Farrington and Welsh, 2005; Petrosino et al., 2003), Scandinavian reviews (for example, Holmberg, 2005) and writings on evidence-based policy and criminal justice in the Scandinavian or Nordic countries (for example, Balvig and Kyvsgaard, 2011; Hansen and Rieper, 2010; Kyvsgaard, 2006; Ogden et al., 2010). Prominent scholars in each of the Scandinavian countries, as well as international scholars focused on experimental and evaluation research, were contacted. Finally, we communicated directly with correctional and policing research agencies in Scandinavia to enquire if their staff were aware of any RCTs – published or not – that had been carried out in any of the Scandinavian countries.

Results

The results from our systematic review are included in Table 2 below. Experiments are classified as either policing, prevention, corrections, court or community experiments, and the table summarizes the author(s), publication year, location of experiment, sample description and size, experimental and control conditions, and crime-related results. We also report the source of the outcome data, the length of the follow-up period and the publication venue.

Table 2.

Summary of the results of the systematic review.

Authors (year)	Location	Category	Sample	Conditions	Results	Source data	Follow-up	Journal
Balvig and Holmberg (2011)	Denmark	Prevention	22 classrooms, n = 390 pupils	T = Classroom instruction and group discussion C = TAU	Lower rates of deviant behaviour (including offending)	Self-report	1 year	Journal of Scandinavian Studies in Crime and Crime Prevention
Berntsen and Christensen (1965)	Denmark	Corrections	252 prisoners	E = Counselling C = TAU	Lower reconviction rate, higher rates of fines, decreased re-incarceration, shorter subsequent sentence length, longer time to failure	National register data	6 years	Scandinavian Studies in Criminology
Ginsberg et al. (2015)	Sweden	Prevention	25 male prisoners with ADHD	E = Methylphenidate pharmacotherapy C = Placebo	Lower reoffending rates	Self-report (offending); unreported (release)	1 and 3 years	Journal of Clinical Psychopharmacology
Gunne and Grönbladh (1981)	Sweden	Prevention	34 drug addicts	E = Methadone maintenance treatment (MMT) C = TAU	Lower re-incarceration rates	Unreported	2 years	Drug & Alcohol Dependence
Lobmaier, Kunøe and Waal (2010a)	Norway	Prevention	46 opioid-involved prison inmates	C = Methadone maintenance treatment (MMT) E =Naltrexone implant	Lower re-incarceration rates for T₁	Norwegian Correctional Service	6 months	European Addiction Research
Löfholm et al. (2009)	Sweden	Prevention	156 youths with diagnosed conduct disorder	T = Multisystemic therapy (MST) C = TAU	Lower arrest rates in both groups	Self-report with confirmation from police records	2 years	Evidence & Policy: A Journal of Research Debate and Practice
Ogden and Halliday-Boykins (2004)	Norway	Prevention	100 youths with antisocial history (25 from each site)	T = Multisystemic therapy (MST) C = TAU	Non-significant decrease in composite measure of deviant behaviour	Self-report	6 months	Child and Adolescent Mental Health
Stevens et al. (2013)	Denmark	Prevention	547 schizophrenic patients	T = Assertive specialized treatment, family involvement and social skills training C = TAU	Non-significant reduction in frequency or prevalence of violent or overall offending	National register data	2 and 5 years	Journal of Clinical Psychiatry

Notes: E = experimental conditions; C = control conditions; T = treatment; TAU = treatment as usual.

Results are statistically significant at the p < .05 threshold unless otherwise specified.

Our systematic review identified eight eligible RCTs. We discuss each in brief below, presenting them in chronological order:

Berntsen and Christensen (1965) describe an RCT conducted with inmates released in 1952/53 from a prison in Copenhagen, Denmark (n = 252). Eligible prisoners were randomly selected to participate in a welfare counselling intervention before release, which largely consisted of assistance in finding work and accommodation, negotiating with official bodies, financial help, and assistance with spouses and relatives. The control group received no special treatment. The outcome of the study, derived from Danish national registers, showed that the treatment group had significantly lower reconviction rates (59 percent vs. 68 percent) after six years. In addition to the complex design, this study is noteworthy for the time in which it was conducted; this research began in the early 1950s, shortly after the publication of Fisher’s (1935) seminal work on the design of experiments. The next randomized experiment in Scandinavia would not be published for over a decade and a half.

Gunne and Grönbladh (1981) detail the results of an RCT evaluating the impact of methadone maintenance therapy (MMT) on intravenous heroin users between the ages of 20 and 24. The experiment employed a waitlist design and enrolment took place from 1953 to 1955 in Uppsala, Sweden. The experimental group received daily methadone (n = 17) while the control group was to receive non-pharmacological treatment (n = 17). However, all controls withdrew from the clinic upon assignment and received no services. After two years, two control individuals (three within the full follow-up period) and zero individuals from the control group were re-incarcerated. The source of these data is unclear. The authors also detail rates of mortality and abstention from drug use, which were the primary outcomes.

Ogden and Halliday-Boykins (2004) report the results of an RCT conducted in four counties in Norway designed to evaluate the impact of multisystemic therapy (MST) on youth conduct. During the study, 100 families (25 from each municipality) were assigned to receive MST (n = 62) or standard child welfare services (n = 38) in a 6:4 ratio to account for treatment attrition. In the reported analyses, measures of self-reported delinquency were collapsed into composite measures and labelled ‘externalizing behaviours’. Scores were significantly lower among treatment group participants relative to comparison cases six months after treatment began (p = .07). Additional outcomes measures included social competence, family cohesion and treatment satisfaction (see also Ogden and Hagen, 2006).

Löfholm et al. (2009) report on an MST-focused experiment conducted in the Swedish municipalities of Stockholm, Göteborg, Malmö and Halmstad. Beginning in 2004, 156 youths aged 12–17 with diagnosed conduct disorder were randomly assigned to receive either MST (treatment) or treatment as usual (control). Youths assigned to the experimental group were immediately contacted by the trained MST team in their geographical area and remained in treatment for an average of 146 days. After 24 months, an intention-to-treat (ITT) analysis demonstrated that the decrease in offending (which could be observed for both groups) was significantly larger within the treatment group (Löfholm et al., 2009: Table 1, p. 384). Self-reported delinquency rates were corroborated using police data of unspecified origin. Additional outcome measures included psychiatric symptomatology, substance abuse, social skills and school attendance (see also Sundell et al., 2008).

Lobmaier et al. (2010b) detail an RCT conducted in five Norwegian prisons between 2005 and 2007. A total of 46 inmates with a history of opioid addiction were randomly allocated to the either the experimental group (naltrexone implant treatment, n = 24) or the control group (methadone maintenance, n = 22). After a six-month follow-up period, an ITT analysis showed a significant reduction in criminal activity in both groups relative to the baseline (p = .009), but not between the two conditions. Recidivism was defined as being incarcerated for at least one day during the study, with data provided by the Norwegian Correctional Service. Lobmaier et al. (2010a) additionally discuss the implementation of the prison-based drug trial, with a focus on attrition and treatment rejection.

Balvig and Holmberg (2011) report on an experiment implemented to impact risky behaviour in 5th and 6th grade Danish school classes in 2003; 22 classrooms from eight schools were randomized into a treatment group of 12 classes (n = 216 pupils) and a control group of 10 classes (n = 174 pupils). The treatment group received a combination of classroom instruction and group discussion focusing on the levels of normative misperceptions about smoking, drug abuse, crime and other risk behaviours, and steps to reduce misconceptions. Data were collected one year later and a significant reduction in numerous self-reported risk behaviours (for example, police contacts, vandalism, shoplifting, fighting and offending) was observed in the treatment group. The study also reported numerous non-crime outcomes including perceptions of friends’ drug use and attitudes regarding alcohol.

Stevens et al. (2013) provide a secondary analysis of a Danish RCT evaluating a multidisciplinary treatment programme for individuals with a diagnosis of schizophrenia. This intervention included case management, medication, social skills training and community engagement. Between 1998 and 2002, 547 psychiatric patients aged 18–45 were recruited from hospitals in Copenhagen and Aarhus and assigned to support from the treatment team (n = 275) or TAU (n = 272). Two and five year follow-ups showed no differences in conviction rates, conviction frequencies, sentence types or time to failure between the two groups (p = .69). All outcome data were derived from the Danish national registers. Longer-term follow-ups, though not directly addressing crime outcomes, failed to find significant clinical differences after 10 years (Secher et al., 2014).

Ginsberg et al. (2015) report on the impact of an experimental evaluation of the effects of methylphenidate, a pharmacological treatment for Attention-Deficit/Hyperactivity Disorder (ADHD), on adult male prisoners in a maximum security Swedish prison. The study design included a 5-week randomized, double-blind, placebo-controlled trial followed by a 47-week extension during which all participants received oral methylphenidate (see also Ginsberg and Lindefors, 2012). Enrolment took place between 2007 and 2011. Follow-up data were collected after one and three years, though only overall study-level descriptive results are provided. The authors report on the release status at follow-up (46 percent and 75 percent, respectively), though the source of these data is unclear, and provide offender self-report data on violent, drug, fraud and driving offences at the same points in time.

Systematic review: Discussion

The eight studies identified in our systematic review provide a unique insight into the current state of criminal justice experiments in Scandinavia. First, only two of the studies were published before 2000. This suggests that RCTs have recently become more prevalent in Scandinavia, though at a scale much smaller than observed elsewhere. Second, there is a roughly equal distribution of experiments between the three countries: three were carried out in Denmark, three in Sweden and two in Norway. However, Denmark stands out as a forerunner: the first RCT was published in 1965 as compared with 1981 in Sweden and as late as 2004 in Norway. Third, our search returned only studies from peer-reviewed journals published in English, meaning that the searches in Scandinavian languages and in non-peer-reviewed outlets yielded no additional findings. It is not the case that evaluations in Scandinavian languages were not available, only that none met the inclusion criteria. This finding suggests that the scarcity of Scandinavian RCTs in previous reviews may not have been downwardly biased.

Notably, seven of the identified studies are explicitly focused on prevention, generally through the provision of a pharmacological or psychological treatment. In fact, three of the studied interventions are drug or drug treatment trials (methylphenidate and methadone) and two are multisystemic therapy (MST) trials. In these articles offending outcomes are considered secondarily. It appears that the reduction of criminal or delinquent behaviour was the primary goal of only two of the underlying experimental studies in our review: Berntsen and Christensen’s Danish correctional experiment from 1965 and Balvig and Holmberg’s Danish norm experiment from 2011.⁶ This is a stark departure from the vast majority of randomized evaluations in criminal justice, in which policies aimed at reducing crime or facilitating reintegration are the explicit focuses for the research design and (re)offending is the primary outcome of interest from the outset. The absence of any qualifying studies on policing is also striking. This is an area, as reflected in our summary of JoEX, in which the development in RCTs has been particularly noteworthy (see Braga et al., 2014; Farrington and Welsh, 2005). This further distinguishes the experimental trends in Scandinavia from those in the US and the UK.

The relative paucity of traditional criminal justice experiments in Scandinavia is interesting for several, interrelated reasons. First, it points to an interesting discrepancy between the overt and public goals of EBP and an absence of ‘typical’ evidence-based research designs. Internationally, experiments have become a key component of the movement towards EBPs (Sherman, 2009; Weisburd, 2010), and key actors in Scandinavia have demonstrated some formal support for the goals of evidence-based criminal justice policies and practices (for example, The Danish Crime Prevention Council, 2014; The Norwegian Ministry of Justice and the Police, 2008; The Swedish Prison and Probation Service, 2016). This has, however, not translated into the implementation of RCTs to date.

Secondly, Scandinavia’s reputation as a world-leader in effective corrections may not be substantiated within the methodologically rigorous evaluation literature. This suggests that system-level characteristics and performance measures (such as national incarceration rates and sentence types; see, for example, Pratt, 2008) are given precedence over micro-level measures focusing on programmatic impact. While macro-level analyses are both relevant and informative, solely relying on this approach limits opportunities to disentangle the impact of various criminal justice and social policy initiatives in generating these overall patterns. In the Scandinavian countries, where the social safety net provided by the welfare state is extensive and uniform, such questions are particularly relevant.

Thirdly, the limited tradition for experimental research in Scandinavia has immediate consequences for the evidence base upon which local policy-makers can rely. If Scandinavian criminal justice agencies do not rigorously evaluate their interventions, the impact – positive, negative or neutral – on offenders’ lives is unclear (Boruch et al., 2000a; Sherman, 2003). This is a particularly salient question when treatment programmes have been imported into Scandinavia from culturally distinct counties. Though a policy may initially have been based on solid evidence, local replication is necessary to validate and understand the expected impact (Cartwright and Hardie, 2012). Another clear advantage of such replications would be a more geographically diverse knowledge base surrounding correctional or justice interventions, thereby contributing to an understanding of not only what works but where these programmes have a demonstrative efficacy.

There are three important qualifications for these assertions. First, we do not suggest that the medical trials included in our review are not valuable sources of information on effective measures to reduce crime. However, the scientific traditions in which these studies are executed have important consequences for the ways in which crime is conceptualized and analysed. Additional epistemological value could be obtained from studies that are conducted by criminal justice scholars, in criminal justice settings, and with the intent to measure crime-related outcomes. Secondly, our focus is explicitly on RCTs, and we have therefore not included the rigorous, well-designed quasi-experimental studies we have encountered in our search. This means that the results reported here are not necessarily representative of the overall status of causal research in Scandinavia, nor that crime policies in the region are not consistently evaluated. Finally, it is not the case that non-experimental or observational research cannot provide information with significant utility in developing policies. There is no clear relationship between the mere quantity of RCTs conducted and the quality of the data they provide in support of EBPs. However, given that determinations of causality remain a keystone of rigorous policy evaluation, the absence of randomized experiments in Scandinavia remains noteworthy.

Conducting randomized experiments: Institutional and cultural premises

Existing accounts of how and why RCTs have been implemented in other criminal justice settings point to some plausible explanations for the paucity of criminal justice RCTs in the Scandinavian countries. A first set of experiences relates to the demand for RCTs in criminal justice. Scholars have noted that the early years of RCTs in the US were characterized by ‘feast and famine’ periods where the prevalence of RCTs fluctuated significantly over relatively short periods of time (Farrington, 2003a; Petrosino et al., 2003; Shepherd, 2003). Farrington (2003a) notes that these shifts coincided with the appointment of key individuals to positions of influence within research agencies. When leaders who advocated for RCTs were appointed, experiments flourished; when opponents were appointed, they vanished (Farrington, 2003a). Relatedly, in the US, the language in many Requests for Proposals distributed by the National Institute of Justice and private sector funders (for example, the Laura and John Arnold Foundation) has directly expressed a preference for experiments where possible. Comparable Scandinavian funding agencies (for example, Scandinavian Research Council for Criminology, Nordforsk, Swedish Research Council) or criminal justice agencies (for example, the correctional services in the three countries) have not expressed similar explicit preferences for RCTs (or other specific research methodologies). For instance, a call from the Norwegian Ministry of Justice and Public Security for an evaluation of the national electronic monitoring policy includes no specific requirements as to how the contracted researchers should determine ‘whether the policy meets the goal of reducing recidivism’ (Norwegian National Notification Database for Public Procurement, 2015). Similarly, in the White Paper stating that the Norwegian Correctional Service should move in a more evidence-based direction, causal designs, including the levels of The Maryland Scale (Sherman et al., 1998), are merely described without any reference being made to the priority or preference for one design or one level over another (The Norwegian Ministry of Justice and the Police, 2008: 71). Given that funders exert a large influence over the choice of research designs among evaluation researchers (Lum and Yang, 2005; Telep et al., 2015), and that RCTs are typically associated with relatively high financial costs, the lack of an explicit preference from funders and other key stakeholders likely directs the research design selection of both academic and public sector researchers away from potentially complex and expensive experiments.

A second set of experiences relates to the supply of RCTs in the field of criminal justice. First, it is important to acknowledge that a successful RCT relies on researchers having the requisite skillset, including in statistics, experimental design and implementation science (Shepherd, 2003: 303). In the US and the UK, for example, the early criminal justice experiments were mounted by psychologists and doctors, whose methodological training places a strong emphasis on causal inference (Cullen and Gendreau, 2001; Farrington, 2003a; Shepherd, 2003). Some have argued that the quantitative tradition has historically been relatively weak in Scandinavian criminology, with preference given to qualitative and critical studies (Lyngstad and Skardhamar, 2011).The quantitative research carried out in these countries has also focused more on longitudinal and ecological studies, including crime trends and youth criminality, than on causal identification and evaluation (Kivivuori, 2014; Wikström, 1996).

The discourse surrounding the application of quantitative research methods (including RCTs) to crime-related problems has also been of a different character in Scandinavia. There has been outspoken scepticism towards the use of quantitative methods and official registers both in university curricula (Finstad and Høigård, 1997: 53) and by influential scholars in the discipline (for example, Christie, 1997).⁷ Although the advantages of RCTs are rarely addressed in Scandinavian evaluation research, the problems of RCTs are. This discourse, manifest in the content of training programmes, mentoring (Braga et al., 2104) and informal socialization (Lum and Yang, 2005), can influence the research activity of generations of scholars. In line with this, none of the criminology departments at six prominent Swedish universities have a strong evaluative or experimental focus (Estrada et al., 2012), and not a single outcome evaluation (either experimental or non-experimental) was published as a Criminological PhD dissertation from Swedish universities between 1997 and 2012 (Olsson and Sundell, 2016). These patterns suggest that there are few quantitative criminologists in Scandinavia with the interest and/or background needed to perform RCTs, and that the professional ideology of Scandinavian criminology is not one wherein RCTs are perceived as a desired, viable or necessary source of knowledge about policy and programme effectiveness (Cullen and Gendreau, 2001). This could, quite naturally, explain the dearth of experiments and experimentalists observed here.

A third potential factor affecting the supply of RCTs can be found in the nature of the relationship between criminology and criminal justice services in Scandinavia. Unlike in medicine, where clinical and evaluation skills are an element of the physician education process, academic criminologists and practitioners have distinctly different training pathways. This may result in a disjunction between the skillsets and ideologies relevant to evaluation research and EBPs. This separation constitutes one of the main obstacles for RCTs in criminal justice (Shepherd, 2003). For example, the education of both police and correctional officers in Scandinavia is carried out at their own respective academies, and, although this has advantages for professionalism and competence, this insular practice could also serve to widen the gap between trained practitioners and academics.⁸ The long-term collaborations that foster many RCTs may simply not be in place.

Scandinavian experimental criminology: Prospects for the future

We have shown that existing criminological research in Scandinavia does not include a significant experimental presence, and we suggest that this may be traced back to the epistemological traditions of crime researchers in this region. This does not, however, mean that the prospects for Scandinavian experimental criminology are weak. In fact, many of the essential institutional, cultural and scientific premises that underlie an experimental approach are either already present or currently developing within the Scandinavian research landscape.

First, and as already highlighted, elected officials and governmental agencies have overtly acknowledged the need for an evidence-based criminal justice policy (Hansen and Rieper, 2010; The Norwegian Ministry of Justice and the Police, 2008; The Swedish Prison and Probation Service, 2016). There is also a growing group of academics acknowledging the value of experimental research (see, for example, Biglan and Ogden, 2008; Hansen and Rieper, 2010; Kivivuori, 2014; Sorensen, 2007), and the use of quasi-experimental research is growing substantially – especially in Denmark (see, for example, Andersen, 2015; Andersen and Andersen, 2014; Larsen, 2017). Interestingly, the Police Executive Programme at Cambridge also has participants from both Sweden and Denmark (Sherman, 2016). Additionally, the former editor of a Scandinavian criminology journal (Journal of Scandinavian Studies in Crime and Crime Prevention) has explicitly solicited experimental research for publication (Kivivuori, 2014), both encouraging this line of work and providing an additional, regional venue for academic dissemination. This suggests that some of the key stakeholders needed to develop and implement experiments are already in place in Scandinavia.

Secondly, we have observed a tentative embrace of random assignment and experimental analyses, even in complex correctional environments and among challenging populations. For example, Ginsberg and colleagues (2015) were able to conduct a 52-week ADHD medical trial in a Swedish maximum security prison, and rehabilitative programming has also been randomly allocated to inmates throughout Scandinavia (for example, Berman, 2004; Forsberg et al., 2011; Gold et al., 2014). Although none of these RCTs included a crime-related outcome, and hence are excluded from this systematic review, there is little reason to suggest that simply including these outcomes would pose additional implementation challenges. The rich and population-wide administrative register data collected throughout Scandinavia (see Lyngstad and Skardhamar, 2011) also provide opportunities to expand the scope and depth of follow-up analyses (see, for example, Stevens et al., 2013).

Finally, our systematic review suggests that the early integration of criminology with the fields of psychology and medicine that preceded the rise of RCTs in both the US and the UK might also be taking place in Scandinavia. The bridging of these scientific disciplines may be important not only because it enables a diffusion of educational methods and training structures, but because it may challenge existing ideas and ideologies about the scope and nature of criminological enquiry and research (see, for example, Cullen and Gendreau, 2001; Farrington, 2003a; Shepherd, 2003). Taken together, these changes suggest that experimental criminology may be approaching a tipping point in Scandinavia. Many of the requisite factors for the successful development and implementation of a crime-focused RCT in Scandinavia are already in place; the challenge remains in linking these disparate elements and actors together.

Conclusion

This study explores the prevalence and trajectory of randomized experiments in the criminal justice literature in the Scandinavian countries. There are several important limitations to our approach. First, we base our assumptions of recent developments in the prevalence of crime-related RCTs internationally on an overview of publications in JoEX. Although an important outlet for crime-related RCTs, it is important to acknowledge that the RCTs published in this journal might not be representative of the complete literature. Secondly, by keeping the entire region of Scandinavia the focus of our study, we are unable to adequately account for the different ways in which research is carried out and in which RCTs are approached within and between the three countries in this region. For instance, although the quantitative tradition can be characterized as being relatively weak in Scandinavia as compared with the US, variations in the relative presence and/or absence of RCT proponents/opponents, early experimental research, and contributors to the nascent quasi-experimental turn, may at the same time vary between the Scandinavian nations. For example, we find that the above-mentioned challenges seem to be are more salient in Norway and less so in Denmark. Next, our reliance on academic research and publicly available reports may omit RCTs, especially replication studies, that were not disseminated in these venues. Finally, limiting the systematic review to RCTs means that we have not included the quasi-experimental literature that exists in these countries. A lack of RCTs, by itself, is not indicative of a weak tradition of evaluation overall. This means that our findings cannot be generalized to the overall status of causal research in Scandinavia. Finally, all systematic reviews are sensitive to an element of publication and other biases reflected in the literatures upon which they rely.

These limitations notwithstanding, our analyses provide four main insights. First, our survey of the 2010–15 volumes of JoEX strongly suggests that the overall number of published RCTs has continued to increase over time. Although not necessarily representative of all publications, we observe evidence of a global adoption of experimental research in criminology, even as the US remains the epicentre of this type of research. Secondly, our multi-lingual systematic review identified eight Scandinavian RCTs that include crime-related outcomes and were published before the end of 2015. Surprisingly, only two of these studies were carried out with a primary goal of reducing criminal or delinquent behaviour; the majority of articles reported on are medical or psychological studies wherein offending outcomes were considered secondarily. This points to a large and untapped potential to conduct randomized experiments in Scandinavia, a scientific endeavour that would likely benefit both local and international theory and policy development. The systematic review also highlights that the observed increase in the rate of criminal justice RCTs in the US cannot be directly extrapolated to all other institutional and geographical settings. Thirdly, we suggest that the relative paucity of criminal justice RCTs in Scandinavia can be linked to the epistemological traditions of criminology with in this region. Manifesting within the professional ideologies of crime scholars, there have been few actors advocating for (but some opposing) the use of random assignment to advance criminological knowledge. Finally, recent developments among key stakeholders may give rise to a cautious optimism surrounding terms of expecting growth for Scandinavian experimental criminology in the near future. Capitalizing on the opportunities for RCTs in Denmark, Norway and Sweden would allow for the development and rigorous testing of the EBPs employed regionally and globally, and provide a more integrated analytical framework for advancing our knowledge of the factors that impact both the onset of and desistance from crime in this unique penal context.

Footnotes

Appendix: Systematic review search strategy

Eligible RCTs were located by searching a combination of databases, agency reports and specific academic journals. Where possible Boolean searches were employed and modified for individual queries to cover the subject regions (Denmark, Norway, Sweden and Scandinavia), as well as to include terminology likely to be included in a report on an RCT: crim* (crime, crimes, criminal, etc.), offend* (offender, offending, etc.), and recid* (recidivism, recidivate, etc.). Accordingly, a sample search string is:

((random* control* trial*) OR experimen*) AND (crim*) AND (norway or norwegian)

Because this study builds on previous reviews of the prevalence of global RCTs, most significantly Farrington and Welsh (2005), database searches were limited to studies published after 2003 and through the end of 2015. Additional restrictions included limitations to academic research (for example, not patents), full text results and/or publication language, where appropriate.

Outside of the English-language academic journals and databases, searches were conducted in the local language and in English. No date limitations were imposed on these searches. Keywords used, which parallel English-language queries, include:

Danish: randomiseret (randomized); eksperiment (experiment); forsøg (trial); tilbagefald (relapse/recidivism); RCT.

Norwegian: randomisert (randomized); eksperiment (experiment); tilbakefall (relapse/recidivism); kriminalitet (crime); RCT.

Swedish: randomisert (randomized); eksperiment (experiment); återfall (relapse); återfall i brott (recidivism); RCT.

Additionally, we explored reference lists for ineligible studies, as well as meta-analyses and systematic reviews and review articles (for example, Farrington, 2003a; Holmberg, 2005; Ogden et al., 2010; Farrington and Welsh, 2005). Publication date was not an exclusion criterion in these instances, resulting in the location of one study published prior to 2003 not identified in previous reviews.

Lastly, this search was carried out between May and December of 2016 by both authors.

Sources searched were as follows and are presented alphabetically:

Databases and indexes: Academic Search Premier (EBSCO), Bibliography of Nordic Criminology, Criminal Justice Abstracts, Google Scholar, International Bibliography of the Social Sciences, JSTOR, PsychINFO, Social Services Abstracts, Sociological Abstracts, Web of Science.

Academic journals: British Journal of Criminology, Crime & Delinquency, Crime and Justice, Criminology, Criminology and Public Policy, EuroVista: Probation and Community Justice, Journal of Criminal Justice, Journal of Experimental Criminology, Journal of Scandinavian Studies in Crime and Crime Prevention, Journal of Offender Rehabilitation, Journal of Research on Crime and Delinquency, Journal of Quantitative Criminology, Justice Quarterly, Jussens Venner, Nordisk politiforskning (Nordic Journal of Studies in Policing), Nordisk Tidsskrift for Kriminalvidenskab (Nordic Journal of Criminal Science), Probation Journal, Tidsskrift for Rettsvitenskap.

Agency databases and websites: Brå (The Swedish National Council for Crime Prevention), CORA (Correctional Service of Norway), Danish National Police, Danish Prison and Probation Service, Det kriminalprevæntive råd, Evalueringsportalen (database of all Norwegian government evaluations; N = 2433), Norwegian Police University College, Rockwoolfonden, Swedish National Police, Swedish Prison and Probation Service.

Overall, 155 potential studies were identified during the search process out of 1484 total hits.⁹ The majority of these studies were excluded after manual review for failing to meet the methodical requirements (random assignment) or a lack of a crime-related outcome (deviance, offending, conviction, incarceration, etc.), regardless of the source of the outcome data (for example, self-report, agency files, register data).¹⁰

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Research Council of Norway [Grant Number 202453 to Synøve N. Andersen] and an award from the Office of Faculty Affairs, Drexel University, USA [to Jordan M. Hyatt].

Notes

References

Andersen

(2014) Losing the stigma of incarceration: Does serving a sentence with electronic monitoring causally improve post-release labor market outcomes? Criminology & Public Policy 13(3): 1–31.

Andersen

(2015) Serving time or serving the community? Exploiting a policy reform to assess the causal effects of community service on income, social benefit dependency and recidivism. Journal of Quantitative Criminology 31(4): 537–563.

*Balvig

Holmberg

(2011) The ripple effect: A randomized trial of a social norms intervention in a Danish middle school setting. Journal of Scandinavian Studies in Criminology and Crime Prevention 12(1): 3–19.

Balvig

Kyvsgaard

(2011) Kriminologi og evidensbaseret forskning. Samfundsøkonomen 1: 25–30.

Berk

(2005) Randomized experiments as the bronze standard. Journal of Experimental Criminology 1(4): 417–433.

Berk

Barnes

Ahlman

Kurtz

(2010) When second best is good enough: A comparison between a true experiment and a regression discontinuity quasi-experiment. Journal of Quantitative Criminology 6(2): 191–208.

Berman

(2004) The reasoning and rehabilitation program: Assessing short- and long-term outcomes among male Swedish prisoners. Journal of Offender Rehabilitation 40(1–2): 85–103.

*Berntsen

Christensen

(1965) A resocialization experiment with short-term offenders. Scandinavian Studies in Criminology 1: 35–54.

Biglan

Ogden

(2008) The evolution of evidence-based practices. European Journal of Behavior Analysis 9(1): 81–95.

10.

Boruch

(1997) Randomized Experiments for Planning and Evaluation: A Practical Guide (Vol. 44). London: SAGE.

11.

Boruch

Snyder

DeMoya

(2000) The importance of randomized field trials. Crime & Delinquency 46(2): 156–180.

12.

Boruch

Victor

Cecil

(2000) Resolving ethical and legal problems in randomized experiments. Crime & Delinquency 46(3): 330–353.

13.

Braga

Welsh

Papachristos

Schnell

Grossman

(2014) The growth of randomized experiments in policing: The vital few and the salience of mentoring. Journal of Experimental Criminology 10(1): 1–28.

14.

Cambridge University (2018). Cambridge Police Executive Programme. URL (accessed 7 July 2018): http://www.crim.cam.ac.uk/courses/police/.

15.

Campbell

(1969) Reforms as experiments. American Psychologist 24(4): 409–249.

16.

Cartwright

Hardie

(2012) Evidence-Based Policy: A Practical Guide to Doing It Better. Oxford: Oxford University Press.

17.

Christie

(1997) Four blocks against insight: Notes on the oversocialization of criminologists. Theoretical Criminology 1(1): 13–23.

18.

Cook

Payne

(2002) Objecting to the objections to using random assignment in educational research. In Mosteller

Boruch

(eds) Evidence Matters: Randomized Trials in Education Research. Washington, DC: Brookings Institute Press, 150–178.

19.

Cullen

Gendreau

(2001) From nothing works to what works: Changing professional ideology in the 21^st century. The Prison Journal 81(3): 313–338.

20.

Estrada

Pettersson

Shannon

(2012) Crime and criminology in Sweden. European Journal of Criminology 9(6): 668–688.

21.

Farrington

(1983) Randomized experiments on crime and justice. In: Tonry

Morris

(eds) Crime and Justice, vol. 4. Chicago: University of Chicago Press, 257–308.

22.

Farrington

(2001) What works in preventing crime? Systematic reviews of experimental and quasi-experimental research. Annals of the American Academy of Political and Social Science 578.

23.

Farrington

(2003a) A short history of randomized experiments in criminology: A meager feast. Evaluation Review 27(3): 218–227.

24.

Farrington

(2003b) Methodological quality standards for evaluation research. Annals of the American Academy of Political and Social Sciences 587: 49–68.

25.

Farrington

Welsh

(2005) Randomized experiments in criminology: What have we learned in the last two decades? Journal of Experimental Criminology 1: 9–38.

26.

Finstad

Høigård

(1997) Kriminologi. Oslo: Pax.

27.

Forsberg

Ernst

Farbring

KÅ

(2011) Learning motivational interviewing in a real-life setting: A randomised controlled trial in the Swedish Prison Service. Criminal Behavior and Mental Health 21: 177–188.

28.

Fisher

(1935) The Design of Experiments. Edinburgh: Oliver and Boyd.

29.

*Ginsberg

Långström

Larsson

Lindefors

(2015) Long-term treatment outcome in adult male prisoners with Attention-Deficit/Hyperactivity Disorder: Three-year naturalistic follow-up of a 52-week Methylphenidate trial. Journal of Clinical Psychopharmacology 35(5): 535–543.

30.

Ginsberg

Lindefors

(2012) Methylphenidate treatment of adult male prison inmates with Attention-Deficit Hyperactive Disorder: Randomised double-blind placebo-controlled trial with open-label extension. British Journal of Psychiatry 200: 68–73.

31.

Gold

Assmus

Hjørnevik

Qvale

Brown

Hansen

Waage

Stige

(2014) Music therapy for prisoners: Pilot randomised controlled trial and implications for evaluating psychosocial interventions. International Journal of Offender Therapy and Comparative Criminology 58(12): 1520–1539.

32.

*Gunne

Grönbladh

(1981) The Swedish Methadone maintenance program: A controlled study. Drug and Alcohol Dependence 7(3): 249–256.

33.

Györy

(2015) European criminology or criminologies of Europe? Criminology in Europe 14(1): 5–9.

34.

Hansen

Rieper

(2010) The politics of evidence-based policy-making: The case of Denmark. German Policy Studies 6(2): 87–113.

35.

Holmberg

(2005) Policing and the feeling of safety: The rise (and fall?) of community policing in the Nordic countries. Journal of Scandinavian Studies in Criminology and Crime Prevention 5(2): 205–219.

36.

Kivivuori

(2014) Experimental research in Nordic criminology. Journal of Scandinavian Studies in Criminology and Crime Prevention 15(2): 103–104.

37.

Kyvsgaard

(2006) Hvad virker-hvad ikke? Kundskabsbaseret kriminalpolitik og praksis. København: Jurist-og Økonomforbundets Forlag.

38.

Lappi-Seppälä

(2012) Penal policies in the Nordic countries 1960–2010. Journal of Scandinavian Studies in Criminology and Crime Prevention 13: 85–111.

39.

Larsen

BØ

(2017) Educational outcomes after serving with electronic monitoring: Results from a natural experiment. Journal of Quantitative Criminology 33(1): 157–178.

40.

Lobmaier

Kunøe

Waal

(2010a) Treatment research in prison: Problems and solutions in a randomized trial. Addiction Research & Theory 18(1): 1–13.

41.

*Lobmaier

Kunøe

Gossop

Katevoll

Waal

(2010b) Naltrexone implants compared to Methadone: Outcomes six months after prison release. European Addiction Research 16(3): 139–145.

42.

*Löfholm

Olsson

Sundell

Hansson

(2009) Multisystemic therapy with conduct-disordered young people: Stability of treatment outcomes two years after intake. Evidence & Policy: A Journal of Research, Debate and Practice 5(4): 373–397.

43.

Lum

Yang

(2005) Why do evaluation research in crime and justice choose non-experimental methods. Journal of Experimental Criminology 1: 191–213.

44.

Lyngstad

Skardhamar

(2011) Nordic register data and their untapped potential for criminological knowledge. In: Tonry

(ed.) Crime and Justice in Scandinavia. Chicago: University of Chicago Press, 613–645.

45.

Maguire

(2007) Crime data and crime statistics. In: Maguire

Morgan

Reiner

(eds) The Oxford Handbook of Criminology. Oxford: Oxford University Press, 241–301.

46.

Mears

(2007) Towards rational and evidence-based crime policy. Journal of Criminal Justice 35(6): 667–682.

47.

Nelken

(2009) Comparative criminal justice. European Journal of Criminology 6(4): 291–311.

48.

Ogden

Hagen

(2006) Multisystemic treatment of serious behaviour problems in youth: Sustainability of effectiveness two years after intake. Child and Adolescent Mental Health 11(3): 142–149.

49.

*Ogden

Halliday-Boykins

(2004) Multisystemic treatment of antisocial adolescents in Norway: replication of clinical outcomes outside of the US. Child and Adolescent Mental Health 9(2): 77–83.

50.

Ogden

Kärki

Teigen

(2010) Linking research, policy and practice in welfare services and education in Norway. Evidence & Policy: A Journal of Research, Debate and Practice 6(2): 161–177.

51.

Olsson

Sundell

(2016) Research that guides practice: Outcome research in Swedish PhD theses across seven disciplines 1997–2012. Prevention Science 17: 525–532.

52.

Petersen

Jeppesen

Thorup

, et al. (2005) A randomised multicentre trial of integrated versus standard treatment for patients with a first episode of psychotic illness. British Medical Journal 331(7517): 602–605.

53.

Petrosino

Boruch

Farrington

Sherman

Weisburd

(2003) Toward evidence-based criminology and criminal justice: Systematic reviews, the Campbell Collaboration, and the Crime and Justice Group. International Journal of Comparative Criminology 3: 42–61.

54.

Pratt

(2008) Scandinavian exceptionalism in an era of penal excess. British Journal of Criminology 48(2): 119–137.

55.

Pratt

Eriksson

(2013) Contrast in Punishment: An Explanation of Anglophone Excess and Nordic Exceptionalism. Abingdon: Routledge.

56.

Reuter

(2013) Why has US drug policy changed so little over 30 years? Crime and Justice 42(1): 75–140.

57.

Rothstein

(2008) Publication bias as a threat to the validity of meta-analytic results. Journal of Experimental Criminology 4(1): 61–81.

58.

Sampson

(2010) Gold standard myths: Observations on the experimental turn in quantitative criminology. Journal of Quantitative Criminology 26(4): 489–500.

59.

Sampson

(2013) Common concerns, unique cultures? European and American criminology. Criminology in Europe 12(1): 12–15.

60.

Secher

Hjorthøj

Austin

Thorup

Jeppesen

Mors

Nordentoft

(2014) Ten-year follow-up of the OPUS specialized early intervention trial for patients with a first episode of psychosis. Schizophrenia Bulletin 41(3): 617–626.

61.

Shepherd

(2003) Explaining feast and famine periods in randomized field trials, medical science and criminology compared. Evaluation Review 27(3): 290–315.

62.

Sherman

(2003) Misleading evidence and evidence-led policy: Making social science more experimental. The Annals of the American Academy of Political and Social Science 589(1): 6–19.

63.

Sherman

(2009) Evidence and liberty: The promise of experimental criminology. Criminology and Criminal Justice 9(1): 5–28.

64.

Sherman

(2016) Evidence-based policing: Demonstrating an introductory video course from Cambridge for potential use across Sweden. Session THU09 at the Stockholm Criminology Symposium, Stockholm, Sweden, June.

65.

Sherman

Farrington

Welsh

MacKenzie

(2002) Preventing crime. In: Sherman

Farrington

Welsh

MacKenzie

(eds) Evidence-Based Crime Prevention. London: Routledge, 1–12.

66.

Sherman

Gottfredson

MacKenzie

Eck

Reuter

Bushway

(1998) Preventing Crime: What Works, What Doesn’t Work, What’s Promising. Washington, DC: National Institute of Justice.

67.

Sorensen

(2007) Scandinavian prospects for a place-based randomized experiment on burglary reduction. Journal of Scandinavian Studies in Criminology and Crime Prevention 8(1): 97–112.

68.

*Stevens

Agerbo

Dean

Mortensen

Nordentoft

(2013) Reduction of crime in first-onset psychosis: A secondary analysis of the OPUS randomized trial. Journal of Clinical Psychiatry 74(5): e439–e444.

69.

Sundell

Hansson

Löfholm

Olsson

Gustle

Kadesjö

(2008) The transportability of multisystemic therapy to Sweden: Short-term results from a randomized trial of conduct-disordered youths. Journal of Family Psychology 22(4): 550–560.

70.

Telep

Garner

Visher

(2015) The production of criminological experiments revisited: The nature and extent of federal support for experimental designs, 2001–2013. Journal of Experimental Criminology 11(4): 541–563.

71.

The Danish Crime Prevention Council (2014) Strategisk handlingsplan 2015–2017. URL (accessed 7 July 2018): https://www.dkr.dk/media/8296/strategisk_handleplan2015-2017.pdf.

72.

The Norwegian Ministry of Justice and the Police (2008) Punishment That Works – Less Crime – A Safer Society. White Paper 37, 2007/2008. Oslo: The Norwegian Department of Justice and the Police.

73.

The Norwegian National Notification Database for Public Procurement (2015) Announcement of competition, DOFFIN reference 2015-303185. URL (accessed 7 July 2018): https://www.doffin.no/en/notice/details/2015-303185.

74.

The Swedish Prison and Probation Service (2016) Ackreditering av kriminavårdens behandlingprogram. URL (accessed 7 July 2018): https://www.kriminalvarden.se/behandling-och-vard/behandlingsprogram/ackreditering/.

75.

Tonry

(2013) Evidence, ideology, and politics in the making of American criminal justice policy. Crime and Justice 42(1): 1–18.

76.

Tonry

(2014) Is there, and should there be, a European criminology? Criminology in Europe 13(2): 2–3.

77.

Visher

Weisburd

(1997) Identifying what works: Recent trends in crime prevention strategies. Crime, Law and Social Change 28(3–4): 223–242.

78.

Weisburd

(2000) Randomized experiments in criminal justice policy: Prospects and problems. Crime & Delinquency 46(2): 181–193.

79.

Weisburd

(2003) Ethical practice and evaluation of interventions in crime and justice: The moral imperative for randomized trials. Evaluation Review 27(3): 336–354.

80.

Weisburd

(2010) Justifying the use of non-experimental methods and disqualifying the use of randomized controlled trials: Challenging folklore in evaluation research in crime and justice. Journal of Experimental Criminology 6: 209–227.

81.

Welsh

Farrington

(2012) Science, politics, and crime prevention: Toward a new crime policy. Journal of Criminal Justice 40(2): 128–133.

82.

Wikström

(1996) Swedish quantitative criminology in the 1990s: A brief overview. Journal of Quantitative Criminology 12(3): 285–295.