Are Alternative Sources of Official Crime Data Interchangeable? A Note on Inter-Agency Consistency

Abstract

The purpose of this study is to determine whether the decision to use one source of official data in lieu of another affects the assessment of social policy on crime. Specifically, we examine the effect of the implementation of stand your ground legislation on state and municipal compilations of residential and non-residential burglaries known to the police within a large, Midwestern city. The interrupted time series analyses of the state agency data reveal that the castle doctrine legislation led to a temporary decline in residential burglaries, but had no effect on non-residential burglaries. In contrast, our analyses of the city agency data indicate that this legislative initiative had no effect on residential burglaries, but did generate a permanent, monthly increase in the number of non-residential burglaries. The implications of these findings for the use of official crime statistics are discussed.

Keywords

crime measurement interrupted time series burglary social policy

This article is concerned with the reliability of crime measurement across formal organizations. Much of the empirical work in this area is comparative. Taking advantage of the onset of the systematic collection of victimization data in the late 1960s (Ennis, 1967), a number of scholars have looked for patterns of agreement and divergence in official counts of crime and those reported in the National Crime [Victimization] Survey (see, for example, Gove, Hughes, & Geerken, 1985; Hindelang, 1974, 1978, 1981; McDowall & Loftin, 2007; Skogan, 1974). More recently, others have examined the correspondence between counts of homicide as reported in the Supplementary Homicide Reports and the Vital Statistics of the United States (Cantor & Cohen, 1980; Pridemore, 2003; Rokaw, Mercy, & Smith, 1990; Wiersema, Loftin, & McDowall, 2000).

This research takes a different track. Rather than examining the extent to which independent governmental agencies produce similar or dissimilar counts of crime, we focus on the consistency in measurement between two agencies that, at least superficially, should be expected to yield virtually identical counts of crime.

Background

This project grew out of our desire to assess the impact of giving civilians more discretion with regard to the use of deadly force to thwart serious, but non-life-threatening, felonies (the passage of so-called castle and/or stand your ground legislation) on the public safety. Until recently, most states adhered to the common law criterion of imminent bodily harm for establishing an affirmative defense to the charge of criminal homicide (Bailey, 1948). However, beginning with Florida’s enactment of the first castle doctrine law in 2005, no less than 23 states have passed some sort of legislation that allows individuals to use lethal force to protect their property from violent and non-violent offenders (Boots, Bihari, & Elliott, 2009; Levin, 2010).

As part of the continuing effort to determine the impact of the various modifications of the justifiable homicide defense on community safety (Chamlin, 2014; Ren, Zhang, & Zhao, 2015), we decided to conduct an interrupted time series analysis to assess the impact of a stand your ground law that permits the use of deadly force to thwart the completion of residential, but not non-residential (commercial), burglaries. To be sure, there is no way of knowing if potential burglars are cognizant of the distinctions made by this legislation regarding residential and non-residential burglaries. However, to the extent that they are aware of the specific details of this legislation, one might anticipate that its implementation would vary across categories of burglary. To the extent that the legislation has its desired effect, we would predict that it would produce a decrease in the amount of residential burglaries (a deterrent effect), but might yield an increase in the amount of non-residential burglaries (a displacement effect).

Although the total number of burglaries for localities within the state is readily obtainable from published sources, disaggregated burglary counts are not. Therefore, we contacted a number of state and local police agencies in the attempt to procure counts of residential and non-residential burglaries. Fortuitously, we were able to acquire disaggregated burglary data for one municipality from two different official sources (the state and local police). Surprisingly, a cursory inspection of the residential and non-residential burglary time series revealed some obvious discrepancies across the data sources.

Consequently, before we can conduct any meaningful assessment of the stand your ground legislation on burglary, we need to determine if the decision to use one source of official data in lieu of another is likely to affect the ensuing analyses. This is the purpose of the present investigation. Specifically, we examine the effect of the implementation of stand your ground legislation (the Home Defense Act) on state and municipal compilations of residential and non-residential burglaries known to the police within a large, Midwestern city.

Method

Data

This study uses autoregressive moving average (ARIMA) techniques to model the impact of Oklahoma’s Home Defense Act on the volume of residential and non-residential burglaries. Each time series, which were derived from two unpublished sources, consists of 144 monthly observations.

The first month for each of the outcome series begins January 2000, approximately 6 years prior to the introduction of the home defense legislation. The last monthly observation ends December, 2011, approximately 5 years subsequent to the implementation of the home defense amendment to the definition of justifiable homicide. Hence, we have a sufficient number of pre-and post-intervention observations to assess the influence of the intervention series on the outcome series (Cochran, Chamlin, & Seth, 1994; Singer & McDowall, 1988).

Outcome Series: Residential and Non-Residential Burglaries

The Home Defense Act was explicitly designed to allow inhabitants of the state to use lethal force to protect their homes, but not commercial property. Hence, we decided to model the influence of this legislation on both residential and non-residential burglaries. Unpublished, monthly counts of residential and non-residential burglaries known to the police were obtained from two sources. We requested and received residential and non-residential burglary time series from the City of Tulsa Police Department (City of Tulsa Police Department, 2013). We also requested and received the same residential and non-residential burglary time series from the Oklahoma State Bureau of Investigation’s Statistical Analysis Center (Oklahoma State Bureau of Investigation, 2013). As the state agency gets the local data directly from the city’s uniform crime reporting division, we anticipate that the findings from the interrupted time series analyses of the state and city burglary counts should be virtually identical.

Intervention Series

The Home Defense Act went into effect on November 1, 2006. To model the influence of this amendment to the penal code on residential and non-residential burglaries, we created a dummy series coded zero for the months prior to implement of this legislation and one thereafter.

Analytic Strategy

We use ARIMA interrupted time series procedures to model the intervention series on the four outcome series. Two features of this analytic technique recommend its use for the purpose at hand.

A fundamental concern associated with the evaluation of the efficacy of any legislative initiative (in our case, the revision of the justifiable homicide defense within the context of residential burglaries) is distinguishing its impact from other social processes that may be influencing an outcome series. It has been long recognized that maturation (ongoing causal processes that operate as a function of the passage of time, rather than the onset of an intervention) is one of the more serious threats to the internal validity of any longitudinal, quasi-experimental design (Campbell, 1957; Campbell & Stanley, 1963; Cook & Shadish, 1994). ARIMA techniques, unlike simple pre- and post-intervention mean or percentage difference tests, explicitly take into account the potentially confounding effects of maturation, as well as other stochastic processes (McDowall, McCleary, Meidinger, & Hay, 1980).

A second advantage of ARIMA modeling techniques over simple pre- and post-intervention change scores, as well as more sophisticated panel designs, is that they allow one to identify and estimate alternative functional forms of the relationship between an intervention series and an outcome series. The latter two analytic approaches assume that the effect of an intervention is well-represented as an abrupt, permanent change in the level of the outcome series (at least for the remainder of the observations for a given time series). Although one can estimate this functional form (as a zero-order transfer function) using ARIMA modeling techniques, one can also examine the relative fit of competing adjustment models. It is possible that the effect of an intervention gradually reaches a new level or that the effect is instantaneous but short-lived (often reflecting a publicity effect). A first-order transfer function can be estimated to model the former pattern of change in the level of a series, whereas a pulse function can be estimated to model the latter (McDowall et al., 1980).

In brief, an ARIMA interrupted transfer function model consists of two parts. The first, the “noise” component, uses information from prior observations of the outcome series to model the systematic variation (autocorrelation) within the outcome series. By applying the appropriate seasonal and non-seasonal differencing, along with the estimation of the appropriate seasonal and non-seasonal autoregressive and moving average parameters (prewhitening), one can separate the confounding influences of other causal processes from those associated with the intervention.

Once a satisfactory noise component is identified and estimated, the intervention component is added to the transfer function equation. If the inclusion of a dummy series for the intervention (coded 0 for the period prior to the onset of the intervention and coded 1 beginning with the observation in which the intervention occurs and thereafter) increases the explanatory power of the model above and beyond that provided by the noise component (Granger causality), then one can conclude that the intervention significantly affects the outcome series (Granger, 1980; McDowall et al., 1980).

ARIMA model building is an iterative process. By successively estimating the noise and intervention components, and subjecting them to a number of diagnostic tests, a final transfer function model can be derived. For the statistical details involved in the identification and estimation of the noise and intervention components of ARIMA interrupted time series models, we refer the reader to readily available published sources (McCleary & Hay, 1980; McDowall et al., 1980).

Results

Figure 1 presents a scatter plot of the number of residential burglaries by data source, whereas Figure 2 presents the same information for non-residential burglaries. Each graph includes a vertical marker that denotes the point in time at which the stand your ground legislation went into effect.

Figure 1.

Number of residential burglaries by data source.

Figure 2.

Number of non-residential burglaries by data source.

The visual examination of these graphs reveals two patterns of interest. To begin, there seem to be some minor discrepancies in reported burglaries across the data sources. Throughout each of the series, there are slightly more city-level, than state-level, residential and non-residential burglaries known to the police.

Second, we see little, if any, evidence that would support the inference that the Home Defense Act has any discernible impact on the volume of burglaries. Consider Figure 1. Throughout the period under investigation the number of residential burglaries drifts upward, regardless of the data source. Thus, it appears that the implementation of the legislative initiative has no impact on the number of residential burglaries.

Figure 2, which presents the scatter plots for non-residential burglaries, is less amenable to interpretation. Approximately 1 year prior to the implementation of modification of the justifiable homicide section of the penal code, the number of non-residential burglaries begins to drift upwards. This pattern of increase holds for more than a year after the new legislation went into effect. Subsequently, both non-residential burglary series seem to drift slightly downward. Hence, there is little reason to infer that legislation that was enacted for the purpose of reducing the number of residential burglaries is responsible for the observed distribution, over time, of non-residential burglaries.

Overall, our examination of the raw burglary series suggests to us that, regardless of some slight discrepancies in the number of residential and non-residential burglaries across data sources, the state- and city-level data are probably interchangeable. Nonetheless, we caution against drawing any strong conclusions based on the simple inspection of the raw time series. It is possible that extraneous causal processes may be obscuring the effect of the legislative initiative on each of the burglary series. We turn now to the interrupted time series analyses, which explicitly take into account the influence of ongoing stochastic processes within an outcome series, to model the differential impact of the Home Defense Act on the four burglary time series.

Interrupted Time Series Analyses

The purpose of this study is to determine whether two, ostensibly equivalent, sources of official data yield comparable (or disparate) findings with regard to the impact of the Home Defense Act on residential and non-residential burglaries. With that objective in mind, we turn now to the interrupted time series analyses.

As there is no theoretical basis for predicting that the impact of the intervention on any of the outcome series should fit one functional form as opposed to another, we estimate and report parameter estimates from the zero-order (immediate and lasting), first-order (gradual and lasting), and the pulse (immediate, but temporary) transfer function equations in Tables 1 through 3, respectively. Within each table, Panel A presents the noise and intervention components of the transfer function equations generated by the analysis of the data procured from the state agency, whereas Panel B presents the same information for the transfer function equations generated by the analysis of the data garnered from the municipal police force.

Table 1.

Noise and Zero-Order Transfer Function Intervention Models for the Effect of the 2006 Home Defense Act on Residential and Non-Residential Burglaries by Data Source.

Panel A. State data.
Noise model	Intervention model
Residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + ω₀I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²) lnY_t	I_t = 0 for observations 1-82
θ₁ = −.702, p < .001	I_t = 1 for observations 83-144
θ₁₂ = −.708, p < .001	ω = −53.37, p < .12 Q = 20.84, df = 19, p < .36
Non-residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + ω₀I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 −B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.699, p < .001	I_t = 1 for observations 83-144
θ₁₂ = −.902, p < .001	ω = 26.92, p < .08 Q = 18.3, df = 19, p < .51
Panel B. Municipal data.
Noise model	Intervention model
Residential burglaries
ARIMA (1,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + ω₀I_t
(1 − θ₁B − Φ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²) lnY_t	I_t = 0 for observations 1-82
θ₁ = −.897, p < .001	I_t = 1 for observations 83-144
Φ₁ = .374, p < .001	ω = −29.21, p < .43
θ₁₂ = .310, p < .001	Q = 19.0, df = 19, p < .46
Non-residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + ω₀I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.732, p < .001	I_t = 1 for observations 83-144
θ₁₂ = .222, p < .01	ω = 32.36, p < .03 Q = 29.3, df = 20, p < .09

Note. $Y_{t}^{*}$ = prewhitened outcome series; a_t = random shock; θ = moving average parameter; Φ = autoregressive parameter; B = backward shift operator; ω = zero-order input parameter of a transfer function; I_t = intervention series; Q = Ljung-Box test statistic for the null hypothesis that the model residuals are distributed as white noise; ARIMA = autoregressive moving average.

Table 2.

Noise and First-Order Transfer Function Intervention Models for the Effect of the 2006 Home Defense Act on Residential and Non-Residential Burglaries by Data Source.

Panel A. State data.
Noise model	Intervention model
Residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + (ω₀/1 − δ₁B)I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.708, p < .001	I_t = 1 for observations 83-144
θ₁₂ = −.709, p < .001	ω = −60.60, p < .19 δ = −0.19, p < .82 Q = 21.1, df = 18, p < .28
Non-residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + (ω₀/1 − δ₁B)I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.714, p < .001	I_t = 1 for observations 83-144
θ₁₂ = −.897, p < .001	ω = 16.34, p < .13 δ = 0.67, p < .008 Q = 17.3, df = 18, p < .51
Panel B. Municipal data.
Noise model	Intervention model
Residential burglaries
ARIMA (1,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + (ω₀/1 − δ₁B)I_t
(1 − θ₁B − Φ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.914, p < .001	I_t = 1 for observations 83-144
Φ₁ = .380, p < .001	ω = 0.58, p < .73
θ₁₂ = .306, p < .001	δ = −1.03, p < .001 Q = 16.1, df = 18, p < .57
Non-residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + (ω₀/1 − δ₁B)I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.737, p < .001	I_t = 1 for observations 83-144
θ₁₂ = .227, p < .01	ω = 19.31, p < .10 δ = 0.62, p < .02 Q = 30.1, df = 19, p < .06

Note. $Y_{t}^{*}$ = prewhitened outcome series; a_t = random shock; θ = moving average parameter; Φ = autoregressive parameter; B = backward shift operator; ω = zero-order input parameter of a transfer function; δ = the first-order output parameter of a transfer function; I_t = intervention series; Q = Ljung-Box test statistic for the null hypothesis that the model residuals are distributed as white noise; ARIMA = autoregressive moving average.

Table 3.

Noise and Pulse Transfer Function Intervention Models for the Effect of the 2006 Home Defense Act on Residential and Non-Residential Burglaries by Data Source.

Panel A. State data.
Noise model	Intervention model
Residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + (θ₀/1 − δ₁B)(1 − B)I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.770, p < .001	I_t = 1 for observations 83-144
θ₁₂ = −.706, p < .001	ω = −96.63, p < .02 δ = 0.72, p < .001 Q = 22.06, df = 18, p < .23
Non-residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + (θ₀/1 − δ₁B)(1 − B)I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.666, p < .001	I_t = 1 for observations 83-144
θ₁₂ = −.888, p < .001	ω = −10.67, p < .30 δ = −0.90, p < .001 Q = 18.9, df = 18, p < .41
Panel B. Municipal data.
Noise model	Intervention model
Residential burglaries
ARIMA (1,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + (θ₀/1 − δ₁B)(1 − B)I_t
(1 − θ₁B − Φ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.918, p < .001	I_t = 1 for observations 83-144
Φ₁ = .389, p < .001	ω = −64.69, p < .19
θ₁₂ = .306, p < .001	δ = 0.84, p < .002 Q = 19.66, df = 18, p < .36
Non-residential burglaries
ARIMA (0,1,1)(0,1,1)₁₂	$Y_{t}^{}$ = a_t* + (θ₀/1 − δ₁B)(1 − B)I_t
(1 − θ₁B)(1 − θ₁₂B¹²) a_t = (1 − B)(1 − B¹²)Y_t	I_t = 0 for observations 1-82
θ₁ = −.751, p < .001	I_t = 1 for observations 83-144
θ₁₂ = .222, p < .01	ω = 35.44, p < .02 δ = 0.98, p < .001 Q = 30.6, df = 19, p < .44

Admittedly, the evaluation of the relative fit of the alternative transfer function models can be somewhat daunting, especially for those unfamiliar with ARIMA modeling procedures. Therefore, to simplify our comparison of the analyses of the state- and city-level data, we also include a brief summary of the findings in Table 4.

Table 4.

Summary of Findings From the Zero-Order, First-Order, and Pulse Transfer Function Intervention Models for the Effect of the 2006 Castle Legislation on Residential and Non-Residential Burglaries by Data Source.

Functional form by location	State data effect	City data effect
Residential burglaries
Zero-order transfer function	Null	Null
First-order transfer function	Null	Null
Pulse transfer function	Significant and negative	Null
Non-residential burglaries
Zero-order transfer function	Approaches significance and positive	Significant and positive
First-order transfer function	Null	Approaches significance and positive
Pulse transfer function	Null	Null

Consider the analyses of the state-level data. Inspection of Panel A within each of the first three tables clearly reveals that the Home Defense Act has no appreciable impact on non-residential burglaries. Regardless of functional form, the parameter estimates associated with the intervention components are insignificant (although the parameter estimate [ω] from the zero-order transfer equation for non-residential burglaries approaches statistical significance [ω = 26.92, p < .08]).

The legislative initiative did, however, lead to an immediate, but temporary, reduction of approximately 100 residential burglaries per month (the best-fitting model is the pulse transfer function). As has been demonstrated elsewhere (McCleary and Hay, 1980, pp. 164-168; McDowall et al., 1980, pp. 80-83), one can calculate how many observations it takes to return to the initial level with the following formula: (δ)ⁱ × (ω), where δ is the parameter estimate of the rate of change back to the pre-intervention level, superscript “i” is the post-intervention observation, and ω is the parameter estimate for the magnitude of change. Inserting the parameter estimates from the pulse function equation in Panel A of Table 3, we find that within 10 months the number of burglaries approaches its pre-intervention level (a total, temporary decline of less than 4% from the pre-intervention cumulative count of 8,960 residential burglaries).

Overall, the ARIMA analyses of the state agency data suggest that the Home Defense Act produced a short-lived reduction in residential burglaries. Clearly, there is no evidence that would indicate that it did any harm. That is to say, it did not lead to an increase in any of the disaggregated burglary series. Unfortunately, as we will discuss below, the same cannot be inferred from the results of the interrupted time series analyses of the municipal burglary data.

Contrary to what we reported above, the intervention series had no impact on residential burglaries, but did generate a change in the level of non-residential burglaries. Specifically, it produced an immediate and lasting increase of approximately 32 non-residential burglaries per month (see Panel B of Table 1 and Table 4).¹ This shift in the expected level of non-residential burglaries is substantial. Expressed as the percent change from the pre-intervention to the post-intervention distribution of non-residential burglaries, it translates into a 25% increase in volume of non-residential burglaries.

In sum, the impact of the Home Defense Act on residential and non-residential burglaries depends on which data source is analyzed. Based on our examination of the state agency data, we would most likely deduce that it led to a temporary decline in residential burglaries, but produced no increase in non-residential burglaries.² Based on the examination of the city agency data, we would conclude that its implementation did not affect residential burglaries, but did generate a monthly increase in the number of non-residential burglaries.

Discussion

Probably the most basic issue facing macro-criminologists seeking to evaluate social theory and/or social policy is the measurement of crime. More often than not, researchers are forced to rely on official data, as codified in the Uniform Crime Reports (UCR), to provide information about the amount of criminal behavior across, and within, jurisdictions. The rationale is simple. Rarely, if ever, are there any alternative, macro-level measures of crime. This is particularly true for those among us who are interested in assessing the impact of changes, over time, in the social environment (e.g., legislation, historical events) on volume of illegal behavior.

Fortunately, comparisons of official statistics with other measures of criminality appear to indicate that official statistics are reasonably valid indicators of common law crime, especially the ones that are perceived to be the most serious by the victims (Cantor & Cohen, 1980; Gove et al., 1985; Hindelang, 1974, 1978, 1981; Wiersema et al., 2000). Reliability, however, may be another matter. We suspect that most of us presume, with little trepidation, that official data, whether they are obtained from federal, state, or local agencies, are virtually interchangeable. The present investigation critically evaluates this implicit assumption by comparing the effects of a change in legislation on official measures of residential and non-residential burglaries from two alternative data sources.

Our visual examination of the raw burglary time series produced no surprises. Although there are some minor discrepancies from month to month in the number of residential and non-residential burglaries, their distributions, over time, are quite similar. Moreover, the comparison of the pre- and post-intervention sections of the two burglary series seem to indicate that the Home Defense Act had no discernible influence on either outcome series (see Figures 1 and 2). However, given the potential confounding (spurious) influence of serial correction within each series, any inferences based entirely on the inspection of the raw time series must be viewed as speculative (Granger & Newbold, 1974).

Although the differences in the raw time series appeared to be inconsequential, the results from the ARIMA transfer equations point to a very different conclusion. Specifically, the findings from the interrupted time series analyses show that state and local data are not interchangeable. Our analyses of the state agency data revealed that the Home Defense Act led to a temporary decline in residential burglaries, but had no effect on non-residential burglaries. In contrast, our analyses of the city agency data indicated that the Home Defense Act had no effect on residential burglaries, but did generate a permanent, monthly increase in the number of non-residential burglaries.

What should we make of the disparities in statistical outcomes across data sources? Some might be tempted to dismiss our findings as an artifact of the research design and/or modeling procedures. We think that this is unlikely. The time series quasi-experiment has long been recognized as the research design of choice for the purpose of assessing the impact of a social policy initiative on behavior (Cook & Campbell, 1979; Cook & Shadish, 1994; Ross & McCleary, 1983). Moreover, as we explained above, ARIMA transfer function models have two advantages over their alternatives. First, they can more effectively control for the confounding influence of serial correlated error and other stochastic processes on an outcome series. Second, they allow the data analyst to model the functional form of the impact of an intervention on the dependent time series (Box & Tiao, 1975; McCleary & Hay, 1980; Ross & McCleary, 1983). Hence, we are confident that the results of our data analyses, which were subjected to numerous diagnostic tests, do not reflect some idiosyncratic methodological anomaly.

Others might be tempted to adopt the stance of Seidman and Couzens (1974) who conclude “. . . that the Uniform Crime Reporting System is useless as a tool for evaluation of social policy” (pp. 484-485). Although we are concerned that researchers, including ourselves, are often too cavalier about the problems associated with crimes known to the police, we are less pessimistic than the more radical critics (Seidman & Couzens, 1974; Selke & Pepinsky, 1982). Our position is much closer to that of McCleary and his colleagues who remind us that official crime counts are socially constructed, reflecting the decisions of victims, investigators, coders, as well as the organizational context within which these decisions are made (McCleary, Nienstedt, & Erven, 1982). Thus, we probably should not be all that surprised when state and local crime control agencies generate divergent enumerations of crimes known to the police (McCleary et al., 1982).

Taken in their entirety, our findings suggest to us the following data collection strategies. First, whenever possible, we need to seek out, obtain, and analyze official crime data from multiple sources. The greater agreement in the enumeration of crime across data sources, the greater the confidence we can have in the reliability of the data. Second, we need to find out, whenever possible, how each of each agency collects and records the amount of crime. Such information can help one to decide which data to utilize when measurement discrepancies do occur.

Last, where one cannot, as in the present situation, determine why alternative sources of data provide divergent counts of crime, we recommend using the most proximate data source available. In a recent examination of the inter-agency consistency in the codification and reporting of homicide data, Pizarro and Zeoli (2013) conclude that discrepancies between information provided by the Federal Bureau of Investigation (FBI)’s Supplementary Homicide Reports and the Newark police department is most likely attributable to the common practice of local departments updating their homicide data but not forwarding this information to the FBI. We suspect that a similar process might be at work here (Pizarro & Zeoli, 2013; Pridemore, 2005).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Mitchell B. Chamlin is professor of Criminal Justice at Texas State University. His research focuses on the elaboration of macro-criminological theory and the evaluation of social policy. He has published over 70 articles in peer-reviewed journals.

Andrea E. Krajewski is a doctoral student at Texas State University. Her research focuses on evaluating the impact of legislative initiatives on crime and crime control.

References

Bailey

R. E.

(1948). The burden of proving self-defense in homicide cases. Journal of Criminal Law and Criminology, 39, 189-192.

Boots

D. P.

Bihari

Elliott

(2009). The state of the castle: An overview of recent trends in state castle doctrine legislation and public policy. Criminal Justice Review, 34, 515-535.

Box

Tiao

G. C.

(1975). Intervention analysis with applications to economic and environmental problems. Journal of the American Statistical Association 70, 70-92.

Campbell

D. T.

(1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297-312.

Campbell

D. T.

Stanley

J. C.

(1963). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin.

Cantor

Cohen

L. E.

(1980). Comparing measures of homicide trends: Methodological and substantive differences in the vital statistics and uniform crime report time series (1933-1975). Social Science Research, 9, 121-145.

Chamlin

M. B.

(2014). An assessment of the intended and unintended consequences of Arizona’s Self-Defense, Home Protection Act. Journal of Crime & Justice, 37, 327-338.

City of Tulsa Police Department. (2013). Unpublished burglary data.

Cochran

J. K.

Chamlin

M. B.

Seth

(1994). Deterrence or brutalization? An impact assessment of Oklahoma’s return to capital punishment. Criminology, 36, 107-134.

10.

Cook

T. D.

Campbell

D. T.

(1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.

11.

Cook

T. D.

Shadish

W. R.

(1994). Social experiments: Some developments over the past fifteen years. Annual Review of Sociology, 45, 545-580.

12.

Ennis

P. H.

(1967). Criminal victimization in the United States: A report of a national survey (No. 2). Chicago, IL: National Opinion Research Center, University of Chicago.

13.

Gove

Hughes

Geerken

(1985). Are uniform crime reports a valid indicator of the index crimes? An affirmative answer with minor qualifications. Criminology, 23, 451-501.

14.

Granger

(1980). Testing for causality: A personal viewpoint. Journal of Economic Development, 2, 329-352.

15.

Granger

Newbold

(1974). Spurious regressions in econometrics. Journal of Econometrics, 2, 111-120.

16.

Hindelang

(1974). The uniform crime reports revisited. Journal of Criminal Justice, 2, 1-17.

17.

Hindelang

(1978). Race and involvement in common law crimes. American Sociological Review, 43, 93-109.

18.

Hindelang

(1981). Variation in rates of offending. American Sociological Review, 46, 461-474.

19.

Levin

(2010). A defensible defense? Reexamining castle doctrine statutes. Harvard Journal on Legislation, 47, 523-553.

20.

McCleary

Hay

R. A.

Jr. (1980). Applied time series analysis for the social sciences. Beverly Hills, CA: Sage.

21.

McCleary

Nienstedt

Erven

(1982). Uniform crime reports as organizational outcomes: Three time series experiments. Social Problems, 29, 361-372.

22.

McDowall

Loftin

(2007). What is convergence, and what do we know about it? In Lynch

J. P.

Addington

L. A.

(Eds.), Understanding crime statistics: Revisiting the divergence of the NCVS and UCR (pp. 93-124). Cambridge, UK: Cambridge University Press.

23.

McDowall

McCleary

Meidinger

E. E.

Hay

R. A.

Jr. (1980). Interrupted time series analysis. Thousand Oaks, CA: Sage.

24.

Oklahoma State Bureau of Investigation (2013). Statistical Analysis Center. Unpublished burglary data.

25.

Pizarro

J. M.

Zeoli

A. M.

(2013). An assessment of the quality of homicide data in the supplementary homicide reports: A research note. Justice Quarterly, 30, 711-731.

26.

Pridemore

W. A.

(2003). Measuring homicide in Russia: A comparison of estimates from the crime and vital statistics reporting systems. Social Science & Medicine, 57, 1343-1354.

27.

Pridemore

W. A.

(2005). A cautionary note on using county-level crime and homicide data. Homicide Studies, 9, 256-268.

28.

Ren

Zhang

Zhao

J. S.

(2015). The deterrent effect of the castle doctrine law on burglary in Texas: A tale of outcomes in Houston and Dallas. Crime & Delinquency, 61(8), 1127-1151.

29.

Rokaw

W. M.

Mercy

J. A.

Smith

J. C.

(1990). Comparing death certificate data with FBI crime reporting statistics on U.S. homicides. Public Health Reports, 105, 447-455.

30.

Ross

H. L.

McCleary

(1983). Methods for studying the impact of drunk driving laws. Accident Analysis & Prevention, 15, 415-428.

31.

Seidman

Couzens

(1974). Getting the crime rate down: Political pressure and crime reporting. Law & Society Review, 8, 457-493.

32.

Selke

Pepinsky

(1982). The politics of police reporting in Indianapolis, 1948-1978. Law and Human Behavior, 6, 327-342.

33.

Singer

S. I.

McDowall

(1988). Criminalizing delinquency: The deterrent effects of the New York juvenile offender law. Law & Society Review, 22, 521-535.

34.

Skogan

W. G.

(1974). The validity of official crime statistics: An empirical investigation. Social Science Quarterly, 55, 25-38.

35.

Wiersema

Loftin

McDowall

(2000). A comparison of supplementary homicide reports and national vital statistics system homicide estimates for U.S. counties. Homicide Studies, 4, 317-340.