Revisiting the IT Productivity Paradox

Abstract

Twenty-five years after Nobel Laureate economist Robert Solow observed “seeing computers everywhere but in the productivity statistics,” the question of productivity gains from information technologies (IT) remains unanswered. This study examines the role of IT on one of the major indicators of police productivity: crime clearance rates. Relying on a two-wave cohort panel research design of roughly 700 police agencies, the study reveals that significant IT advances were made between the pre and post time periods in the provision of computerized crime data, crime analysis capabilities, and real-time communications. Nonetheless, using multiple hierarchical regression analysis, the study provides robust evidence for suggesting that computerization had little influence on productivity gains. The results of this study raise several very important issues pertaining to the goals of public organizations. While this study is limited to policing, a narrow time period, and internal IT systems, the results are nonetheless noteworthy. The research illustrates that conventional explanations for the IT productivity paradox do little to explain the shortfall. In closing, the article offers rival, but yet untested, explanations that may prove worthy of additional research.

Keywords

IT productivity paradox public sector police

Introduction

Twenty-five years after Nobel Laureate economist Robert Solow observed “seeing computers everywhere but in the productivity statistics” the question of productivity gains from information technologies (IT) remains unanswered. With governments investing roughly $160 billion annually on IT (Nunziata, 2010), the failure to provide definitive insights is lamentable.

This study examines the role of IT on police productivity. Relying on a two-wave cohort-based research design of roughly 700 police agencies, IT changes are tracked and then measured against one of the most longstanding measures of police performance: crime clearance rates. The design is especially pivotal because it allows comparisons to be made pre and post computerization. Moreover, the study goes beyond the simple formulation of “computer usage” and measures outcomes in light of: (a) the provision of a wide range of computerized databases for tracking criminal activity, (b) advances in computerized crime analysis functions, and (c) improvements in computerized communications. In other words, gains in criminal data, crime analysis, and internal communications should equate to positive changes in productivity and be witnessed in improvements in crime clearance rates.

With more than a quarter century of experience behind us, and ongoing allegations of an IT productivity paradox (Goldfinch, 2007; Thatcher, Brower, & Mason, 2006), the topic is especially salient. For the most part, empirical research on IT and public sector productivity has been lacking and when present has done little to offer consistent and sage advice. As discussed below, on the issue of productivity gains, the research is especially vague. Yet the topic is important because the costs of IT are not purely financial. Other costs (such as foregone opportunities, switching costs, security and privacy, computer crime, identity theft, virus attacks, hacking, etc.) all take a toll.

The discussion below begins with a discussion of IT and its influence on organizational productivity. Unfortunately, with the advent of e-government, scholars have shifted their research agendas so the discussion of organizational payoffs from “back office” systems has gone largely dormant. The data analysis discussed below focuses exclusively on internal software solutions that are designed to promote organizational productivity as opposed to externally facing solutions designed to engender customer engagement. After the literature review, the research questions, research methods, and findings follow. The discussion section explores the IT productivity paradox in light of the specific research findings. Explanations for the productivity paradox are then examined. The article closes with suggestions for future research.

The IT Productivity Paradox

Solow’s (1987) assertion of an IT productivity paradox back in 1987 drew immense attention and remains the subject of inquiry (Lee & Perry, 2002; Thatcher et al., 2006). The paradox was first broached when Solow found that rapid growth in computing was accompanied by declining labor productivity. Predictions that IT would lead to a fundamental shift in organizational operations were called into question.

Despite impressive financial investments, the subject of payoffs on work operations remains unanswered. According to King and Kraemer (2006) computing is often seen as a “catalyst that can and should be used to bring about dramatic change and transformation in government” (p. 1). And yet the question as to whether IT has acted as a catalyst for administrative reform, remains controversial. The predominant answer seems to be: It depends.

Recognizing that the pool of studies on IT in the public sector is relatively small and aging quickly, the dilemma of the paradox remains in full swing. A spate of research took place in the 1980’s and 1990’s, but over the recent past the topic has received little consistent empirical attention. Literature on IT in the public sector has largely focused on three areas, innovation diffusion, implementation, and less frequently, outcomes. For the most part, implementation has received the lion’s share of the attention. And for the last 10 years, e-government has held a cherished spot. Studies of IT and its influence on internal operations are relatively scant.

By and large the literature on productivity gains ranges from euphoric optimism to melancholic pessimism. Ho (2002) is perhaps one of the more optimistic. He predicts a paradigm shift in government operations—one from standardization and departmentalization to coordinated networks and external collaborations. Dunleavy, Margetts, Bastow, and Tinkler (2006) apparently agree. In their article titled “New Public Management is Dead—Long Live Digital-Era Governance,” the authors claim that “digital-era changes have triggered numerous significant shifts” not the least of which is disintermediation, or the decoupling of tasks from a sequential assembly line orientation to parallel processing where multiple tasks can be conducted in tandem.

Empirical studies that link explicitly to objective outcomes are rare. A noteworthy exception is Lee and Perry’s (2002) examination of the 50 states. They looked at IT investments and gross state product. Their findings illustrate that IT investments did prove instrumental in stimulating economic performance.

Welch and Pandey (2006) acknowledge that research on the potential for IT to cut red tape is heavily context specific. But they do find that higher intranet reliance leads to lower red tape which should transfer to productivity payoffs. They also speculate that information quality and intranet reliance improves as the qualities of the technology are explored and understood. In another example, Norris and Moon’s (2005) research on local governments finds that IT improves organizational efficiency, accuracy, timeliness, and effectiveness.

King and Kraemer (2006) argue, “There is little dispute that IT is beneficial to the organization’s that use it, especially in the area of productivity. Benefits come in the way of day-to-day tasks but are simply useful adaptations of the technology to improve administrative performance” (p. 8). But they also identify that IT has little influence on reforming current administrative controls.

Others call attention to the potential capabilities of IT but claim that benefits are moderated by a number of organizational forces. Norris’ (2010) work is perhaps the most pessimistic. Norris predicts that e-government will follow IT’s path and will not provide any public sector transformations and all that will be garnered is more of the same. Addressing the question of IT and administrative reform, King and Kraemer claim that IT has been relatively instrumental in addressing high-volume-transaction-based operations but the benefits are not bold or innovative.

Despite Solow’s assertions, a failure rate of roughly 40% (Goldfinch, 2007), and a lack of identifiable benefits, investments in IT solutions continue unabated. Moon and Bretschnieder (2002) claim that IT is seen as a panacea for improving managerial efficiency. Bewildered and confused scholars such as Foley and Alfonso (2009) identify that “despite the fact that little robust evidence exists . . . the efficiency benefits of IT have been universally accepted by policy makers” (p. 371). Moreover, Morgeson, VanAmburg, and Mithas (2010) agree claiming practitioners have misplaced their trust in technology. Moynihan and Lavertu (2012) examine the cognitive biases that drive public managers to adopt technology solutions. They claim that public officials are driven by utopian technological determinism, cyber optimism, and idolization (Garson, 2006; Goldfinch, 2007). As Dawes (2008) claims: “IT has permeated nearly every aspect of government” but apparently the payoff is far from certain.

Karr-Wisniewski and Lu (2010) capture the paradox in their statement: “Individuals very often must face the dilemma of technology use—increased usage of technology tools does not always lead to increased work productivity; rather, sometimes it actually can be counterproductive” (p. 1061). In a seminal article, two decades ago Brynjolfsson (1993) offered five reasons for the possibility of a productivity paradox. His first explanation is the mismeasurement of outputs and inputs. Purportedly, the benefits of IT may be quite large but that a proper index of its value has yet to be discovered. His argument is that the field lacks the proper measurements to reveal benefits. The next explanatory factor is a potential time lag due to a learning curve. Here the idea is that IT implementation is associated with a learning curve and that organizations require time to learn the technology before benefits will be demonstrated. Brynjolfsson’s (1993) third explanation addresses national-level statistics and is associated with the redistribution of benefits and the dissipation of profits. Under this scenario, those investing in IT witness benefits privately but at the expense of others, thus the return on investment at the national level is a wash. Apparently, “information is particularly vulnerable to rent dissipation, in which one firm’s gain comes entirely at the expense of others” (p. 75). Thus, no “new wealth” is created.

The next explanation is mismanagement of the IT implementation effort. This threat suggests that some sort of mismanagement occurred in the adoption of the technology that led to low returns. Studies of IT failures and abandonments suggest that as many as 60% of projects experience problems (Goldfinch, 2007). Brynjolfsson’s (1993) final explanation is that IT acts as an important intermediary effect and, thus, influences organizational processes but not outcomes.

The research below seeks to test for the presence of the paradox in light of controlling for many of Brynjolfsson’s explanations. Ideally, the research will provide greater insights on IT and productivity and act as a springboard for future research. In sum, the question of IT payoffs appears contingent on a host of factors. Alternatively, longstanding research suggests that IT can be an instrument of innovation but it can also solidify the power of the ruling elite by reinforcing the hierarchical structure of bureaucratic organizations (Attewell & Rule, 1984; King & Kraemer, 2006; Norris, 2010; Pinsonneault & Kraemer, 2002). Previous findings that IT can solidify existing power structures are especially intriguing for this research. If in fact IT does support the ruling elite, one might expect the realization of performance gains . . . it seems unlikely that executives would seek to undermine productivity payoffs given the potential for political and financial gains. Executives are often rewarded handsomely for improved performance. This study examines the role of IT on police productivity. Relying on a 2-wave cohort based research design of roughly 700 police agencies, IT changes are tracked and then measured against one of the most longstanding measures of police performance: crime clearance rates.

Method

Data Sources and Sample

This research uses two major data collection efforts. Every several years, the Bureau of Justice Statistics conducts The Law Enforcement Management and Administrative Statistics (LEMAS) survey. The survey captures information from law enforcement agencies on topics including expenditures, job functions, education and training, community policing, and computers and information systems. The most recent LEMAS survey was conducted in CY2007. Prior to CY2007, the previous survey was conducted in CY2003. As the computer and information systems questions were comparable for the two surveys, change rates in computerization activities could be isolated. The independent variables (discussed further below) derive from the LEMAS surveys.

Uniform Crime Reports from the Federal Bureau of Investigation for CY2003 and CY2007 provide the dependent variables for the study. A reported crime is considered “cleared” when at least one suspect is charged and/or arrested for the crime or offense. Each month, police agencies are required to submit statistics to the Federal Bureau of Investigation on the number and type of crimes that occurred and whether the crime was cleared by an arrest. Where the sheer “number of crimes” is an important metric, “crime clearance rates” have been used for assessing police performance for over three decades and tends to be an important metric of choice (Garicano & Heaton, 2010; Halachmi & Bouckaert, 1996). The benefit of examining crime clearance rates is grounded in the fact that crime rates are a function of an entire system of political, social, and economic attributes. But clearing the crime with an arrest is limited to police performance. Therefore, it has become a major indicator of police productivity.

Because agency identification codes are routinely captured, the LEMAS surveys could be merged with the Uniform Crime Reporting database. Thus, the study sample consists of the 734 police agencies that responded to both the CY2003 and CY2007 surveys along with their respective crime clearance rates. Pairwise t-tests revealed no appreciable difference between the sample and the non-responders in light of state of origin. Therefore, all states appear adequately represented in the sample. However, as indicated in Table 1, ANOVA test comparisons between the sample and the non-responders showed statistically significant differences on population served, operating budget, number of sworn personnel, and minimum educational requirements (p ≤ .000, .000, .000, .01, respectively). The fact that the sample was significantly larger on population served, operating budget, number of sworn personnel is not surprising given that many of the nation’s largest departments responded (i.e., New York, Los Angeles, Chicago, Houston, etc.). But it does question the generalizability of the findings to smaller police departments.

Table 1.

Comparison Between Sample and Non-Responders.

Variable	Sample status	n	M		Sum of squares	df	Mean square	F	Significance
Population served	Not in sample	1,255	28,293	Between groups	5,354,787,400,319	1	5,354,787,400,319	86.39	.000
	In Sample	734	1,35,820	Within groups	123,168,035,811,405	1,987	61,986,932,970
	Total	1,989	67,973	Total	128,522,823,211,723	1,988
Operating budget	Not in sample	1,255	72,49,962	Between groups	514,068,227,948,226,000	1	514,068,227,948,226,000	48.17	.000
	In sample	734	4,05,66,348	Within groups	21,206,537,946,680,200,000	1,987	10,672,641,140,755,000
	Total	1,989	1,95,44,697	Total	21,720,606,174,628,400,000	1,988
Number of sworn personnel	Not in sample	1,255	56	Between groups	38,965,355	1	38,965.355	43.96	.000
	In sample	734	346	Within groups	1,761,323,531	1,987	886,424
	Total	1,989	163	Total	1,800,288,886	1,988
Minimum education requirement	Not in sample	1,241	3.63	Between groups	4	1	4.407	6.29	.012
Minimum education requirement	In sample	727	3.53	Within groups	1,378	1,966	0.701	6.29	.012

Research Design

As alluded to above, the research relies on a two-wave cohort panel and incorporates gain score changes to measure advances. Gain score analysis (sometimes referred to as change score analysis) answers the question: “how do groups, on average, differ in gains?” Critics of gain score analysis claim that measurement unreliability can result due to individual differences. However, Rogosa (1988) counters with “If all individuals grow at nearly the same rate, gain scores show that you cannot detect individual differences that do not exist . . . The difference score is an unbiased estimate of true change” (p. 180). Rogosa demonstrates that gain scores can provide both a reliable and unbiased estimate of true change. Thus, the variables identified below are measured by the differences between CY2003 and CY2007. The expectation is that those agencies that computerized in the period between the CY2003 to CY2007 surveys would encounter improvements in the gain scores of their crime clearance rate over and above their counterparts and their pre-computerization time period. The cohort design is especially useful because it allows the ability to measure computerization at two points in time and isolate the influences before and after while also allowing the group that had “no computerization changes” to serve as a comparison group.

Independent Variables

The LEMAS survey asks a number of questions pertaining to computerization. Thus, the research is able to go beyond the simple question of whether computers are used and tap three overall contributions from computerization. Three broad categories of questions were examined. The first category reflects the provision of “criminal-related computerized databases” that were made available on-site in the police department and in the field to patrolling officers. The distinction between departmental and field-level database access is important. Patrol (field-level) officers are usually the first responders to a crime incident where an arrest is made or reports are taken and then forwarded to investigations. Detectives investigate crime events where the perpetrator(s) is unknown. In combination, the on-site detectives and the field-level patrol officers are responsible for the crime clearance rates. Percentages were calculated for 12 variables to approximate “availability on-site” and 7 variables for “availability in the field.” The appendix provides the survey items used in the study.

The second category of questions focuses on the provision of “computers to conduct crime analysis” for solving criminal activities. Again, percentages were calculated for eight crime analysis functions including whether computers were used for problem-solving, analyzing crime rates, determining crime hot spots, and conducting criminal investigations (i.e., 0% indicating that computers were not used for any of the tasks and 100% indicating that computers were used on all the tasks).

The third category of questions pertained to using “computers to advance communications.” Percentages were calculated for whether computers were used to “communicate in near real time with other members of the organization,” to “provide real-time access to generated reports,” and to “provide real-time communications on intelligence gathering activities.” For each set of independent variables, gain scores were obtained for the percent differences between the 2003 and 2007 LEMAS surveys.

Dependent Variables

Crime clearance rates serve as the dependent variables in the models below. The gain scores are based on the difference in the percent cleared for 2003 and 2007. To prevent obfuscation of underlying trends, clearance rates were obtained for each crime type: murder, manslaughter, rape, robbery, assault, burglary, and vehicle theft. It is important to note that for each given crime type analyzed below, if there were zero crimes in CY2003 or in CY2007, then the clearance rate defaulted to missing.

Control Variables

Again, gain score analysis was used for operationalizing the control variables. Changes between CY2003 and CY2007 on the police jurisdiction’s (a) percent unemployment, (b) median family income, and (c) percent poverty rate were recorded. Also changes in department operational budget and number of sworn officers were added as controls to rule out competing hypotheses pertaining to resource availability.

Model Specification

Hierarchical regression was used to test for productivity improvements from computerization. In hierarchical regression, the predictors are added to the model sequentially. The sequential addition allows the researcher to observe changes in the model’s correlation coefficients so the size of the effect can be isolated. The models below report a two-step analysis with the controls as the first step and the independent variables for the second step. Confidence intervals for the regression coefficients were also obtained to isolate effect size.

Results

Descriptive Statistics

Table 2 provides the means, standard deviations, and average gain scores for each of the summative indices and the control variables. The first test was to determine the extent of the computerization changes that occurred between CY2003 and CY2007. Table 2 provides the results for each individual variable. Given that database availability is a binary variable, the paired McNemar’s nonparametric test was used to test the computerization differences between CY2003 and CY2007.

Table 2.

Paired Sample t-Test.

	M	SD	SE M	95% confidence interval of the difference		t	df	Significance
	M	SD	SE M	Lower bound	Upper bound	t	df	Significance
Unemployment 2007–Unemployment 2003	−1.33	0.94	0.03	−1.40	−1.26	−38.35	733	.000
Percent Poverty 2007−Percent Poverty 2003	0.56	1.91	0.07	0.43	0.70	8.01	732	.000
Median Income 2007–Median Income 2003	$8,083	$3,919	$145	$7,799	$8,367	55.85	732	.000
Operating Budget 2007–Operating Budget 2003	$7,859,505	$20,908,557	$771,749	$6,344,403	$9,374,606	10.18	733	.000
Sworn Employees 2007–Sworn Employees 2003^a	4.05	63.24	2.33	−0.53	8.63	1.74	733	.083

per 1,000 population.

In terms of on-site data availability, the largest gain was in the availability of data on firearms, fingerprints, summonses, and traffic stops (32%, 28%, 17%, and 15%, respectively). Data availability appeared to lose ground in the case of incident reports, stolen property reports, arrest reports, and citation reports. Warrant reports and biometric reports were the only two that did not witness a significant change rate between the two time periods. For field-level data availability, significant and positive changes were noted on all seven data sets with an average gain rate of between 15% and 11%. “Motor vehicle records,” “driving records,” and “warrants” demonstrated approximately 85% adoption levels.

For the computerized crime analysis functions only “computer use for problem solving” failed to change in a positive direction holding steady at roughly 34% of agencies. The greatest change rates were noted in using computers for “inter-agency information sharing,” “crime mapping,” “booking,” and “investigations.” Nonetheless, “crime analysis” and “crime investigation” show the greatest overall penetration at roughly 80% of police departments.

On the computerized communications front, all three recognized positive and significant growth rates. Communication of “infield report writing” witnessed the largest gain at roughly 30%. Computerization of infield report writing is an important contributor to clearance rates because it improves communication of crime events by allowing quick access to reports. All three showed a 70% adoption rate by the agencies.

Pearson correlation coefficients are provided in Table 3. Not surprisingly, unemployment, poverty rate, and income all show significant correlations. Changes in the operational budget also reflect a significant correlation with community unemployment, poverty, and income rates. “Computerized analysis functions” is also positively correlated with infield and on-site files along with communications with the field. Alternatively, changes in the rate of “on-site computerized files” do not demonstrate statistical significance with changes in “infield file rates.” These findings suggest that organizations that computerize typically pursue improvements in data, analysis, and communications. The link between communications and analysis is especially strong (r = .63).

Table 3.

Data, Analysis, and Communications Change Rates.

	2003 percent	2007 percent	Gain score percent	McNemar’s significance
Criminal data availability
On-site
Firearms	7	39	32	.000
Fingerprints	39	67	28	.000
Summonses	40	57	17	.000
Traffic stops	62	77	15	.000
Terrorism	25	29	4	.038
Calls for service	89	92	3	.004
Biometric	2	3	1	—
Warrants	67	68	1	—
Citations	77	68	−9	.000
Arrests	91	78	−13	.000
Stolen property	77	57	−20	.000
Incident	91	69	−22	.000
Infield
Address history	49	64	15	.000
Driving records	69	82	13	.000
Protective orders	53	66	13	.000
Inter-agency	44	57	13	.000
Criminal history	39	51	12	.000
Motor vehicle records	78	89	11	.000
Warrants	71	82	11	.000
Crime analysis functions
Inter-agency information	51	72	21	.000
Crime mapping	58	75	17	.000
Booking	43	58	15	.000
Investigations	72	84	12	.000
Community problems	52	61	9	.000
Hotspot	41	50	9	.000
Crime analysis	72	80	8	.000
Problem solving	33	36	3	—
Technical upgrade	52	53	1	—
Communications
Reports	47	71	24	.000
Infield	51	71	20	.000
Intelligence	52	70	18	.000

Hierarchical Regression

Diagnostic tests for normality, linearity, multicollinearity, homoscedasticity, and the existence of outliers ruled out violations of the assumptions for the hierarchical regressions. As indicated in Table 4, the largest correlation is between communications and analysis (r = 0.63) suggesting a potential problem with multicollinearlity; however, the variance inflation factor (VIF) tests revealed that the largest VIF value was two, much lower than the typical cutoff point of four. Thus, multicollinearity does not appear to provide a problem for the standard errors.

Table 4.

Variable Correlations.

	1	2	3	4	5	6	7	8
1. Unemployment
2. Poverty rate	.224**
3. Income	−.127**	−.588**
4. Number of officers	−.023	−.052	−.019
5. Budget	−.111**	−.114**	.108**	.006
6. Analysis functions	−.059	−.051	.046	−.035	.061
7. Computerized files infield	.074*	.059	−.025	−.033	−.047	.106**
8. Computerized files on-site	.021	−.062	.044	−.036	.104**	.239**	.049
9. Communications w/field	−.058	−.037	.01	.038	.03	.631**	.202**	.120**

p ≤ .05. **p ≤ .01.***p ≤ .001.

The results of the hierarchical regressions provide important insights on the “productivity paradox.” Table 5 provides the results of the hierarchical regression tests. Of the seven models (change rates in murder, rape, robbery, assault, burglary, larceny, and vehicle theft), only three statistically significant relationships were witnessed. Positive growth in the poverty rate had a significant influence on burglary and larceny clearance rates (p ≤ .05 and p ≤ .05, respectively). The only other statistical relationship was a negative relationship between on-site computerized files and percent change in the murder clearance rates (^*p ≤ .05). None of the other relationships, including the adjusted R² and the F tests, achieved statistical significance.

Table 5.

Summary of Hierarchical Regression Analysis for Crime Clearance Rates.

	B	SE	β	95% confidence interval for B		B	SE	β	95% confidence interval for B		Interval distance
	B	SE	β	Lower bound	Upper bound	B	SE	β	Lower bound	Upper bound	Interval distance
Dependent variable: Percent change in murder clearance rates (n = 394)
(Constant)	−0.74	7.52		−15.51	14.04	0.34	7.75		−14.9	15.57
Unemployment	1.18	2.42	0.03	−3.59	5.94	1.55	2.42	0.03	−3.21	6.31	9.52
Poverty rate	0.22	1.37	0.01	−2.47	2.92	0.13	1.37	0.01	−2.57	2.83	5.4
Income	0	0	0.04	0	0	0	0	0.04	0	0	0
Number of officers	0.01	0.03	0.02	−0.04	0.06	0	0.03	0.01	−0.05	0.06	0.11
Budget	0	0	0.01	0	0	0	0	0.02	0	0	0
Analysis functions						−1.06	0.96	−0.08	−2.95	0.84	3.79
Computerized files infield						−0.03	0.99	0	−1.97	1.91	3.88
Computerized files on-site						−1.55	0.7	−0.11*	−2.92	−0.18	2.74
Communications w/field						2.09	2.64	0.05	−3.1	7.28	10.38
Adjusted R²	−.01					0
F	0.15					0.92
Dependent variable: Percent change in rape clearance rates (n = 587)
(Constant)	−2.95	4.43		−11.65	5.75	−3.92	4.55		−12.86	5.02
Unemployment	−0.04	1.46	0	−2.91	2.83	−0.02	1.47	0	−2.9	2.86	5.76
Poverty rate	0.63	0.87	0.04	−1.09	2.34	0.57	0.87	0.03	−1.15	2.29	3.44
Income	0	0	−0.01	0	0	0	0	−0.01	0	0	0
Number of officers	0	0.02	0.01	−0.03	0.04	0	0.02	0	−0.04	0.04	0.08
Budget	0	0	0.01	0	0	0	0	0.02	0	0	0
Analysis functions						−0.88	0.6	−0.08	−2.06	0.29	2.35
Computerized files infield						0.44	0.63	0.03	−0.8	1.67	2.47
Computerized files on-site						−0.53	0.45	−0.05	−1.42	0.36	1.78
Communications w/field						2.38	1.63	0.08	−0.81	5.58	6.39
Adjusted R²	−.007					−.005
F	0.21					0.7
Dependent variable: Percent change in robbery clearance rates (n = 587)
(Constant)	0.68	2.49		−4.21	5.57	1.05	2.57		−4	6.09
Unemployment	−0.03	0.83	0	−1.65	1.6	0	0.83	0	−1.63	1.64	3.27
Poverty rate	0.2	0.49	0.02	−0.77	1.16	0.19	0.49	0.02	−0.78	1.16	1.94
Income	0	0	−0.02	0	0	0	0	−0.02	0	0	0
Number of officers	0	0.01	0	−0.02	0.02	0	0.01	0	−0.02	0.02	0.04
Budget	0	0	0.01	0	0	0	0	0.01	0	0	0
Analysis functions						0.31	0.33	0.05	−0.35	0.96	1.31
Computerized files infield						−0.04	0.35	−0.01	−0.72	0.65	1.37
Computerized files on-site						−0.16	0.26	−0.03	−0.66	0.34	1
Communications w/field						−0.7	0.9	−0.04	−2.47	1.07	3.54
Adjusted R²	−.007					−.011
F	0.16					0.22
Dependent variable: Percent change in assault clearance rates (n = 617)
(Constant)	−0.84	2.29		−5.33	3.66	−2	2.35		−6.62	2.62
Unemployment	−0.72	0.77	−0.04	−2.22	0.79	−0.7	0.77	−0.04	−2.21	0.81	3.02
Poverty rate	−0.01	0.46	0	−0.91	0.88	−0.03	0.46	0	−0.92	0.87	1.79
Income	0	0	−0.04	0	0	0	0	−0.04	0	0	0
Number of officers	0	0.01	0.02	−0.02	0.03	0	0.01	0.01	−0.02	0.02	0.04
Budget	0	0	0	0	0	0	0	0	0	0	0
Analysis functions						−0.08	0.31	−0.01	−0.68	0.52	1.2
Computerized files infield						0.39	0.32	0.05	−0.23	1.02	1.25
Computerized files on-site						−0.19	0.24	−0.03	−0.67	0.28	0.95
Communications w/field						1.23	0.83	0.08	−0.39	2.86	3.25
Adjusted R²	−.005					−.002
F	0.39					0.88
Dependent variable: Percent change in burglary clearance rates (n = 606)
(Constant)	−0.68	1.45		−3.52	2.16	−1.08	1.49		−4.01	1.84
Unemployment	0.68	0.48	0.06	−0.27	1.63	0.72	0.49	0.06	−0.24	1.67	1.91
Poverty rate	0.6	0.29	0.1	0.03	1.17	0.61	0.29	0.10*	0.04	1.17	1.13
Income	0	0	0.04	0	0	0	0	0.05	0	0	0
Number of officers	0	0.01	0	−0.01	0.01	0	0.01	0	−0.01	0.01	0.02
Budget	0	0	0.01	0	0	0	0	0.01	0	0	0
Analysis functions						−0.24	0.19	−0.07	−0.62	0.14	0.76
Computerized files infield						−0.08	0.2	−0.02	−0.47	0.32	0.79
Computerized files on-site						−0.07	0.15	−0.02	−0.37	0.23	0.6
Communications w/field						0.95	0.53	0.09	−0.08	1.98	2.06
Adjusted R²	.005					.004
F	1.64					1.32
Dependent variable: Percent change in Larceny clearance rates (n = 618)
(Constant)	−0.52	1.22		−2.92	1.87	−0.66	1.26		−3.13	1.81
Unemployment	0.32	0.41	0.03	−0.48	1.12	0.38	0.41	0.04	−0.43	1.18	1.61
Poverty rate	0.47	0.24	0.1	−0.01	0.95	0.48	0.24	0.10*	0	0.96	0.96
Income	0	0	0.02	0	0	0	0	0.03	0	0	0
Number of officers	0.01	0.01	0.03	−0.01	0.02	0	0.01	0.03	−0.01	0.02	0.03
Budget	0	0	0.04	0	0	0	0	0.04	0	0	0
Analysis functions						−0.13	0.16	−0.04	−0.45	0.2	0.65
Computerized files infield						−0.22	0.17	−0.05	−0.55	0.12	0.67
Computerized files on-site						−0.05	0.13	−0.02	−0.31	0.2	0.51
Communications w/field						0.62	0.44	0.07	−0.25	1.49	1.74
Adjusted R²	.003					.002
F	1.34					1.11
Dependent variable: Percent change in vehicle theft clearance rates (n = 610)
(Constant)	−2.31	1.92		−6.08	1.47	−2.68	1.98		−6.57	1.21
Unemployment	0.72	0.64	0.05	−0.55	1.98	0.69	0.65	0.04	−0.58	1.97	2.55
Poverty rate	0.59	0.38	0.08	−0.16	1.34	0.58	0.38	0.08	−0.17	1.34	1.51
Income	0	0	0.08	0	0	0	0	0.08	0	0	0
Number of officers	0	0.01	−0.01	−0.02	0.02	0	0.01	−0.01	−0.02	0.02	0.04
Budget	0	0	0.01	0	0	0	0	0.01	0	0	0
Analysis functions						−0.24	0.26	−0.05	−0.74	0.26	1
Computerized files infield						0.15	0.27	0.02	−0.38	0.68	1.06
Computerized files on-site						0	0.2	0	−0.39	0.4	0.79
Communications w/field						0.61	0.7	0.05	−0.76	1.97	2.73
Adjusted R²	0					−.004
F	1.03					0.73

p ≤ .05. **p ≤ .01.***p ≤ .001.

Confidence intervals are also reported below because they reflect how much the population parameters could deviate from the value in the presence of a null hypothesis—it provides an idea of how close the population differences are to zero. Loftus (1996) argues that confidence intervals provide the ability to accept the null hypothesis because they indicate whether some of the slopes of some of the respondents are likely to be meaningful.

“Communications with the Field” had an upper bound confidence interval of roughly seven with “murder clearance rates” and approximately six with “rape clearance rates.” These findings suggest that, based on a 95% confidence interval, the true mean falls below a 7% increase in clearance rates. None of the upper bound confidence intervals in any of the other relationships exceeded 3%.

The narrow intervals, and the minimal upper bound confidence intervals, provide evidence to reject the occurrence of a Type 1 error (incorrect rejection of a true null hypothesis). In other words, the evidence for no statistical significance between computerized data, analysis, and communications and crime clearance rates is fairly robust. Where null results can be a function of a number of research design and measurement problems (i.e., statistical power, unreliable measures, measures that lack construct validity, confounding variables that mask relationships, etc.), the research measures, the research design, and the size of the population suggests that the only appropriate statistical conclusion is that there is no evidence to suggest that the null could be false.

Given the lack of findings pertaining to crime clearance rates, one last hierarchical regression was obtained to test whether computerization improved the department’s analytical capacity. Per Table 6, two of the variables achieved statistical significance: “on-site files” and “communications with the field” (p ≤ .001). Despite the significance, the size of the effect is fairly small (upper bound confidence intervals of 0.22 and 1.82, respectively).

Table 6.

Summary of Hierarchical Regression Analysis for Variables Predicting Number of Changes in Analytical Functions (n = 676).

	B	SE	β	95% confidence interval for B		B	SE	β	95% confidence interval for B		Interval distance
	B	SE	β	Lower bound	Upper bound	B	SE	β	Lower bound	Upper bound	Interval distance
(Constant)	0.96	0.37		0.24	1.68	−0.43	0.29		−0.99	0.14
Unemployment	−0.15	0.12	−0.05	−0.38	0.09	−0.07	0.09	−0.02	−0.25	0.11	0.36
Poverty rate	−0.04	0.07	−0.03	−0.19	0.1	0.01	0.06	0	−0.1	0.12	0.22
Income	0	0	0.02	0	0	0	0	0.03	0	0	0
Number of officers	0	0	−0.04	−0.01	0	0	0	−0.05	−0.01	0	0.01
Budget	0	0	0.05	0	0	0	0	0.02	0	0	0
Computerized files infield						−0.03	0.04	−0.03	−0.11	0.04	0.15
Computerized files on-site						0.17	0.03	0.16***	0.11	0.22	0.11
Communications w/field						1.67	0.08	0.62***	1.51	1.82	0.31
Adjusted R²	.009					.42
F	1.36					67.59***

p ≤ .05. **p ≤ .01.***p ≤ .001.

Finally, Table 7 provides the percent clearance rates for each crime type. Average clearance rates appear less than noteworthy. Paired sample t-tests showed little improvement over CY2003 (see Table 8). With the exception of clearance rates for rape (−2.7%) and assault (−1.4%), which showed a reduction in clearance rates, the remaining crime types failed to achieve a statistical difference between the two time periods.

Table 7.

Percent Clearance Rates of Major Crimes (2007).

	N	M	SD
CY2007 Murder	493	59.36	41.38
CY2007 Rape	646	35.90	29.78
CY2007 Robbery	665	28.62	18.87
CY2007 Assault	693	49.67	23.06
CY2007 Burglary	691	13.40	10.37
CY2007 Larceny	693	18.57	10.24
CY2007 Vehicle theft	688	14.98	15.16

Table 8.

Paired Differences for Crime Clearance Rates.

	M	SD	SE M	95% confidence interval of the difference		t	df	Significance (2-tailed)
	M	SD	SE M	Lower bound	Upper bound	t	df	Significance (2-tailed)
CY2007 Murder–CY2003 Murder	1.58	43.49	2.11	−2.57	5.73	0.75	423	.46
CY2007 Rape–CY2003 Rape	−2.70	32.20	1.31	−5.28	−0.12	−2.05	600	.04
CY2007 Robbery–CY2003 Robbery	0.23	18.55	0.74	−1.21	1.68	0.32	634	.75
CY2007 Assault–CY2003 Assault	−1.43	17.49	0.68	−2.76	−0.10	−2.11	668	.04
CY2007 Burglary–CY2003 Burglary	−0.32	11.10	0.43	−1.16	0.53	−0.74	667	.46
CY2007 Larceny–CY2003 Larceny	−0.12	9.37	0.36	−0.83	0.59	−0.33	669	.75
CY2007 Vehicle theft–CY2003 Vehicle theft	−0.54	14.68	0.57	−1.66	0.58	−0.94	661	.35

Discussion and Conclusion

In short, the findings identify that for the approximately 700 police agencies in this study, a significant amount of computerization occurred during the CY2003 to CY2007 time period. At the same time, the results also suggest that crime clearance rates (the primary measure for assessing police performance) did not demonstrate any significant improvements from the computerization effort. Moreover, there was very little change in the crime clearance rates between the pre- and post-computerization periods.

As discussed above, Brynjolfsson (1993) offers five explanations for why research has failed to identify productivity gains from computing. In this study, the redistribution and dissipation of benefits argument that Brynjolfsson offered is ruled out as a threat because it lacks relevance in the public sector. Under this scenario, due to the competitive nature of the private sector, one firm benefits at the cost of another and thus no “new wealth” can be generated. However, in the government sector, “new wealth” can be generated. Gains in service delivery, that is, improvements in the numbers of the population served, can be classified as “new wealth.” In essence, government is immune to the dissipation of benefits threat. Because the public sector lacks market competition, the problem of redistribution and dissipation of benefits is rendered moot. In contrast, given the nature of public goods, the public setting provides a rich environment for isolating productivity gains from IT because the public sector can generate “new wealth.”

The next explanation for the lack of productivity improvements is attributed to a learning curve and lag effect. However, the research design eliminates lagged effects as a possible explanation for the failure to witness productivity improvements. The study is based on the gain scores for CY2003 and CY2007. Gain scores were also tested for the CY2003 and CY2009 time points to rule out any possibility of a lagged effect. The findings were comparable with the original null findings and thus are not reported. The third explanation, intermediary affects, could act as a rival explanation, but if intermediary effects did occur one might safely expect to see these benefits demonstrated in personnel or budgetary savings. So this explanation does not seem entirely plausible. Given that there was no reduction in the number of personnel, and the operating budget actually increased for the average agency, it is safe to eliminate intermediary efficiency gains as a rival explanation.

Brynjolfsson’s two remaining explanations, mismeasurement and mismanagement, can also be eliminated. Brynjolfsson argues that ITs influence on productivity is underestimated because organizations fail to develop adequate productivity measures. In this study, both the independent and dependent variables appear to be measured with enough precision to provide good indicators of computerization changes and productivity advances. Moreover, a number of different measures were obtained on both the independent and dependent variables thereby increasing the reliability of the measures. In addition, the measures were valid measures that are accepted at-large as indicators of law enforcement performance.

With respect to mismanagement, Brynjolfsson (1993) speaks directly to the project management implementation challenges that often stymie information technology efforts. In this research, it is unlikely that the null findings can be attributed to mismanagement. Under a mismanagement scenario, one would expect to see extreme scores in the confidence intervals. Meaning that, some agencies would do well while others would not. One would expect low scores for the mismanaged efforts and high scores for the well-managed initiatives. However, these extremes were not witnessed.

This study is perhaps one of the most rigorous to date, given its objective measures and its research design. The sample witnessed significant changes in their use of IT but did not demonstrate any note worthy productivity gains. Not even a small signal of benefits was uncovered. Moreover, for this sample, Brynjolfsson’s five explanations for why productivity gains did not accrue proves limited. In sum, this study demonstrated the presence of the IT productivity paradox; and the question remains “Why do managers continue to invest in IT when benefits are not forthcoming?”

The answer to the paradox may lie in the notion of “mixed motives.” Longstanding research indicates that unlike the private sector, public agencies pursue a variety of competing goals and productivity is often sacrificed for social or political objectives. Desires for transparency, accountability, and responsibility are often balanced against desires for efficiency and productivity. Back in 1981, Feldman and March wrote a thought-provoking article on the role of information in organizations as signal and symbol. They present a number of possible explanations for behavior that appears peculiar, such as the lack of any tangible performance gains from information. Identifying reason, rationality, and intelligence as key bureaucratic values, they call attention to the symbolic significance of information posturing. They contend that information has important symbolic value that conveys a perception of competence and social efficacy. King and Kraemer (2006) agree by arguing that “managers use IT to symbolize professionalism and rationality in their management practices” (p. 4).

Building on the symbolic significance, Rindova et al. (2005) claim that reputation serves as an invaluable asset for garnering resources. Perceptions of legitimacy signal competence, quality, and prominence regardless of performance (Wang, 2010). Police agencies may feel pressure to project an image of technological advancement to reassure stakeholders that their tools are capable of resolving and warding off crime. In a culture of technological determinism, the failure to automate may bring criticism. Institutional theory suggests that the pursuit of reputational legitimacy is in fact a rational action and sees little surprise that it might supersede the desire for tangible performance improvements.

Norris (2010) argues that e-government, and IT in large, has had very little influence on productivity or performance and it is unlikely to provide any in the near future. And scholars are dismayed by utopian views of technology (Foley and Alfonso, 2009; Moon and Bretschnieder, 2002; Moynihan and Lavertu, 2012). But, if IT is sought for its legitimacy and reputational benefits, then perhaps Brynjolfsson is correct, IT is mismeasured. Under a legitimacy scenario, investment in IT would be expected, but the benefits would not accrue where one traditionally looks, in the performance metrics. If IT is pursued for its reputational advantages, rather than as a mechanism for productivity advances, achieving productivity gains would simply be an accidental by-product.

While this study is limited to policing, a narrow time period, and internal IT systems, the results are nonetheless noteworthy. This study calls attention to the need for additional research on the underlying motives, incentives, and expectations that shape IT adoption and implementation. Public organizations invest heavily in IT. And, in terms of this sample, there is little evidence to suggest that productivity gains, in the traditional sense, will occur from these investments.

Perhaps a shift in perspectives is needed. Examining the productivity paradox from multiple frames of reference may shed additional light on the role of IT in public organizations. Future research efforts may help to untangle these complicated forces and, thus, put the dilemma of the paradox to rest. The challenge for scholars is that the search for answers will not be easy. It is likely to require a variety of different methodologies and insights from a number of theoretical domains. The challenge for practitioners and citizens is reconciling what constitutes productivity. But almost certainly, given the fiscal resources that IT requires, the discussion of what constitutes productivity, and how to measure it, is clearly in order.

Footnotes

Appendix

Acknowledgements

The author acknowledges and thanks the anonymous reviewers for their valuable comments and suggestions.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biography

Mary Maureen Brown is a Professor of Public Administration at UNC Charlotte. Her past experiences include visiting scientist at the Software Engineering Institute at Carnegie Mellon University. Her research interests focus on information technology and its implications on public sector organizations.

References

Attewell

Rule

(1984). Computing and organizations: What we know and what we don’t know. Communications of the ACM, 27, 1184-1192.

Brynjolfsson

(1993). The productivity paradox of information technology. Communications of the ACM, 36(12), 67-77.

Dawes

(2008). The evolution and continuing challenges of e-governance. Public Administration Review, 68(Suppl. S1) 86-102.

Dunleavy

Margetts

Bastow

Tinkler

(2006). New public management is dead—Long live digital-era governance. Journal of Public Administration Research and Theory, 16, 467-494.

Feldman

M. S.

March

J. G.

(1981). Information in Organizations as Signal and Symbol. Administrative Science Quarterly, 26, 171-186.

Foley

Alfonso

(2009). E-government and the transformation agenda. Public Administration, 87, 371-396.

Garicano

Heaton

(2010). Information technology, organization, and productivity in the public sector: Evidence from police departments. Journal of Labor Economics, 28, 167-201.

Garson

G. D.

(2006). Public Information Technology and E-Governance: Managing the Virtual State. Burlington, MA: Jones & Bartlett Learning.

Goldfinch

(2007). Pessimism, computer failure, and information systems development in the public sector. Public Administration Review, 67, 917-929.

10.

Halachmi

Bouckaert

(1996). Organizational performance and measurement in the public sector: Toward service, effort and accomplishment reporting. Westport, CT: Greenwood Publishing.

11.

(2002). Reinventing local governments and the e-government initiative. Public Administration Review, 62, 434-444.

12.

Karr-Wisniewski

(2010). When more is too much: Operationalizing technology overload and exploring its impact on knowledge worker productivity. Computers in Human Behavior, 26, 1061-1072.

13.

King

J. L.

Kraemer

K. L.

(2006). Information technology and administrative reform: Will e-government be different? International Journal of Electronic Government Research, 2(1), 1-20.

14.

Lee

Perry

(2002). Are computers boosting productivity? A test of paradox in state government. Journal of Public Administration Research and Theory, 12, 77-102.

15.

Loftus

G. R.

(1996). Psychology will be a much better science when we change the way we analyze data. Current Directions in Psychological Science, 5, 161-171.

16.

Moon

Bretschneider

(2002). Does the perception of red tape constrain it innovativeness in organizations? Unexpected results from a simultaneous equation model and implications. Journal of Public Administration Research and Theory, 12, 273-291.

17.

Morgeson

VanAmburg

Mithas

(2010). Misplaced trust? Exploring the structure of the e-government-citizen trust relationship. Journal of Public Administration Research and Theory, 21, 257-283.

18.

Moynihan

Lavertu

(2012). Cognitive biases in governing technology preferences in election administration. Public Administration Review, 72, 68-77.

19.

Norris

D. F.

(2010). E-government 2020: Plus ça change, plus c’est la meme chose. Public Administration Review, 70(Suppl. S1), s180-s181.

20.

Norris

D. F.

Moon

(2005). Advancing e-government at the grass roots: Tortoise or Hare? Public Administration Review, 65, 1, 64-75.

21.

Nunziata

(2010). Government IT spending: $61.5 Billion by 2015. Retrieved from http://www.cioinsight.com/c/a/Research/Government-IT-Spending-615-Billion-by-2015-465251/

22.

Pinsonneault

Kraemer

K. L.

(2002). Information technology and middle management downsizing: A tale of two cities. Organization Science, 13, 191-208.

23.

Rindova

V. P.

Williamson

I. O.

Petkova

A. P.

Server

J. M.

(2005). Being Good or Being Known: An Empirical Examination of the Dimensions, Antecedents and Consequences of Organizational Reputation. Academy of Management Journal, 48, 1033–1049.

24.

Rogosa

(1988). Myths about longitudinal research. In Schaie

K. W.

Campbell

R. T.

Meredith

W. M.

Rawlings

S. C.

(Eds.), Methodological issues in aging research (pp. 171-209). New York, NY: Springer.

25.

Solow

(1987, July 12). “We’d better watch out.” New York Times Book Review, p. 36.

26.

Thatcher

Brower

Mason

(2006). Organizational fields and the diffusion of information technologies within and across the nonprofit and public sectors. American Review of Public Administration, 36, 437-454.

27.

Wang

(2010). Chasing the hottest IT: Effects of information technology fashion on organizations. MIS Quarterly, 34, 63-85.

28.

Welch

Pandey

(2006). E-government and bureaucracy: Toward a better understanding of intranet implementation and its effect on red tape. Journal of Public Administration Research and Theory, 17, 379-404.