Abstract
In April of 2016, National Association for Civilian Oversight of Law Enforcement (NACOLE) and John Jay College partnered to sponsor the Academic Symposium “Building Public Trust: Generating Evidence to Enhance Police Accountability and Legitimacy.” This essay introduces the Criminal Justice Policy Review Special Issue featuring peer-reviewed, empirical research papers first presented at the Symposium. We provide context for the Symposium in relation to contemporary national discourse on police accountability and legitimacy. In addition, we review each of the papers presented at the Symposium, and provide in-depth reviews of each of the manuscripts included in the Special Issue.
Keywords
Context: Guiding Civilian Oversight Into the Public Spotlight
Following the successes of its first Annual Academic Symposium, in partnership with Seattle University (Hendrickson & Olson, 2015), the National Association for Civilian Oversight of Law Enforcement (NACOLE) approached John Jay College of Criminal Justice with the challenge—and opportunity—of expanding on the template. The follow-up Academic Symposium that took place on April 22, 2016, on John Jay’s New York City campus proceeded against the backdrop of a number of dramatic and highly publicized incidents in the field of police accountability. To name just a few of these, the release of the Ferguson report (U.S. Department of Justice, Civil Rights Division, 2015) and the contentious process that resulted in a federal consent decree (United States of America v. The City of Ferguson, 2016), the proposed shuttering of the Chicago Independent Police Review Authority (IPRA) in the wake of the court-ordered release of horrific video footage of the shooting of Laquan McDonald (Hinkel & Cohen, 2016), the public dialogue around police accountability structures and ongoing Federal Review of the San Francisco Police Department following the shooting of Mario Woods and other high-profile incidents (The Blue Ribbon Panel on Transparency, Accountability, and Fairness in Law Enforcement, 2016), and the concerted pushback by law enforcement unions and professional bodies against the Police Executive Research Forum’s “30 Guiding Principles on Use of Force” (Canterbury & Cunningham, 2016; Police Executive Research Forum, 2016).
Any one of these circumstances would represent an extraordinary level of public and official attention on issues of police legitimacy and accountability. Taken together, they highlight the status of these issues in the contemporary public imagination. In the 2016 America of the Black Lives Matter movement, a presidential campaign in which 1980s-era notions of “law and order” have become the subject of stump speeches and a topic of debate, and large city district attorneys being voted out of office for their decisions not to prosecute police involved in the killings of civilians, the need for high-quality data to guide the debate and drive the reform of policy and practice is greater than ever. The NACOLE/John Jay College Academic Symposium was designed to help meet this intense need, and the papers contained in this special issue are a continuation of that effort.
The Role of NACOLE and Its Academic Partners in Building an Evidence Base for Police Accountability
Established in 1995, NACOLE has hosted annual conferences and regional training for oversight practitioners and others interested in oversight. For oversight to continue to grow and be successful, it must be critically examined. As a result, NACOLE developed an academic symposium series signaling its commitment to research and analysis, and its interest in promoting scholarly research into oversight practices. The series is intended to bring academics and researchers together with oversight practitioners, police leaders, community representatives, and other stakeholders.
When NACOLE decided to put together its first academic symposium in February of 2015, it was in direct response to the growing interest in civilian oversight around the nation and, in particular, its expanding role in more than just complaint review and officer discipline. It reflected the organization’s interest in advancing more evidence-based research about oversight practices. For its inaugural event, NACOLE partnered with Seattle University’s (SU) School of Law and the SU Department of Criminal Justice. This symposium focused on the role of civilian oversight agencies in developing policy, influencing effective police communication strategies, advancing neighborhood safety, identifying standards of police leadership, and other subjects outside of police misconduct and discipline. Papers considered community and police perceptions of civilian oversight, data used by oversight practitioners to assess police practices, and how police body cameras may affect existing oversight practices in new or unexplored ways. The event also targeted a mix of practitioners and researchers, and concluded with a facilitated discussions centered on how to further research partnerships between the two groups.
From these discussions, a variety of issues were raised for consideration for future symposia, including the following: (a) Is it possible to create a set of shared definitions relating to oversight models and roles to help facilitate research? (b) What kind of quantitative and qualitative data will help us understand the effectiveness of civilian oversight? (c) What is NACOLE’s role in facilitating research on civilian oversight and police practices? As criminologists and oversight practitioners collaborate on research, we can better understand how civilians can most effectively contribute to constitutional policing and enhanced trust between police and the communities they serve.
The second symposium, held in partnership with John Jay College of Criminal Justice in April of 2016, focused more intently on discussing the specific application of research to the practice of civilian oversight and exploring rigorous academic research and scholarship with the potential to contribute to an evidence base and research agenda on police accountability and legitimacy. For this, its second annual academic symposium, NACOLE sought to increase its contribution to the growing evidence base of methodologically sound and peer-reviewed research in the field of civilian oversight and related matters; expose an audience of practitioners, policy experts, and advocates to this research base; and generate discussion between and among these distinct stakeholders on areas in need of further research and evaluation. The research was more closely curated for academic content and its potential for making a scientifically sound contribution to research and scholarship.
The program consisted of four panels on the topics of “Integrity in Procedural Justice,” “Examining Issues Regarding Use of Force,” “The Challenges of Implementing Body-Worn Camera Programs,” and “Accountability and Legitimacy Through Data.” A civilian law enforcement practitioner moderated each panel, leading the panelists in discussions about their research related to each topic. The panelists represented a wide range of disciplines, including academic research, law, statistics, law enforcement oversight, and state and federal policy.
The lack of abundant research into the effectiveness of different civilian oversight models and practices limits our understanding about what works and what does not in oversight, as well as how to improve its effectiveness. Effective oversight leads to more effective policing, and NACOLE is committed to supporting and highlighting the importance of research in this area. Citizen review is now found in cities and counties both large and small, and in every geographic region of the nation. And as the reach of oversight grows, it must work in tandem with academics to make sure that it is as effective as possible. The academic symposium series, which will continue to evolve as the field grows and changes, will play an important role going forward with its contributions to the growing body of research on civilian oversight.
We dedicate the remainder of this article to comprehensive reviews of the research presented at the NACOLE/John Jay College Symposium. Some of these research presentations—in general, those treated at the greatest length—are included within this special issue as finished, peer-reviewed academic papers. All of the research discussed herein, however, is available in some form—that is, presentation slides or live video recordings of the event itself—on duplicative 1 resource sites hosted by John Jay College 2 and NACOLE, 3 respectively.
Panel 1: Integrity in Procedural Justice
The term “procedural justice” has become central to the national conversation around police reform. It is a byword for presenting justice as a two-way street: The distance between the official/authoritative definition of justice, and the definition held by those who are its object, provides a key measure of police legitimacy. The perception of police as legitimate, in turn, conditions the degree of cooperation law enforcement agencies (LEAs) can expect from the communities for whose safety they are responsible: “If authorities have legitimacy they can function effectively; if they lack it it is difficult and perhaps impossible for them to regulate public behavior” (Tyler, 1990/2006, p. 57).
For LEAs, there are a multitude of reasons for undertaking efforts to enhance the delivery of the kind of procedural justice their communities desire, and in turn to enhance their own legitimacy. The most fundamental moral reason values legitimacy for its own sake. Although “justice” is the definition of a slippery concept, it is an idea with thousands of years of cultural and historical weight behind it. Plato dedicates much of The Republic to pinning down the concept, arriving eventually at a definition somewhere between “excellence of the soul” (Anderson, 1873/2001, p. 42) and “having what belongs to you and doing what is proper to you” (Anderson, 1873/2001, p. 150). Such a foundational definition remains widely shared and gives the many stakeholders in the law enforcement endeavor a strong mutually agreed upon starting point for articulating their expectations of criminal justice/law enforcement practices and processes.
A second reason for enhancing law enforcement legitimacy lies with the operational benefits described by Tyler above. Positive relationships between the police and the policed make innumerable aspects of law enforcement practice run more smoothly—to put it simply, legitimacy makes the work of law enforcement easier and the operations of law enforcement more effective. The benefits of procedural justice and legitimacy extend beyond the confines of the police–citizen encounters that make up the functional exchange or transaction of law enforcement practice. Legitimate LEAs may receive stronger political support, less turnover at the top of the chain of command, better funding, and a more collaborative legislative environment—meaning legitimacy can even have an indirect effect on the very laws that an LEA is charged with enforcing (Sunshine & Tyler, 2003).
Finally, a legitimate LEA invested in procedural justice itself provides a strong example of both the inherent virtue and the potential benefits of respect for law, justice, and fairness alike. Whether or not providing such an example to the community carries with it tangible benefits, such as a reduction in crime rates or higher community-wide quality of life, is difficult to prove through social-scientific evidence gathering and analysis. At the very least, however, internal law enforcement legitimacy provides benefit simply by preventing the detriments of its converse: Corruption or criminality within the ranks of an LEA by definition contributes directly to criminality within the community, through the crimes committed and facilitated by corrupt law enforcement personnel. Legitimacy is certainly enhanced, then, through the prevention and swift, transparent resolution of incidences of corruption.
Two papers examining the twin issues of police legitimacy and procedural justice are included herein, both of which were presented as part of the Integrity in Procedural Justice panel that kicked off the 2016 NACOLE Academic Symposium. 4 The first, Robert Worden and Sara McLean’s Measuring, Managing, and Enhancing Procedural Justice in Policing: Promise and Pitfalls, presents the results of the Finn Institute 5 study of a novel procedural justice-enhancing intervention applied in the police departments of Schenectady and Syracuse, New York, respectively. The second, Sanja Kutnjak, Maria Haberfeld, and Robert Peacock’s Decoding the Code of Silence, presents results from an 11-agency survey of 604 American police officers, using an innovative questioning tactic to provide a measure of officers’ views of different kinds of misconduct and their likelihood of reporting it.
Measuring, Managing, and Enhancing Procedural Justice in Policing: Promise and Pitfalls
In 2011, Worden and McLean began an 18-month, 36-wave survey process in Schenectady and Syracuse, questioning community members identified in police records of calls for service, stops, and arrests about their subjective perceptions of the procedural justice they experience in the encounter. The resulting 3,603 surveys provided both a baseline measure of community members’ perceptions of procedural justice and a measure of their perceptions after an intervention designed to be procedural justice-enhancing. Using the surveys themselves, the researchers distilled community members’ procedural justice “feedback” for each department’s command staff at their regular monthly CompStat meetings.
How this information was used to change procedure or influence officer behavior (i.e., the “intervention”) was left up to each department’s command staff, while also being monitored through regular interviews of both command staff and patrol officers, along with regular reviews of footage captured by patrol car dashboard cameras in Schenectady. The researchers’ intent, of course, was to provide multiple, overlapping measures of the intervention’s effects. The potential for legitimacy-enhancing effects lay with the police managers who were provided regular information regarding the perception of their officers and their work by the community. What would managers do with this information?
As it turned out, managers’ responses fell along a spectrum: Those who effectively did nothing to draw their patrol officers’ attention to the procedural justice aspects of their interactions with community members formed one end of the spectrum, and those who routinely addressed these issues occupied the other. Even at the latter, positive end of the continuum, however, “managers operationalized their mandate . . . by telling subordinates to: ‘be nice,’ ‘be polite,’ and ‘explain yourself’” (Worden & McLean, 2018, p. 160).
Exhortations by line managers to “be nice,” of course, represent a fairly shallow procedural justice intervention, a limitation that helps to explain why Worden and McLean found a lack of effect across most of their subjects and measures, with the important exception of “one of Schenectady’s three patrol platoons” (Worden & McLean, 2018, p. 166). Between the limits of managerial influence, the lack of theoretically or empirically designed training tools, and the slow pace of change in community-wide attitudes, 6 it is realistic to expect the size of any significant effect to be minute.
In the Schenectady patrol platoon that did show an effect, the researchers observe that everyone from command staff to “first-line supervisors . . . were all on board” (Worden & McLean, 2018, p. 167)—a powerful testament to the importance of a cohesive organizational culture in effecting positive change in any organization, LEAs most certainly included. Such cohesive organizational cultures are difficult to come by. Yet an argument can be made that the very paramilitary structures and cultural ethos that are often cited as factors that impede LEAs from taking on a community-oriented “guardian” role (ostensibly instead encouraging an unproductive “us vs. them” mind-set) hold the potential to underpin levels of organizational cohesion that might otherwise be out of reach to less structured social service organizations. The next paper we will consider examines one of law enforcement’s paramilitary cultural ethos—the “code of silence”—to find that the strength of this supposed tradition derives from law enforcement officers’ sense of cohesion and mutual moral standards, rather than vice versa. As we shall see, the power and potential of this cohesion are arguably one of law enforcement’s greatest untapped resources.
Decoding the Code of Silence
In 2013 and 2014, Kutnjak, Haberfeld, and Peacock “surveyed 11 U.S. police agencies, primarily located in the Midwest and on the East Coast” (Kutnjak Ivkovic, Haberfeld, & Peacock, 2018, p. 179). The sample included police agencies of a wide range of sizes, from the very small (fewer than 25 officers) to the very large (more than 500 officers), in a distribution similar to that of LEAs nationwide. With a 34% response rate, the researchers were confident in the representativeness of their sample, and with 604 total surveys, equally confident of the statistical power of their results.
Such statistical confidence, however, could easily be rendered meaningless if the researchers could not be similarly confident of the veracity of their data. Can law enforcement officers be relied upon to answer truthfully questions related to their own misconduct? Historically, this question has posed one of the greatest challenges facing research into police corruption: Officers are, after all, steeped in both the culture of the aforementioned “code of silence” and a procedural aversion to self-incrimination. No matter how many assurances of anonymity a questioner might provide, would a smart officer ever admit out loud to his own wrongdoing? Under such circumstances, how could a research team ever expect to avoid severely undercounting the incidence of misconduct?
Kutnjak and her coauthors’ solution to this problem was both simple and ingenious: They avoided demographic or other identifying questions, framed misconduct issues as hypothetical scenarios, and, rather than asking respondents to discuss the likelihood of their own participation in the scenarios described, portrayed the participants as “fellow officers” within the respondent’s department. Thus, respondents were asked directly only about their own hypothetical likelihood of reporting hypothetical misconduct, not the likelihood of their committing hypothetical misconduct.
Much of what the authors found was simply and straightforwardly heartening, such as the very low (less than 5%) proportion of respondents who stated they would not report “the most serious examples of police misconduct in the questionnaire (including both corruption and the use of excessive force)” (Kutnjak Ivkovic, Haberfeld, & Peacock, 2018, p. 185). Such results suggest that the “code of silence” rarely supersedes strongly and commonly held moral frameworks. That some of these serious forms of misconduct—in particular, the use of excessive force—are especially salient to the contemporary discourse around civilian oversight should not escape the reader’s attention. What are the implications of battle lines being drawn around conduct that most police officers are themselves quick to condemn? For oversight, this is a reality that raises questions about how better to approach rank and file officers as allies, and how to hold responsible the organizational structures—whether police unions, law enforcement or political leadership, prosecutors, or others—that might seek to defend or obscure indefensible conduct.
Equally important among Kutnjak and her coauthors’ findings was their discovery that “[t]he strongest predictor of the respondents’ unwillingness to report is their assessment that other officers in the agency would not be willing to report” (Kutnjak Ivkovic, Haberfeld, & Peacock, 2018, p. 187). Such a momentous result should give us pause, not as some cynical confirmation of the valence of peer pressure into adulthood or within the confines of paramilitary culture, but as a surprising indication of what police culture is, what it is not, and—most importantly—what it could be.
On the negative end of the spectrum, police culture does indeed appear to be a motivating force strong enough to overcome the compunctions implicit in the commonly and strongly held moral frameworks discussed above. Importantly, however, this is neither universally nor intrinsically the case. The “code of silence” is not the manifestation of some fundamental feature of policing’s history, social function, or organizational nature. Instead, it is more likely to be an ill-considered “side effect” of policing’s paramilitary structure, arising unintentionally from the camaraderie and sense of collective purpose such structures are designed to build and sustain in the face of potential life and death situations in the field.
We say “ill-considered” because paramilitary structures can be—and often are—harnessed in the service of nobler collective goals than self-preservation, mutual protection, and willful blindness to misconduct among one’s fellows. It is clear that Worden and McLean uncovered this sort of goal-focused discipline in the Schenectady Police Department, and other LEAs around the country have used it to similarly positive effect. Its potential, however, has yet to be fully explored. Civilian oversight’s future success may depend in part on how well it can be measured, understood, and harnessed in the service of the goals of community policing.
Panel 2: Examining Issues Regarding Use of Force
As we discussed above, use of force is perhaps the central issue of contention in contemporary U.S. discourse around law enforcement reform. As such, it has also become the primary focus of many of the nation’s civilian oversight bodies, where it was not already. Until recently, however, use of force by LEAs was a poorly understood phenomenon in America’s jurisdictional patchwork, even to the extent of understanding the basic scope of the problem or the universe of incidents. At the local level, the situation is often similar, with a lack of generally accepted standards about what information to record regarding use of force incidents, and how or when to communicate these incidents to the public.
Our presenters on this panel offered solutions for this confusion on both ends of the spectrum. Cynthia Barnett-Ryan of the FBI’s Crime Statistics Management Unit, and her coauthor, Shelley Hyland of the U.S. Department of Justice’s (DOJ) Bureau of Justice Statistics, presented the federal government’s plan for Improving the National Collection of Law Enforcement Use of Force Data. With a focus on the local level, Matt Hickman of Seattle University and Bob Scales of Sanford, Olson & Scales, LLC, presented Exploring the Utility of Force Factors for Community Engagement About Use of Force. Both sets of presenters, to their great credit, gave equal weight to improving methods of data collection and enhancing the end user experience. Data on police use of force that are complete, comprehensive, and communicated clearly are a prerequisite to holding LEAs, individual officers, and civilian political authorities accountable. The presentations in this panel took an important step toward fulfilling that prerequisite.
Improving the National Collection of Law Enforcement Use of Force Data
Barnett-Ryan and Hyland provided attendees at the Second Annual NACOLE Academic Symposium a detailed preview of the Bureau of Justice Statistics’ improved collection procedures for its Arrest-Related Deaths (ARD) Program. 7 The proposed procedures have since been released in the Federal Register and remained, at the time of writing, under public comment. 8 The intent of these new procedures is to capture the entire universe of arrest-related deaths in the United States on a quarterly basis and to report this comprehensive data promptly and clearly to the public.
Perhaps the most innovative feature of the effort as presented is its willingness to incorporate open-source data as a source for validation and a check on agency reporting. 9 Multiple parallel efforts to monitor killings by police have recently been undertaken in the media, including The Guardian’s Counted project 10 and the Washington Post’s police shootings database, 11 among others. Barnett-Ryan and Hyland’s willingness, along with their fellow researchers, to embrace these projects and incorporate their data will no doubt stand as a long-term strength of the revised ARD. It shows that an engaged and active fourth estate can both serve a civilian oversight function and impact government policy and practice in meaningful ways.
Exploring the Utility of Force Factors for Community Engagement About Use of Force
Hickman and Scales presented a demonstration of the proprietary Police Force Analysis System, 12 using publicly available data from the Spokane (Washington) Police Department. Along with an impressive dashboard laying out the Department’s universe of use of force incidents along multiple variables—temporal, geographic, call type, reporting officer, and so on—the presenters proposed the concept of “force factors” as a way to communicate to the public a consideration that seems all too often to be missing from much public discourse on police use of force: proportionality.
In Graham v. Connor (1989), the Supreme Court held that the Fourth Amendment’s “objective reasonableness” standard was the appropriate measure for determining whether or not a law enforcement officer’s use of force in a given incident was appropriate or excessive. Proportionality is the essential component of “objectively reasonable” force, in that the law enforcement officer is justified in using a level of force proportional to the force exerted by the suspect. Hickman and Scales propose a ranked numerical typology for use of force, from 1 at the lowest (i.e., restraint) to 4 at the highest (i.e., deadly force). By focusing on incidents where there is a clear disparity between the force applied by the officer versus that used by the suspect, civilian oversight agencies have the means to better invest limited resources, and (conceivably) better respond to public anger.
Panel 3: The Challenges of Implementing Body-Worn Camera (BWC) Programs
BWCs are widely seen as a promising technology for enhancing police accountability. They are not, however, a panacea, a foolproof solution to the problems of excessive force or a shortcut to community trust. Successful BWC initiatives incorporate training and clear usage policies, and engage the communities in which they will be used in a dialogue about what to expect—and what they can and cannot accomplish.
The organizers of the Second Annual NACOLE Academic Symposium were spoiled for choice of presentations on BWC research, so great is the interest within the academic community in their potential. The two presentations ultimately selected were fortuitously focused on the practical and theoretical aspects of implementing BWC programs: Jennifer Fratello and Matthew Buttice of the Office of the Independent Monitor for the City and County of Denver with Body-Worn Camera Implementation Challenges and Outcomes: Lessons Learned From a Pilot Study, and Maria Ponomarenko and Barry Friedman of New York University’s Project on Body-Worn Cameras and Civilian Oversight: A Case Study on Camden.
BWC Implementation Challenges and Outcomes: Lessons Learned From a Pilot Study
Like any technological innovation, BWCs are only as good as the procedures and structures set up around their use. Fratello and Buttice presented their analysis of the Denver Police Department’s Body-Worn Camera Pilot Project, 13 shedding light on the ways that the structures and procedures specific to the pilot succeeded and failed.
Of particular interest in the Denver BWC Pilot was the fact that, of the 80 use of force incidents that occurred during the rollout period, only 21 (26%) were recorded. Of the incidents not recorded, 35 (44%) were not captured because (a) supervisory personnel were not required to wear the cameras and (b) officers were not required to wear the cameras off-duty. These statistics illustrate a reality of contemporary policing that civilian oversight professionals must grapple with: In a public safety field where salaries are decidedly working class and moonlighting is common (O’Hara & Sainato, 2015), the lines between official authority and private applications of coercive force will inevitably blur. In this context, standards for off-duty behavior—including, potentially, BWC requirements for certain kinds of private security positions—are equally important for sworn officers as those they are bound by when in uniform. 14
BWCs and Civilian Oversight: A Case Study on Camden
In contrast to Fratello and Buttice’s portrayal of the pilot phase of a BWC initiative, Ponomarenko and Friedman examine a point even earlier in the implementation process—namely, the community-input period for a proposed BWC initiative of the Camden County, New Jersey Police Department.
Under the auspices of New York University’s Policing Project, 15 Ponomarenko and Friedman coordinated the public input process for the Camden BWC initiative, covering such issues as privacy, notification, scope, and public release of recordings. Camden’s community members were, perhaps unsurprisingly, generally in favor of the initiative, but also expressed a number of privacy concerns that might otherwise have received short shrift. Community members were concerned about their own appearance on video footage and requested that officers notify community members during interactions in which the camera would be recording. Similarly, community members wanted a straightforward means to access footage in which they appeared. The latter concern highlights the range of resources necessary to support various aspects of the storage, organization, and retrieval of footage—costs that might otherwise be forgotten in the rush to implement BWCs.
Panel 4: Accountability and Legitimacy Through Data
The fourth and final panel, Accountability and Legitimacy Through Data, focused on the importance of collecting and disseminating objective and measurable evidence to enhance police oversight. Recent technological advancement and increased public attention to issues of transparency have given rise to new and improved collections of data on a diverse range of topics.
Although national collection of crime data goes as far back as the creation of the Uniform Crime Report (UCR) in 1930, 16 local LEAs have increasingly used data to map crime patterns and respond to crime trends since the New York City Police Department (NYPD) popularized CompStat, their comparative crime statistics tracking mechanism, in the mid-1990s. 17 Over the past 20 years, police data collection nationwide has broadened to include more than merely arrests, summonses, and incidents. Police departments, enabled by more efficient and sophisticated technology, are collecting data on a wide array of variables, including geographically coded shooting incidents, calls for service, complaints against officers, circumstances surrounding police use of force, deaths in police custody, force used against police officers, and demographic data on offenders, victims, and law enforcement officers. Most recently, footage from BWCs has arisen as an entirely new form of data for LEAs to collect and analyze.
In some cases, LEA data collection has been shaped by executive policy initiatives. For instance, in May 2015, the White House launched the Police Data Initiative (PDI), a derivative of the President’s Task Force on 21st Century Policing. 18 The PDI involves a collective of data specialists and law enforcement practitioners covering 53 jurisdictions who work together to increase transparency and accountability through the use of police data. Some of the applications of data are internal, using metrics to improve early warning systems to identify officers who may have problematic interactions with the public, or automating tools to scour audio and video from BWCs to locate footage related to training outcomes. Other initiatives are aimed at improving transparency by creating publicly available statistics gateways like the Police Foundation’s Public Safety Open Data Portal, 19 which contains select data sets from agencies participating in the PDI.
A number of individual police forces have also been subject to mandated releases of key data sets. In Chicago, the court in the 2014 case Kalven v. Chicago 20 ordered that records of police misconduct be opened to the public, ultimately resulting in the publication of more than 50,000 misconduct complaints in the Citizens Police Data Project 21 by the Invisible Institute. Following a DOJ investigation that discovered “systemic deficiencies and practices” resulting in unreasonable police use of force in Cleveland, 22 the city entered into a consent decree requiring data on use of force justifications to be made public. Similarly, the consent decree agreed upon between the DOJ and the Ferguson Police Department in 2016 23 broadens requirements for crime, use-of-force, and court data to be made publicly available. Similar past consent decrees have resulted in open data in New Orleans, Washington, D.C., and Los Angeles. 24
Despite improvements in the release of police data to the public, there are limitations on what is currently available. As a result, some policy accountability advocates and journalists have begun to compile their own quantitative resources. For instance, because national data sets are often incomplete, the above-mentioned Guardian and Washington Post efforts have both supplemented publicly available databases with news reports and investigative reporting to estimate the number of people killed by law enforcement. The Black Lives Matter–affiliated group Mapping Police Violence 25 uses graphic data visualizations to depict statistics gathered from crowdsourced databases. These examples underscore the need for transparency in police data production. When police agencies are not mandated to share data on key metrics like police shootings, use of force, and misconduct, the resulting data picture is often only a partial look at these important issues, impairing independent oversight and accountability. More work is needed to bring police data to the public, and more advanced analysis of currently available data sets is required to make conclusions and recommendations related to some of the most discussed issues in police oversight.
The discussants of Panel 4 took on these issues, focusing on the need for improved data collection and more sophisticated analytical approaches in two key areas for law enforcement accountability and oversight: police use of force and early warning systems. These papers, both presented herein, come at the issue of police data from two very different approaches. The first, Jon Shane’s Improving Police Use of Force: A Closer Look at Data Collection, explores the historical context of gaps in understanding of police use of force as a result of incomplete data and recommends remedies to correct this issue. The second, Helsby et al.’s Early Intervention Systems: Predicting Adverse Interactions Between Police and the Public, is an attempt at improving the accuracy of early intervention systems (EISs) that identify officers who may be at risk of adverse police–public interactions via a prototype data-driven EIS using improved data sources and machine learning techniques.
Improving Police Use of Force: A Closer Look at Data Collection
Professor Jon Shane of John Jay College of Criminal Justice in New York City outlines a number of data collection and interpretation issues in his policy paper 26 on police use of force. Shane argues that the lack of a standardized national data collection effort on police use of force has resulted in incomplete and misinterpreted statistics and political rhetoric, particularly with regard to the role of suspect race in force incidents. Moreover, he notes that current data sets are incomplete, often based on voluntary sharing of idiosyncratically collected and submitted data from local LEAs, and contain only aggregate information rather than incident-based information. Incomplete data, he stresses, result in incomplete public policy options based on ill-defined problems.
Ultimately, Shane’s article is an argument for a national, incident-level use of force data that is similar to the National Incident-Based Reporting System (NIBRS). Shane maintains that incident-level data would serve as a counterweight for both police departments’ institutionalist/traditionalist conceptualizations of force incidents as being overwhelmingly justified and the public’s “emotional arguments that mirror a moral panic of sorts” in response to the same incidents (Shane, 2018, p. 131).
Shane explores a case study of what kinds of research might be done with individual-level data by using data on deaths as a result of police force compiled by the Washington Post’s “Fatal Encounters” project; estimates from Eith and Durose’s (2011) Contacts Between Police and the Public, 2008; and U.S. Census data. Shane’s findings underscore his core thesis that more context in data is needed. For example, although he finds that the rate of a fatal encounter with police differs by race—for people who are White, it is 249.2 per million, compared with 727.2 for people who are Black, 380.9 for Hispanics, and 97.9 for people who are Asian—alone, these rates do not indicate whether the level of force used was potentially appropriate in any individual situation. Furthermore, although unarmed individuals were more likely to be both shot and tasered, there is no way to know the order of these events, leaving open the possibility that an offender was first tasered, then shot. As Shane points out, this may actually indicate “proper escalation of force insofar as the offender did not respond to less-lethal options and continued to attack the officer” (Shane, 2018, p. 138).
Shane’s piece concludes with the speculation that national, incident-based use of force data could become available through the pending legislation H.R. 306, the National Statistics on Deadly Force Transparency Act of 2015, or via state-based databases funded though federal grants authorized by the also-pending Police Reporting Information, Data and Evidence Act of 2015 (S.1476). Although less than optimistic about local law enforcement’s desire to follow through with a national, or even state-level, collection of data, Shane makes a reasoned case for the need for police departments to do better with their metrics on use of force. As he argues, it benefits both the public and the police departments themselves:
Without data, police leaders do not have a mechanism to train their officers on emerging trends in assaults and risky confrontations, or strengthen training to avoid confrontations and accidents . . . [and] are denied the benefit of knowing how their police agency compares with similar police agencies on use of force incidents and how the specific circumstances and environment may have contributed to the event. (p. 4)
Early Intervention Systems: Predicting Adverse Interactions Between Police and the Public
While Shane’s article is a call for better data, Helsby et al.’s piece is a provisional attempt at providing better data, albeit surrounding a different issue entirely: EISs. 27 EISs are an attempt to identify officers at risk of “adverse events,” such as citizen complaints or excessive uses of force, for police departments to engage in risk management activities like counseling or training before more serious problems arise. Helsby and colleagues worked with the Charlotte-Mecklenburg Police Department (CMPD) in North Carolina, which has an existing EIS that employs behavioral thresholds to flag officers. Their research was initiated as part of the White House’s Police Data Initiative and is an example of a successful academic/practitioner/government partnership in police accountability.
Existing EISs, Helsby et al. argue, often calculate risk based on insufficient or oversimplified metrics that “fail to capture the complex nature of behavioral patterns and the context in which these events play out” (Helsby et al., 2018, p. 194). For instance, CMPD’s EISs failed to differentiate between officers working in high-crime and low-crime areas and times of day, and excluded factors like prior suspensions from the force. To develop a better predictive model, Helsby and colleagues used both more thoughtful, evidence-based metrics and machine learning algorithms, which learn from historical data.
EISs suffer from a number of issues that this type of algorithm corrects, Helsby et al. contend. Inconsistent use of systems, difficult-to-customize programs, the ease with which officers can game threshold-based EISs by avoiding reporting prohibited actions, and the use of binary risk flags instead of probability-based risk scores all make current EISs less effective. Machine learning models are easily retrained when given new data, and risk parameters are so complex that they are harder for officers to guess. Helsby et al. go one step further, incorporating into their algorithm a vast number of predictor variables from prior research related to officer characteristics, situational factors, and neighborhood factors. These data range from internal affairs information and employee records to records associated with dispatch events, criminal complaints, and arrests—and even include information from secondary employment (“moonlighting”) and training records.
Not only does Helsby et al.’s sophisticated modeling seem an improvement on current protocols—their model increased the true positive rate by ~12% and decreased the false negative rate by ~32% compared with the existing EIS used by CMPD—their unique approach reveals some interesting findings with regard to situational factors. For instance, the risk of adverse outcomes was increased by dispatches that were initiated by officers themselves instead of via 911 calls, an increased number of hours on duty, and indicators of a more aggressive policing style (as measured by the officer’s discretionary arrest rate).
With a more advanced EIS like the one designed by Helsby and her colleagues, police departments may be better equipped to use data to individually tailor interventions to provide officers with appropriate training and support. Moreover, if particular units or divisions are flagged with higher risk scores, group interventions like retraining may be called for. The comparative nature of risk scores based on, in part, situational factors, may prove destigmatizing on their own: Officers would no longer be deemed “problematic” or “not problematic” by EIS, but instead, relative risk can be discussed in terms of ameliorating officer stress or improving training on de-escalation techniques.
In the juxtaposition of the papers presented in Panel 4, there is both a call and an answer, with Jon Shane issuing a plea for better data surrounding use of force incidents, and Jennifer Helsby and colleagues exemplifying what can be done with better data in early intervention systems when partnering with law enforcement. Taken together, these papers are a lesson to police departments and oversight professionals alike. Enhancing transparency and professionalism of police requires an honest and open look at the best possible evidence we can measure—together.
Closing Thoughts
The papers presented at the NACOLE/John Jay College Academic Symposium, and particularly those included in this issue, present compelling cases for both the necessity of improved data collection and the potential benefits to be gained from it—for LEAs and the public alike. The models presented by this research broadly call for oversight to be treated as an activity integral to the efforts of law enforcement itself. The independence and civilian leadership of law enforcement oversight are a procedural necessity rather than a functional one, designed to ensure that oversight decision making is guided by community standards and universal ideals rather than factionally defined, narrow self-interest. By continuing to build an evidence base in police accountability and legitimacy, the criminal justice academic community can help to move the dialogue on civilian-led oversight past the drama of the contemporary moment, and into a future of rational policy and effective practice.
Footnotes
Acknowledgements
The authors would like to thank the 2016 National Association for Civilian Oversight of Law Enforcement (NACOLE)/John Jay College Academic Symposium keynote and plenary speakers, Dr. Tom Tyler (Yale) and Dr. Lorie Fridell (University of South Florida); the oversight professionals who volunteered to serve as discussants on Symposium panels, Nicholas E. Mitchell (Independent Monitor for the City and County of Denver, Colorado), Michael Gennaco (Principal, OIR Group), Alexander A. Bustamante (Inspector General, Los Angeles Police Commission), and Walter Katz (Independent Police Auditor, San Jose, California); hosts John Jay College of Criminal Justice President Jeremy Travis, Associate Provost and Dean of Research Dr. Anthony Carpi, and his staff in the Office for the Advancement of Research; Symposium Co-Chair Inspector General Philip K. Eure and his staff at the Office of the Inspector General for the New York Police Department (NYPD); and the staff, board, and past presidents of NACOLE, in particular, Cameron McElhiney, Director of Education and Training.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.