National Data on Citizen Complaints About Police Use of Force

Abstract

National data on citizen complaints about police use of force were collected by the Bureau of Justice Statistics (BJS) in 2003 and 2007. These data are a critical component of the Department of Justice’s (DOJ) overall response to 42 USC 14142, which requires the Attorney General to “acquire data about the use of excessive force by law enforcement officers.” The BJS data have the potential to help support democratic policing, provide baseline data on use of force for comparative statistical reporting and research purposes, and enable strong local checks on police abuses, provided their validity and reliability can be demonstrated. This study sought to assess the validity and reliability of the BJS data. Findings indicate that the BJS data suffer from serious measurement flaws, do not provide a valid and reliable basis for comparative statistical reporting and research purposes, and should not be relied on for purposes of litigation.

Keywords

citizen complaints use of force police data quality pattern or practice

“ . . . pattern or practice, whatever that means . . . ”

—Jonathan Smith, Chief of the Special Litigation Section, U.S. Department of Justice (DOJ), giving opening remarks at the National Association for Civilian Oversight of Law Enforcement (NACOLE)/Seattle University Academic Symposium, February 6, 2015 (emphasis added).

To truly engage in “democratic policing,” citizens must examine law enforcement activity to ensure that the democratic ideals of transparency, accountability, and—perhaps most important—fundamental fairness are reflected in police behavior (Bayley, 2006; Hickman, 2009; Luna, 2000; Sklansky, 2008; Skolnick, 1999). Democratic policing is therefore best viewed as an ongoing process rather than an achievable end in itself. While the United States presently uses a broad array of social and economic indicators to gauge the overall “health” of the nation, it has a much more limited set of indicators concerning the behavior of the police and the quality of law enforcement. We should seek to develop not only evidence that our police are enforcing the law while obeying the law but also basic indicators about the quality of police–citizen interactions, the potential role of extralegal factors in policing, and particularly the use of force given its centrality to the police role (Bittner, 1970). National data on public perceptions of police behavior and legitimacy (Tyler, 2002) are also essential to understanding democratic policing.

In the wake of recent high-profile incidents (including the cases of Michael Brown in Ferguson, Missouri, and Eric Garner in Staten Island, New York), it is disturbing and even somewhat hypocritical that the nation known as the leading global advocate of democracy (and democratic policing) actually collects and reports very little of this type of information. As the late Professor and former New York Police Department (NYPD) Officer James Fyfe (2002) remarked, “ . . . we still live in a society in which the best data on police use of force come to us not from the government or from scholars, but from the Washington Post” (p. 99). Likewise, Robert Kane (2007) has noted that,

. . . it is both ironic and unacceptable that in American democratic society, the police, which function as the most visible representatives of the crime control bureaucracy, collect data on members of the public in the form of arrest and complaint reports without systematically distributing comprehensive data on their own activities that produced those crime statistics. (p. 776)

These assessments were valid when written, and remain so today. National media reaction to the incidents in Ferguson and New York included some outrage at the fact that we simply do not know how often police in the United States use deadly force on its citizens (e.g., Doyle, 2014; Lowery, 2014), and the U.S. Attorney General also recently expressed dismay over the lack of data on police use of force (Berman, 2015).

What do we know about the nature and extent of police use of force in the United States, and how do we know it? In a review of the literature conducted as part of an effort to construct an improved national estimate of police use of nonlethal force, Hickman, Piquero, and Garner (2008) found that the majority of studies producing an incident-based rate of police use of force were based on data from a single jurisdiction, and the methods were quite diverse as were the units of analysis. Somewhat unsurprisingly, across 36 studies reporting on the amount of nonlethal force used by the police, rates varied from about a tenth of 1% up to almost 32%. While recognizing the inherent value of a diversity of methodological approaches, Engel (2008) noted that the literature reflects a failure to adequately conceptualize and measure police use of force, and that current approaches to understanding force on a national level are not achieving those goals, though some may question whether precision is a worthwhile goal (e.g., see Klinger, 2008; Smith, 2008).

The lack of national data and the limited state of knowledge about police use of force is all the more disturbing considering that for the past 20 or more years, the U.S. Attorney General has been legally obligated to produce such information and has failed to make any substantial progress toward that requirement. Section 210402 of the Violent Crime Control and Law Enforcement Act of 1994, codified as 42 USC 14142, requires the Attorney General to “acquire data about the use of excessive force by law enforcement officers” and to “publish an annual summary of the data acquired under this section.” So where are these annual summaries of police use of excessive force?

In this study, we focus on a particularly important source of data bearing on democratic policing—citizen’s complaints about police use of force. While there is arguably a somewhat “loose” connection between citizen complaints about the use of force and official uses of force (particularly given that public perceptions of police use of force may be very different from how force may be viewed under agency policy and/or from the Court’s perspective), citizen complaints have long been viewed as a strong indicator for police chiefs to gauge the quality of police–citizen interactions, whether factual or not (National Advisory Commission on Criminal Justice Standards and Goals, 1973), and the International Association of Chiefs of Police (IACP) has defined officially sustained complaints about use of force as incidents of excessive force (IACP, 2001). The Bureau of Justice Statistics (BJS) collected data on citizen complaints about police use of force from police departments in two waves of their Law Enforcement Management and Administrative Statistics (LEMAS) survey program. The present study sought to address the validity and reliability of these data by examining within-agency change across the two survey waves and attempting to validate data for agencies reporting extreme changes, as well as attempting to validate data for a random sample of agencies. We begin with a review of the literature on national-level data about police use of force, and specifically citizen complaints (including sub-national data). We then turn to our study methodology, the results, and a discussion of the findings and their importance for policy and practice.

Literature Review

National-Level Data on Use of Force

The principal federal response to the use of force data collection requirement mentioned earlier (42 USC 14142), and the only systematic, national-level data collection on police–citizen interactions at present, is the Police Public Contact Survey (PPCS) administered by BJS. The PPCS is a supplement to the National Crime Victimization Survey (NCVS), and the goal is to provide national estimates of the incidence and prevalence of citizen contacts with the police. The PPCS also seeks to describe the nature of those contacts, including whether the police used or threatened the use of force, the specific actions they took, as well as any potentially provoking citizen behaviors. The PPCS has been administered every 3 years since 1999.

The 2002 BJS Survey of Inmates in Local Jails (SILJ) also provided some information about police use of force, although BJS has never analyzed or reported on these data. The SILJ is a computer-assisted personal interview conducted with a nationally representative sample of jail inmates, and covering a broad range of topics (for details, see James, 2004). Importantly, the SILJ contained items that parallel those included in the PPCS; thus, jail inmates are asked about the use or threat of force experienced at the time of their arrest. Analysis of the inmate data demonstrates that the PPCS underestimates force due to the exclusion of the recently incarcerated (the PPCS accounts for about 87% of the total force incidents derived from both sources), and that the inmate sample is more likely to experience force, a much higher level of force, and is more likely to report injury from force (Hickman et al., 2008). However, demographic characteristics are substantively similar across the two data sources: Males, Blacks, and youth are more likely to experience force.

The BJS Deaths in Custody Reporting Program (DCRP) has provided national data on arrest-related deaths. DCRP data on law enforcement homicides (both justifiable and unjustifiable) are substantively similar to those obtained by the FBI’s Supplemental Homicide Reports (SHR) although concerns over under-counting plague both data collections due to the voluntary nature of reporting, problems with classification of homicides as justifiable or non-justifiable, and other data quality issues. Specifically, 97% of law enforcement homicides involved a male subject, the average age was 33 years, more than 80% were killed by a handgun, and about 30% involved a Black subject (Mumola, 2007), similar to SHR statistics. The DCRP data indicate that about 56% of all arrest-related deaths involved a minority person (suicide is the only category of arrest-related deaths in which Whites are the majority; Mumola, 2007).

Finally, the BJS LEMAS survey, which is presently the most systematic and comprehensive source of national data on law enforcement personnel, expenditures and pay, operations, equipment, and policies and procedures, has also collected data on use of force. The LEMAS surveys have been conducted roughly every 3 years since 1987, and provide national estimates for all state and local law enforcement agencies based on a representative sample of about 3,000 agencies. In the 2003 and 2007 iterations of LEMAS, data were collected on formal citizen complaints about police use of force. The LEMAS data provide information on the volume and rate of complaints, as well as complaint dispositions. Importantly, the LEMAS platform can also be used in conjunction with other data (such as Census Bureau data) to explore issues such as minority representation and structural correlates of citizen complaints (Hickman & Piquero, 2009). The LEMAS complaints data have also found diverse application in the field, including being used to establish “baselines” for larger police departments in their annual reports, as well as in Monell litigation involving police departments. We will return to these issues later.

The LEMAS items on citizen complaints were motivated in part by Fyfe’s (2002) article on the lack of data on the use of deadly force (which was part of a larger National Academy of Sciences panel focused on BJS police data series). In consultation with Fyfe and other scholars engaged in the study of police use of force, it became clear that one viable option was to harness the LEMAS survey for these purposes. These discussions led to the development of items that were incorporated into the 2003 LEMAS survey instrument for the purpose of collecting official departmental data on citizen complaints about police use of force. These data were collected from large agencies (those with 100 or more officers) as an initial effort that could potentially lead to broader or more detailed use of force data collection. This initial foray was successful in that it provided evidence that police departments were, if nothing else, at least generally willing and able to provide this type of information.

The 2007 LEMAS data (referencing calendar year 2006 complaints data) were expanded beyond large agencies to include sampled smaller agencies. However, the complaints data were restricted to include the total received, pending, sustained, or having some other, non-specified disposition. This is unfortunate given that there is substantial interest in both the academic and practitioner communities regarding unfounded complaints in particular, but it is also recognized that omnibus surveys often have to sacrifice detail for broad scope. The 2006 complaints data were never reported on by BJS. LEMAS data were not collected as scheduled for 2010, and the 2012 collection is being processed as of this writing.

Citizen Complaints

The academic literature on citizen complaints can be organized in terms of studies about the receipt and processing of complaints (Brandl, Stroshine, & Frank, 2001; Dugan & Breda, 1991; Griswold, 1994; Hassell & Archbold, 2010; Hickman, 2006; Hickman, Piquero, & Greene, 2000; Hudson, 1970; Independent Commission on the Los Angeles Police Department, 1991; Lersch & Mieczkowski, 1996; Liederbach, Boyd, Taylor, & Kawucha, 2007; Littlejohn, 1981; Pate & Fridell, 1993; Wagner, 1980a, 1980b; Worrall, 2002), the characteristics of officers who are the subjects of complaints (Brandl et al., 2001; Hassell & Archbold, 2010; Hickman et al., 2000; Lersch & Mieczkowski, 1996, 2000; Liederbach et al., 2007; Pate & Fridell, 1993; Wagner, 1980b), and the characteristics of complainants (Hudson, 1970; Lersch & Mieczkowski, 1996; Liederbach et al., 2007; Pate & Fridell, 1993; Wagner, 1980a). There is also literature focused on the structural correlates of complaint volumes and dispositions, such as agency organizational characteristics, the presence of external review entities, agency policies and procedures, and community demographics (Brandl et al., 2001; Cao, 1999; Cao, Deng, & Barton, 2000; Griswold, 1994; Hassell & Archbold, 2010; Hickman, 2006; Hickman & Piquero, 2009; Hickman et al., 2000; Lawton, Hickman, Piquero, & Greene, 2001; Lersch, 1998; Lersch & Mieczkowski, 1996, 2000; Shjarback, 2015; Worrall, 2002). Finally, there have been at least two national-level data collection efforts reported in the literature (Hickman, 2006; Pate & Fridell, 1993).

One of the challenges in synthesizing this literature is that police complaint processes are somewhat idiosyncratic, owing to different intake processes, unique codes of conduct, varying types and layers of decision-making processes, different degrees of internal and external review, and differing policies and procedures, some of which are negotiated with collective bargaining units. Despite historic attempts by Presidential Commissions, the IACP, and others during the 1970s to encourage some degree of standardization and consistency as part of the process of formalizing police complaints and disciplinary systems, much of the idiosyncrasy that remains is tied to the individual department histories and traditions. Nevertheless, the available literature on complaint processing indicates that, in general:

A relatively small proportion of officers are responsible for a large proportion of complaints, but this has generally been linked to arrest activity and other measures of productivity;

On-view incidents and more “proactive” policing assignments may be more likely to lead to complaints than dispatched calls;

Minority citizens complain in numbers disproportionately greater than their representation in the population served, and this is particularly true for Black citizens;

Minority officers are disproportionately the subject of citizen complaints, as well as internal (police-initiated) complaints;

There is wide variation in sustain rates across agencies, some of which is attributable to structural differences, but also to idiosyncrasies in the processing of complaints;

In general, sustain rates are lower for physical force complaints as compared with other types of complaints;

Internal (police-initiated) complaints tend to have higher sustain rates than citizen complaints.

The idiosyncrasies in complaint processing noted above should give rise to concern over national-level data collection efforts. This is not to say that such efforts should not be undertaken; rather, a careful program of research and development is needed to understand whether the data can be collected, and with what degree of validity and reliability.

Probably the first national-level study of citizen complaints was conducted by Pate and Fridell (1993) for the Police Foundation. Their study focused on police use of force broadly (including agency policies and procedures, training, official reports, citizen complaints, and so forth), and collected data via an establishment survey of 1,111 agencies. A total of 840 agencies provided data on citizen complaints about police use of force. Collectively, they reported 15,608 complaints during 1991, with rates of complaint varying by size and type of agency. The study also examined the demographic characteristics of complaints, officers, and the populations served.

The 2003 LEMAS complaints data are referenced to calendar year 2002, and provide interesting data for researchers and for police departments. For example, large municipal police departments received a total of 22,238 citizen complaints about police use of force during 2002, corresponding to overall rates of 45 force complaints per agency, 9.5 per 100 full-time sworn officers, and 2.4 per 10,000 residents (Hickman, 2006). Eight percent of these complaints were officially sustained, meaning there was sufficient evidence to justify disciplinary action against the subject officer(s). Other dispositions included not sustained (37%), meaning that there was insufficient evidence to prove the allegation; unfounded (25%), meaning that the complaint was not based on facts, or the reported incident did not occur; exonerated (21%), meaning that the incident occurred, but the officer’s action was deemed lawful and proper; and 9% had some other disposition, such as the complaint being withdrawn. As previously mentioned, BJS collected a second wave in 2007 (referenced to calendar year 2006) but never reported on them.

Validity Concerns

LEMAS suffers not only from the common problems of self-administered questionnaires (such as memory, socially desirable responding, fatigue, indifference) but also from those problems relatively unique to establishment surveys, such as inadequate record keeping or the “force fitting” of agency records to match the criteria of survey items, and error attributable to an individual’s response to less quantitative items on behalf of the organization. A commonly expressed concern about the LEMAS program is that BJS could invest more in validation research.

Walker and Katz (1995) examined bias crime units in 16 municipal police departments in the central region of the United States. These 16 agencies were selected in part because they had reported having a bias crime unit on the 1990 LEMAS survey. Walker & Katz conducted telephone interviews with the officers in charge of the bias crime units in those agencies. However, they found that only four of these agencies actually had such a unit, six other agencies had designated officers in other units to handle the bias crime function as needed, and the remainder had no unit, designated officers, or special procedures for bias crimes. Others have raised concerns about the agency and employee counts (Maguire, Snipes, Uchida, & Townsend, 1998; Uchida & King, 2002). Maguire et al. (1998), for example, noted discrepancies between the number of agencies and officers enumerated across the Uniform Crime Reports (UCR), LEMAS, and data collected by the Office of Community-Oriented Policing Services, suggesting that LEMAS substantially undercounted agencies and officers in the mid-1990s. Langworthy (2002) raised concerns about the reliability of data items pertaining to the number of patrol beats, as well as low item response rates for calls for service data.

BJS has largely relied on the research community for assessments of validity and reliability, and BJS is generally responsive to this research. For example, in direct response to Walker and Katz (1995), BJS modified LEMAS survey items concerning special units to incorporate expanded response options that did not force an agency into an “either-or” situation. The present study is in the same spirit and tradition of reliance on the research community for assistance in establishing data quality. The next section discusses the study methodology, and is followed by the results and a discussion of the findings.

Method

The study reported here included three phases: (a) merging the 2002 and 2006 complaints data (from the 2003 and 2007 waves of LEMAS data) for large municipal police departments and conducting basic descriptive analyses of those data, (b) assessing the reliability of the complaints data collection by conducting a within-agency comparison of the 2 years of data, and (c) assessment of the validity of the data using a subsample of agencies for external validation.

The merging process was complicated by the fact that the agency identification fields designed by the Census Bureau and in use for several prior waves of the LEMAS survey were not used by the data collection agent in the 2007 LEMAS survey. However, data for large municipal police agencies (i.e., self-representing agencies) were successfully merged at the agency level. The study was necessarily limited to large agencies because the 2002 complaints data were collected only for these agencies, whereas the 2006 data were expanded into the LEMAS sample of non-self-representing agencies.

Following the merging of the two waves of data, the within-agency changes across the 2 time points were calculated. We used a distributional rationale for identifying relatively large changes from 2002 to 2006 in both the volume of complaints and sustained complaints, as well as rates (per 100 officers). Where very large changes were observed (i.e., above the 90th percentile increase or below the 10th percentile decrease), we attempted to explain those changes using publically available reports (such as police department annual reports to the public), any publically accessible records and/or data, as well as direct contacts with agency personnel via telephone, mail, or electronic inquiry. This approach primarily addresses reliability (the within-agency distributional difference across the two waves serves as a reasonable indicator) but also addresses validity in the assessment of the underlying reasons for substantial changes.

In the third phase, a simple random sample of 100 agencies was selected for external validation of complaints data using publically available reports, records, and/or data, as well as direct contacts with agency personnel via telephone, mail, or electronic inquiry. A key outcome of interest is the proportion of sampled agencies for which complaints data reported to LEMAS corresponded with the external criterion. This can be considered both in exact terms as well as within a subjectively acceptable range, allowing that some minor differences may be expected if an agency had a large number of cases pending disposition at the time they reported data to LEMAS, and/or a significant lag between that point in time and the time that external measures such as annual reports were generated.

Results

Descriptive Analysis of Merged Waves of LEMAS Data

Because BJS never reported on the 2006 complaints data, and we are unaware of any other published analyses of these data, we begin by providing descriptive statistics for the merged data. Although there were 496 agencies reporting complaints data in 2002, when the waves of LEMAS were merged, we found that a total of 417 agencies had reported complaints data for both 2002 and 2006. Among these 417 agencies, there was an overall increase in the total number of complaints about police use of force from 17,300 complaints in 2002, to 20,330 complaints in 2006 (Table 1). There were also increases in the overall rate of force complaints per 100 officers, per 100 officers responding to calls for service, and per 10,000 residents served.

Table 1.

Citizen Complaints About Police Use of Force, 2002 and 2006, by Number of Full-Time Sworn (Agencies Reporting in Both Waves Only).

2002
Full-time sworn	Number of agencies	Number of FTS	FTS responding to CFS	Population 2003	Total complaints	Complaints per agency	Per 100 FTS	Per 100 FTS CFS	Per 10,000 pop
Total	417	197,671	119,790	79,677,197	17,300	41	8.8	14.4	2.17
1,000 or more	32	109,255	63,632	34,622,295	11,424	357	10.5	18.0	3.30
500-999	30	20,816	12,761	9,372,281	1,912	64	9.2	15.0	2.04
250-499	76	25,783	16,663	12,353,960	1,930	25	7.5	11.6	1.56
100-249	279	41,817	26,734	23,328,661	2,034	7	4.9	7.6	0.87
2006
Full-time sworn	Number of agencies	Number of FTS	FTS responding to CFS	Population 2007	Total complaints	Complaints per agency	Per 100 FTS	Per 100 FTS CFS	Per 10,000 pop
Total	417	200,457	119,525	81,936,925	20,330	49	10.1	17.0	2.48
1,000 or more	32	108,580	62,457	35,393,317	12,698	397	11.7	20.3	3.59
500-999	30	21,505	12,348	9,644,348	2,351	78	10.9	19.0	2.44
250-499	76	26,853	17,198	12,839,674	2,116	28	7.9	12.3	1.65
100-249	279	43,519	27,522	24,059,586	3,165	11	7.3	11.5	1.32

Note. Full-time sworn categories for 2006 are based on 2003 LEMAS designation. FTS = full-time sworn; CFS = calls for service; LEMAS = Law Enforcement Management and Administrative Statistics.

Overall, the change in the number of use of force complaints from 2002 to 2006 (3,030) amounts to a 17.5% increase, ranging from a 9.6% increase among agencies having 250 to 499 officers, up to a 55.6% increase among agencies having 100 to 249 officers (Table 2). While there was an overall increase in the number of full-time sworn officers during the same time period, there was a slight decrease in the number of officers responding to calls for service. Populations served increased 2.8% overall. The corresponding rates of complaint on an officer and resident basis increased overall among agencies in all size categories.

Table 2.

Percentage Change in Personnel, Population Served, and Numbers and Rates of Use of Force Complaints, 2002-2006.

Full-time sworn	Number of FTS	FTS responding to CFS	Population	Total complaints	Complaints per agency	Per 100 FTS	Per 100 FTS CFS	Per 10,000 pop
Total (%)	1.4	−0.2	2.8	17.5	17.5	15.9	17.8	14.3
1,000 or more	−0.6	−1.8	2.2	11.2	11.2	11.8	13.2	8.7
500-999	3.3	−3.2	2.9	23.0	23.0	19.0	27.1	19.5
250-499	4.2	3.2	3.9	9.6	9.6	5.3	6.2	5.5
100-249	4.1	2.9	3.1	55.6	55.6	49.5	51.1	50.9

Note. FTS = full-time sworn; CFS = calls for service.

We also examined complaints that were officially sustained. Among the 417 agencies, there was an overall increase in the total number of sustained complaints from 1,240 complaints in 2002, to 1,863 complaints in 2006 (Table 3). Two agencies did not report sustained complaints in 2006. Overall, the change in the number of sustained use of force complaints from 2002 to 2006 amounts to a 50.2% increase, with wide variation from 5.8% among agencies having 250 to 499 officers, up to 171.4% among agencies having 100 to 249 officers.

Table 3.

Sustained Complaints, 2002 and 2006—Agencies Reporting 2006 Data Only.

2002
Full-time sworn	Total sustained complaints	Complaints per agency	Per 100 FTS	Per 100 FTS CFS	Per 10,000 pop
Total	1,240	3	0.6	1.0	0.16
1,000 or more	612	19	0.6	1.0	0.18
500-999	193	6	0.9	1.5	0.21
250-499	274	4	1.1	1.6	0.22
100-249	161	1	0.4	0.6	0.07
2006
Full-time sworn	Total sustained complaints	Complaints per agency	Per 100 FTS	Per 100 FTS CFS	Per 10,000 pop
Total	1,863	4	0.9	1.6	0.23
1,000 or more	845	27	0.8	1.4	0.24
500-999	291	10	1.4	2.4	0.30
250-499	290	4	1.1	1.7	0.23
100-249	437	2	1.0	1.6	0.18

Note. Full-time sworn categories for 2006 are based on 2003 LEMAS designation. Two agencies did not report sustained complaints in 2006. FTS = full-time sworn; CFS = calls for service; LEMAS = Law Enforcement Management and Administrative Statistics.

Complaint sustain rates (i.e., the percentage of complaints that are sustained) were also explored. We first subtracted any complaints that were still pending a final disposition from the total number of complaints in the denominator. Overall, the sustain rate increased slightly, from 7.8% in 2002 to 9.7% in 2006 (Table 4).

Table 4.

Complaint Sustain Rates, 2002 and 2006.

2002
Full-time sworn	Total complaints	Total pending	Total sustained complaints	Sustain rate
Total	17,300	1,334	1,240	7.8
1,000 or more	11,424	1,028	612	5.9
500-999	1,912	130	193	10.8
250-499	1,930	114	274	15.1
100-249	2,034	62	161	8.2
2006
Full-time sworn	Total complaints	Total pending	Total sustained complaints	Sustain rate
Total	20,330	1,054	1,863	9.7
1,000 or more	12,698	315	845	6.8
500-999	2,351	450	291	15.3
250-499	2,116	116	290	14.5
100-249	3,165	173	437	14.6

Note. Full-time sworn categories for 2006 are based on 2003 LEMAS designation. Two agencies did not report sustained or pending complaints in 2006. LEMAS = Law Enforcement Management and Administrative Statistics.

Analysis of Within-Agency Change

We next examined the distribution of within-agency changes in the raw number of complaints across the two waves. Central tendency and dispersion statistics for the raw change in complaints from 2002 to 2006, by size of agency, are provided in Table 5. As can be seen, there are some extreme changes in all size categories, ranging overall from a decline of about 1,000 complaints to an increase of more than 3,000 complaints among the largest agencies. We focused on those agencies having the most extreme changes in total complaints (i.e., changes above the 90th percentile or below the 10th percentile) within categories of agency size. As previously mentioned, we attempted to explain those changes using publically available reports, records, and/or data, as well as direct contacts with agencies. When we were able to directly verify the data reported for 2002 and 2006, we noted the verification numbers. More commonly, in telephone and/or email contacts with relevant agency personnel (e.g., internal affairs), estimates were provided based on their knowledge of current and historical complaint processing. We also noted the nature of the problem when there were discrepancies (e.g., reporting the total number of complaints rather than the subset of complaints involving the use of force). We also provide a few case examples of specific agencies that highlight some common problems encountered, as well as the impact on overall figures reported in LEMAS.

Table 5.

Descriptive Statistics for Raw Change in Total Complaints, 2002-2006, by Number of Full-Time Sworn.

Full-time sworn	Agencies	M	Median	SD	Minimum	Percentiles					Maximum
Full-time sworn	Agencies	M	Median	SD	Minimum	10	25	50	75	90	Maximum
Total	417	7	0	183	−1,012	−19	−5	0	4	28	3,213
1,000 or more	32	40	−35	636	−1,012	−217	−99	−35	9	324	3,213
500-999	30	15	−2.5	184	−295	−104	−16	−2.5	15.5	105	901
250-499	76	2	−1	54	−200	−24	−10	−1	10	38	229
100-249	279	4	0	24.5	−61	−7	−3	0	4	15	205

Agencies having 1,000 or more officers

Among the largest agencies, six had what could be characterized as extreme changes (as defined above) relative to other agencies in the same size category. Inaccurate data for these six agencies alone inflate the total number of complaints about police use of force reported in 2002 LEMAS data by an estimated 4,351 complaints, or 25% of the total 17,300 complaints for all agencies that reported in both waves of the data collection, and by 38% for agencies within the same size category. For 2006, the inflation is an estimated 4,854 complaints, or 24% of the total 20,330 reported.

The largest raw increase in complaints among this agency size group was reported by the NYPD (an increase of 3,213 complaints, or 72%, from 2002 to 2006). However, in comparing the reported figures for the NYPD in LEMAS with available data reported by the NYC Civilian Complaint Review Board (CCRB), it is apparent that the LEMAS figures do not reflect the total number of complaints about police use of force. The NYC CCRB reports 4,612 complaints in 2002, and 7,669 complaints in 2006, involving Force, Abuse of Authority, Discourtesy, and Offensive Language (FADO). The figures reported in LEMAS are 4,450 and 7,663, for 2002 and 2006, respectively, suggesting that the NYPD likely reported the total FADO complaints from the CCRB in the LEMAS survey. The NYC CCRB reports that 2,336 complaints had at least one force allegation in 2002, and this increased to 4,159 in 2006 (an increase of 1,823 complaints, or 78%). This provides a good example of the “total vs. subset problem” (providing the total number of complaints rather than the requested subset of force complaints), and it also sheds light on the potential “cases vs. allegations problem” (complaint cases vs. allegations within complaint cases).

The largest raw decrease in complaints among this agency size group was reported by the Detroit Police Department (a decrease of 1,012 complaints, or 86%, from 1,172 reported in the 2002 LEMAS to 160 reported in 2006). Communication with the Detroit Board of Police Commissioners confirms that the 1,172 figure for 2002 is approximately the total number of citizen complaints or allegations in that year (their records show 1,113 total complaints, and 1,154 total allegations in 2002, 160 of which were force allegations), while the 160 figure for 2006 is the accurate total use of force allegations in that year; thus, there were 160 force-related allegations in both years. This is an example of the “total vs. subset” problem in 2002, and the “cases vs. allegations” problem in both 2002 and 2006.

This inflation will of course influence the overall rates on an agency, officer, and population basis, as well as the rates within the agency size group. For example, within this size group for 2002, the rates of force complaint would be revised from 357 per agency to 221, from 10.5 per 100 full-time sworn to 6.5, from 18.0 per 100 full-time sworn responding to calls for service to 11.1, and from 3.30 per 10,000 population to 2.04. Within this size group for 2006, the rates of force complaint would be revised from 397 per agency to 245, from 11.7 per 100 full-time sworn to 7.2, from 20.3 per 100 full-time sworn responding to calls for service to 12.6, and from 3.59 per 10,000 population to 2.22 (see Table 6).

Table 6.

Revisions to 2002 and 2006 Complaints Data When Extreme Agencies Are Corrected (Corrected Figures in Parentheses).

2002
Full-time sworn	Number of agencies	Number of FTS	FTS responding to CFS	Population 2003	Total complaints	Complaints per agency	Per 100 FTS	Per 100 FTS CFS	Per 10,000 pop
Total	417	197,671	119,790	79,677,197	17,300 (11,681)	41 (28)	8.8 (5.9)	14.4 (9.8)	2.17 (1.47)
1,000 or more	32	109,255	63,632	34,622,295	11,424 (7,073)	357 (221)	10.5 (6.5)	18.0 (11.1)	3.30 (2.04)
500-999	30	20,816	12,761	9,372,281	1,912 (1,537)	64 (51)	9.2 (7.4)	15.0 (12.0)	2.04 (1.64)
250-499	76	25,783	16,663	12,353,960	1,930 (1,310)	25 (17)	7.5 (5.1)	11.6 (7.9)	1.56 (1.06)
100-249	279	41,817	26,734	23,328,661	2,034 (1,761)	7 (6)	4.9 (4.2)	7.6 (6.6)	0.87 (0.75)
2006
Full-time sworn	Number of agencies	Number of FTS	FTS responding to CFS	Population 2007	Total complaints	Complaints per agency	Per 100 FTS	Per 100 FTS CFS	Per 10,000 pop
Total	417	200,457	119,525	81,936,925	20,330 (12,585)	49 (30)	10.1 (6.3)	17.0 (10.5)	2.48 (1.54)
1,000 or more	32	108,580	62,457	35,393,317	12,698 (7,844)	397 (245)	11.7 (7.2)	20.3 (12.6)	3.59 (2.22)
500-999	30	21,505	12,348	9,644,348	2,351 (1,409)	78 (47)	10.9 (6.6)	19.0 (11.4)	2.44 (1.46)
250-499	76	26,853	17,198	12,839,674	2,116 (1,307)	28 (17)	7.9 (4.9)	12.3 (7.6)	1.65 (1.02)
100-249	279	43,519	27,522	24,059,586	3,165 (2,025)	11 (7)	7.3 (4.7)	11.5 (7.4)	1.32 (0.84)

Note. Full-time sworn categories for 2006 are based on 2003 LEMAS designation. FTS = full-time sworn; CFS = calls for service; LEMAS = Law Enforcement Management and Administrative Statistics.

Agencies having 500 to 999 officers

The largest raw increase in complaints among agencies of this size group was reported by the Oakland (California) Police Department (901 complaints, or an increase of 570%). In comparing the reported figures for Oakland with available data from that agency’s Internal Affairs Division, it is apparent that the 2006 LEMAS figures for Oakland do not reflect the total number of complaints about police use of force. The LEMAS data indicate 158 complaints about use of force in 2002, and 1,059 such complaints in 2006. Annual reports from Oakland’s Internal Affairs Division indicate 167 for 2003 (the earliest year for which data were available) and 247 in 2006. The larger number for 2006 likely reflects the total number of all citizen complaints received in that year (reported in annual reports as 1,008), another example of the “total vs. subset problem.”

The largest raw decrease in complaints in this agency size group was reported by the Albuquerque Police Department (from 364 in 2002, to 69 in 2006, or a decrease of 81%). The department was unable to confirm the LEMAS data, and they have been under investigation by the DOJ civil right division, with a recent findings letter dated April 2014. A recent annual report indicates about 130 total complaints during a 6-month period. An older report by the Police Assessment Resource Center (PARC) indicates approximately 200 complaints per year from 1999 to 2001, and 21 allegations of excessive force in citizen complaints for 2000, and 48 allegations in 2001. The 364 in 2002 is likely a total number for both internal investigations and external complaints (allegations), while the 69 in 2006 is possibly the correct external allegations of use of force.

Inaccurate data for agencies reporting extreme changes in this size category inflate the total number of complaints about police use of force reported in 2002 LEMAS data by an estimated 375 complaints, and for 2006, the inflation is an estimated 942 complaints. Within this size group for 2002, the rates of force complaint would be revised from 64 per agency to 51, from 9.2 per 100 full-time sworn to 7.4, from 15.0 per 100 full-time sworn responding to calls for service to 12.0, and from 2.04 per 10,000 population to 1.64. For 2006, the rates of force complaint would be revised from 78 per agency to 47, from 10.9 per 100 full-time sworn to 6.6, from 19.0 per 100 full-time sworn responding to calls for service to 11.4, and from 2.44 per 10,000 population to 1.46 (see Table 6).

Agencies having 250 to 499 officers

The Professional Standards Bureau (PSB) for the police department in Dayton, Ohio, indicated that they had a 4-year records retention policy and that searches of paper records for 2002 and 2006 would not produce anything. A legacy computer system was able to pull some electronic data for 2006, and they reported a total of 57 citizen complaints in 2006. PSB personnel suggested that the 24 complaints about police use of force reported in LEMAS for 2006 could be accurate. With regard to the 2002 data, PSB personnel indicated that the LEMAS figure (91) was more consistent with the total officer-reported uses of force. For example, in 2006, there were 72 total officer-reported uses of force.

Inaccurate data for agencies in this size category inflate the total number of complaints about police use of force reported in 2002 LEMAS data by an estimated 620 complaints, and for 2006, the inflation is an estimated 809 complaints. Within this size group for 2002, the rates of force complaint would be revised from 25 per agency to 17, from 7.5 per 100 full-time sworn to 5.1, from 11.6 per 100 full-time sworn responding to calls for service to 7.9, and from 1.56 per 10,000 population to 1.06. For 2006, the rates of force complaint would be revised from 28 per agency to 17, from 7.9 per 100 full-time sworn to 4.9, from 12.3 per 100 full-time sworn responding to calls for service to 7.6, and from 1.65 per 10,000 population to 1.02 (see Table 6).

Agencies having 100 to 249 officers

Within this agency size group, it was much more challenging to validate data, but there were many examples of erratic reporting, such as for the police departments in Wilmington, North Carolina (18 complaints in 2002, but 223 in 2006) and Warner Robbins, Georgia (66 complaints in 2002, but just 5 in 2006). Inaccurate data for these agencies are conservatively estimated to inflate the total number of complaints about police use of force reported in 2002 LEMAS data by 273 complaints, and for 2006, the inflation is an estimated 1,140 complaints. Within this size group for 2002, the rates of force complaint would be revised from 7 per agency to 6, from 4.9 per 100 full-time sworn to 4.2, from 7.6 per 100 full-time sworn responding to calls for service to 6.6, and from 0.87 per 10,000 population to 0.75. For 2006, the rates of force complaint would be revised from 11 per agency to 7, from 7.3 per 100 full-time sworn to 4.7, from 11.5 per 100 full-time sworn responding to calls for service to 7.4, and from 1.32 per 10,000 population to 0.84 (see Table 6).

Summary

The net effect of Inaccurate reporting on the BJS complaints data is substantial inflation in both the 2002 and 2006 waves. For 2002, the rates of force complaint would be revised from 41 per agency to 28, from 8.8 per 100 full-time sworn to 5.9, from 14.4 per 100 full-time sworn responding to calls for service to 9.8, and from 2.17 per 10,000 population to 1.47. For 2006, the rates of force complaint would be revised from 49 per agency to 30, from 10.1 per 100 full-time sworn to 6.3, from 17.0 per 100 full-time sworn responding to calls for service to 10.5, and from 2.48 per 10,000 population to 1.54 (see Table 6).

Random Sample for External Validation

Focusing only on those agencies having relatively extreme within-agency changes will of course mask more subtle inaccurate variations in reported data for other agencies. For this reason, we selected a random sample of 100 agencies for validation. Of the 100 agencies contacted, 39 were actually able to fulfill our data request. Twenty-seven agencies reported that they purged their records due to record retention laws, or citizen complaints were not recorded during the years investigated. Among the agencies that reported they purged their records, there was little consistency in the length of time records were accessible (the shortest was 3 years). Despite extensive follow-up, 31 agencies were unable to fulfill our request during the study period because of limited resources and/or backlogs of public data requests, or agencies were simply non-responsive to correspondence attempted by electronic inquiry, mail, or telephone contact. Other challenges included restrictive rules on public data requests, costs associated with records retrieval, and organizational complexities (e.g., agencies integrated with other departments such as corrections, firefighters, and medical responders). Finally, 3 of the 100 agencies explicitly refused to participate in the study.

Among agencies able to provide data, the data received only rarely matched the data reported in LEMAS. In fact, only 3 of the 39 agencies reported complaints data that matched LEMAS data for both years; another 10 reported data that matched 1 of the 2 years. Sixteen of the remaining 26 agencies reported data that were either 50% higher or lower than the data reported in LEMAS, ranging from an under-count of 89 to an over-count of 82. The net effect was inflation in both years (119 complaints in 2002, or 19% of the total complaints reported by these agencies; and 64 complaints in 2006, or 10% of the total reported by these agencies).

Discussion

The problems encountered in this study of citizen complaints point to serious measurement concerns. Agencies in 2002 were asked to “Enter the current dispositions for all formal citizen complaints received during 2002 regarding use of force” and were provided with response blanks to report the total number of complaints, as well as those that were unfounded, exonerated, not sustained, sustained, pending, or had some other disposition. In 2006, agencies were asked about complaints using the exact same wording; however, the response categories were limited to total, sustained, pending, or other disposition. Clearly, the item wording is not sufficiently precise, or is not being interpreted and used by respondents as intended, to yield valid and reliable measurement. The types of problems we encountered are listed below, with some agencies having one or more of the listed problems, in one or both years of reported data:

Reporting total citizen complaints rather than the subset involving use of force;

Combining citizen complaints with internal complaints;

Combining or replacing citizen complaints with officer-reported uses of force;

Reporting total force allegations within complaint cases, rather than cases involving any force allegation;

Reporting total complaints investigated, rather than complaints received; and

Undetected data entry error (either agency or data collection agent).

In sum, the findings of this study indicate that the LEMAS data on citizen complaints about police use of force suffer from serious, if not fatal, measurement flaws. The LEMAS data on citizen complaints do not provide a valid and reliable basis for comparative statistical reporting and research purposes, and should not be used for these purposes. The LEMAS data also should not be relied on for purposes of litigation involving the conduct of law enforcement agencies.

Recent Monell litigation points to increasing reliance on statistical evidence of police organizational wrongdoing (Futterman, Mather, & Miles, 2008). Plaintiffs must show that they suffered a deprivation of their Constitutional rights, that it was caused by some policy or custom of the city to act or fail to act, and the action or failure to act was a result of “deliberate indifference” to the plaintiff’s rights. Increasingly, lawyers are turning to statistical arguments in support of a policy or custom and/or in support of arguments about deliberate indifference. The City of Chicago recently experienced a wave of litigation in this style using the BJS complaints data, where it was argued that as their complaint sustain rate was lower than the baseline in the BJS data, the City was not adequately policing itself, and this contributed to an environment in which officers acted with impunity. The present study shows that the BJS data do not provide a valid and reliable basis for this kind of comparative analysis.

With the passage of the 1994 Violent Crime Control and Law Enforcement Act, the U.S. DOJ was granted authority to pursue civil litigation against police departments that display a “pattern or practice” of Constitutional violations, including use of excessive force (42 USC 14141). This is an important and powerful mechanism, and a necessary check on police behavior that helps ensure that the police are held accountable to the public. But there is no clear context for these investigations, and it remains unclear what constitutes a pattern or practice of use of excessive force. As previously noted, the same Act legally required the Attorney General to collect and publish annual summaries of data on use of excessive force (42 USC 14142). While we could find no evidence bearing on legislative intent, perhaps the intent of this pair of statutes was to enable the rational investigation of police use of force, as the data collected under USC 14142 could provide an empirical basis for pattern or practice investigations carried out under section USC 14141.

In the findings letter issued following their recent investigation of the Albuquerque Police Department, the Civil Rights Division opines that the law does not require statistical evidence or a specific number of incidents or acts to find a pattern or practice of use of excessive force. Indeed, they have maintained this position for several years. Yet, they noted in their investigation in Albuquerque that fatal police shootings were statistically high for a city of that size. Although the Civil Rights Division holds that such evidence is not required, they often rely on it in establishing a pattern or practice. Returning to the very telling quote from the Chief of the Special Litigation Section (at the beginning of this article), which has the responsibility for enforcement under section 14141, it remains unclear what constitutes a pattern or practice of use of excessive force. At present, a pattern or practice is whatever the DOJ says it is.

In the roughly 20 years since the passage of the 1994 Crime Act, there has been little significant progress toward the annual reporting requirement. Absent substantial political will and resources, it may be that the Federal government will simply not be the leader on national data collection regarding police use of force. A strict reading of 42 USC 14142 reveals no requirement that the Federal government take the lead role, or any particular role, in the design or execution of primary data collection on police use of excessive force. The U.S. Attorney General could satisfy the requirement by compiling and reporting on various government-sponsored and non-government-initiated efforts. A process for assessing and reporting on data quality would likely be necessary if DOJ were to serve as a clearinghouse for these data collection efforts, because even with disclaimers, DOJ may be seen as extending its imprimatur to these efforts.

Perhaps privately funded, citizen-led, or other grassroots efforts will lead the way on national data collection. Media aggregation seems to be a promising area. For example, the Cato Institute’s National Police Misconduct Statistics and Reporting Project (NPMSRP; www.policemisconduct.net) is based on media reports of police misconduct incidents. In their latest annual statistical report (2010), they reported tracking 4,861 incidents involving 6,613 officers; about one quarter of the officers were involved in excessive force reports. The NPMSRP misconduct data were recently used as the dependent variable in an agency-level analysis reported by Eitle, D’Alessio, and Stolzenberg (2014).

A related endeavor has been undertaken by policing scholar Phillip Stinson, who developed a system for harvesting news reports of police officers who have been arrested. These data have enabled him to study such diverse topics as police sexual misconduct (Stinson, Liederbach, Brewer, & Mathna, 2014), drunk driving (Stinson, Liederbach, Brewer, & Todak, 2013), drug-related corruption (Stinson, Liederbach, Brewer, Schmalzried, et al., 2013), and most recently, violence (Stinson, Brewer, & Bridges, 2015).

Another media-driven effort is “Killed by Police” (www.killedbypolice.net), which aggregates “corporate news reports of people killed by nonmilitary law enforcement officers, whether in the line of duty or not, and regardless of reason or method.”

A survey-based national-level data collection, the National Police Research Platform project (www.nationalpoliceresearch.org), is led by Dennis Rosenbaum, policing scholar and the present Monitor for the DOJ settlement agreement in Portland, Oregon. As of this writing, the project has recruited some 130 agencies across the country to participate in data collection and relies on a consortium of scholars to help direct and report on findings. A Police–Community Interaction (PCI) survey has recently been field-tested, which tries to capture procedural justice behaviors, as well as conversational etiquette, emotional control, use of force, helpfulness, task competence, and empathy. Perhaps the national platform project will produce some information about use of force in the near future.

The UCLA Center for Policing Equity (cpe.psych.ucla.edu) is a research consortium focused on addressing issues of racial and gender equity in a variety of areas in policing, including the use of force, by supporting research collaborations between scholars and police agencies. The Center serves as a catalyst for change and will hopefully lead to more widespread data collection efforts by police departments.

Although the law surrounding videotaping of the police remains quite controversial in some jurisdictions, smart phone applications for recording video and audio of police–citizen interactions are becoming increasingly available. The Civil Liberties Unions of New York, New Jersey, and California have phone applications that are freely distributed. The New York version is called “Stop and Frisk Watch” and allows the user to record interactions, as well as to monitor the use of the application by others nearby, and to submit the video with survey data to the New York Civil Liberties Union. Another application that has received quite a bit of media attention, if only because it was created by three teenagers in Georgia, is “Five-O,” which in addition to video and audio allows users to rate specific officers, and has community boards for different counties to support data collection by community activists.

It seems likely that the future of national data collection on police use of force will include the widespread collection of video materials—whether official video from police body cameras and dash cams, or from citizen videos via smart phone applications or other smart devices—and content analysis of the video material. Data collected in parallel with the video material, such as user-submitted date/time, number of individuals involved, apparent race, and so on, will also be useful. Perhaps social scientists will soon rely heavily on data harvested from individual’s smart devices, either actively or passively, to study use of force incidents. To be sure, privacy concerns abound. But the time is rapidly approaching where we might learn more about an individual from his or her smart phone than we could by asking them questions or observing their behavior.

Conclusion

The results of this study suggest that much work remains to be done on collecting national data about police use of force both in general and specifically with regard to citizen’s complaints. Lest we seem overly critical of the federal efforts, it is important to also note that President Obama has empaneled a Task Force on 21st Century Policing, and that panel has already held hearings and issued an interim report. The Task Force made several recommendations concerning the collection of data on use of excessive force under 42 USC 14142, as well as the need for improved national data on the use of deadly force. The Task Force is likely to take up the need for national data on police use of force in their future meetings. For its part, BJS is planning a meeting for late 2015 that will once again bring academics, police practitioners, and others together to assist in determining what types of data should be collected and how best to proceed. It is somewhat disheartening that it takes high-profile incidents like those in Ferguson, Missouri, and New York to motivate political leadership on these issues, but it is critical to maintain the pressure and momentum if the legally mandated goal of annual national statistics on police use of excessive force is to be satisfied.

BJS has discontinued the collection of citizen complaints data in LEMAS. This is perhaps unfortunate from a research and development perspective, as the collection of these data could be improved with additional work. In the most recent iteration (2012), agencies are instead asked to report the numbers of recorded “use of force incidents” as well as “separate reports from individual officers or deputies.” These data will also need to undergo rigorous assessments of validity and reliability before they are used for comparative purposes.

Footnotes

Authors’ Note

Points of view are those of the authors and do not necessarily represent those of the U.S. Department of Justice.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by a grant from the National Institute of Justice (Award 2013-R2-CX-0035).

Author Biographies

Matthew J. Hickman, PhD is an associate professor in the Department of Criminal Justice at Seattle University. His research focuses broadly on issues in policing, quantitative research methodology, and the impact of forensic sciences on the administration of justice. He received his doctoral degree from Temple University in Philadelphia, PA.

Jane E. Poore is a graduate student at Seattle University completing her Master of Arts in Criminal Justice. Her research interests focus on issues in policing, use of force, offender reentry, and the management of individuals with mental health and substance use conditions within the criminal justice system. She received her bachelor’s degree from Seattle University in Seattle, WA.

References

Bayley

(2006). Changing the guard: Developing democratic police abroad. New York, NY: Oxford University Press.

Berman

(2015, January 15). Holder: We need better data on police shootings and officer deaths. The Washington Post. Retrieved from http://www.washingtonpost.com/news/post-nation/wp/2015/01/15/holder-we-need-better-data-on-police-shootings-and-officer-deaths/

Bittner

(1970). The functions of the police in modern society. Bethesda, MD: National Institute of Mental Health.

Brandl

Stroshine

Frank

(2001). Who are the complaint-prone officers? An examination of the relationship between police officers’ attributes, arrest activity, assignment, and citizens’ complaints about excessive force. Journal of Criminal Justice, 29, 521-529.

Cao

(1999). Curbing police brutality: What works? A reanalysis of citizen complaints at the organizational level (Final report to the National Institute of Justice, Grant No. 98-IJ-CX-0064). Retrieved from https://www.ncjrs.gov/pdffiles1/nij/grants/192518.pdf

Cao

Deng

Barton

(2000). A test of Lundman’s organizational product thesis with data on citizen complaints. Policing: An International Journal of Police Strategies & Management, 23, 356-373.

Doyle

(2014, August 20). Data on police shootings is hard to find. McClatchy DC. Retrieved from http://www.mcclatchydc.com/news/crime/article24772057.html

Dugan

Breda

(1991). Complaints about police officers: A comparison among types and agencies. Journal of Criminal Justice, 19, 165-171.

Eitle

D’Alessio

Stolzenberg

(2014). The effect of organizational and environmental factors on police misconduct. Police Quarterly, 17, 103-126.

10.

Engel

(2008). Revisiting critical issues in police use-of-force research. Criminology & Public Policy, 7, 557-561.

11.

Futterman

Mather

Miles

(2008). The use of statistical evidence to address police supervisory and disciplinary practices: The Chicago Police Department’s broken system. DePaul Journal for Social Justice, 1, 251-291.

12.

Fyfe

(2002). Too many missing cases: Holes in our knowledge about police use of force. Justice Research and Policy, 4, 87-102.

13.

Griswold

(1994). Complaints against the police: Predicting dispositions. Journal of Criminal Justice, 22, 215-221.

14.

Hassell

Archbold

(2010). Widening the scope on complaints of police misconduct. Policing: An International Journal of Police Strategies & Management, 33, 473-489.

15.

Hickman

(2006). Citizen complaints about police use of force. Washington, DC: Bureau of Justice Statistics.

16.

Hickman

(2009). Democratic policing: How would we know it if we saw it? In Rice

S. K.

White

M. D.

(Eds.), Race, ethnicity, and policing: New and essential readings (pp. 489-504). New York, NY: New York University Press.

17.

Hickman

Piquero

(2009). Organizational, administrative, and environmental correlates of complaints about police use of force: Does minority representation matter? Crime & Delinquency, 55, 3-27.

18.

Hickman

Piquero

Garner

(2008). Toward a national estimate of police use of nonlethal force. Criminology & Public Policy, 7, 563-604.

19.

Hickman

Piquero

Greene

(2000). Does community policing generate greater numbers and different types of citizen complaints than traditional policing? Police Quarterly, 3, 70-84.

20.

Hudson

(1970). Police-citizen encounters that lead to citizen complaints. Social Problems, 18, 179-193.

21.

Independent Commission on the Los Angeles Police Department. (1991). Report of the independent commission on the Los Angeles police department. Los Angeles, CA: International Creative Management.

22.

International Association of Chiefs of Police. (2001). Police Use of Force in America, 2001. Alexandria, VA: Author.

23.

James

(2004). Profile of jail inmates, 2002. Washington, DC: Bureau of Justice Statistics.

24.

Kane

(2007). Collect and release data on coercive police actions. Criminology & Public Policy, 6, 563-604.

25.

Klinger

(2008). On the importance of sound measures of forceful police actions. Criminology & Public Policy, 7, 605-617.

26.

Langworthy

(2002). LEMAS: A comparative organizational research platform. Justice Research and Policy, 4, 21-38.

27.

Lawton

Hickman

Piquero

Greene

(2001). Using GIS to analyze complaints against police: A research note. Justice Research and Policy, 3, 95-108.

28.

Lersch

(1998). Police misconduct and malpractice: A critical analysis of citizens’ complaints. Policing: An International Journal of Police Strategies & Management, 21, 80-96.

29.

Lersch

Mieczkowski

(1996). Who are the problem-prone officers? An analysis of citizen complaints. American Journal of Police, 15, 23-44.

30.

Lersch

Mieczkowski

(2000). An examination of the convergence and divergence of internal and external allegations of misconduct filed against police officers. Policing: An International Journal of Police Strategies & Management, 23, 54-68.

31.

Liederbach

Boyd

Taylor

Kawucha

(2007). Is it an inside job? An examination of internal affairs complaint investigation files and the production of nonsustained findings. Criminal Justice Policy Review, 18, 353-377.

32.

Littlejohn

(1981). The civilian police commission: A deterrent of police misconduct. University of Detroit Journal of Urban Law, 59, 5-62.

33.

Lowery

(2014, September 8). How many police shootings a year? No one knows. The Washington Post. Retrieved from http://www.washingtonpost.com/news/post-nation/wp/2014/09/08/how-many-police-shootings-a-year-no-one-knows/

34.

Luna

(2000). Transparent policing. Iowa Law Review, 85, 1107-1194.

35.

Maguire

Snipes

Uchida

Townsend

(1998). Counting cops: Estimating the number of police departments and police officers in the United States. Policing: An International Journal of Police Strategies & Management, 21, 97-120.

36.

Mumola

(2007). Arrest-related deaths in the United States, 2003-2005. Washington, DC: Bureau of Justice Statistics.

37.

National Advisory Commission on Criminal Justice Standards and Goals. (1973). Police. Washington, DC: U.S. Government Printing Office.

38.

Pate

Fridell

(1993). Police use of force: Official reports, citizen complaints, and legal consequences (Vol. 1). Washington, DC: Police Foundation.

39.

Shjarback

(2015, March 4). Emerging early intervention systems: An agency-specific pre-post comparison of formal citizen complaints of use of force. Policing: A Journal of Policy and Practice. doi:10.1093/police/pav006

40.

Sklansky

(2008). Democracy and the police. Stanford, CA: Stanford University Press.

41.

Skolnick

(1999). On democratic policing. Washington, DC: Police Foundation.

42.

Smith

(2008). Toward a national use-of-force data collection system: One small (and focused) step is better than a giant leap. Criminology & Public Policy, 7, 619-627.

43.

Stinson

Brewer

Bridges

(2015, March). Violence-related police crime arrests in the United States, 2005–2011. Presented at the annual meeting of the Academy of Criminal Justice Sciences, Orlando, FL.

44.

Stinson

Liederbach

Brewer

Mathna

(2014). Police sexual misconduct: A national scale study of arrested officers. Criminal Justice Policy Review. doi:10.1177/0887403414526231

45.

Stinson

Liederbach

Brewer

Schmalzried

Mathna

Long

(2013). A study of drug-related police corruption arrests. Policing: An International Journal of Police Strategies & Management, 36, 491-511.

46.

Stinson

Liederbach

Brewer

Todak

(2013). Drink, drive, go to jail? A study of police officers arrested for drunk driving. Journal of Crime & Justice, 37, 356-376.

47.

Tyler

(2002). A national survey for monitoring police legitimacy. Justice Research and Policy, 4, 71-86.

48.

Uchida

King

(2002). Police employee data: Elements and validity. Justice Research and Policy, 4, 11-19.

49.

Wagner

(1980a). Citizen complaints against the police: The accused officer. Journal of Police Science and Administration, 8, 373-377.

50.

Wagner

(1980b). Citizen complaints against the police: The complainant. Journal of Police Science and Administration, 8, 247-252.

51.

Walker

Katz

(1995). Less than meets the eye: Police department bias-crime units. American Journal of Police, 14, 29-48.

52.

Worrall

(2002). If you build it, they will come: Consequences of improved citizen complaint review procedures. Crime & Delinquency, 48, 355-379.