Testing for Racial Profiling With the Veil-of-Darkness Method

Introduction

Innumerable state and local police agencies collect data on the traffic stops that their officers make, as part of an effort to address public concerns about racial profiling. Analyses of those data, for the purpose of drawing inferences about the nature and source(s) of any racial disparities, confront thorny challenges. Ideally, clues about racial bias would emerge from a comparison of the characteristics of the people stopped by the police with the characteristics of the people who could have been legitimately stopped; discrepancies between the former and the latter might suggest that police stops were influenced by factors other than the behavior of the citizens involved. ¹ Unfortunately, however, satisfactorily valid information on the latter population, which represents a suitable “benchmark,” is at best difficult and costly to come by; this is the widely discussed benchmarking problem in analyses of racial bias in police stops.

The “veil-of-darkness” method, devised by Grogger and Ridgeway (2006), is an innovative and low-cost approach to resolving many of these issues. The basic idea is to use changes in natural lighting to establish a benchmark, on the assumption that after dark, police officers suffer a degraded ability to detect motorists’ race; the pattern of stops during darkness represents the presumptively more race-neutral benchmark, against which the pattern of stops during daytime can be compared. The comparison is limited to stops that occur “near the boundary of daylight and darkness,” in what has been called the “intertwilight” period, lest the analysis confound differences in officers’ decisions to stop with changes in the composition of the driving (and violator) population across the hours of the day. To our knowledge, this method has been applied in analyses of stops in only three cities: Oakland (Grogger & Ridgeway, 2006; Oakland Police Department, 2004, pp. 40-43), Cincinnati (Ridgeway, 2009), and Minneapolis (Ritter & Bael, 2009). It may be useful in many sites that seek answers to questions about racial profiling by police, and we believe that it should be considered as one—or the—approach in cities and departments that are examining patterns of vehicle stops by police. The potential utility of the method would be enhanced by the accumulation of the analytical results that its application yields, and so in this article we report the results of our application of the veil-of-darkness method to data on vehicle stops by the Syracuse (NY) police.

Tillyer, Engel, and Wooldredge (2008) observe that “[w]hile there is some consensus in the research community that residential census populations are the least reliable of the benchmarks available, there is no such consensus regarding the validity of other techniques” (p. 143). If a consensus is to form, it will likely take shape only with the execution and dissemination of individual studies like this one. No one study can by itself establish a sufficient base of knowledge to support assertions about valid (and invalid) benchmarks, and so as in the past, further advances in our knowledge about racial profiling and the analysis of racial profiling will turn mainly on the incremental accumulation of findings across individual studies.

Public concern about racial profiling by Syracuse police prompted the collection of data on stops in that city, beginning in 2001. We examine stops conducted and recorded between 2006 and 2009, inclusive. First we review the development and previous applications of the veil-of-darkness method. Then, we describe stops by Syracuse police: the temporal and spatial distributions of the stops, the assignments of the officers who made the stops, the reasons for the stops, and the characteristics of the people who are stopped. Then we present the veil-of-darkness analysis of Syracuse stops.

The Veil-of-Darkness Method

Noting the costs of many benchmarking techniques, such as observations of vehicular traffic, and their inevitable flaws (see Engel, Calnon, & Bernard, 2002, pp. 256-258; also see Engel & Calnon, 2004; Tillyer, Engel, & Cherkauskas, 2010), Grogger and Ridgeway propose a fairly simple test based primarily on a simple and plausible assumption: If police are more prone to stop African American drivers, evidence of their bias will be more pronounced among stops made in daylight, when drivers’ race can be more readily detected. They stress that the method need not presume that race is completely obscured without natural light (or that race is completely visible with it), but rather only that officers’ ability to discern drivers’ race is impaired in darkness; that is, the race neutrality of nighttime stops is relative, not absolute. This is an assumption that has intuitive appeal, and it is also consistent with the experiences of several researchers who, for the construction of benchmarks, sought to detect and record the race of drivers. As Ridgeway (2009, p. 12) reports, Greenwald (2001) abandoned plans to conduct nighttime observations of traffic on learning that the race of only 6% of drivers could be determined around dusk, and Lamberth’s (2003) traffic survey required auxiliary lighting for nighttime observations. Likewise, Lange, Johnson, and Voas (2005, pp. 201-202) used two large strobe lights in taking high-resolution photographs of vehicles on the New Jersey Turnpike.

It would be difficult to overstate the importance of valid, credible benchmarks in analyzing data on police stops for evidence of racial bias. A host of factors other than racial bias—some organizational, such as the allocation of patrol resources across police beats, and some individual—may affect the number of stops conducted by police and their distribution across social space (Engel et al., 2002; Stroshine, Alpert, & Dunham, 2008). Plausibly eliminating those other factors as explanations for racial disparities is the analytical burden borne by any analysis of stops. Moreover, any racial bias would affect officers’ behavior (consciously and deliberately or unconsciously and stereotypically) at the margin, net of (or contingent on) other factors, such as the seriousness of violations (see, for example, Fyfe, 1988; Smith, Visher, & Davidson, 1984; Worden, 1995; more generally see National Research Council, 2004, pp. 122-126); even with a perfect calibration of any racial bias, then, we would not in most cases expect it to be very large in magnitude. Against this analytical backdrop, the shortcomings of the benchmarks used in previous analyses are thrown into stark relief. Some analyses of police stops compare the racial composition of those who are stopped with the racial composition of the residential population, even though it is typically acknowledged that the comparison is liable to be misleading for several reasons. ² A number of studies have conducted field research in an effort to form a benchmark for comparison that more plausibly captures the characteristics of the violator population, sampling times and places at which the race of passing motorists is tabulated, and even providing for the use of radar or “rolling surveys” to tabulate violators’ race. These approaches provide benchmarks that are superior to census data on the residential population, but none of them captures all of the legitimate reasons for police stops, and most may be better suited to highway traffic enforcement than to city policing. Worse still, the costs of constructing these benchmarks can be considerable. ³

The veil-of-darkness method may offer a solution as simple as it is inexpensive, but as Grogger and Ridgeway caution, some additional assumptions or controls are necessary. One is to limit the analysis to the “intertwilight” period: Depending on the time of year, stops made at any point during this time frame may have been initiated in darkness or during daylight, but in this delimited period of time during each day, we might presume that the driving population does not change dramatically. In addition, however, they impose additional controls for changes in the driving population across times of the day as well as seasons of the year. In particular, Ridgeway (2009) takes advantage of the semiannual changes to and from Daylight Savings Time (DST), which allows an analyst to still more effectively hold time of day constant—and with it, presumably, patterns of driving—as natural lighting abruptly shifts. This DST-focused analysis, delimited to 30 days before and after the switches to and from DST, sacrifices a large degree of statistical power, but it may be advantageous in controlling for seasonal variation in driving patterns. Grogger and Ridgeway also provide for statistical controls for clock time and for geographic areas across which police deployments can vary, in analyzing intertwilight stops across the entire year. Furthermore, the analysis may be confined to stops for moving violations, given that some kinds of equipment violations (e.g., malfunctioning headlights) are uniquely nighttime violations, and it is conceivable that the incidence of such equipment violations is also correlated with drivers’ race. ⁴

To our knowledge, the veil-of-darkness method has been applied previously only in Oakland, Cincinnati, and Minneapolis. ⁵ The RAND corporation analyzed stop data collected in Oakland from June through December, 2003 (Oakland Police Department, 2004). During that time, Oakland police recorded information on 7,607 stops, though with substantial underreporting. ⁶ The analysis focused on stops for moving violations in Oakland’s intertwilight period, from 5:19 p.m. to 9:06 p.m. Stops between sunset and the end of civil twilight were excluded, inasmuch as the visibility at that time of day was ambiguous. In all, 976 stops were subjected to analysis. To control for potential changes in the driving population within the intertwilight period, additional analysis focused on 1 hour before and after the end of civil twilight. To control for seasonal changes in the driving population during intertwilight hours, another analysis focused on only stops during October and November. In each analysis, Blacks were somewhat less likely to be stopped during the day, contrary to the pattern that would be observed if officers engaged in racial profiling.

Grogger and Ridgeway (2006) analyzed the same Oakland stop data. From the 7,607 stops, Grogger and Ridgeway excluded 329 that were made pursuant to a criminal investigation, and an additional 776 with missing data. From among the remaining, usable stops, they focused mainly on 1,130 made in the intertwilight period, including stops for nonmoving (equipment or registration) violations. They note, however, that their results were insensitive to the inclusion or exclusion of stops for nonmoving violations, which they suggest would generally comprise a small proportion of all stops (p. 886). They too found no evidence of racial bias.

Ridgeway (2009) analyzed vehicle stops by Cincinnati police, including those in 2008 and those in the 2003-2008 period. His analysis of 2008 stops focused primarily on 598 stops for moving violations in the intertwilight period—5:50 p.m. to 8:06 p.m.—and within 30 days of the spring and fall switches to and from DST. Additional analysis examined 5,036 stops for moving violations in the intertwilight period across the entire year, which represented about 14% of the stops for moving violations at any time of the day and about 9% of the vehicle stops. Ridgeway’s analysis of 2003-2008 stops included 3,726 stops within 30 days of a DST switch and 28,927 across all seasons of the 6 years. None of these analyses yielded evidence of racial profiling, and in 2008, Blacks were less likely to be stopped during the daytime.

Ritter and Bael (2009) analyzed data on 53,559 stops by the Minneapolis police in 2002. The “official report” on Minneapolis (Council on Crime and Justice and Institute on Race and Poverty, 2003), which relied on comparisons of the racial composition of the drivers who were stopped with that of the (approximated) driving population, concluded that Blacks and Latinos were stopped with disproportionate frequency. Noting the respects in which the benchmark failed to eliminate alternative explanations for racial disparities, Ritter and Bael applied the veil-of-darkness method to the same data. Focusing on intertwilight stops and controlling statistically for the time of day, they report substantively and statistically significant differences in the probabilities with which Blacks and Latinos are stopped in daylight rather than darkness, and the differences are uniformly consistent with the racial profiling proposition.

Stops by Syracuse Police

Located in central New York, Syracuse is a city of 25 sq. miles and nearly 150,000 people (based on the 2000 census), one quarter of whom were Black, and nearly two thirds of whom were White, 5% were Hispanic. Syracuse is the urban center of a metropolitan county of more than 450,000 people, and both north–south and east–west limited-access highways intersect near the geographic center of the city. Syracuse is served by a police department of nearly 500 sworn officers.

We replicate Ridgeway’s (2009) veil-of-darkness analysis of Cincinnati stops, albeit in a city with a substantially longer intertwilight period, during which (proportionally) much larger numbers of stops were made. The Syracuse Police Department (SPD) provided data on stops conducted in a 4-year period, from 2006 through 2009. ⁸ Information about these stops is drawn from two sources: citizen contact forms, which are completed by Syracuse police when they have enforcement-related contacts with citizens that do not eventuate in an arrest and arrest reports for “on-view” arrests (i.e., arrests made in incidents that officers initiated rather than pursuant to a dispatch). ⁹ The citizen contact forms include much of the information that is needed for analysis of the kind normally performed on this subject, though these records lack some information about searches, particularly the reason for the search. Moreover, the arrest records do not capture information about the reasons for stops, nor do they contain information about searches. ¹⁰ Hence we examine stops but not poststop outcomes.

In addition, we examine the differences among officers assigned to different units, including specialized traffic units and crime-suppression units. SPD personnel manually coded information on officers’ assignments for 1 of the 4 years (2009), identifying those assigned to either the traffic division or to the Crime Reduction Team (CRT), so that we could examine patterns by officers in each of these units separately. The traffic division is responsible for proactive enforcement of state vehicle and traffic laws and vehicle-related city ordinances. The CRT focuses on reducing violent and other crime, is deployed mainly to high-crime areas, and uses proactive patrol tactics.

We have confined our focus to vehicle stops, on the presumption that the veil-of-darkness method does not apply plausibly to pedestrian stops. ¹¹ Syracuse police made more than 50,000 vehicle stops in the 4 years examined here, 87% of which were documented on a citizen contact form. Table 1 shows the distribution of stops across times of the day, overall and, for 2009, by officer assignment. We would of course expect that the numbers of vehicle stops would fluctuate with both the volume of vehicular traffic and the deployment of police, and so it is no surprise that stops are most numerous during the afternoon commuting hours and the evening hours. Stops in the intertwilight period comprise about one third of the stops in Syracuse.

OPEN IN VIEWER

Table 1

Vehicle Stops by Time of Day (2006-2009) and by Officer Assignment (2009)

	Total 2006-2009		Traffic 2009		CRT 2009		Other 2009		Total 2009
Time of day 2009	Frequency	%	Frequency	%	Frequency	%	Frequency	%	Frequency	%
7:00 a.m.-10:59 a.m.	4,162	8.2	398	4.8	0	0	602	11.2	1,000	6.7
11:00 a.m.-2:59 p.m.	5,698	11.3	1,161	14.0	25	1.9	580	10.8	1,766	11.8
3:00 p.m.-6:59 p.m.	13,099	26.0	2,032	24.6	327	24.3	1,099	20.5	3,458	23.1
7:00 p.m.-10:59 p.m.	14,349	28.4	2,969	35.9	749	55.6	864	16.1	4,582	30.6
11:00 p.m.-2:59 a.m.	10,887	21.6	1,662	20.1	246	18.3	1,642	30.6	3,550	23.7
3:00 a.m.-6:59 a.m.	2,259	4.5	51	0.6	0	0	572	10.7	623	4.2
Total	50,454		8,273		1,347		5,359		14,979
Intertwilight	17,254	34.2	3,055	36.9	774	57.5	1,235	23.0	5,064	33.8

Note: CRT = Crime Reduction Team.

To some degree the numbers and the mix of vehicle stops vary across the hours of the day with the deployment of different units. CRT officers make virtually all of their stops between 3 p.m. and 3 a.m., with 80% of them between 3 p.m. and 11 p.m., although CRT stops represent only 15.3% of all of the vehicle stops. Most of the vehicle stops by traffic officers are made between 3 p.m. and 3 a.m. Between 3 a.m. and 3 p.m., the majority of vehicle stops—52%—are made by officers who are not assigned to one of these specialized units. Stops in the intertwilight period are disproportionately made by traffic and CRT officers, compared with other times of the day.

We also note that vehicle stops are widely distributed spatially, but they are more densely concentrated in the central parts of the city, and especially along major traffic arterials. ¹² When the spatial distribution is disaggregated by officers’ assignments, they exhibit differences that are consistent with the units’ respective missions. Stops by CRT officers are more concentrated spatially, and in areas that tend to have higher rates of crime. Stops by traffic officers are more widely dispersed and tend to be congruent with major traffic arterials. Stops by officers with neither of these assignments are also rather widely dispersed but appear to cluster more in higher crime areas than the traffic officers’ stops do, which may reflect deployment patterns.

Intertwilight Stops

Our analytic focus, in testing for evidence of racial bias in stops, is on the stops made during the intertwilight period. The intertwilight period is marked by the earliest time at which civil twilight ends during the year—in Syracuse, that is 5:02 p.m. in December—and the latest time at which civil twilight ends—in Syracuse, 9:23 p.m. in June. (Vehicle stops during the morning intertwilight period—approximately 5 a.m. to 7 a.m. in Syracuse—are far less numerous, and we do not include them in our veil-of-darkness analysis.) The driving population might change some over this period of time, as we discuss below, but not as much as the population would be expected to change across all of the hours of the day, and so to a degree, the focus on the intertwilight period serves to control for officers’ opportunities to make stops. Here, we briefly describe these stops: the reasons for the stops, the assignments of the officers who make them, and the characteristics of the people who are stopped. One would be properly concerned about the degree to which stops during the intertwilight period resemble those made during the other hours of the day, so these comparisons set an important context for interpreting the results of the veil-of-darkness analysis.

Table 2 displays the reasons for the stops, as officers recorded them, across all 4 years and for 2009 for officers with different assignments; intertwilight stops are shown in each cell with the corresponding numbers and percentages of all stops, regardless of time of day, appearing within parentheses. The reasons for stops during the intertwilight period mirror those for stops more generally: at least three quarters of the stops were made for traffic violations, and the proportion is probably higher still, assuming that a fraction of the stops ending in arrests began as stops for traffic violations and through investigation (e.g., warrant checks or searches) officers established probable cause for arrest. Fewer suspicion stops are made in the intertwilight period than at other times of the day, but they are few in number generally.

OPEN IN VIEWER

Table 2

Reasons for Intertwilight Vehicle Stops, Overall (2006-2009) and by Officer Assignment (2009)

Reason for stops	Total 2006-2009		Traffic		CRT		Other		Total
	Frequency	%	Frequency	%	Frequency	%	Frequency	%	Frequency	%
Traffic violation	14,211 (42,084)	82.4 (83.4)	2,999 (7,922)	98.2 (95.7)	666 (1,149)	86.0 (85.3)	1,088 (4,545)	88.1 (84.8)	4,753 (13,616)	93.9 (90.9)
Other unknown—On-view arrest	2,645 (6,493)	15.3 (12.9)	50 (333)	1.6 (4.0)	79 (149)	10.2 (11.1)	83 (351)	6.7 (6.5)	212 (833)	4.2 (5.6)
Suspicious person/vehicle	166 (1,151)	1.0 (2.3)	1 (8)	0.0 (0.1)	13 (20)	1.7 (1.5)	32 (325)	2.6 (6.1)	46 (353)	0.9 (2.4)
Other	232 (728)	1.3 (1.4)	5 (11)	0.2 (0.1)	0 (29)	0 (2.2)	32 (138)	2.6 (2.6)	53 (178)	1.0 (1.2)
Total	17,254 (50,546)		3,055 (8,274)		774 (1,347)		1,235 (5,359)		5,064 (14,980)

Note: Total numbers of stops at any time of day shown in parenthesis. CRT = Crime Reduction Team.

When the reasons for stops are disaggregated by officers’ assignments, we likewise find that the stops during the intertwilight period display a pattern that resembles that of stops more generally (see Table 2), and also that the stops made by traffic officers, CRT officers, and all other officers, respectively, exhibit somewhat different patterns that are consistent with their respective missions. ¹³ Overall (across all hours of the day), approximately 55% of the stops were made by officers assigned to the traffic division, and all but very small fractions of the stops made by traffic officers were, as expected, for traffic violations. A larger proportion of the stops made by CRT officers either end in arrest or are of suspicious vehicles or people; given the nature of these officers’ assignments, this too is as expected. We infer that other officers (mainly patrol officers) also make, proportionally, fewer stops merely for traffic violations than traffic division officers do, and compared with CRT officers, a smaller fraction of these stops resulted in arrest.

The people stopped during the intertwilight period differ somewhat, as a group, from the larger population of those who are stopped. ¹⁴ A slim majority of the people stopped in this period are African American, and compared with the stopped population as a whole, they are disproportionately African American, as well as disproportionately younger and male (see Table 3). However, these differences are not large in magnitude.

OPEN IN VIEWER

Table 3

Characteristics of People Stopped in Intertwilight Period

Characteristics	2006-2009		2009		Traffic		CRT		Other
	Frequency	%	Frequency	%	Frequency	%	Frequency	%	Frequency	%
African American	10,342 (28,514)	51.5 (47.8)	3,135 (8,712)	51.8 (48.2)	1,444 (3,410)	44.4 (38.9)	858 (1,539)	70.5 (72.6)	833 (3,763)	52.5 (52.3)
White	9,152 (29,148)	45.6 (48.9)	2,741 (8,708)	45.3 (48.1)	1,704 (5,012)	52.4 (57.1)	340 (553)	27.9 (26.1)	697 (3,143)	43.9 (43.7)
Male	14,191 (42,046)	70.7 (70.5)	4,200 (12,599)	69.4 (69.6)	2,138 (5,851)	65.8 (66.7)	961 (1,662)	79.0 (78.4)	1,101 (5,086)	69.4 (70.6)
16-24 years old	7,657 (22,362)	38.1 (37.5)	2,226 (6,541)	36.8 (36.2)	1,028 (2,793)	31.6 (31.8)	581 (1,045)	47.7 (49.3)	617 (2,703)	38.9 (37.5)
25-35 years old	6,170 (17,746)	30.7 (29.7)	1,812 (5,402)	29.9 (29.9)	990 (2,686)	30.5 (30.6)	408 (663)	33.5 (31.3)	414 (2,053)	26.1 (28.5)
36-55 years old	5,021 (15,481)	25.0 (26.0)	1,661 (5,030)	27.4 (27.8)	1,015 (2,694)	31.2 (30.7)	206 (371)	16.9 (17.5)	440 (1,965)	27.7 (27.3)
Total	20,086 (59,656)		6,053 (18,093)		3,249 (8,774)		1,217 (2,119)		1,587 (7,200)

Note: Total numbers of stops at any time of day shown in parenthesis. CRT = Crime Reduction Team.

The composition of the stopped population differs some for officers with different assignments. A majority of the people stopped by traffic division officers are White; compared with the people stopped by other police units, a somewhat smaller proportion of those stopped by the traffic division are men, and the distribution of ages is more nearly even. The people stopped by CRT officers are disproportionately African American, male, and young, as one would expect given their unit’s mission: These officers work mainly in higher crime areas, which tend to have a greater residential representation of minorities, and offenders are disproportionately young men.

Testing for Racial Bias in Stops

The simplest application of the veil-of-darkness method is to compare two proportions: of those who are stopped during daylight, the proportion who are African American; and of those who are stopped during darkness, the proportion who are African American. If police are biased against African Americans in making stops, then the former proportion will be larger than the latter. Figure 1 displays these proportions as a line graph. For each half hour interval between 5 p.m. and 9:30 p.m., the solid line represents the percentage of stops in daylight that involved at least one Black occupant, whereas the dashed line represents the percentage of stops in darkness that involved at least one Black occupant, with the scale of percentages on the left axis.

OPEN IN VIEWER

Figure 1.

Percent stops of African Americans, daylight and darkness

Two facts are immediately apparent from the line graph. First, the proportion of stops involving a Black occupant varies some across the intertwilight period, increasing from 5:00 until about 7:00, and then holding roughly steady. We attribute this pattern mainly to a changing composition of the driving population, though it could be partly a function of police deployment. ¹⁵ Second, and more importantly, the proportions of stops involving Black occupants during daylight and darkness, respectively, are in the main quite similar during each clock-time interval, with the possible exception of the 8:30-8:59 interval, when 50% of the nighttime stops, and 57% of the daytime stops, involved one or more African Americans. The daytime–nighttime disparity in the 8:30-8:59 interval is consistent with a pattern of racial bias, but otherwise the disparities are small and several are in the opposite direction, leading us to infer that this one larger difference is a statistical anomaly.

A more analytically powerful—but less visually intuitive—approach is to statistically control for variation in time and place, so that we can better isolate the effect of daylight on the probability that an African American will be stopped. We use the technique of logistic regression to hold clock time (in 15-minute intervals), day of week, and police beat constant. From the regression results, we compute an odds ratio that indicates how many times more likely it is that daylight stops involved an African American citizen, compared with nighttime stops. An odds ratio of 1.0 tells us that African Americans were no more (and no less) likely to be stopped during daylight than in darkness. An odds ratio greater than 1.0 is consistent with an inference of racial bias, signifying that African Americans were more likely to be stopped during daylight than in darkness. An odds ratio of less than 1.0 signifies that African Americans were less likely to be stopped during daylight than in darkness, a pattern contrary to the proposition that stops are racially biased. We conduct this analysis for each year and for all 4 years together, with the relevant results displayed in Table 4. ¹⁶

OPEN IN VIEWER

Table 4

Comparison of the Odds of African American Versus Other Occupants Being Stopped Between Daylight and Dark

Year	Full year			Within 30 days of DST
	Odds ratio (p value)	95% confidence interval	Number of stops	Odds ratio (p value)	95% confidence interval	Number of stops
All	0.963 (.319)	[0.894, 1.037]	17,172	1.076 (.432)	[0.897, 1.290]	5,470
2006	0.890 (.102)	[0.775, 1.023]	4,707	1.012 (.934)	[0.761, 1.346]	1,606
2007	0.969 (.703)	[0.822, 1.142]	3,494	0.818 (.287)	[0.565, 1.184]	1,059
2008	1.062 (.466)	[0.904, 1.247]	3,908	1.541 (.019)	[1.075, 2.211]	1,210
2009	1.023 (.753)	[0.886, 1.023]	5,063	1.175 (.325)	[0.852, 1.621]	1,595

Note: DST = Daylight Savings Time.

The odds ratios neither for the individual years nor for the entire 4-year period indicate that African Americans are more likely to be stopped during daylight than during darkness. ¹⁷ The overall odds ratio and the odds ratios for 2 of the 4 years are all less than 1.0, contrary to the proposition that police target African Americans for stops, though none of them can be reliably distinguished from 1.0. The odds ratios for the remaining 2 years are both greater than 1.0, but the difference is small substantively and, again, neither odds ratio can be reliably distinguished from 1.0 statistically. ¹⁸ These results are, then, consistent with the conclusion that Syracuse police have not exhibited racial bias in making vehicle stops. ¹⁹

Patterns by Officer Assignment

As we noted above, CRT officers’ stops tend to be concentrated in high-crime areas, whereas traffic officers’ stops are more spatially dispersed but clustered along major traffic arterials. With these differences in the nature of the areas and the roads on which they make stops, we might expect to find some differences in the characteristics of the people whom they stop, which was observed in Table 3, above. These differences remain with controls for police beat in a logistic regression: CRT officers are twice as likely to stop a Black motorist as other (mainly patrol) officers are, whereas traffic officers are two-thirds as likely to stop a Black motorist. ²¹ When we conduct separate analyses of CRT, traffic, and other officers, respectively, we find that the odds ratios conform with our a priori expectations; see Table 5. The odds ratio for CRT officers is not only larger than 1.0, but also substantively large (1.376) and greater than those for other officers; the odds ratio for traffic officers is the smallest of the three. All of the odds ratios, however, are within a 90% confidence interval of 1.0 and of one another. We can reject neither the null hypothesis of race-neutral stops for any of these three groups nor the null hypothesis that officers with different assignments exhibit the same pattern of stops, but in view of the estimated odds ratios, Ns, and confidence intervals, we cannot conclude that all three sets of officers are unbiased in their stops.

OPEN IN VIEWER

Table 5

Comparison of the Odds of African American Versus Other Occupants Being Stopped, by Officer Assignment and Between Daylight and Dark

Officer assignment	Main effect—Odds ratio (p value)	Daylight effect—Odds ratio (p value)	95% confidence interval	Number of stops
Traffic	0.677 (.000)	0.993 (.943)	[0.817, 1.207]	3,055
CRT	2.087 (.000)	1.376 (.130)	[0.910, 2.079]	774
Other	reference	1.164 (.330)	[0.858, 1.578]	1,234

Note: CRT = Crime Reduction Team.

Conclusions

Addressing the reality and the perceptions of biased policing calls for serious efforts to detect patterns of racially biased police practices, determine the origins of any bias that is detected, and establish fair and equitable practices. But detecting bias in police practice is subject to daunting analytic challenges in discriminating biases—subtle biases—from systematic but legitimate influences as well as stochastic effects. Policing is a complex set of tasks, with ambiguous and even conflicting goals. Information about police–citizen encounters is never as rich as the events are nuanced, and analysis requires simplifying assumptions that can sometimes yield misleading results. Erroneous findings could eventuate in unwarranted and unproductive—even counterproductive—policy interventions or in inaction that leaves serious concerns unresolved. Social scientists have made herculean efforts over the past decade to identify the analytical pitfalls of racial profiling research and devise methods that avoid those pitfalls, and even with good faith efforts by cities and researchers to fund and implement those methods, the research seldom if ever supports definitive conclusions. Firm conclusions are inevitably elusive, given the inability of any analyst to establish indisputably valid benchmarks. Moreover, the best benchmarks are not inexpensive to form. A more economical approach that promises to generate equally (or more) credible results is needed, one that can be applied even in cash-strapped municipalities whose concern about racial profiling by police outstrips their ability to pay for benchmarking data collection.

The veil-of-darkness method might be that approach. It may be as close to a strong quasi-experiment as research on racial profiling can come, minimizing the variables that should be measured and statistically controlled; economical studies of racial profiling cannot measure and analyze all or even most of the factors that affect officers’ decisions to stop (Engel et al., 2002; Stroshine et al., 2008). In our judgment, the veil-of-darkness approach affords the most useful, cost-effective benchmark yet devised. Although no analysis of data, of the kind examined here, can definitively establish that officers’ decisions to stop citizens are—or are not—influenced by racial bias, the preponderantly null finding yielded by the veil-of-darkness analysis of Syracuse stops is, we believe, fairly persuasive evidence of racial neutrality by Syracuse police in making traffic stops. By themselves, these findings are of particular interest mainly to the people of Syracuse, but like the findings of other analyses of individual cities, they are also of value to the field more generally, as our knowledge of policing practices and methods of analysis deepens with the accumulation of studies like this one. ²²