Abstract
The manuscript entitled “Updating Perceptions of (In)Justice” was published in the Journal of Research in Crime and Delinquency in 2016. The research investigated five factors that were hypothesized to be related to changes in procedural justice: 1) prior judgments of procedural justice, 2) direct experiences of arrest, 3) vicarious experiences of arrest, 4) individual arrest history, and 5) age. Data from the Pathways to Desistance Study were used to test these hypotheses using multilevel models and led to the following summative conclusion: judgments of procedural justice are not static; rather, they are a function of prior judgments, personal and vicarious experiences of arrest (more specifically, the arrest of family members) and age.
After its publication, I became aware of skip patterns in the primary measure of interest – the pre-existing procedural justice scale–that may have affected the results. Notably, while there are 38 possible items that make up the procedural justice scale, up to 24 of those items can be skipped at each wave of data collection if the subject was not picked up by the police (resulting in a skip pattern consisting of 12 items) or if the subject did not appear in court (resulting in a skip pattern consisting of 12 items). Thus, items comprising the scale are not consistent across waves and are dependent upon contacts with the criminal justice system.
In late Fall 2017, the individual items in the procedural justice scale became available through ICPSR (36800; prior to this, they were not available). With the individual items, it was possible to more closely examine the nature of the skip patterns for the procedural justice scale and subsequently determine if they may have affected the results presented previously. An assessment of the data revealed that between 73% and 83% of subjects with valid data at each wave legitimately skipped 12 questions in the procedural justice scale because they were not picked up by the police. Of subjects with valid data at each wave, between 67% and 72% did not appear in court and thus skipped 12 items on the procedural justice scale. Furthermore, between 45% and 68% of subjects with valid data at each wave legitimately skipped 24 questions on the procedural justice scale because they were not picked up by the police or had a court appearance. This assessment also uncovered a third and fourth skip pattern. Specifically, if a subject did not report having any friends who had been arrested (based on the answer to a previous question in the Pathways survey instrument), one procedural justice scale item was legitimately skipped. Between 18-39% of subjects with valid data at a given wave legitimately skipped this item. Moreover, if the subject did not report having any friends who had been to court (based on an answer to a previous question in the Pathways survey instrument), one procedural justice scale item was skipped resulting in between 20-39% of subjects with valid data at a given wave legitimately skipping this item. Thus, only 12 items from the procedural justice scale were asked of all subjects at each wave of data collection. Ten of these items pertain to the procedural justice element “impartiality” and two of the items relate to the ability of an individual to appeal decisions by criminal justice agents. The reliability of these ten items was assessed across waves and ranged from .66-.84. With the 12 items, I created a new procedural justice scale akin to the methodology used by the original Pathways working group – after reverse coding the necessary items, the mean score was taken from the 12 questions that the subject answered. However, not all subjects answered all twelve of the procedural justice items Thus, a second procedural justice scale was also created. If the subject answered at least nine (or 75%) of the 12 procedural justice questions, the mean of the items was computed (resulting in a loss of between .2% of subjects to 14% of subjects in each wave). 1
The original analysis was then re-estimated using the two new scales of procedural justice. When using the scale of procedural justice that included all subjects who answered at least one of the 12 procedural justice items, there was still evidence of substantial within-individual variation in judgments of procedural justice, which is consistent with the published study. Also consistent with the previous analysis, both lagged measures of the procedural justice scale (t-1 and t-2) were positive, and significantly related to current judgments of procedural justice. Notably, though, personal experiences of arrest were no longer a statistically significant predictor of changes in judgments of procedural justice (previously exhibiting a positive, significant effect) nor was age (previously exhibiting a decreasing negative effect). Additionally, whereas only familial arrests were negatively related to judgments of procedural justice in the previous analysis, the re-analysis of the data with the new measure of procedural justice demonstrated that both familial arrests and peer arrests were negatively related to judgments of procedural justice. Finally, the re-analysis of the data with the new measure of procedural justice failed to find that one’s arrest history conditioned the effect of new arrest experiences on changes in judgments of procedural justice.
As a form of sensitivity analysis, a second set of analyses of the original data using the measure of procedural justice that required the subjects to answer at least nine of the 12 items in the scale was also conducted. Similar to the two previous analyses conducted, there was evidence that judgments of procedural justice significantly vary over time within individuals and prior judgements of procedural justice (as indicated by lagged measures of procedural justice taken at t-1 and t-2) were positively related to changes in judgements of procedural justice. None of the other hypothesized factors – direct experiences of arrest, vicarious experiences of arrest, arrest history, or age – were related to changes in judgements of procedural justice when using the more stringent criteria to make the scale of procedural justice.
In conclusion, the skip patterns inherent to the pre-existing procedural justice scale that is available in the Pathways data affected some results presented in “Updating Perceptions of (In)Justice” (Augustyn, 2016), while others remain consistent. Subsequent research using these data should account for how the various skip patterns and measurement of the individual items in the procedural justice scale may affect the conclusions made regarding judgements of procedural justice. Given that the individual items are now publicly available for analysis, it would be beneficial for scholars to conduct sensitivity analyses (or primary analyses) with scales created from the individual measures to determine the robustness of results with measures subject to skip patterns.