Sex Offender Populations and Clinical Efficacy

Substantive Concerns

At the heart of Rosky’s substantive concerns is the premise that only research tying polygraphy outcomes to one indicator of behavioral improvement—actual recidivism—is useful or worthwhile. Undoubtedly, strong evidence linking polygraphy outcomes to actual recidivism rates is an ideal and valuable outcome—a point we explicitly make in our paper. However, it seems strange to argue for a focus on a single outcome when there are likely numerous ways to assess the efficacy of sex offender treatment, including self-reports. Self-report measures in criminological contexts are not without precedence or substantiation (e.g., Piquero, Schubert, & Brame, 2014). Moreover, self-report measures are used in a sizable quantity of clinical contexts (e.g., Haroz, Ybarra, & Eaton, 2014; Head et al., 2013; Miilunpalo, Vuori, Oja, Pasanen, & Urponen, 1997). We do, however, acknowledge the serious nature of sexual offending, and the need to target reduction in recidivism—an additional point we make in our original paper.

In any case, our study was not designed to resolve the long-lasting debate between proponents and opponents of polygraphy—nor do we claim anywhere in our paper that it does. We want to reiterate that point here: Our paper cannot and does not resolve the debate regarding the use of polygraphy in sex offender treatment. Moreover, our study was not designed to assess any direct indicator of the clinical efficacy of polygraphy. Instead, our study was designed to answer one simple question: Do adult and juvenile offenders differ in their rate of passing sexual history disclosure polygraphs? It is unrealistic and unfounded to expect our study to do anything more. Had we set out to ask the research questions Rosky posited as being paramount and failed to adequately answer said questions, his concerns would be more reasonable. As nearly 50% of juvenile sex offender treatment agencies in the United States use polygraphy as a clinical tool (McGrath, Cumming, Burchard, Zeoli, & Ellerby, 2010), using available data to compare and contrast adult and juvenile offenders is relevant in our view and, apparently, in the view of the peer reviewers of our original manuscript.

Given Rosky’s premise, it is also surprising that he suggested we include additional predictors in our logistic regression model. After all, Rosky stated that any assessment of polygraphy other than its direct link to recidivism is a “useless exercise.” However, we agree that future studies should assess the influence of race/ethnicity and offense type on polygraph outcomes—luxuries our current data set could not afford us. We explicitly mentioned these suggestions for future research in our discussion section.

In addition, the arguments we extended in our discussion section were not meant to dissuade researchers from using recidivism rates as an outcome in future research. We believe that recidivism research is highly useful, and we believe that more research is needed in this area. Our arguments were simply made to pre-emptively address some of the concerns that polygraphy opponents might have about the merits of our study. We introduced these arguments primarily to nest our study within the current polygraphy debate, rather than ignoring it. Unfortunately, our data did not include indicators of offender recidivism, rendering impossible the inclusion of this indicator in our analysis. This represents a common limitation among studies in the polygraphy literature.

In regard to Rosky’s general concerns about polygraphy, including the potential for false-positive or false-negative results, we offer several responses. First, each offender who initially failed his or her polygraph in our study admitted to not being truthful and eventually passed a later examination. The only possible exceptions were those incarcerated as a result of offenses completed between the initial failed polygraph and the completion of a second polygraph. Second, Rosky noted a concern about offenders being placed into treatments or receiving sanctions as a result of false-positive polygraph outcomes. The examinees in our study were already in treatment, and the state in which treatment was conducted only required that treatment progress be delayed as a result of a failed full-disclosure polygraph. Third, all polygraph examinations were conducted by an examiner with a proper state license, decades of experience in conducting examinations, and time served on the licensing board for Deception Detection Examiners. Fourth, Rosky stated, “It seems reasonable that some passed merely because they weren’t asked the right questions about their sexual histories” (p. XX). In actuality, all offenders were asked to complete a very detailed pre-polygraph sexual history questionnaire, and the polygraph examiner asked whether the offender had committed any sexual offenses that were not included in his or her questionnaire. Thus, regardless of age or history, each polygraph examination was suitable for each offender’s history. Again, disclosed sexual histories were not used to further sanction or charge offenders with additional crimes, but were used as a state-mandated treatment tool. Moreover, offender risk assessments provided by instruments such as the Static 99 are responsive to past sexual offenses, and polygraph examinations can help verify offender sexual histories. If a more accurate method became available for such verification, treatment providers would likely embrace this new method.

Methodological Concerns

Many of the methodological concerns expressed by Rosky represent either a misunderstanding of our research question, study design, and sample, or a disregard for the explicit declarations we made with respect to our study limitations. He specifically commented on the potential nonindependence of observations, our measure for age, and our sample. We address each of these concerns in turn.

In terms of the potential nonindependence of our study observations, we mentioned in our study that hierarchical linear modeling (HLM), or multilevel modeling, was not necessary, as our data did not have a hierarchical structure (Raudenbush & Bryk, 2002). Specifically, only a single measurement occasion was analyzed for each participant, and all participants were nested within the same polygraph examiner. Had there been multiple measurement occasions, and had there been multiple polygraph examiners, then measurement occasions would have been nested within participants, and participants would have been nested within examiners (Heck & Thomas, 2009; Hox, 2010; Raudenbush & Bryk, 2002). Rosky’s suggestion is akin to asking an education researcher to use HLM in an effort to model teacher and school effects in a sample of individuals who have the same teacher and share the same school—it is simply useless because there is no higher-level variation to estimate.

In response to our rationale for not using HLM, Rosky stated, “not being able to do a particular analysis to account for certain types of variability does not mean that type of variability doesn’t exist and doesn’t affect the analyses” (p. XX). In general, this is true, but this statement is not accurate as it pertains to our study and the partition of outcome variance to between- and within-examiner components. For instance, the intraclass correlation coefficient (ICC) is used to assess the so-called cluster effect of a particular set of data (Raudenbush & Bryk, 2002). The ICC is calculated using the following equation:

ρ = τ 00 σ 2 + τ 00 ,

where τ₀₀ represents the between-group variance (i.e., between examiners), and σ² represents the within-group (i.e., within examiners) variance. Because our data were derived from a single polygraph examiner, there was no between-group variance to estimate. Thus, we were left with within-group, or within-examiner, variance only. This equation both illustrates the nonclustered nature of our data and disconfirms Rosky’s claim that not having multiple polygraph examiners increases the likelihood of nonindependent observations within our data set. For the sake of additional precaution, we assessed the extent to which the date of the polygraph might have influenced our model results by including a variable for time in the logistic regression. The substantive results of this model were the same. We also addressed the influence of other potential unobserved sources of data clustering by using robust standard errors. Results from the robust logistic regression yielded the same substantive findings. In summary, there was no variation in agency, locale, or examiner to estimate in our original study. The suggestion that we use HLM results from either a lack of knowledge about HLM or a misunderstanding about our data.

Rosky also expressed concerns regarding our measure for participant age. Specifically, he argued that categorical groupings of age potentially overlook individual developmental characteristics and nuances. This is precisely why we used a continuous measure of biological age in our logistic regression model. The categorical grouping for age was used only in our preliminary bivariate analysis. Although our models did not account for each participant’s developmental characteristics, this criterion seems unrealistic and incongruent with the broader polygraphy literature.

Rosky also pointed out that our sample was nonrandom. Indeed, we used a nonprobability sample. In fact, we stated the following in our paper: “Our participants were not gathered through random sampling, and thus our results yielded neither national representation nor generalizability—a common issue within the sex offender literature” (Jensen, Shafer, Roby, & Roby, 2014, p.13). Thus, we are being criticized for something that we readily admitted ourselves. We did note, however, that the unique clinical nature of our data might be useful for understanding the implications of our results—a point that we will not concede. Rosky also labeled our sample “small,” with potential adverse consequences for the statistical power of our analytical model. Our sample was comprised of 238 adult and 86 juvenile offenders (total N = 324). In most statistical contexts in which maximum likelihood estimation is used, a sample of 300 cases or more is considered adequate (refer to Kline, 2011, pp. 11-12). This is particularly true when one is analyzing a very simple model, such as ours. Furthermore, because our logistic regression model used a continuous form of age, our statistical power was not diminished due to the relatively lower number of juveniles in our sample.

Conclusion

Overall, it appears Rosky has accused us of not answering well a question we were not trying to ask. From that perspective, he is right: Our study does not do a good job answering a question about recidivism because our paper never attempted to focus on that question. Instead, our study was designed to answer a simple research question in a substantive area ripe for additional research. Some of the methodological concerns Rosky pointed out appear to represent a misunderstanding of statistical principles or the data we analyzed. Many of his other concerns were explicitly addressed in the original manuscript. Despite our transparency, Rosky decided to submit a paper in which he reiterated the limitations of our study. As mentioned previously, we also alluded to the ongoing polygraphy debate not to resolve it, but to provide context for our study. Regardless of any disagreements Rosky might have with our study, we respect and value his skepticism. Skepticism within the scientific community is a necessary ingredient for continual empirical and clinical growth and discovery. In that spirit, we are excited to see new polygraph research emerge in the future.