Cross-Cultural Fairness of the PCL: SV: A New Zealand Analysis

Abstract

Several studies have investigated the predictive validity of the Hare Psychopathy Checklists (PCL); however, the majority have used North American samples. The PCL can inform important decisions about people’s futures and therefore it is imperative to validate its psychometric properties within the jurisdictions it is used, particularly for Indigenous and non-European populations. In New Zealand, Botha and Polaschek (2023) concluded that the PCL: Screening Version (PCL:SV; Hart et al., 1999) demonstrated cross-ethnic predictive invariance using traditional discrimination statistics. The present study extends on this research by reporting on a novel statistic for assessing fairness in risk tool performance, cross-area under the curve (xAUC), as well as error rate balance and predictive parity of the PCL:SV for the prediction of any and violent 5-year recidivism outcomes in European and Māori subsamples (n = 288). In contrast to previous research, we found discrepancies, between European and Māori, on xAUC, error rate balance, and predictive parity.

Keywords

psychopathy PCL:SV cross-cultural fairness predictive validity risk assessment xAUC

Get full access to this article

View all access options for this article.

References

Angwin

Larson

Mattu

Kirchner

(2016). Machine bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

Ashford

L. J.

Spivak

B. L.

Ogloff

J. R.

Shepherd

S. M.

(2022). The cross-cultural fairness of the LS/RNR: An Australian analysis. Law and Human Behavior, 46(3), 214–226. https://doi.org/10.1037/lhb0000486

Ashford

L. J.

Spivak

B. L.

Shepherd

S. M.

(2021). Racial fairness in violence risk instruments: A review of the literature. Psychology, Crime & Law, 28(1), 911–941. https://doi.org/10.1080/1068316X.2021.1972108

Berk

Heidari

Jabbari

Kearns

Roth

(2021). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1), 3–44. https://doi.org/10.1177/0049124118782533

Botha

Nichols

(2024). Process evaluation of the special treatment units for violent offending (STU:VO): Within-treatment change. Unpublished manuscripts.

Botha

Polaschek

D. L. L.

(2023). Predictive validity of the Psychopathy Checklist: Screening Version (PCL:SV) in a post-release sample of New Zealand male prisoners. Psychological Services. https://doi.org/10.1037/ser0000885

Botha

Polaschek

D. L. L.

Wilson

(2024). Measurement invariance and convergent validity of the Psychopathy Checklist: Screening Version (PCL:SV) across Indigenous Māori and European samples in Aotearoa/New Zealand. Manuscript submitted for publication.

Chouldechova

(2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047

Cooke

D. J.

Michie

Hart

S. D.

Hare

R. D.

(1999). Evaluating the screening version of the Hare Psychopathy Checklist—Revised (PCL:SV): An item response theory analysis. Psychological Assessment, 11(1), 3–13. https://doi.org/10.1037/1040-3590.11.1.3

10.

Day

Tamatea

A. J.

Casey

Geia

(2018). Assessing violence risk with Aboriginal and Torres Strait Islander offenders: Considerations for forensic practice. Psychiatry, Psychology and Law, 25(3), 452–464. https://doi.org/10.1080/13218719.2018.1467804

11.

DeSorcy

D. R.

Olver

M. E.

Wormith

J. S.

(2017). Working alliance and psychopathy: Linkages to treatment outcome in a sample of treated sexual offenders. Journal of Interpersonal Violence, 35(7–8), 1739–1760. https://dx-doi-org.web.bisu.edu.cn/10.1177/0886260517698822

12.

Eckhouse

Lum

Conti-Cook

Ciccolini

(2019). Layers of bias: A unified approach for understanding problems with risk assessment. Criminal Justice and Behaviour, 46(2), 355. https://doi.org/10.1177/0093854818820890

13.

Edens

J. F.

Petrila

Kelley

S. E.

(2018). Legal and ethical issues in the assessment and treatment of psychopathy In Patrick

C. J.

(Ed.), Handbook of psychopathy (2nd ed., pp. 732–751). The Guilford Press.

14.

Ewert v. Canada (2015) FC 1093.

15.

Fergusson

D. M.

Horwood

L. J.

Swain-Campbell

(2003). Ethnicity and criminal convictions: Results of a 21-year longitudinal study. The Australian and New Zealand Journal of Criminology, 36(3), 354–367. https://doi.org/10.1375/acri.36.3.354

16.

Fox

B. H.

Jennings

W. G.

Farrington

D. P.

(2015). Bringing psychopathy into developmental and life-course criminology theories and research. Journal of Criminal Justice, 43(4), 274–289.

17.

Freedman

(2001). False prediction of future dangerousness: Error rates and Psychopathy Checklist-Revised. Journal of the American Academy of Psychiatry and the Law, 29, 89–95. https://dl.acm.org/doi/10.5555/3454287.3454596

18.

Hanley

J. A.

McNeil

B. J.

(1983). A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology, 148(3), 839–843. https://doi.org/10.1148/radiology.148.3.6878708

19.

Hare

R. D.

Clark

Grann

Thornton

(2000). Psychopathy and the predictive validity of the PCL-R: An international perspective. Behavioural Sciences & the Law, 18(5), 623–645. https://doi.org/10.1002/1099-0798(200010)18:5<623::aid-bsl409>3.0.co;2-w

20.

Hare

R. D.

(2003). Manual for the revised psychopathy checklist (2nd ed.). Multi-Health Systems.

21.

Hart

S. D.

(2016). Culture and violence risk assessment: The case of Ewert v. Canada. Journal of Threat Assessment and Management, 3(2), 76–96. https://doi.org/10.1037/tam0000068

22.

Hart

S. D.

Cox

D. N.

Hare

R. D.

(1995). The Hare Psychopathy Checklist: Screening Version. Multi-Health Systems.

23.

Justice Sector. (2022). Long-term insights on imprisonment, 1960 to 2050 (pp. 1–124). Justice Sector. https://www.justice.govt.nz/assets/Documents/Publications/14.02.2023-LTIB_Report_extended_final_v5_Web.pdf

24.

Kallus

Zhou

(2019). The fairness of risk scores beyond classification: Bipartite ranking and the xAUC metric. Advances in Neural Information Processing Systems, 32, 1–16. https://doi.org/10.48550/arXiv.1902.05826

25.

Kleinberg

Mullainathan

Raghavan

(2016). Inherent trade-offs in the fair determination of risk scores. Innovations in Theoretical Computer Science. https://doi.org/10.48550/arXiv.1609.05807

26.

Lowder

E. M.

Morrison

M. M.

Kroner

D. G.

Desmarais

S. L.

(2019). Racial bias and LSI-R assessments in probation sentencing and outcomes. Criminal Justice and Behavior, 46(2), 210–233. https://doi.org/10.1177/0093854818789977

27.

Nettlebeck

Smandych

(2010). Policing Indigenous peoples on two colonial frontiers: Australia’s mounted police and Canada’s North-West mounted police. The Australian and New Zealand Journal of Criminology, 43(2), 356–375. https://journals-sagepub-com-s.web.bisu.edu.cn/doi/pdf/10.1375/acri.43.2.356

28.

Neumann

C. S.

Hare

R. D.

(2008). Psychopathic traits in a large community sample: Links to violence, alcohol use, and intelligence. Journal of Consulting and Clinical Psychology, 76(5), 893–899.

29.

Olver

M. E.

Neumann

C. S.

Sewall

L. A.

Lewis

Hare

R. D.

Wong

S. C. P.

(2018). A comprehensive examination of the psychometric properties of the Hare Psychopathy Checklist-Revised in a Canadian multisite sample of Indigenous and non-Indigenous offenders. Psychological Assessment, 30(6), 779–792. https://dx-doi-org.web.bisu.edu.cn/10.1037/pas0000533

30.

Olver

M. E.

Stockdale

K. C.

Helmus

L. M.

Woods

Termeer

Prince

(2024). Too risky to use, or too risky not to? Lessons learned from over 30 years of research on forensic risk assessment with Indigenous persons. Psychological Bulletin, 150(5), 487–553. https://doi.org/10.1037/bul0000414

31.

Olver

M. E.

Stockdale

K. C.

Neumann

C. S.

Hare

R. D.

Mokros

Baskin-Sommers

Brand

Folino

Gacono

Gray

N. S.

Kiehl

Knight

Leon-Mayer

Logan

Meloy

J. R.

Roy

Salekin

R. T.

Snowden

Thomson

. . .Yoon

(2020). Reliability and validity of the Psychopathy Checklist-Revised in the assessment of risk for institutional violence: A cautionary note on DeMatteo et al. (2020). Psychology, Public Policy, and Law, 26(4), 490–510. https://doi.org/10.1037/law0000256

32.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2014). Thirty years of research on the level of service scales: A meta-analytic examination of predictive accuracy and sources of variability. Psychological Assessment, 26(1), 156. https://doi.org/10.1037/a0035080

33.

Perry

(2006). Nobody trusts them! Under- and over-policing Native American communities. Critical Criminology, 14, 411–444. https://doi.org/10.1007/s10612-006-9007-z

34.

Shepherd

S. M.

Adams

McEntyre

Walker

(2014). Violence risk assessment in Australian aboriginal offender populations: A review of the literature. Psychology, Public Policy, and Law, 20(3), 281–293. https://doi.org/10.1037/law0000017

35.

Shepherd

S. M.

Lewis-Fernandez

(2016). Forensic risk assessment and cultural diversity: Contemporary challenges and future directions. Psychology, Public Policy, and Law, 22(4), 427. https://doi.org/10.1037/law0000102

36.

Singh

J. P.

Grann

Fazel

(2011). A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clinical Psychology Review, 31(3), 499–513. https://doi.org/10.1016/j.cpr.2010.11.009

37.

Spruit

van der Put

Gubbels

Bindels

(2017). Age differences in the severity, impact and relative importance of dynamic risk factors for recidivism. Journal of Criminal Justice, 50, 69–77.

38.

Taber

K. S.

(2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48, 1273–1296. https://doi.org/10.1007/s11165-016-9602-2

39.

Vitacco

M. J.

Neumann

C. S.

Jackson

R. L.

(2005). Testing a four-factor model of psychopathy and its association with ethnicity, gender, intelligence, and violence. Journal of Consulting and Clinical Psychology, 73, 466–476. https://dx-doi-org.web.bisu.edu.cn/10.1037/0022-006X.73.3.466

40.

Wilson

H. A.

Gutierrez

(2014). Does one size fit all? A meta-analysis examining the predictive ability of the Level of Service Inventory (LSI) with Aboriginal offenders. Criminal Justice and Behavior, 41(2), 196–219. https://doi.org/10.1177/0093854813500958

41.

Wilson

N. J.

(2003). The utility of the Psychopathy Checklist-Screening Version for predicting serious violent recidivism in a New Zealand offender sample [Thesis, Doctor of Philosophy (PhD)]. The University of Waikato. https://hdl.handle.net/10289/13978