Assessing the Validity of the Self-Appraisal Questionnaire in Differentiating High-Risk and Violent Female Offenders

Abstract

The Self-Appraisal Questionnaire (SAQ) holds promise as a self-report measure that predicts the risk of recidivism and assesses treatment needs for incarcerated populations. However, its validity has been questioned for use with females. Using a random sample of 543 incarcerated women, we assessed the validity of the SAQ by examining differences in scale scores and Receiver Operating Characteristic curves using multiple measures of violent behavior among women. Self-reported violence, versus a violent conviction, was a strong predictor of SAQ scores, but many of those in the most violent group did not meet the published cut scores that indicate treatment need, suggesting the need for adjusting these thresholds for women.

Keywords

Self-Appraisal Questionnaire incarcerated women violence assessment treatment

Introduction

The prediction of violent behavior is an important public safety goal for corrections personnel as they make decisions regarding release from prison. The primary question is whether the individual poses a risk to society, and, if so, the nature of that risk. Offender risk assessment instruments are used by criminal justice officials as one tool to assist in the prediction of recidivism and violent behavior for those exiting prison and jail. Several iterations or generations of these instruments have been developed over the years to improve violence prediction, as well as the assessment of treatment and programming needs. Generally, these instruments have been developed on male populations, and there have been questions regarding the validity of these instruments in determining the specific needs and risks of women (see review by Hannah-Moffat, 2009). However, the use of these instruments to predict risk of violent re-offending among women has received little attention. A recent meta-analysis of the outcomes attributable to various instruments and methods used to predict violent behavior mentions females only twice, both in relation to the poor generalizability of a particular method or instrument to women (Campbell, French, & Gendreau, 2009).

This dearth of research may be related to differences in male versus female offending; women are arrested for 19% of all crimes included in the Violent Crime Index (homicide, rape, robbery, and aggravated assault; Uniform Crime Reports, 2009), and violent recidivism is very rare among women (Deschenes, Owen, & Crow, 2007). Nonetheless, there has been increased attention to perpetration of violence by females, whether or not that violence results in formal criminal justice involvement (Schwartz, Steffensmeier, & Feldmeyer, 2009; Steffensmeier, Schwartz, Zhong, & Ackerman, 2005). This increased attention is likely to have parole boards and other correctional administrators seeking tools that accurately assist them in assessing and managing risk for women (Manchak, Skeem, Douglas, & Siranosian, 2009), particularly in predicting risk of re-offending by women serving time for violent offenses (Hannah-Moffat & Yule, 2011). Moreover, the greater focus on women as perpetrators of violence provides a catalyst for scholars and correctional administrators to assess unique risk factors and specific needs for interventions focused on women’s assaultive behavior (Collins, 2010).

The aforementioned meta-analysis on prediction of violence (Campbell et al., 2009) highlights a self-report risk instrument, the Self-Appraisal Questionnaire (SAQ), as holding promise in the prediction of violent recidivism. However, the authors note that additional research is required to test various subgroups of offender populations, particularly among female offenders, because effect sizes in their meta-analysis were often restricted to male samples. The current study is part of a broader research project assessing long-term outcomes for incarcerated women with violent offenses. A short-term goal, and focus of this article, is to determine the utility of the SAQ in assessing the risks and needs of women during incarceration.

Background

Often, the perpetration of a violent crime by a woman is an isolated event occurring within the context of family and intimate relationships (Goetting, 1988; Kruttschnitt, 2002; Mann, 1990, 1996). Nearly half (44%) of all homicides perpetrated by women involve an intimate partner, with an additional one third of victims described as acquaintances (Pollock & Davis, 2005). However, for other women, the use of violence may be more patterned over the life course (Kruttschnitt, 2002; Kubiak, Kim, Fedock, & Bybee, 2013) and related to their victimization histories (Kubiak, Rivera, & Bybee, 2011; Pollock, Mullings, & Crouch, 2006). Perceptions of increasing violence among women may be largely unfounded (Schwartz et al., 2009), but there is a dearth of empirical evidence examining the risk of violent offending/re-offending among women.

Over the past 50 years, risk prediction has changed—from relying on unstructured clinical judgments of risk that were prone to bias and errors (first generation risk assessments)—to the latest instruments that integrate various functions of risk management, selection of treatment or intervention, and assessment of change during the rehabilitation process (fourth generation; Andrade, O’Neill, & Diener, 2009; Campbell et al., 2009). Whereas the first generation of risk instruments only assessed static factors (e.g., those unlikely to change such as past criminal history), the more recently developed third-/fourth-generation tools incorporate dynamic factors (e.g., those that can be changed and, if changed, may lead to desistance from crime) and are able to assess risk as well as treatment needs (Smith, Cullen, & Latessa, 2009). In addition, these risk instruments have multiple methods of administration. For example, some utilize a self-report paper and pencil method of administration, whereas others rely on a professional rater and/or a file audit. Each administration method can be conducted at admission or at various stages across criminal justice involvement. Furthermore, some instruments assess a single risk construct (e.g., violent recidivism); others assess multiple constructs (i.e., institutional behavior, recidivism, and treatment need; Campbell et al., 2009).

Morash (2009) succinctly summarized the multiple purposes of risk and needs assessment instruments: (a) predicting prison misconduct; (b) identifying the needs of offenders that are amenable to change; (c) providing interventions to address those needs; (d) identifying the needs that are most predictive of rules violations and recidivism and using this information to make parole decisions; (e) identifying those most at risk of recidivism; and (f) developing plans of correctional intervention that will address needs. Acknowledging these multiple uses, it seems important that criminal justice administrators choose a tool that is reliable and valid for both men and women across all or most of these purposes. Although many risk instruments have been found to be reliable with good predictive utility, almost every risk assessment tool currently in use was designed for and tested more extensively on male offenders. Although some argue that empirical evidence supports the predictive ability of these instruments for both males and females (Bonta, Law, & Hanson, 1998; Smith et al., 2009), others have challenged that risk factors for men and women differ and/or differentially predict recidivism (Hannah-Moffat, 2009; Shute, 2007; Van Voorhis, Wright, Salisbury, & Bauman, 2010).

The two instruments that are widely used within criminal justice institutions around the country (Holtfreter & Cupp, 2007) are the Level of Service Inventory–Revised version (LSI-R; Andrews & Bonta, 2001, 2006) and the Correctional Offender Management Profiling for Alternative Sanction (COMPAS; Brennan, Dieterich, & Oliver, 2006; Brennan, Dieterich, & Ehret, 2009). Studies predicting recidivism among women with these tools are mixed. A meta-analysis assessing gender differences on the LSI utilized 25 unique data sets that include female offenders to test its effectiveness in predicting recidivism (Smith et al., 2009). Although there were several strong effect sizes for both men and women, none of the studies used violent recidivism as an outcome. Rather, recidivism outcomes included probation/parole violation, re-arrest, re-conviction, and re-incarceration. Another study assessing recidivism among 70 female and 1,035 male serious violent offenders in the state of Washington found that the LSI-R favorably predicted general recidivism for both men and women (Manchak et al., 2009). However, there are several important caveats. First, the published scores for designation of “high risk” groupings were ineffective for women, and the authors suggested a customized approach to establishing risk classifications based on individual sample characteristics was needed (Manchak et al., 2009). Second, although total scores were found to be predictive, the financial subscale was the strongest predictor of recidivism for women. Unfortunately, later versions of the tool (Level of Service/Case Management Inventory) deleted this scale (Andrews, Bonta, & Wormith, 2006). A study of the COMPAS using survival analysis predicts recidivism equally well for females and males. However, the sample size for women was small (n = 29), and the area under the curve (AUC) for risk recidivism in women was somewhat low at .66 (Brennan et al., 2009).

A third tool, the SAQ has not been widely adopted but has shown promise in the few available studies (see Campbell et al., 2009; Loza, Dhaliwal, Kroner, & Loza-Fanous, 2000; Loza & Green, 2003; Loza & Loza-Fanous, 2002). The SAQ is a 72-item, true/false self-report assessment tool with eight subscales specifically designed to predict violent and nonviolent recidivism. Inmates are given a total score across the subscales that place them into a risk category. Previous studies demonstrated the validity of the SAQ by correlating SAQ subscale scores with various measures, such as General Statistical Information on Recidivism (GSIR), the Violence Risk Appraisal Guide (VRAG), LSI-R, and Psychopathology Checklist–Revised (PCL-R), that resulted in differentiating violent and nonviolent offenders (Loza et al., 2000). Moreover, there were correlations between the Offender Intake Assessment (OIA) and the SAQ when used to assign individuals into treatment programs (Loza & Loza-Fanous, 2002). However, the samples in all of these studies included only male offenders. Although originally designed for male offenders, the SAQ has also shown promise for use with females. Assessing the validity of the SAQ using LSI-R among incarcerated women in two countries (Singapore and the United States), Loza, Neo, Shahinfar, and Loza-Fanous (2005) found similar concurrent validity in female offenders as previous studies found in male offenders. The SAQ was also found to be valid for predicting recidivism for female and male offenders (Loza & Green, 2003; Loza et al., 2005; Loza & Loza-Fanous, 2002). Female offenders categorized as “high-risk offenders” recidivated more frequently and sooner than the group with the more moderate risk range, and the moderate group recidivated more frequently and sooner than those in the low risk group (Loza et al., 2005). It should be noted that the recidivism study was completed on women from Singapore only, using incarceration for a new conviction during a 1-year period post release as the measure of recidivism. There was no attention to the violent or nonviolent nature of the new offense.

In addition to risk assessment, the authors of the SAQ argue that it is multifaceted and can also be used for security level assignment and treatment determination within the institutional setting (Loza & Loza-Fanous, 2002). Moreover, nearly half of the items on the SAQ are dynamic and can also be used to measure benefits of treatment. However, utility of the SAQ within the institutional setting has not been assessed for females.

The utility of risk/needs assessment instruments within the institutional setting is an important consideration because it is often the presence or absence of a violent or assaultive offense that is used to categorize offenders within prisons and jails. Those convicted of a violent offense may be mandated to special programming designed to reduce risk for violent re-offending upon release. However, arrests and convictions represent known criminal behavior but tell us little about violent behavior that does not result in arrest or other legal sanctions. In their seminal research following 328 females addicted to heroin involved in a methadone maintenance program, Anglin and Hser (1987) used both self-report and official crime data to track illegal behavior post treatment. They found that drug-involved women commit more crime than they are caught for and that criminal behavior escalates as addiction severity increases. Similarly, in an analysis of several national longitudinal databases, Steffensmeier and colleagues (2005) found that adolescent males and females self-report much more violence than arrest records indicate.

Current Study

The current study aims to assess (a) the validity of the SAQ in differentiating risk among a random sample of incarcerated women and (b) the utility of the SAQ in determining the need for institutional programming. Our research questions are as follows:

Research Question 1: Does the SAQ differentiate risk between women with a violent versus nonviolent offense?

Research Question 2: How does the inclusion of self-reported violent behaviors alter the differentiation among women?

Research Question 3: Is the SAQ useful in determining the need for programming and other interventions for high-risk women?

Method

Sampling and Procedures

Women were randomly selected following stratification across security levels and the presence/absence of a substance use disorder (SUD). Security levels include Levels I (low), II (medium), and IV (high) with the majority of women within the institution in Levels I and II. Due to the high correlation between drug use and violence (Chermack et al., 2009; Murray et al., 2008), stratification by SUD was determined by a score indicating dependence on the Substance Abuse Subtle Screening Inventory (SASSI; Lazowski, Miller, Boye, & Miller, 1998), administered at prison admission.

Lists of women, organized by presence/absence of SUD and security level, were generated by prison staff, and women were randomly selected from the list in numbers reflecting their proportion to the prison population. Because the highest security level was sparsely populated, all of the women in this level were included. In total, 822 women were selected from a total statewide institutional population of women of 1,875 placed on the institutional “call out” list requiring them to be present at a pre-specified time and place for an overview of the survey. Research staff explained the study, data collection, and anonymity guarantee prior to providing informational materials. Waiver of written consent and all data collection procedures were approved by the University Institutional Review Board to ensure anonymity.

Because other requirements of the institution (e.g., medical appointments) took priority over the survey, not all women on the selection list attended the informational/data collection meeting. Of those that were “called out,” 71% (n = 580) participated and completed all or part of the survey. Due to the anonymous nature of the survey, there is no information as to the similarity or differences between those who did and those who did not attend or participate.

Missing Data

Among 580 surveys obtained, 6 cases were eliminated because they were missing responses to more than 80% of the items. In addition, 31 cases were eliminated because they were missing information about the nature of the current offense, a variable used for grouping cases, resulting in 543 (94%) usable surveys. These remaining surveys contained a modest amount of missing data (5.24% of the data matrix). We estimated missing data using expectation maximization (EM) to maximize statistical power for analysis and reduce bias that can be caused by ad hoc methods of handling missing data (Little & Rubin, 2002).

Sample Demographics

The average age of women, upon entering prison, was 32 years (SD = 9.9) and the women’s current mean age was 37 years (SD = 10.5; see Table 1). The average prison stay at the time of the data collection was 6 years (6.5). Other demographic information, including race, were not collected to further ensure anonymity. Institutional data reflect that 50% of the population is of minority status (i.e., African American, Latino).

Table 1.

Sample Characteristic (N = 543).

	M	SD
Age
Current age	37.72	10.52
Age at incarceration	31.67	9.89
Length of incarceration	6.04	6.51

	%	n

Offense type
Violent offense	67.3	365
Nonviolent offense	33.7	178
Self-reported uncaught violence^a (any)	45.1	245
Committed robbery with a gun or knife	9.2	50
Committed robbery without a weapon	15.3	83
Injured or killed someone	16	87
Threatened another with a gun or knife	18.2	99
Shot a gun or cut with a knife	14.2	77
Sexually assaulted another	1.3	7
Beat up another person	29.5	160

% indicates the presence of one or more of the uncaught violent behaviors listed below.

Measures

Risk assessment

The SAQ has eight subscales (criminal tendencies, conduct problems, criminal history, alcohol and drug abuse, antisocial associates, anger, antisocial personality problems, and a validity subscale), with the number of items in each varying from 3 to 27. Previous validity studies found low internal consistency for women on several subscales (i.e., antisocial associates, criminal history, and antisocial personality). Dichotomous responses, true (1) or false (0), are summed with higher scores indicating a greater level of risk and predicting violent and nonviolent recidivism (Loza et al., 2005). Only 67 of the 72 items are used for the total, excluding the anger subscale due to the author’s belief in the unreliability of anger as a predictor of recidivism (Loza & Loza-Fanous, 1999a, 1999b).

The range of the total scores in this sample was 2 to 64 (M = 25, SD = 12.41). Coefficients of reliability for subscales were diverse. Subscales with fewer items had relatively low reliability (antisocial associates, 3 items, .44; antisocial personality, 5 items, .57; and criminal history, 6 items, .64), whereas those with a greater number of items had higher (criminal tendency, 27 items, .81; conduct problem, 18 items, .88; and alcohol and drug abuse, 8 items, .77). The coefficient of reliability for the SAQ total score was .92.

Violent offense/behavior

Commission of a violent offense was defined based on a current (61%, n = 333) or past (18%, n = 96) conviction involving robbery, assault, homicide, or sex offense with force. In total, 67% (n = 365) of the women had been convicted of either a current or past assaultive offense. The remaining third of the sample (n = 178) were categorized as nonviolent with offenses such as burglary, drug possession/use, drug trafficking/sales, driving under the influence (DUI), fraud, property/larceny, and sex offense without force (see Table 1).

In addition, self-reported violent behavior was defined as engagement in behavior that would be considered criminal if you were caught, replicating items from a study by Pollock and colleagues (2006). Women were asked to mark “any activity they participated in, but did not get caught for, in the year before they came to prison.” Items, with frequencies, are included in Table 1, with 245 women reporting engagement in at least one of these behaviors.

Analysis

Analyses were conducted to assess the validity of the SAQ in distinguishing between groups of female offenders who differed in the use of violence, using two constructs of violence: (a) presence or absence of a current or past violent conviction and (b) presence or absence of self-reported violent behavior. Based on these categorizations, and later a combination of both, differences on subscale and total SAQ scores among women were compared using t tests and ANOVA with post hoc Tukey’s honestly significant difference (HSD) test. In addition, Receiver Operating Characteristic (ROC) analysis (Lasko, Bhagwat, Zou, & Ohno-Machado, 2005) was used to test the effectiveness of the full range of SAQ scores in distinguishing between women using both constructs of violence. Chi-square and Marascuilo procedure were conducted to compare differences in proportions of women with treatment needs among incarcerated women.

Results

To answer the first research question, whether the SAQ differentiates between women with and without violent offenses, women were divided into two groups based on their conviction category: violent (n = 365) versus nonviolent (n = 178). Comparing SAQ subscale scores, there was only one significant difference between groups (see Table 2). Women who committed a violent offense had significantly higher scores on the conduct problems subscale than those who were convicted of a nonviolent offense (M = 6.80 and 5.90, respectively, t = 2.06, p = .04). In terms of SAQ total score, women who were convicted of violent offenses had a slightly higher mean score than those who committed nonviolent offenses (M = 25 and 24, respectively), but the difference was not significant.

Table 2.

Comparison of SAQ Scores Between Women With and Without Violent Offenses.

			Offense type
	Total (N = 543)		Nonviolent (n = 178)		Violent (n = 165)
SAQ subscales	M	SD	M	SD	M	SD	t(222)
Criminal tendencies	9.47	4.87	8.93	4.71	9.73	4.93	−1.80
Conduct problems	6.50	4.81	5.90	4.63	6.80	4.88	−2.06*
Criminal history	2.11	1.62	2.16	1.59	2.09	1.64	0.49
Alcohol and drug abuse	3.49	2.29	3.69	2.22	3.39	2.33	1.46
Antisocial associates	1.65	0.98	1.58	0.97	1.68	0.99	−1.09
Antisocial personality	1.78	1.39	1.89	1.34	1.72	1.42	1.34
SAQ total	25.00	12.41	24.16	11.91	25.41	12.64	−1.10

Note. SAQ = Self-Appraisal Questionnaire.

p < .05.

In a further attempt to assess the validity of the SAQ in differentiating between those with and without a violent conviction, a ROC analysis was conducted. The result showed that SAQ had low accuracy in detecting women with a violent conviction with AUC = .52, 95% confidence interval (CI) = [0.47, 0.57] (see Figure 1). At the optimal cut point of 30.5, sensitivity and specificity were .39 and .86, respectively, and more than half of the women (53%) were misclassified.

Figure 1.

ROC curves assessing validity of two types of violence indicators on SAQ scores.

To answer the second research question, assessing the importance of self-reported violent behavior in validating the SAQ, we began with an ROC analysis, again dividing women into two groups: those with self-reported violent behavior (n = 245, 45%) and those without this behavior (n = 298, 55%). The SAQ demonstrated moderate accuracy in identifying women who were engaged in self-reported violence with AUC = .75, 95% CI [0.71, 0.79]. Using the optimal cut point of 30.5, sensitivity and specificity of the SAQ in detecting involvement in self-reported violence were .54 and .84, respectively, with 30% of the women misclassified. Figure 1 shows the differences of the AUC of the both ROC analyses. By comparing two ROC graphs and AUCs, we found that the SAQ was more accurate in distinguishing women using self-reported violent behavior than in distinguishing the presence or absence of a violent conviction. The difference between two AUCs was statistically significant (z = −6.94, p < .001).

Because the ROC analyses show stronger validity of the SAQ based on self-reported violence, using the combination of conviction type (violent vs. nonviolent) and self-reported violence (presence/absence) might enhance the validity of the SAQ in predicting risk of future engagement in violent behavior. Based on this assumption, we created four groups based on the two constructs: Group A, women without either a violent offense or self-reported violence (no violence; n = 126); Group B, women without a violent offense who engaged in self-reported violence (self-reported violence only; n = 52); Group C, women convicted of a violent offense, without being involved in self-reported violence (isolated violence; n = 172); and Group D, women convicted of a violent offense and engaging in self-reported violent behavior in the community (patterned violence; n = 193). Table 3 describes the variations of SAQ scores across these four groups.

Table 3.

Comparison of SAQ Scores Across Four Groups Based on Offense Type and Self-Reported Violence.

	Nonviolent conviction		Violent conviction
	No violence (n = 126)	Self-reported violence (n = 52)	Isolated violence (n = 172)	Patterned violence (n = 193)
SAQ subscales	M	M	M	M	F(3, 539)
Criminal tendencies	7.84_a	11.58_b	7.70_a	11.54_b	31.70**
Conduct problems	4.62_a	9.00_b	4.84_a	8.55_b	35.25**
Criminal history	1.86_a	2.90_b	1.56_a	2.56_b	18.51**
Alcohol and drug abuse	3.33_b	4.58_c	2.61_a	4.08_c	18.39**
Antisocial associate	1.48_a	1.83	1.53_a	1.81_b	4.39**
Antisocial personality problem	1.66_b	2.46_c	1.16_a	2.22_c	25.21**
SAQ total	20.79_a	32.35_b	19.41_a	30.77_b	45.39**

Note. Means in the same row that do not share subscripts differ at p < .05 in the Tukey honestly significant difference (HSD) comparison. A mean that does not have a subscript does not differ from any other means at p < .05 in the Tukey HSD comparison. SAQ = Self-Appraisal Questionnaire.

p < .01.

On most subscales and on the total SAQ score, women in the no-violence group (A) and isolated violence group (C) were more similar to each other and statistically different from women in the self-reported violence (B) and patterned violence (D) groups, who were more like each other. In fact, women in the isolated violence group were lower on two of the subscales (alcohol and drug abuse and antisocial personality) than women in the no-violence (A) group.

To answer our third research question regarding the utility of the SAQ in appropriately assigning women to institutional programming based on their assessed needs, we began by examining the published cut scores indicating need for programming and then determined the proportion of women in each of the four previously defined groups that met those cut score criteria. Individuals, who have subscale scores within the cut score range, should be assigned to an intervention aimed at that particular need. Table 4 illustrates the proportion of women in each violence group who would be designated for institutional programming based on an individual’s total scores and the published cut scores. The published cut scores for the SAQ (Loza, 2005) associated with each subscale are contained in the first column of Table 4. In total, more than half of the women demonstrated a need for intervention around their antisocial associates (n = 308, 57%; e.g., programs that deal with peer pressure) and addiction (n = 272, 50% at or above elevated risk range on alcohol and drug abuse subscale). Furthermore, 40% of all women (n = 217) needed individual counseling to explore risk factors associated with their criminal history and a third (32%) needed interventions related to conduct problems and antisocial personality. Only one out of five women (n = 118, 22%) would be recommended to programming for criminal tendencies.

Table 4.

Proportions of Women Meeting Established Cut Scores Denoting Need for Treatment Across Four Groups.

		(A) No violence	(B) Self-reported violence	(C) Isolated violence	(D) Patterned violence	Total across groups	$χ_{d i f f}^{2}$
SAQ subscales	Cut score range^a	%	%	%	%	%	(3, N = 543)	Group difference^b
Criminal tendencies	14-17	8.7	36.5	9.3	37.3	21.7	62.37***	A = C < B = D
Conduct problems	9-18	15.1	50.0	22.1	47.7	32.2	53.64***	A = C < B = D
Criminal history	3-6	37.3	67.3	22.1	50.3	40.0	48.00***	C < A < B, C < D
Alcohol and drug abuse	4-8	51.6	75.0	32.6	58.0	50.1	39.03***	C < D < B, C < D
Antisocial associates	2-3	52.4	61.5	54.1	60.6	56.7	3.15	No difference
Antisocial personality problem	3-5	25.4	55.8	14.0	46.6	32.2	60.52***	A = C < B = D

Note. SAQ = Self-Appraisal Questionnaire.

Elevated score range indicating treatment need in each subscale (Loza, 2005).

Differences in proportions of women with treatment needs among multiple groups are significant at p < .05 in the Marascuilo procedure (Marascuilo & McSweeney, 1977).

***

p < .001.

Examining the need for intervention across the four violence groups, women in the no-violence (Group A) and isolated violence (Group C) groups were more alike in their need for programming, significantly differing from both groups engaging in self-reported violence (Groups B and D)—that were more like each other. In fact, on every subscale except antisocial associates, there were highly significant differences between groups. However, it should also be noted that at least half of the women in the patterned violence group (Group D) do not meet cut score criteria to receive intervention based on their scores on four out of six subscales (criminal tendencies, conduct disorders, criminal history, antisocial personality problem).

Because higher SAQ total scores indicate greater level of risk for recidivism, our final examination uses the SAQ total score to determine membership in a risk category: Low Risk (0-11), Low/Moderate Risk (11-22), High-Moderate (23-42), and High (43-67). The SAQ differentiates risk categories across the four violence groups (A-D), with a greater proportion of women in groups B and D in the highest risk category—significantly differing from women in Groups A and C who were more likely to be in low or low/moderate category of risk (see Figure 2). Women in the patterned violence group (D) demonstrated the most elevated risk with 21% meeting criteria for the highest risk group compared with 14% in the uncaught violence group (B) and 2% in the no-violence (A) and isolated violence (C) groups.

Figure 2.

Proportions of women in each risk category across four groups (A-D).

Discussion

The main purpose of the current study was to assess the utility of the SAQ in differentiating risk within the institutional setting and to examine the reliability of this tool in assessing the needs of women. To determine whether the SAQ differentiated risk among incarcerated women, we initially grouped them based on the presence/absence of an assaultive offense. These distinctions are often made within correctional facilities, particularly as to the need for programming (e.g., Assaultive Offender Programming). However, the actuarial assessment using the SAQ based on this distinction found only one significant difference (i.e., conduct disorders) across all of the subscale and total scores between groups. The conduct disorder subscale assesses violence prior to age 15 with queries that assess aggression in school or toward family members. Although it seems likely that early signs of aggression would differentiate violent versus nonviolent offenders similar to studies of males (Glover, Nicholson, Hemmati, Bernfeld, & Quinsey, 2002), it seems unlikely that this would be the only risk factor identified, particularly when studies of males show several significant differences on multiple SAQ subscales between violent and nonviolent offenders (Loza, Conley, & Warren, 2004; Loza et al., 2000).

In subsequent analyses, using self-reported violent behavior in the year prior to incarceration as a predictor rather than the “caught” behavior implied by the conviction, the validity of the SAQ increases as demonstrated by the AUC of the ROC analysis. The success of the SAQ in differentiating those with violent behaviors suggests sensitivity to the underlying risk factors, as compared with the obviously known violence associated with an assaultive conviction. Using both points of information, self-reported violent behavior as well as conviction for a violent offense, allows a more precise differentiation among four groups, further substantiating that the use of a formal conviction on a violent offense alone may be inconclusive in predicting future risk. In fact, women who were convicted of a violent offense and did not report any other violent behavior (Group C, isolated violence) may represent a group that embodies few risks of repeat offending, violent or otherwise.

This finding mirrors a recent conclusion in a study using mental health and personality attributes to differentiate women using the same dichotomy of violent/nonviolent offense; there were few differences between groups in measures of anger, serious mental illness, and impulsivity when using conviction type only compared with self-reported violent behaviors (see Kubiak et al., 2013 ). This suggests that simple categorization based on offense—whether by criminal justice personnel or researchers—may mask important within-group differences that may be salient to understanding women’s use of violence as well as their criminogenic risk related to recidivism. Women who have committed a violent offense but who have no recent history of other convicted or unconvicted violence reflect a profile that is much more similar to women with no violence history, and as such, may pose minimal risk of violent or nonviolent recidivism. Interestingly, SAQ subscales and total risk scores seemed to discern these nuance as the risk scores were significantly lower for women in the no-violence (Group A) and isolated violence (Group C) groups.

Although the overall internal consistency of the summative total score of the SAQ was high (.92), the lower internal consistency of some of the SAQ subscales is problematic. Three of the SAQ subscales (criminal history, antisocial associate, and antisocial personality problem) demonstrated lower than commonly acceptable standards of .7 and above (Nunnally, 1978). Loza and his colleagues (Loza et al., 2000; Loza, Conley, & Warren, 2004; Loza et al., 2005) found similar issues in studies of both males and females and pointed to the relatively few items in some of the subscales and small sample sizes as possible explanations for low internal consistency. That our findings replicate these problems may point to the need for further developmental work on the instrument and may be one of the reasons for problems related to the ability of the measure to indicate a need for treatment or programming interventions.

As stated previously, SAQ subscales measure the individualized treatment needs related to specific risk factors. Although there were significant differences in the proportions of women who met current published cut scores across groups, many of the women with the highest propensity for violence did not meet cut score criteria that would denote a need for programming/treatment. On one hand, these results suggest that the SAQ differentiates the need for services and that this differentiation was not related to offense type (violent vs. nonviolent). Almost half of the women who were involved in patterned violent behaviors needed treatment services associated with antisocial personality and criminal tendencies compared with one out of five women in the isolated violence group. On the other hand, the study suggests that these cut scores, validated in studies of male offenders, may need to be adjusted for females. The proportions of women categorized as needing services, based exclusively on cut scores, did not appear to coincide with the need for intervention. For example, only one third of women in the patterned violence group (D) were identified as needing services for criminal tendencies, although it appears that these women repeatedly engaged in violent behavior. This argues for a modification to the SAQ subscale cut scores for women so that treatment designation might more accurately reflect the need for treatment among women with histories of assaultive offenses. Distinctive characteristics differentiate the use of violence among female offenders as compared with male offenders (Collins, 2010), and between violent and nonviolent female offenders (Kubiak et al., 2013; Pollock & Davis, 2005). Violent criminal history has been found as a predictor of violent recidivism with males but was not the case with females (Collins, 2010). In addition, differences between male violent and nonviolent recidivists included factors such as antisocial personality, use of alcohol, and being unmarried. However, Pollock and Davis (2005) and Pollock et al. (2006) found that women with a history of past victimization, substance abuse, and certain personality attributes were more likely to commit violent offenses or engaged in self-reported violent behavior as compared with other incarcerated women. This suggests that both between- and within-group differences for incarcerated women would require a more customized approach for risk and treatment assessment.

Despite these shortcomings, the SAQ is promising because it includes dynamic risk factors, proven to be better in predicting validity for violent recidivism as compared with static risk items alone (Campbell et al., 2009). Moreover, among self-report measures, the SAQ was deemed promising in a comparison of risk instruments in predicting future violence, although self-report measures have less predictive validity than those based on file-and-interview methods of assessment (Campbell et al., 2009). However, other research argues that self-report assessments can have comparable predictive validity if they are content-relevant (Walters, 2006) and may be more efficient when taking into consideration costs related to staff time, resulting in a more prudent method of assessment.

In addition to efficiencies offered through the use of self-report actuarial method, there may be other factors associated with their use and effectiveness. For example, there is little known about decision making across parole boards as it pertains to women who have committed violent offenses (see Hannah-Moffat & Yule, 2011), but there are some studies that point to a negative gender bias for women who violate the gender norm of nonviolence (Comack & Balfour, 2004; Erez, 1992; Gilbert, 2002; Silverstein, 2006). The use of a data-driven process to assess risk of violent re-offending may offer little comfort to parole board members weighing decisions regarding public safety on community re-entry. Certainly, studies supporting the validity of such self-report instruments may increase assurances that such a tool can provide an unbiased assessment of risk and facilitate decisions regarding prison release and parole readiness.

This study provides some assurances in assessing risk among female offenders; however, it is not without limitation. This analysis relies purely on self-report as both the SAQ and the measure of uncaught violence use this method on inquiry. Although self-report methods for measuring undetected criminal behavior appear to have acceptable validity, there is still considerable underreporting (Thornberry & Krohn, 2000). Self-report can offer objectivity by avoiding the possible misinterpretations by a third person and illusory correlation, which means that one may see relationships when no such relationship exists (Chapman & Chapman, 1969). Although all self-reports are vulnerable to deliberate bias and misrepresentation by participants (Stone et al., 1999), researchers used several strategies to minimize this risk. These strategies include the use of an anonymous survey that cannot be linked back to the individual, the selection of a random sample, and the utilization of an external university-based research team rather than an internal department of corrections’ study, which may have enhanced the reliability of the responses. The high proportion of women who reported engaging in self-reported violent behavior (45%) may be one indication of honesty among the participants.

Conclusion

Because the adage that previous violence predicts future violence is generally true (Andrade et al., 2009; Collins, 2010), administrators in jails and prisons often distinguish between those with and without violent/assaultive offenses as one mechanism to assist them with decisions regarding security level, programming, and release. In contrast to these practices, recent studies have found that a previous violent offense is not a strong predictor of future violence in women (Collins, 2010; Deschenes et al., 2007). Unfortunately, there has been little attention to predicting risk of violence, within the institution or in the community, among women.

As the first step in a longitudinal study using the SAQ, we found that this actuarial instrument was able to differentiate risk among incarcerated women when using two indicators of violence simultaneously—violent conviction and self-reported violence. Initially, the SAQ poorly differentiated risk with the use of the conviction only, finding very few differences between those with and without violent offense on the majority of subscales or overall score. It appears that women with a violent conviction comprised distinct groups—those whose use of violence was isolated incident and those whose use of violence was more patterned. Those whose use of violence was isolated were much more similar on SAQ scores to women who had neither a violent conviction nor self-reported violence. This suggests that some women with convictions for violent offenses pose a minimal risk of recidivism. Conversely, we found some women without a conviction for a violent offense who were involved in self-reported violent behavior seemed to be at higher risk for recidivism based on SAQ scores. These women might benefit from a preventive intervention or programming that their nonviolent offense status probably does not allow them to access within an institutional setting.

Although the use of the SAQ can assist prison staff and parole board members with decisions regarding risk, modifications are required to recalibrate the cut scores that assess the need for services within prison. Current cut scores negate referrals for many high-risk women, and the absence of appropriate intervention services will do little to diminish a high-risk status, perhaps increasing lengths of incarceration or failing to equip women with the skills necessary for successful community re-entry.

Longitudinal research is underway that will inform the predictive validity of the SAQ in determining recidivism. In addition, further research is required to validate the utility of the SAQ in predicting recidivism—violent or otherwise—on various subpopulations of female offenders, including women of minority status or those entering the system as youthful offenders.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received financial support for the research through a state contract.

Author Biographies

Sheryl Pimlott Kubiak is a professor of social work, Michigan State University. Her research examines the intersections of mental health, substance abuse, and criminal justice at individual and systems levels.

Woo Jong Kim is a PhD student in the Michigan State University Social Work program. Her research interests include issues of social policy, especially for families of young children, and those relevant to social work with incarcerated women, such as risk assessment.

Deborah Bybee is professor of ecological/community psychology, Michigan State University, specializing in research design and statistical methodology. Her substantive interests include violence against women, trauma, and mental health.

Lee Eshelman is a PhD student in the Clinical Psychology program at Miami University. Her research interests include the long-term outcomes of sexual assault, domestic violence, and child abuse.

References

Andrade

O’Neill

Diener

R. B.

(2009). Violence risk assessment and risk management: A historical overview and clinical application. In Andrade

J. T.

(Ed.), Handbook of violence risk assessment and treatment: New approaches for mental health professionals (pp. 3-40). New York, NY: Springer.

Andrews

D. A.

Bonta

J. A.

(2001). Level of Service Inventory–Revised (LSI-R): User’s manual. North Tonawanda, NY: Multi-Health Systems.

Andrews

D. A.

Bonta

J. A.

(2006). The psychology of criminal conduct (4th ed.). Cincinnati, OH: Anderson.

Andrews

D. A.

Bonta

J. A.

Wormith

J. S.

(2006). The recent past and near future of risk and/or need assessment. Crime & Delinquency, 52, 7-27.

Anglin

D. M.

Hser

(1987). Addicted women and crime. Criminology, 25, 359-397.

Bonta

J. A.

Law

Hanson

R. K.

(1998). The prediction of criminal and violent recidivism among mentally disordered offenders: A meta-analysis. Psychological Bulletin, 123, 123-142.

Brennan

Dieterich

Ehret

(2009). Evaluating the predictive validity of the COMPAS risk needs assessment system. Criminal Justice and Behavior, 36, 21-40.

Brennan

Dieterich

Oliver

W. L.

(2006). COMPAS: Technical manual and psychometric report version 5.0. Traverse City, MI: Northpointe Institute.

Campbell

M. A.

French

Gendreau

(2009). The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment. Criminal Justice and Behavior, 36, 567-590.

10.

Chapman

L. J.

Chapman

J. P.

(1969). Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. Journal of Abnormal Psychology, 74, 271-280.

11.

Chermack

S. T.

Murray

R. L.

Winters

J. J.

Walton

M. A.

Booth

B. M.

Blow

F. C.

(2009). Treatment needs of men and women with violence problems in substance use disorder treatment. Substance Use & Misuse, 44, 1236-1262.

12.

Collins

R. E.

(2010). Effect of gender on violent and nonviolent recidivism: A meta-analysis. Journal of Criminal Justice, 38, 675-684.

13.

Comack

Balfour

(2004). The power to criminalize: Violence, inequality and the law. Halifax, Nova Scotia, Canada: Fernwood Publishing.

14.

Deschenes

E. P.

Owen

Crow

(2007). Recidivism among female prisoners: Secondary analysis of the 1994 BJS recidivism data set. NCJ 216950, NIJ Sponsored,Report for the U.S. Department of Justice, Washington, D.C..

15.

Erez

(1992). Dangerous men, evil women: Gender and parole decision-making. Justice Quarterly, 9, 105-126.

16.

Gilbert

P. R.

(2002). Discourses of female violence and societal gender stereotypes. Violence Against Women, 8, 1271-1300.

17.

Glover

A. J. J.

Nicholson

D. E.

Hemmati

Bernfeld

G. A.

Quinsey

V. L.

(2002). A comparison of predictors of general and violence recidivism among high-risk federal offenders. Criminal Justice and Behavior, 29, 235-249.

18.

Goetting

(1988). Patterns of homicide among women. Journal of Interpersonal Violence, 3, 3-20.

19.

Hannah-Moffat

(2009). Gridlock or mutability: Reconsidering “gender” and risk assessment. Criminology & Public Policy, 8, 209-219.

20.

Hannah-Moffat

Yule

(2011). Gaining insight, changing attitudes and managing release decisions for women convicted of violent crimes. Punishment & Society, 13, 149-175.

21.

Holtfreter

Cupp

(2007). Gender and risk assessment: The empirical status of the LSI-R for women. Journal of Contemporary Criminal Justice, 23, 363-382.

22.

Kruttschnitt

(2002). Women’s involvement in serious interpersonal violence. Aggression and Violent Behavior, 7, 529-565.

23.

Kubiak

S. P.

Kim

W. J.

Fedock

Bybee

(2013). Differences among incarcerated women with assaultive offenses: Isolated versus patterned use of violence. Journal of Interpersonal Violence, 28, 2462-2490.

24.

Lasko

T. A.

Bhagwat

J. G.

Zou

K. H.

Ohno-Machado

(2005). The use of receiver operating characteristic curves in biomedical informatics. Journal of Biomedical Informatics, 38, 404-415.

25.

Lazowski

L. E.

Miller

F. G.

Boye

M. W.

Miller

G. A.

(1998). Efficacy of the Substance Abuse Subtle Screening Inventory–3 (SASSI-3) in identifying substance dependence disorders in clinical settings. Journal of Personality Assessment, 71, 114-128.

26.

Little

R. J. A.

Rubin

D. B.

(2002). Statistical analysis with missing data (2nd ed.). Hoboken, NJ: John Wiley.

27.

Loza

(2005). Self-Appraisal Questionnaire. Toronto, Ontario, Canada: Multi-Health Systems.

28.

Loza

Conley

Warren

(2004). Concurrent cross-validation of the Self-Appraisal Questionnaire: A tool for assessing violent and nonviolent recidivism and institutional adjustment on a sample of North Carolina offenders. International journal of offender therapy and comparative criminology, 48, 85-95.

29.

Loza

Cumbleton

Shahinfar

Neo

L. H.

Evans

Conley

(2004). Cross-validation of the Self-Appraisal Questionnaire (SAQ): An offender risk assessment measure on Australian, British, Canadian, North Carolinian, Pennsylvanian, and Singaporeans offenders. Journal of Interpersonal Violence, 18, 1-18.

30.

Loza

Dhaliwal

Kroner

D. G.

Loza-Fanous

(2000). Reliability, construct, and concurrent validities of the Self-Appraisal Questionnaire: A tool for assessing violent and nonviolent recidivism. Criminal Justice and Behavior, 27, 356-374.

31.

Loza

Green

(2003). The Self-Appraisal Questionnaire: A self-report measure for predicting recidivism versus clinician-administered measures: A 5-year follow-up study. Journal of Interpersonal Violence, 18, 781-797.

32.

Loza

Loza-Fanous

(1999a). Anger and prediction of violent and nonviolent offenders’ recidivism. Journal of Interpersonal Violence, 14, 1014-1029.

33.

Loza

Loza-Fanous

(1999b). The fallacy of reducing rape and violent recidivism by treating anger. International Journal of Offender Therapy and Comparative Criminology, 43, 492-502.

34.

Loza

Loza-Fanous

(2002). The effectiveness of the Self-Appraisal Questionnaire as an offenders classification measure. Journal of Interpersonal Violence, 17, 3-13.

35.

Loza

Neo

L. H.

Shahinfar

Loza-Fanous

(2005). Cross-validation of the self-appraisal questionnaire: A tool for assessing violent and nonviolent recidivism with female offenders. International Journal of Offender Therapy and Comparative Criminology, 49, 547-560.

36.

Manchak

S. M.

Skeem

J. L.

Douglas

K. S.

Siranosian

(2009). Does gender moderate the predictive utility of the Level of Service Inventory–Revised (LSI-R) for serious violent offenders? Criminal Justice and Behavior, 36, 425-442.

37.

Mann

(1990). Female homicide and substance abuse: Is there a connection? Women and Criminal Justice, 1, 87-109.

38.

Mann

(1996). When women kill. Albany: State University of New York Albany Press.

39.

Marascuilo

L. A.

McSweeney

(1977). Nonparametric and distribution-free methods for the social sciences. Monterey, CA: Brooks/Cole Publishing.

40.

Morash

(2009). A great debate over using the Level of Service Inventory–Revised (LSI-R) with women offenders. Criminology and Public Policy, 8, 173-181.

41.

Murray

R. L.

Chermack

S. T.

Walton

M. A.

Winters

Booth

B. M.

Blow

F. C.

(2008). Psychological aggression, physical aggression, and injury in nonpartner relationship among men and women in treatment for substance-use disorders. Journal of Studies on Alcohol and Drugs, 69, 896-905.

42.

Nunnally

J. C.

(1978). Psychometric theory (2nd ed.). New York, NY: McGraw-Hill.

43.

Pollock

J. M.

Davis

S. M.

(2005). The continuing myth of the violent female offender. Criminal Justice Review, 30(5), 5-28.

44.

Pollock

J. M.

Mullings

J. L.

Crouch

B. M.

(2006). Violent women: Findings from the Texas women inmates study. Journal of Interpersonal Violence, 21, 485-502.

45.

Schwartz

Steffensmeier

D. J.

Feldmeyer

(2009). Assessing trends in women’s violence via data triangulation: Arrests, convictions, incarcerations, and victim reports. Social Problems, 56, 494-525.

46.

Shute

(2007). Parole and risk assessment. In Padefield

(Ed.), Who to release? Parole, fairness and criminal justice (pp. 21-42). Abingdon, UK: Willan Publishing.

47.

Silverstein

(2006). Justice in genderland: Through a parole looking glass. Symbolic Interaction, 29, 393-410.

48.

Smith

Cullen

F. T.

Latessa

E. J.

(2009). Can 14,737 women be wrong? A meta-analysis of the LSI-R and recidivism for female offenders. Criminology & Public Policy, 8, 183-208.

49.

Steffensmeier

D. J.

Schwartz

Zhong

Ackerman

(2005). An assessment of recent trends in girls’ violence using diverse longitudinal sources: Is the gender gap closing? Criminology, 43, 355-406.

50.

Stone

A. A.

Turkkan

J. S.

Bachrach

C. A.

Jobe

J. B.

Kurtzman

H. S.

Cain

V. S.

(1999). The science of self-report: Implications for research and practice. Mahwah, NJ: Lawrence Erlbaum.

51.

Thornberry

Krohn

(2000). The self report method for measuring delinquency and crime. In Duffee

(Ed.), Measurement and analysis of crime and justice (pp. 33-84). Washington, DC: U.S. Department of Justice.

52.

Uniform Crime Reports. (2009). Crime in the United States, 2008. Washington, DC: U.S. Department of Justice, Federal Bureau of Investigation.

53.

Van Voorhis

Wright

E. M.

Salisbury

Bauman

(2010). Women’s risk factors and their contributions to existing risk/needs assessment: The current status of a gender-responsive supplement. Criminal Justice and Behavior, 37, 261-288.

54.

Walters

G. D.

(2006). Risk-appraisal versus self-report in the prediction of criminal justice outcomes: A meta-analysis. Criminal Justice and Behavior, 33, 279-304.