Assessment of Risk for Seclusion Among Forensic Inpatients: Validation and Modification of the Risk of Administrative Segregation Tool (RAST)

Abstract

Seclusion is used in psychiatric care to protect patients and staff or to manage aggression but may have adverse effects. The ability to identify at-risk patients could help reduce seclusion. This study tested the Risk of Administrative Segregation Tool’s (RAST) ability to predict any seclusions among 229 male forensic inpatients followed for up to 1 year of hospitalization, and days spent secluded, controlling for length of stay. RAST scores were lower than in correctional samples. The RAST did not predict seclusions in Year 1, but modification of three items to fit the forensic population (RAST-F) offered a small improvement. Among 62 patients hospitalized for more than 1 year, the RAST significantly predicted seclusions in Year 2, and the modifications improved prediction. The present modest findings support the RAST’s potential to help identify patients most in need of clinical efforts to avert seclusion. Replication in larger samples, including female patients, is needed.

Keywords

forensic mental health risk seclusion segregation

Seclusion in psychiatric care has been defined as “the involuntary confinement of a patient alone in a room or area from which the patient is physically prevented from leaving” (U.S. Department of Health and Human Services, cited by Knox & Holloman, 2012, pp. 35-36). Patients can be secluded because of aggressive or self-injurious behavior. They may also be secluded pre-emptively, to protect the patient or others (e.g., Bergk, Einsiedler, Flammer, & Steinert, 2011; Van Der Merwe, Muir-Cochrane, Jones, Tziggili, & Bowers, 2013; Velpry & Eyraud, 2014). As a result, the most common reason given for implementing seclusion is patient agitation or disorientation (e.g., 42%, Kaltiala-Heino, Tuohimäki, Korkeila, & Lehtinen, 2003; 37%-46%, Keski-Valkama et al., 2010), rather than overt aggression. Yet patients experience seclusion as coercive and potentially traumatizing (e.g., Muskett, 2014; Van Der Merwe et al., 2013), and poor technique or monitoring has resulted in patient injuries and even death during seclusion (e.g., Busch & Shore, 2000). These concerns have led to widespread attempts to reduce psychiatric seclusion (e.g., Knox & Holloman, 2012, and see reviews by Muskett, 2014; Scanlan, 2010; and Stewart, Van Der Merwe, Bowers, Simpson, & Jones, 2010). Programs developed with the aim of reducing seclusion have reported some success (e.g., Fletcher et al., 2017).

The ability to identify at-risk patients could aid efforts to reduce seclusion, if tied to proactive alternatives to managing problematic behavior. Identifying at-risk individuals is foundational to the Risk–Need–Responsivity (RNR) principles of effective correctional intervention, and an important first step to implementing strategies to reduce risk (e.g., Bonta & Andrews, 2016). Individual risk factors for seclusion among adult psychiatric patients include male gender (c.f. Cullen et al., 2018), younger age, psychotic disorder, prior seclusions, legal detention, and violence risk (e.g., Bullock, McKenna, Kelly, Furness, & Tacey, 2014; Cullen et al., 2018; Luckhoff et al., 2013; McKenna, McEvedy, Maguire, Ryan, & Furness, 2017; Noorthoorn et al., 2015). Surveys of psychiatric staff reveal that they sometimes justify seclusion on grounds that it is thought to have a therapeutic effect on patients, or is necessary to maintain a therapeutic milieu (Van Der Merwe et al., 2013; Velpry & Eyraud, 2014). Use of seclusion for symptom management is relatively common in forensic units, that is, psychiatric units serving patients undergoing court-ordered assessment or treatment. McKenna and colleagues reported that 62% of seclusions on forensic units were attributed to symptom management, compared with 47% of seclusions in general psychiatric units. Furthermore, they observed that seclusions in forensic units lasted significantly longer (median 24 hr 18 min) than in general psychiatric care (median 16 hr 42 min). A significant difference in the likelihood of seclusion has also been reported, with 14% of forensic inpatients and 7% of civil inpatients being confined to a room (Seto, Harris, & Rice, 2004). Thus, there is value in assessing patients’ risk of seclusion, particularly in forensic settings.

We are aware of no previous study of statistical prediction of seclusion among forensic patients. Among correctional inmates, research has been conducted to identify the risk of segregation, imposed when an inmate’s behavior jeopardizes the safety or security of the inmate, other inmates, or the institution (Helmus, 2015). An actuarial instrument, the Risk of Administrative Segregation Tool (RAST), was constructed and cross-validated on a total sample of 16,701 male or female offenders incarcerated with custodial sentences of at least 2 years (Helmus, 2015). The outcome of interest was segregation for at least 6 consecutive days, to identify offenders at risk of remaining in segregation after a mandatory 5-day review (Helmus, 2015, p. 89). In the development sample of 11,110 inmates, 24% of offenders met this outcome criterion. Six RAST items were selected (see Table 1) from a large pool of more than 400 variables based on their predictive accuracy as well as face validity, ease of scoring, and availability of information within the Canadian federal corrections system. The RAST was validated on a separate sample of 5,501 inmates, with a large predictive effect (area under the curve [AUC] = .80) and adequate calibration with the development sample. The RAST total score has a potential range of 0 to 13, although in development, the highest observed score was 12 and the mean score was 6.5 (SD = 2.30). Inter-rater reliability of the RAST was not examined.

Table 1.

RAST, RAST-F, Mental Health Measures, and VRAG Score in Full Sample (N = 229) and Subsample of Patients Hospitalized More Than 1 Year (N = 62).

Variable	Full sample				Subsample
Variable	M	SD	N	%	M	SD	N	%
RAST items
1. Age category (0-3)	1.70	0.93			1.68	0.95
0 = 50 or older			32	14			9	15
1 = 40 to 49.9			46	20			14	23
2 = 25 to 39.9			109	48			27	44
3 = under 25			42	18			12	19
2. Prior conviction sentencing dates (0-2)	0.86	0.87			0.90	0.86
0 = 0 to 1			104	45			26	42
1 = 2 to 4			53	23			16	26
2 = 5 or more			72	31			20	32
3. Admission to administrative segregation in prior federal sentence (0-2)	0.02	0.14			0.00	0.00
0 = no, or no federal sentences			225	98			58	94
1 = yes			4	2			4	7
4. Sentence length for index offense (where applicable) (0-4)	0.00	0.00			0.00	—
0 = 2 years			4	2			1	2
2 = more than 2 years but less than 3 years			0	0			0	0
3 = at least 3 years but less than 10 years			0	0			0	0
4 = 10 years or more (including indeterminate)			0	0			0	0
5. Criminal versatility in current convictions, number of offense type categories (0-2)	0.01	0.09			0.02	0.13
0 = 0 categories			227	99			61	98
1 = 1 to 2 categories			2	1			1	2
2 = at least 3 categories			0	0			0	0
6. Prior conviction for violence (0-1)	0.55	0.50			0.55	0.50
0 = no			103	45			28	45
1 = yes			126	55			34	54
RAST total score (0-13)	3.13	1.50			3.11	1.36
RAST-F modified items
3. Seclusion in previous forensic hospitalization due to behavior or presentation	0.21	0.41			0.42	0.50
0 = no			182	79			36	58
1 = yes			47	21			26	42
4. Transferred from another psychiatric hospital	0.13	0.34			0.40	0.50
0 = no			199	87			37	60
1 = yes			30	13			25	40
5. Criminal versatility in current charges, number of offense type categories (0-2)	0.97	0.61			0.87	0.67
0 = 0 categories			46	20			18	29
1 = 1-2 categories			144	63			34	55
2 = at least 3 categories			39	17			10	16
RAST-F total score (0-10)	4.42	1.88			4.79	1.90
Diagnosis (yes/no)
Schizophrenia			136	59			37	60
Any psychosis			178	78			55	89
Mood disorder			25	11			2	3
Personality disorder			31	14			12	19
PIC problem counts
Psychosis symptoms (0-6)	2.71	1.97			2.15	2.05
Institutional management problems (0-18)	2.60	2.56			3.02	2.75
Mood problems (0-7)	1.81	1.47			1.45	1.22
Social withdrawal (0-7)	1.35	1.25			1.42	1.21
VRAG score (−26 to +38)	6.56	12.12			6.41	11.64

Note. RAST = Risk of Administrative Segregation Tool; VRAG = Violence Risk Appraisal Guide; PIC = Problem Identification Checklist.

Correctional and forensic populations have some similarities. Many persons who undergo pre-trial psychiatric assessments in forensic hospitals are subsequently convicted (Rice, Harris, Lang, & Bell, 1990), and at the same time, psychiatric disorders are common among correctional inmates (Correctional Service Canada, 2014; although often undetected, for example, Martin, Colman, Simpson, & McKenzie, 2013). Offenders found not criminally responsible on account of mental disorder, and therefore seen in forensic units, are more likely than convicted offenders to have previous psychiatric admissions and to be diagnosed with a serious mental illness involving psychosis, whereas convicted offenders are more likely to have alcohol problems and to be diagnosed with a personality disorder (Rice et al., 1990).

There are also both similarities and differences between correctional segregation and psychiatric seclusion. In Canadian corrections, administrative segregation is invoked when the safety and security of the individual, other persons, or the environment are of concern, and it is not necessary that the inmate has already acted in a manner that jeopardizes security (e.g., Corrections and Conditional Release Act, 1992, S. 31 (3)). Psychiatric seclusion, too, may often be used proactively, such as when inpatients are agitated rather than overtly aggressive (Kaltiala-Heino et al., 2003; 37-46%, Keski-Valkama et al., 2010). However, correctional segregation has legally defined uses, limits, and standards (Corrections and Conditional Release Act, 1992, S. 31-37), whereas psychiatric seclusion is governed by local hospital policies, although attempts have been made to standardize its practice through position statements by professional health care provider associations (e.g., Chaimowitz, 2011). Furthermore, and prescribed limits notwithstanding, correctional segregation appears to last longer, on average, than psychiatric seclusion: whereas 71% of correctional segregations last less than 30 days (Public Safety Canada, 2017), McKenna and colleagues (2017) reported that 62% of psychiatric seclusions last less than 1 day.

For these reasons, it cannot be assumed that segregation and seclusion are the same phenomena or that a procedure to identify risk of administrative segregation in corrections will necessarily be useful in the management of psychiatric seclusion. Therefore, we conducted the present study to evaluate the predictive utility of the RAST for seclusion of forensic psychiatric patients. Furthermore, the database used by Helmus (2015) had limited information on mental health variables, precluding thorough examination of the contribution of mental health variables, particularly to the incremental prediction of segregation. Therefore, we tested the incremental contribution of mental health symptoms and behaviors, gathered from patients’ medical records. In addition, because existing research reveals that psychiatric seclusions may be used in response to violent behavior, or concerns about potential violence, we also tested the Violence Risk Appraisal Guide (VRAG; for example, Harris, Rice, Quinsey, & Cormier, 2015). The VRAG, an actuarial tool originally developed to predict violent recidivism among men undergoing inpatient forensic assessment, is also associated with institutional violence (e.g., Fitzgerald et al., 2013; Vitacco, Gonsalves, Tomony, Smith, & Lishner, 2012; though see Hogan & Olver, 2016). Our rationale was that, if an existing tool routinely used to assess forensic patients’ risk of violence was also associated with risk of seclusion during their hospitalization, then the RAST would have to show incremental validity over this tool to be of practical value. Similarly, we examined a measure of clinical symptoms and problematic behavior, the Problem Identification Checklist (PIC) that has been shown to predict clinical and forensic outcomes, to see whether the RAST could show incremental validity over this tool.

Present Study

In the present study, we examined predictors of seclusion among men admitted to inpatient forensic assessment at one psychiatric hospital. We tested the RAST’s ability to predict any seclusion and number of days in seclusion among male forensic inpatients during up to 1 year of hospitalization. Based on our familiarity with the forensic setting, we anticipated that some RAST items would have a low prevalence in a forensic patient sample; therefore, we planned also to examine a modified version of the RAST, altering items that pertained to custodial sentences and criminal convictions to be more relevant, and then comparing the two scores. For clarity, we refer to the modified RAST as the RAST–Forensic version (RAST-F). Among men hospitalized beyond 1 year, we tested the RAST and RAST-F’s prediction of seclusions during Year 2. Finally, we explored the potential incremental contribution of clinical symptoms and behaviors recorded during the first year of hospitalization to the prediction of seclusions during Year 2.

Method

Sample

Male forensic patients were sampled from all admissions to an inpatient assessment program in a forensic hospital in Ontario, Canada, from the years 2009 through 2012 (Hilton, Ham, & Green, 2018). All men (aged 16 or older) had been charged for at least one criminal offense that precipitated their current admission (the index offense). Admissions were most often for a court-ordered assessment of fitness to stand trial or eligibility for a defense of Not Criminally Responsible on account of mental disorder (NCR), lasting at least 1 month. These psycholegal questions are similar to those for competence to stand trial or so-called insanity defenses in other jurisdictions. Other patients were admitted after being found unfit or NCR, either directly from court or from other institutions. Our data did not include whether patients were subsequently convicted.

In a previous study of men admitted in 2009 or 2010, a subsample of 122 was identified that had sufficient documented information about physical health to permit analysis of changes in health over time (Hilton, Ham, Lang, & Harris, 2015). Having sufficient information meant that (a) the patient’s length of stay was at least one calendar month, (b) clinical reporting of patient progress (including medical health) was complete for that period of time and documented on the medical record. The present study sampled these 122 men and 111 additional patients admitted in 2011 or 2012 who met the same inclusion criteria. We excluded four patients whose medical records were inaccessible at the time of the present study due to legal inquiries, preventing coding of the RAST, resulting in a sample of 229 selected from a total admission cohort of 638 (36%). The patients in the present sample were significantly more likely to have been secluded in their first year of hospitalization than the rest of the admission cohort with available data (50% vs. 41%), χ² (1, 523) = 4.24, p = .040.

Measures

All data were extracted from the medical records. These records contained formal reports of the forensic assessment process, as well as extensive information gathered from criminal records, review boards and other sources, and routine clinical progress reports and observations documented at least weekly (and often more than daily). As reported in the following, we achieved good inter-rater reliability.

Seclusion

The primary dependent measure was seclusion during the first year of hospitalization, up to 365 days after admission or up to discharge, whichever occurred first. Among patients hospitalized beyond Year 1, we also coded seclusions in the second year of hospitalization, up to 730 days after admission or discharge within Year 2. Seclusion was defined as any documented occasion on which a patient was confined to a locked room because of his behavior. Seclusions may occur within the patient’s room or in a separate room designed to have no removable fixtures, padded walls, and an acrylic window (our available data did not distinguish between seclusions in regular and specially designed rooms). We excluded routine seclusion for assessment upon admission unless it exceeded 5 days (such extensions were normally due to the patient’s disruptive or aggressive behavior). We also excluded physical or chemical restraint in the absence of seclusion, and brief, planned confinements involving time-outs in patient rooms as part of written “individual management plans” that were not recorded as seclusion. We were particularly concerned to identify risk of prolonged seclusion in response to problematic patient behavior, rather than brief, planned seclusion, as the former represents a greater burden to human and material resources (e.g., need for continual observation of secluded patients, extraction of patients for room cleaning). McKenna and colleagues (2017) observed that most seclusions in psychiatric care lasted less than one day but longer among forensic patients. We counted the total duration of all seclusions in days, rounding up from 12 hr.

RAST

We first coded the six RAST items according to the scoring manual (Helmus, Johnson, & Harris, 2014). Item 1 (age at admission) was coded from the date of the current admission. Item 2 (prior convictions) was coded from a list of criminal charges and convictions maintained by a national police database. Item 3 (admission to administrative segregation in a previous federal sentence) included any such segregation in a federal correctional institution prior to the current admission, and was coded as “0” for patients without a prior federal sentence. Item 4 (current sentence length) included only federal sentences for the index offense, and was inapplicable (and scored 0) in all but four cases. Item 5 (criminal versatility in current convictions) was also based on the index offense or any current convictions (e.g., patients transferred from correctional institutions). This item was scored using seven categories of criminal charges: abduction/kidnapping, administration of justice (e.g., failure to comply), public order offences, weapons/explosives, assault (excluding homicide), robbery, and property offenses (Helmus et al., 2014). This item was scored “0” for any patient without a current conviction. Item 6 (prior conviction for violence) was coded from the criminal history and included convictions prior to the index offense that met the RAST criteria for violence (Helmus et al., 2014).

Modified RAST Items (RAST-F)

Consideration was given to ease of scoring and face validity in the original design of the RAST (Helmus, 2015), and we did the same in the present study in our modification of three items. This decision was made a priori and based on our clinical experience and familiarity with the criminal and psychosocial history typically contained within medical records in the study institution. Item 3 (federal segregation) did not apply to much of our sample and, for those patients who had a previous federal sentence, correctional reports were often not available or did not indicate whether segregation had occurred. Therefore, we created a substitute item, admission to seclusion in a previous forensic hospitalization. We scored this item as “1” if the patient had previously been admitted to a forensic psychiatric inpatient unit (including transfers from custodial institutions) and secluded because of his behavior or presentation, and “0” if the patient had no previous forensic admission or no seclusion because of behavior or presentation.

A second modification was made for Item 4 (current sentence length). We examined psychiatric transfer (n = 32), which we coded as “1” if the current admission was from another psychiatric hospital and “0” if not. A third modification was made for Item 5 (criminal versatility in current convictions). We coded charges for the index offense and scored the number of charge categories using the same criteria as in the original RAST item regarding convictions. The total RAST-F score had a potential range of 0 to 10.

Violence risk

The VRAG (Harris et al., 2015) is an actuarial instrument for the assessment of violence risk, developed and validated using samples of men admitted to an inpatient forensic assessment unit, including men who were subsequently convicted and men who were found NCR or who received psychiatric treatment. The VRAG has shown predictive accuracy across a range of samples, locations, and violent outcomes, including institutional violence and other misconducts that may lead to seclusion. For example, Harris and colleagues (2015) reviewed 10 studies of the VRAG’s ability to predict institutional violence and misconducts in hospital and correctional settings, reporting an average AUC of approximately .70, where .71 is considered a large effect size (Rice & Harris, 2005). The VRAG has a possible total score ranging from −26 to +38 and comprises 12 items, each with weighted scores contributing from 2 to 17 points: lived with both parents to age 16, elementary school maladjustment, alcohol problems, never married, nonviolent criminal history, failure on prior conditional release, age at index offense, victim injury, any female victim in index offense, personality disorder, schizophrenia, and psychopathy. We scored the VRAG from file information, including the criminal history and psychosocial history reports gathered for the forensic assessment. We used the scoring criteria as published in Harris et al. (2015), including item substitution or prorating for up to four items where needed; valid scores were available for 226 (99%) patients. For the psychopathy item, we did not assess patients ourselves but used the scores reported by clinical psychologists (in 86 cases, 35%) or substituted the Child and Adolescent Taxon Scale, as recommended by Harris et al. (2015), which we coded from information available in the medical record. The Violence Risk Appraisal Guide–Revised (VRAG-R; Rice, Harris, & Lang, 2013) was not available at the time of coding (see “Procedure”).

Clinical measures

Other independent measures concerned psychiatric diagnosis at admission and problem behaviors and symptoms exhibited during the hospital stay. We coded the diagnosis documented by the psychiatrist on completion of the forensic assessment, grouped into the following categories: schizophrenia, other psychosis, mood disorder, substance use disorder, personality disorder, paraphilia, or other (e.g., developmental disorder). Patients could have diagnoses in multiple categories, and we coded up to five diagnoses per patient. Diagnosis of schizophrenia was of particular interest because of its expected prevalence in this forensic psychiatric sample compared with correctional samples (e.g., Rice et al., 1990) and its association with symptoms potentially related to aggression or other problematic behavior that may lead to use of seclusion.

Problem behaviors and symptoms were coded using the PIC, a significant predictor of clinical and forensic outcomes in previous research in samples of psychiatric inpatients and outpatients (e.g., Harris et al., 2015; Nagi, Ostapiuk, Craig, Hacker, & Beech, 2009). The PIC is intended to be scored from clinical evidence during a 6-month period; we coded the PIC based on the 6 months preceding the patient’s anniversary date, or the total time in hospital if less than 6 months. The PIC has four scales, derived from factor analytic research with forensic inpatients (e.g., Rice, Harris, & Quinsey, 1996), which represent sets of problems relating to psychotic behaviors, institutional management problems, mood problems, and social withdrawal. In previous research, these scales have been coded with inter-rater reliability coefficients typically of .80 or over (e.g., Harris, Rice, & Quinsey, 1993; Quinsey, Coleman, Jones, & Altrows, 1997) and produced internal consistencies of .68 to .90 (Quinsey et al., 1997).

We simplified the scoring from that described in Harris and colleagues (2015) by summing the number of items present within each PIC scale to create a problem count. Items that applied were coded as 1, otherwise they were coded as 0. There are six psychotic behavior items: unusual thought content, hallucinatory behavior, conceptual disorganization, psychotic actions (e.g., stereotypes, bizarre mannerisms, obviously inappropriate laughter), inappropriate suspicion, and grandiosity. There are 18 institutional management problem items (e.g., impulsivity, criminal attitudes, medication noncompliance, poor self-care and personal hygiene). There are seven mood problem items—agitation, anxiety, mania, anger, blunted affect, depression, excessive feelings of guilt—and seven social withdrawal items—poor use of leisure time, unpopular, avoids contact with others, inactivity, excessive shyness, refusal to participate in nonmedical therapy, and preoccupation with staying in the institution.

Procedure

The hospital’s institutional ethics review board reviewed the research protocol and approved use of medical records without patient consent, according to the Tri-Council Policy Guidelines for waiver of consent (see http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/chapter3-chapitre3/#toc03-1b).

We collected research data in three stages. First, each patient’s medical record was retrieved shortly after admission in 2009-2012 to code information about his psychosocial history, psychological and risk assessments, current diagnosis, and presentation within the first 30 days of hospitalization, drawing on the extensive multidisciplinary assessment, daily care notes, and other information gathered as part of the forensic assessment. Admission coding for the present sample was conducted by an experienced research psychometrist (n = 165, 71%), the second author (n = 57, 25%), or a research student (n = 11, 5%) following training and evaluation of inter-rater reliability with the primary coder.

Second, the medical record was retrieved again after either discharge or the admission anniversary date to code information about the patient’s clinical presentation, behavior, and events (including seclusions) during the first year of hospitalization. This coding was conducted primarily by the second author (n = 54, 24%) or one of six research students between 2009 and 2017, each of whom was trained and supervised by the first author. Inter-rater reliability with the second author was evaluated.

Third, the medical records were retrieved again in January to March 2017. The RAST and RAST-F were coded by the first and second authors using information pertaining to pre-admission criminal and psychiatric history. RAST coders were masked to other coded study data, and to information about patients’ presentation and seclusion that was contained in other sections of the medical record.

Inter-Rater Reliability

Following training, each new coder independently coded 10 cases, masked to the coding by the primary coder for that coding stage. Intra-class correlation coefficients (mixed methods, absolute agreement, single measures) showed excellent agreement for the number of seclusions, mean ICC = .98; total days in seclusion, mean ICC = .97; RAST total score, ICC = .98; RAST-F total score, ICC = .98; and VRAG total score, ICC = .99. The PIC counts yielded lower average levels of agreement, psychotic symptoms ICC = .75; institutional management problems, ICC = .74; mood problems, ICC = .62; social withdrawal, ICC = .53; hence students’ coding of clinical symptoms and behavior was routinely reviewed for consensus.

Analytical Plan

We first described the distribution of scores on the independent variables and seclusion outcomes. Correlational analyses were used to explore associations between seclusion and scores on the RAST and RAST-F, VRAG, intake clinical variables, and PIC problem counts. We then tested the RAST and RAST-F’s prediction of any seclusion using AUC values from receiver operating characteristic analysis. Remaining analyses focused on total time spent in seclusion during hospitalization as the dependent measure, controlling for time at risk, in an effort to ensure the RAST was related not just to any seclusion but also to duration of seclusion. In separate linear regression analyses, using transformed data where indicated for reducing skewness, we entered time at risk (length of stay, capped at 365 days) and either the RAST score or the modified RAST score. We also planned to test the ability of the RAST to improve on the VRAG by entering the VRAG total score in the first step of a linear regression and then subjecting the best RAST measure from preceding analyses to a stepwise selection procedure.

Among patients whose hospital stay was over 1 year, we tested the RAST and RAST-F scores’ ability to predict seclusion in the second year of in-hospital follow-up, using a similar procedure. For these analyses, length of stay was capped at 730 days. The ability of clinical variables to improve the statistical model of seclusions was tested by entering the best RAST measure and length of stay in the first step of linear regression of Year 2 seclusion days and then subjecting Year 1 PIC scales to stepwise selection.

Results

The mean age at admission was 36 years (SD = 12). Length of stay within the first year of admission ranged from 28 to 365 days (the cap), M = 180 (SD = 134), median = 111, skewness = 0.46 (SE = 0.16), kurtosis = −1.59 (SE = 0.32). Half the patients (n = 115, 50%) were secluded at least once in Year 1, and these men were secluded an average of 29 days in total (SD = 55). In the whole sample, number of days secluded within the first year of admission ranged from 0 to 354 and was positively skewed, M = 14.35 (SD = 41.52), median = 1, skewness = 5.90 (SE = 0.16), kurtosis = 40.78 (SE = 0.32).

Table 1 shows descriptive and correlational analyses of the principal variables and PIC scales. The four PIC scales yielded internal reliability coefficients of α = .67 (psychosis), .66 (management problems), .33 (mood), and .42 (social withdrawal). In bivariate analyses, PIC psychosis symptoms, institutional management problems, and mood problems in Year 1 were associated with any seclusions and total days in seclusion within the same time period (Table 2). Of note, neither diagnosis of schizophrenia nor the VRAG were significantly associated with seclusions (Table 2). Therefore, analyses of the RAST’s incremental predictive ability over the VRAG are not reported here.

Table 2.

RAST, Mental Health Measures, and VRAG Association with Seclusions in Years 1 and 2.

Variable (and potential range)	Year 1 (N = 229)			Year 2 (N = 62)
	Any seclusion		Days secluded (Rho)	Any seclusion		Days secluded (Rho)
	AUC	[95% CI]	Days secluded (Rho)	AUC	[95% CI]	Days secluded (Rho)
RAST items
1. Age category	.576*	[0.502, 0.650]	.158*	.557	[0.414, 0.740]	.137
2. Prior convictions	.486	[0.411, 0.561]	−.036	.617	[0.463, 0.771]	.208
3. Prior administrative segregation	.490	[0.413, 0.567]	−.074	.500	[0.332, 0.668]	—
4. Index sentence length	—	—	—	—	—	—
5. Criminal versatility in current convictions	.500	[0.425, 0.575]	−.039	.489	[0.331, 0.647]	−.080
6. Prior conviction for violence	.530	[0.455, 0.605]	.060	.662	[0.516, 0.807]	.299*
RAST total score	.541	[0.465, 0.616]	.069	.738	[0.590, 0.872]	.383**
RAST-F modified items
3. Prior forensic seclusion	.535	[0.459, 0.611]	.111	.635	[0.481, 0.790]	.285*
4. Transferred from psychiatric hospital	.517	[0.442, 0.592]	.056	.568	[0.409, 0.727]	.190
5. Criminal versatility in current charges	.475	[0.400, 0.550]	−.046	.444	[0.280, 0.609]	−.055
RAST-F total score	.543	[0.468, 0.618]	.083	.693	[0.540, 0.847]	.349**
Diagnosis
Schizophrenia	.497	[0.422, 0.572]	−.005	.549	[0.392, 0.707]	.113
Any psychosis	.523	[0.488, 0.598]	.062	.501	[0.342, 0.661]	.039
Mood disorder	.478	[0.403, 0.533]	−.058	.516	[0.355, 0.678]	.032
Personality disorder	.495	[0.420, 0.570]	−.043	.481	[0.323, 0.639]	−.054
PIC problem counts (in same time period as seclusion)
Psychosis symptoms	.672**	[0.544, 0.699]	.275**	.628	[0.471, 0.785]	.258*
Institutional management problems	.785***	[0.725, 0.844]	.539***	.711*	[0.574, 0.849]	.334**
Mood problems	.677***	[0.608, 0.746]	.327***	.645	[0.492, 0.797]	.202
Social withdrawal	.566	[0.492, 0.640]	.121	.490	[0.337, 0.643]	−.029
VRAG score	.523	[0.447, 0.599]	.009	.596	[0.437, 0.754]	.151

Note. RAST = Risk of Administrative Segregation Tool; VRAG = Violence Risk Appraisal Guide; AUC = area under the curve; CI = confidence interval; PIC = Problem Identification Checklist.

p < .05. **p < .01. ***p < .001.

RAST and RAST-F

The RAST mean score was 3.13 (SD = 1.50), with a median of 3 and an observed range of 0 to 6. There was no substantial deviation from normal distribution, skew = −0.178 (SE = 0.161), kurtosis = −0.613 (SE = 0.320). The observed maximum RAST score was half of that observed for the correctional offenders comprising the RAST development and validation samples; indeed, it was lower than the correctional offenders’ median score (Helmus, 2015).

The RAST-F mean score was 4.42 (SD = 1.88) with a median of 5 and an observed range of 0 to 9. Again, there was no substantial deviation from normal distribution, skew = 0.051 (SE = 0.161), kurtosis = −0.612 (SE = 0.320). RAST and RAST-F scores were strongly and positively correlated, r = .87, p < .001. Table 1 shows that the modified items exhibited a higher base rate than the items they replaced.

Prediction of Total Days Secluded

Neither the RAST nor RAST-F significantly predicted any seclusion within one year of admission, AUC = .541 (SE = 0.038), 95% confidence interval (CI) [0.465, 0.616], and AUC = .543 (SE = 0.038), 95% CI [0.468, 0.618], respectively. The linear regression model with the original RAST score explained 8% of the variance in the (square-root transformed) number of days secluded in Year 1, R² =.084, F(2, 226) = 10.36, p < .001. Length of stay predicted total number of days secluded after controlling for other variables in the model (β = .284, p < .001) but the RAST score did not (β = .057, p = .372). The model with the RAST-F score offered some improvement, explaining 10% of the variance, R² =.096, F(2, 226) = 11.97, p < .001. Length of stay again predicted number of days secluded after controlling for other variables in the model (β = .269, p < .001), and the modified RAST score approached significance (β = .124, p = .054).

Prediction of Total Days Secluded in Year 2

Among the 62 men whose hospitalizations extended into a second year, length of stay ranged from 420 to 730 days (the cap), M = 634 (SD = 119) median = 730, skewness = −0.74 (SE = 0.30), kurtosis = −1.11 (SE = 0.60). This variable was not transformed for analysis. Eighteen patients (29%) were secluded at least once in Year 2. Total days secluded in the second year ranged from 0 to 35, M = 12.27 (SD = 47.40), median = 0, skewness = 6.37 (SE = 0.30), kurtosis = 44.80 (SE = 0.60).

The mean RAST score in this subsample was 3.11 (SD = 1.36); RAST-F, M = 4.79 (SD = 1.90). Medium to large predictive effect sizes were observed for any seclusion within the second year of admission using both the RAST, AUC = .731 (SE = 0.072), 95% CI [0.590, 0.872], and the RAST-F, AUC = .693 (SE = 0.078), 95% CI [.540, .847]. Of note, these effects were larger than prediction of seclusions within the first year for this subsample, significantly so for the RAST-F: Year 1 seclusion RAST AUC = .618 (SE = 0.075), 95% CI [0.471, 0.764], RAST-F AUC = .527, (SE = 0.075), 95% CI [0.380, 0.675]. Table 2 shows predictive effects for all RAST items and modified RAST-F items for Year 2 seclusion.

The linear regression model with the original RAST score explained 18% of the variance in the (square-root transformed) number of days secluded in Year 2, R² =.180, F(2, 59) = 6.49, p = .003. Length of stay in Year 2 predicted total number of days secluded in Year 2 after controlling for other variables in the model (β = .315, p = .010), and unlike in Year 1, the RAST score also made a significant contribution (β = .259, p = .033). An improved model was obtained when using the RAST-F score, explaining 23% of the variance, R² = .233, F(2, 59) = 8.99, p < .001. Again, length of stay predicted number of days secluded after controlling for other variables in the model (β = .262, p = .028), as did the RAST-F score (β = .354, p = .004).

Incremental prediction of mental health variables

PIC institutional management problems and psychosis measured in Year 1 were significant bivariate predictors of seclusions in year 2 (Table 2). Predictive effects of PIC problems were smaller for year 2 than year 1 seclusions, in which institutional management (AUC = .745), mood problems (AUC = .714) and psychosis (AUC = .659) all had significant predictive effects (social withdrawal AUC = .541). After controlling for length of stay in year 2 and RAST-F score, entering year 1 PIC problems did not result in a significantly improved model, R² = .301, F(6, 55) = 3.95, p = .002, R² change = .068, F (4, 55) change = 1.33, p = .270. RAST-F was the only statistically significant unique predictor, controlling for other variables in the model (β = .289, p = .024), followed by PIC psychosis symptoms, (β = .259, p = .080) and length of stay (β = .219, p = .085).

Categories of Risk for Seclusion

Following Helmus (2015), who presented options for dividing RAST scores into three categories to aid score interpretation (a five-category option was also examined; Helmus, 2015), we did the same for the RAST-F to help interpret risk of seclusion. We divided the sample into those scoring at the mean RAST-F score (4 or 5), those scoring below 4, and those scoring above 5. Table 3 shows the approximately equal distribution of cases across the three groups, percentage of cases having any seclusion in Year 1, and the increasing number of seclusions and total days secluded in Year 1 across groups. Table 4 shows the same information for Year 2.

Table 3.

Sample Distribution and Seclusions in Year 1 as a Function of RAST-F Risk Group Scores.

RAST-F group	DistributionN (%)		RAST-F ScoreM (SD)		Seclusions			Verbal risk category
RAST-F group	DistributionN (%)		RAST-F ScoreM (SD)		Any Seclusions (N, %, 95% CI)	Number of Seclusions (M, SD, 95% CI)	Total Days Secluded (M, SD, 95% CI)	Verbal risk category
1	81	(35)	2.35	(0.76)	42	0.80 (1.36)	9.91 (24.97)	“Low”
1					[31, 53]	[0.50, 1.10]	[4.39, 15.44]
2	77	(34)	4.58	(0.50)	57	1.34 (1.76)	10.42 (18.78)	“Medium”
2					[46, 68]	[0.94, 1.74]	[6.15, 14.68]
3	71	(31)	6.61	(0.84)	52	1.72 (3.33)	23.69 (66.26)	“High”
3					[40, 64]	[0.93, 2.51]	[8.01, 39.37]

Note. Verbal risk categories should not be used for communication or documentation in the absence of statistical information. RAST = Risk of Administrative Segregation Tool; CI = confidence interval.

Table 4.

Sample Distribution and Seclusions in Year 2 as a Function of RAST-F Risk Group Scores.

RAST-F group	DistributionN (%)		RAST-F scoreM (SD)		Seclusions			Verbal risk category
RAST-F group	DistributionN (%)		RAST-F scoreM (SD)		Any Seclusions (N, %, 95% CI)	Number of Seclusions (M, SD, 95% CI)	Total Days Secluded (M, SD, 95% CI)	Verbal risk category
1	16	(26)	2.31	(0.60)	13	0.44 (1.50)	0.75 (2.75)	“Low”
1					[−6, 31]	[−0.36, 1.24]	[4.39, 15.44]
2	24	(39)	4.58	(0.24)	21	0.88 (2.09)	5.13 (17.33)	“Medium”
2					[3, 38]	[−0.01, 1.76]	[−2.19, 12.44]
3	22	(36)	6.82	(0.96)	50	1.64 (2.46)	28.45 (75.89)	“High”
3					[27, 73]	[0.55, 2.73]	[−5.19, 62.10]

Discussion

In this study of 229 male forensic inpatients, half the men were secluded during their first year or less of hospitalization because of their behavior or presentation. Seclusion was associated not only with longer length of stay but also with measures obtainable on admission. The RAST-F, consisting of the RAST with three items modified to better suit the forensic application, had a small contribution that approached significance in a model of men at risk of prolonged seclusion in their first year of forensic hospitalization. Both versions of the tool significantly predicted seclusions within a second year of hospitalization, in a subsample of 62 patients. These promising findings represent the first independent validation of the RAST, and its first test outside of a correctional setting. Most of the predictive value was provided by two RAST items, age on admission and seclusion in previous forensic hospitalizations, which was modified from the original item, admission to administrative segregation in a previous federal sentence. Thus, the present study provides modest support for the RAST and RAST-F tools in this forensic sample, driven by two of its items. Validation of the RAST-F and its items in other forensic samples, including female patients, would be of value.

The present study permitted testing of the incremental contribution of mental health variables, which were limited in the RAST developmental research (Helmus, 2015). PIC symptoms of psychosis, mood, and behavior problems in the hospital had significant associations with days of seclusion when measured in the same time period (during Year 1) but did not significantly predict seclusions in the subsequent year when tested in patients whose hospital stays extended into Year 2. Some variability across the RAST items notwithstanding, the RAST had more or less stable predictive accuracy in this subsample across Years 1 and 2, whereas the PIC problems were strongly “predictive” for seclusions in Year 1 (in an overlapping time frame) but weaker in Year 2, no longer contributing significantly to prediction. Thus, it appears that when looking at longer-term assessment of seclusion risk, initial clinical presentation becomes less valuable, and the more static risk measure becomes more valuable. These findings suggest that symptoms may be a proximate reason for seclusion (or possible consequence) rather than predictors of future seclusions. Nevertheless, the present study offers some hope that it may be possible to identify patients in need of the highest intensity treatment or other interventions designed to avert seclusion, before a host of problematic symptoms are exhibited, and before prolonged seclusion becomes a challenge. Helmus (2015) observed that further research examining mental health predictors of administrative segregation was warranted, and we add that the RAST-F’s ability to identify segregation risk among mentally disordered inmates merits investigation.

RAST scores were substantially lower in the present sample of men undergoing forensic assessment than previously reported for correctional offenders serving sentences of 2 years or more (Helmus, 2015). This difference, and the item modifications necessary to improve the RAST’s validity in this study, indicates a distinction between forensic and correctional populations, despite some overlap in terms of mental health problems and criminal histories. The present study thus suggests that transferring an assessment tool such as the RAST from one population to another must be done cautiously, and requires validation for the new population.

A related matter is the different frequency and duration of seclusion. In the present sample, 50% of men were secluded within their first year, and secluded men spent an average of 29 days in seclusion. This contrasts with the base rate in Helmus’s correctional sample, of whom 24% were segregated for at least 6 days within their first 2 years and spent an average of 97 days segregated (Helmus, 2015). These statistics are not directly comparable, and our current dataset does not lend itself to examining the proportion of patients secluded for at least 6 consecutive days within their first 2 years of hospitalization. Nevertheless, these differences suggest that forensic seclusion does not substantially align with correctional segregation. Furthermore, although seclusion was common, we found only small prediction effects for the RAST, which measures individual characteristics. This finding suggests that studying environmental or systemic variables (e.g., staff-to-patient ratio, staff skill set, physical environment, policy and training) may enhance our understanding of factors contributing to seclusion.

Limitations and Research Implications

Compared with the original RAST development and validation, the present study had a far smaller sample, entirely male, and all cases were drawn from a single institution. Our sampling procedure produced a sample at greater risk of seclusion than those excluded on the basis of convenience, although we note that a substantial minority of patients (45% of the overall cohort for whom data are available) experienced at least one seclusion. Nevertheless, our sample included more than 100 cases with seclusion and more than 100 without, surpassing the minimum number of events and nonevents recommended for adequately powered validations of regression models in new samples (Vergouwe, Steyerberg, Eijkemans, & Habbema, 2005).

Several methodological differences limit our ability to compare our test of the original RAST with previous research. We examined any seclusions and duration of seclusions, whereas the RAST was constructed to predict segregations of at least six days (Helmus, 2015). We effectively omitted Item 4, sentence length for index offense. This item achieved a moderate predictive effect in the original construction research, in the whole sample and multiple subsample tests (Helmus, 2015), but it was inapplicable to most of our sample. Two other items, admission to administrative segregation in a previous federal sentence and versatility in current convictions, had very low base rates in our forensic psychiatric sample.

Conversely, prior convictions (affecting Items 2 and 6) were prevalent in our sample and did not meet our a priori criteria for modification, which related to ease of scoring and face validity for the forensic population. Nevertheless, our decision not to adapt them to capture prior findings of NCR likely underestimates our sample’s criminal history, which these items were designed to capture on the original RAST. These items comprise a third of the RAST items. As a result, the RAST and RAST-F’s predictive accuracy in the present study may have been diminished. It will be important in future research to consider replacing prior convictions with prior violent charges, or counting prior NCR findings as convictions, in the manner used for other risk assessment such as the Static-99 (e.g., Phenix et al., 2016).

Of the RAST items, only younger age bore a significant bivariate relationship with being secluded in the first year of hospitalization. Younger age has a well-established association with criminal outcomes among offenders with and without mental disorder (e.g., Bonta, Law, & Hanson, 1998) and was previously reported to be a risk factor for seclusion among psychiatric inpatients (e.g., Bullock et al., 2014; Cullen et al., 2018). The present study was designed only to test the applicability of an existing tool to a psychiatric population, and we did not explore what additional characteristics might better predict seclusions in lieu of the RAST items, other than the scoring modifications and potential incremental contribution of mental-health-related variables. Further studies, using considerably larger samples than the present one, would be needed to conduct the necessary developmental research in a forensic population to parallel the work by Helmus (2015) in the correctional population. Meanwhile, the present study modestly adds to the growing evidence that patient characteristics other than clinical presentation can provide important cues to risk of seclusion among psychiatric inpatients.

The VRAG, which in previous research has been shown to have moderate but not unequivocal predictive accuracy for institutional violence (e.g., Hogan & Olver, 2016), did not predict seclusions in the present study. Replication of this test with measures shown to be associated with institutional violence and misconducts (such as the Historical Clinical Risk Management-20; e.g., Fitzgerald et al., 2013; Kroner & Mills, 2001) would be valuable. Researchers may also consider examining actual violence against staff or other patients or inmates for their potential to identify patients at risk of forensic seclusion. Our data did not allow us to distinguish pre-emptive seclusion from reactive seclusion. However, we note that existing literature indicates that agitation, not overt aggression, is the most common precipitating factor in seclusion (e.g., Kaltiala-Heino et al., 2003; Keski-Valkama et al., 2010), suggesting that use of violence risk assessment tools to identify patients at risk of seclusions may be unfruitful.

Our test of the contribution of mental health variables to the prediction of seclusion was based on a substantially reduced sample (N = 62) for whom Year 2 data were available. This was necessary to ensure the mental health measures pertained to a time period separate from and prior to the seclusion outcomes. Furthermore, the contribution of mental health variables was tested in terms of incremental prediction, which requires greater statistical power, and the effect size for problems related to psychosis symptoms was similar to that for the RAST-F, although it fell short of statistical significance. Therefore, we recommend further research into the incremental contribution of mental health measures, using larger samples. Future studies that assess mental health symptoms shortly after admission (e.g., within the first month of hospitalization) could better evaluate their incremental contribution to the prediction of seclusion over the long term.

As well as being a smaller sample, the patients included in the Year 2 analyses differed from the rest of the sample in their substantially longer length of stay than most patients admitted for forensic assessment, and likely in other characteristics too. It is possible that the 62 patients who were hospitalized beyond the first year are a unique subgroup that are more similar to the correctional population on which the RAST was developed; however, the subsample RAST scores are not substantially different from the full sample, making it less plausible that the subsample is more similar to a correctional offender population. Nevertheless, patients with longer stays represent a particular demand on the forensic health care system, and there may be value in being able to identify those at risk of prolonged seclusion over their forensic “career.”

We focused on seclusion and did not include physical or chemical restraint, because they were not included in the available data. Restraint has been the subject of recent prediction research (e.g., Dazzi et al., 2017) and reduction efforts (e.g., Smith et al., 2015). Restraint and seclusion are complementary procedures for containing problematic behavior among forensic inpatients, so future research investigating whether the RAST predicts restraint, or is specific to risk of seclusion, could help inform forensic practice in identifying risk of either intervention.

We did not prospectively investigate the effects of prolonged seclusion on patients’ mental health, although we observed associations between mental health variables and days in seclusion within the same time frame. Most existing research on seclusion concerns correctional inmates in administrative segregation (e.g., Gendreau & Labrecque, 2016), for which there is evidence of small to moderate adverse effects on psychological and physical health outcomes (e.g., Morgan et al., 2016). Weight gain has been associated with days in seclusion in a subsample of the present study (Hilton et al., 2015), and further research with expanded measures of physical health is under way.

Clinical Implications

In forensic psychiatric care, there has been little previous attention to identifying patients at risk of seclusion. Recent endeavors have been made, though, to reduce the use of seclusion, with some success reported (e.g., Ching, Daffern, Martin, & Thomas, 2010; Fletcher et al., 2017; Maguire, Young, & Martin, 2012; Putkonen et al., 2013; Qurashi, Johnson, Shaw, & Johnson, 2010). For example, Van de Sande and colleagues (2011) reported that using daily and weekly assessments to monitor psychiatric symptoms and risk of aggression reduced aggression and consequent use of seclusion among 207 male and female acute psychiatric inpatients in the Netherlands, compared with 251 patients on other wards without this practice. The use of a tool such as the RAST-F, or its most predictive items (age and seclusion in previous admission), may augment efforts to reduce seclusion by allowing clinicians to focus on patients who are at greatest risk, developing individualized interventions based on behavior therapy or other techniques, to prevent problems that can lead to seclusion. This approach is consistent with the RNR principles of effective intervention, in which identifying at-risk individuals is an important precursor of successful attempts to reduce risk. By suggesting this approach, we do not intend to negate the strong associations among mental health variables and seclusion risk among psychiatric patients. Rather, we suggest it may be possible to identify patients most in need of clinical efforts to avert seclusion on admission, so that decisions related to seclusion can be planned, rather than reactive. The RAST-F shows some promise as an aid to the clinical management of this risk.

Footnotes

Acknowledgements

We are grateful to Carol Lang for her extensive contribution to coding development and data extraction, and to Maaike Helmus for sharing RAST research and coding documents. We thank Alecia Dretzkat, Carol Lang, Chelsea Turan, Courtney Duthie, Craig Rafla, Jenna Rutherford, Oleg Belanovsky, and Sonja Dey for research assistance, and Waypoint’s clinical program and clinical information staff for assistance. We thank Maaike Helmus, Shari McKee, and Craig Rafla who provided helpful comments on an earlier manuscript.

Authors’ Note

N. Zoe Hilton is also affiliated with University of Toronto, Toronto, Ontario, Canada.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by a grant from Public Safety Canada. The views expressed are those of the authors and not necessarily those of Public Safety Canada.

References

Bergk

Einsiedler

Flammer

Steinert

(2011). A randomized controlled comparison of seclusion and mechanical restraint in inpatient settings. Psychiatric Services, 62, 1310-1317.

Bonta

Andrews

D. A.

(2016). The psychology of criminal conduct (6th ed.). New York, NY: Routledge.

Bonta

Law

Hanson

(1998). The prediction of criminal and violent recidivism among mentally disordered offenders: A meta-analysis. Psychological Bulletin, 123, 123-142.

Bullock

McKenna

Kelly

Furness

Tacey

(2014). When reduction strategies are put in place and mental health consumers are still secluded: An analysis of clinical and sociodemographic characteristics. International Journal of Mental Health Nursing, 23, 506-512. doi:10.1111/inm.12078

Busch

A. B.

Shore

M. F.

(2000). Seclusion and restraint: A review of the recent literature. Harvard Review Psychiatry, 8, 261-270.

Chaimowitz

(2011). The use of seclusion and restraint in psychiatry. Canadian Journal of Psychiatry, 56(8), 1-2. Retrieved from https://www.cpa-apc.org/wp-content/uploads/Seclusion_and_Restraints-2011-31s-web-FIN-EN.pdf

Ching

Daffern

Martin

Thomas

(2010). Reducing the use of seclusion in a forensic psychiatric hospital: assessing the impact on aggression, therapeutic climate and staff confidence. The Journal of Forensic Psychiatry & Psychology, 21, 737-760. doi:10.1080/14789941003681361

Corrections and Conditional Release Act. (S.C. 1992, c. 20). Retrieved from https://laws-lois.justice.gc.ca/eng/acts/c-44.6/

Correctional Service Canada. (2014, November). Research results - mental health. Retrieved from http://www.csc-scc.gc.ca/publications/005007-3030-eng.shtml

10.

Cullen

A. E.

Bowers

Khondoker

Pettit

Achilla

Koeser

. . . Tulloch

A. D.

(2018). Factors associated with use of psychiatric intensive care and seclusion in adult inpatient mental health services. Epidemiology and Psychiatric Services, 27, 51-61. doi:10.1017/S2045796016000731

11.

Dazzi

Tarsitani

Di Nunzio

Trinci

Scifoni

Ducci

(2017). Psychopathological assessment of risk of restraint in acute psychiatric patients. Journal of Nervous and Mental Disease, 205, 458-465. doi:10.1097/NMD.0000000000000672

12.

Fitzgerald

Gray

N. S.

Alexander

R. T.

Bagshaw

Chesterman

Huckle

. . . Snowden

R. J.

(2013). Predicting institutional violence in offenders with intellectual disabilities: The predictive efficacy of the VRAG and the HCR-20. Journal of Applied Research in Intellectual Disabilities, 26, 384-393. doi:10.1111/jar.12032

13.

Fletcher

Spittal

Brophy

Tibble

Kinner

Elsom

(2017). Outcomes of the Victorian Safewards trial in 13 wards: Impact on seclusion rates and fidelity measurement. International Journal of Mental Health Nursing, 26, 461-471. doi:10.1111/inm.12380

14.

Gendreau

Labrecque

R. M.

(2016). The effects of administrative segregation: A lesson in knowledge cumulation. In Wooldredge

Smith

(Eds.), The Oxford handbook of prisons and imprisonment (pp. 1-34). Oxford University Press. doi:10.1093/oxfordhb/9780199948154.013.36

15.

Harris

G. T.

Rice

M. E.

Quinsey

V. L.

(1993). Violent recidivism of mentally disordered offenders: The development of a statistical prediction instrument. Criminal Justice and Behavior, 20, 315-335.

16.

Harris

G. T.

Rice

M. E.

Quinsey

V. L.

Cormier

C. A.

(2015). Violent offenders: Appraising and managing risk (3rd ed.). Washington, DC: American Psychological Association.

17.

Helmus

Johnson

Harris

A. J. R.

(2014). Risk of Administrative Segregation Tool (RAST) Scoring Manual (Draft) (Unpublished report) Research Branch, Correctional Services Canada, Ottawa, Ontario.

18.

Helmus

L. M.

(2015). Developing and Validating a Risk Assessment Scale to Predict Inmate Placements in Administrative Segregation in the Correctional Service of Canada (Unpublished doctoral thesis). Carleton University, Ottawa, Ontario, Canada.

19.

Hilton

N. Z.

Ham

Green

(2018). The roles of antisociality and neurodevelopmental problems in criminal violence and clinical outcomes among male forensic inpatients. Criminal Justice and Behavior, 45, 293-315. doi:10.1177/0093854817745911

20.

Hilton

N. Z.

Ham

Lang

Harris

G. T.

(2015). Weight gain and its correlates among forensic inpatients. Canadian Journal of Psychiatry, 60, 232-238. doi:10.1177/070674371506000505

21.

Hogan

N. R.

Olver

M. E.

(2016). Assessing risk for aggression in forensic psychiatric inpatients: An examination of five measures. Law and Human Behavior, 40, 233-243. doi:10.1037/lhb0000179

22.

Kaltiala-Heino

Tuohimäki

Korkeila

Lehtinen

(2003). Reasons for using seclusion and restraint in psychiatric inpatient care. International Journal of Law and Psychiatry, 26, 139-149.

23.

Keski-Valkama

Sailas

Eronen

Koivisto

A. M.

Lönnqvist

Kaltiala-Heino

(2010). The reasons for using restraint and seclusion in psychiatric inpatient care: A nationwide 15-year study. Nordic Journal of Psychiatry, 64, 136-144.

24.

Knox

D. K.

Holloman

G. H.

(2012). Use and avoidance of seclusion and restraint: consensus statement of the american association for emergency psychiatry project Beta seclusion and restraint workgroup. The western journal of emergency medicine, 13(1), 35-40. doi:10.5811/westjem.2011.9.6867

25.

Kroner

D. G.

Mills

J. F.

(2001). The accuracy of five risk appraisal instruments in predicting institutional misconduct and new convictions. Criminal Justice and Behavior, 28, 471-489.

26.

Luckhoff

Jordaan

Swart

Cloete

K. J.

Koen

Niehaus

D. J. H.

(2013). Retrospective review of trends in assaults and seclusion at an acute psychiatric ward over a 5-year period. Journal of Psychiatric and Mental Health Nursing, 20, 687-695. doi:10.1111/jpm.12006

27.

Maguire

Young

Martin

(2012). Seclusion reduction in a forensic mental health setting. Journal of Psychiatric and Mental Health Nursing, 19, 97-106. doi:10.1111/j.1365-2850.2011.01753.x

28.

Martin

M. S.

Colman

Simpson

A. I. F.

McKenzie

(2013). Mental health screening tools in correctional institutions: A systematic review. BMC Psychiatry, 13, Article 275. doi:10.1186/1471-244X-13-275

29.

McKenna

McEvedy

Maguire

Ryan

Furness

(2017). Prolonged use of seclusion and mechanical restraint in mental health services: A statewide retrospective cohort study. International Journal of Mental Health Nursing, 26, 491-499. doi:10.1111/inm.12383

30.

Morgan

R. D.

Gendreau

Smith

Gray

A. L.

Labrecque

R. M.

MacLean

. . . Mills

J. F.

(2016). Quantitative syntheses of the effects of administrative segregation on inmates’ well-being. Psychology, Public Policy, and Law, 22, 439-461. doi:10.1037/law0000089

31.

Muskett

(2014). Trauma-informed care in inpatient mental health settings: A review of the literature. International Journal of Mental Health Nursing, 23, 51-59. doi:10.1111/inm.12012

32.

Nagi

Ostapiuk

E. B.

Craig

L. A.

Hacker

Beech

A. R.

(2009). Using the revised Problem Identification Checklist to predict inpatient and community violence: A pilot study. British Journal of Forensic Practice, 11, 4-13.

33.

Noorthoorn

Lepping

Janssen

Hoogendoorn

Nijman

Widdershoven

Steinert

(2015). One-year incidence and prevalence of seclusion: Dutch findings in an international perspective. Social Psychiatry and Psychiatric Epidemiology, 50, 1857-1869. doi:10.1007/s00127-015-1094-2

34.

Phenix

Fernandez

Harris

A. J. R.

Helmus

Hanson

R. K.

Thornton

(2016). Static-99R coding rules revised–2016. Retrieved from http://www.static99.org/pdfdocs/Coding_manual_2016_v2.pdf

35.

Putkonen

Kuivalainen

Repo-Tiihonen

Ryynänen

O, P.

Kautiainen

Tiihonen

(2012). Cluster-randomized controlled trial of reducing seclusion and restraint in secured care of men with schizophrenia. Psychiatric Services, 64, 850-855. doi: 10.1176/appi.ps.201200393

36.

Public Safety Canada. (2017, April). 2016 annual report: Corrections and conditional release statistical overview. Retrieved from https://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/ccrso-2016/ccrso-2016-en.pdf

37.

Quinsey

V. L.

Coleman

Jones

G. B.

Altrows

I. F.

(1997). Proximal antecedents of eloping and reoffending among supervised mentally disordered offenders. Journal of Interpersonal Violence, 12, 794-813. doi:10.1177/088626097012006002

38.

Qurashi

Johnson

Shaw

Johnson

(2010). Reduction in the use of seclusion in a high secure hospital: A retrospective analysis. Journal of Psychiatric Intensive Care, 6, 109-115. doi:10.1017/S174264640999015X

39.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC Area, Cohen’s d, and r. Law and Human Behavior, 29, 615-620. doi:10.1007/s10979-005-6832-7

40.

Rice

M. E.

Harris

G. T.

Lang

(2013). Validation of and revision to the VRAG and SORAG: The Violence Risk Appraisal Guide–Revised (VRAG-R). Psychological Assessment, 25, 951-965. doi:10.1037/a0032878

41.

Rice

M. E.

Harris

G. T.

Lang

Bell

(1990). Recidivism among male insanity acquittees. Journal of Psychiatry & Law, 18, 379-403.

42.

Rice

M. E.

Harris

G. T.

Quinsey

V. L.

(1996). Treatment for forensic patients. In Sales

B. D.

Shah

S. A.

(Eds.), Mental health and law: Research, policy, and services (pp. 141-189). Durham, NC: Carolina Academic Press.

43.

Scanlan

(2010). Interventions to reduce the use of seclusion and restraint in inpatient psychiatric settings: What we know so far a review of the literature. International Journal of Social Psychiatry, 56, 412-423. doi:10.1177/0020764009106630

44.

Seto

M. C.

Harris

G. T.

Rice

M. E.

(2004). The criminogenic, clinical, and social problems of forensic and civil psychiatric patients. Law and Human Behavior, 28, 577-586.

45.

Smith

G. M.

Ashbridge

D. M.

Altenor

Steinmetz

Davis

R. H.

Mader

Adair

D. K.

(2015). Relationship between seclusion and restraint reduction and assaults in Pennsylvania’s forensic services centers: 2001-2010. Psychiatric Services, 66, 1326-1332. doi:10.1176/appi.ps.201400378

46.

Stewart

Van Der Merwe

Bowers

Simpson

Jones

(2010). A review of interventions to reduce mechanical restraint and seclusion among adult psychiatric inpatients. Issues in Mental Health Nursing, 31, 413-424. doi:10.3109/01612840903484113

47.

Van de Sande

Nijman

H. L. I.

Noorthoorn

E. O.

Wierdsma

A. I.

Hellendoorn

van der Staak

C. P. F.

Mulder

C. L.

(2011). Aggression and seclusion on acute psychiatric wards: Effect of short-term risk assessment. British Journal of Psychiatry, 199, 473-478. doi:10.1192/bjp.bp.111.095141

48.

Van Der Merwe

Muir-Cochrane

Jones

Tziggili

Bowers

. (2013). Improving seclusion practice: Implications of a review of staff and patient views. Journal of Psychiatric and Mental Health Nursing, 20, 203-215.

49.

Velpry

Eyraud

(2014). Confinement and psychiatric care: A comparison between high-security units for prisoners and for difficult patients in France. Culture, Medicine, and Psychiatry, 38, 550-577. doi:10.1007/s11013-014-9400-0

50.

Vergouwe

Steyerberg

E. W.

Eijkemans

M. J. C.

Habbema

J. D. F.

(2005). Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. Journal of Clinical Epidemiology, 58, 475-483. doi:10.1016/j.jclinepi.2004.06.017

51.

Vitacco

M. J.

Gonsalves

Tomony

Smith

B. E. R.

Lishner

D. A.

(2012). Can standardized measures of risk predict inpatient violence? Combining static and dynamic variables to improve accuracy. Criminal Justice and Behavior, 39, 589-606. doi:10.1177/0093854812436786