Outcomes of Victorian Prevention and Recovery Care Services: A matched pairs comparison

Abstract

Objective:

In Victoria, Prevention and Recovery Care Services have been established to provide a partial alternative to inpatient admissions through short-term residential mental health care in the community. This study set out to determine whether Prevention and Recovery Care Services are achieving their objectives in relation to reducing service use and costs, fostering least restrictive care and leading to positive clinical outcomes.

Methods:

We matched 621 consumers whose index admission in 2014 was to a Prevention and Recovery Care (‘PARCS consumers’) with 621 similar consumers whose index admission in the same year was to an acute inpatient unit and who had no Prevention and Recovery Care stays for the study period (‘inpatient-only consumers’). We used routinely collected data to compare them on a range of outcomes.

Results:

Prevention and Recovery Care Services consumers made less subsequent use of acute inpatient services and, on balance, incurred costs that were similar to or lower than inpatient-only consumers. They were also less likely to spend time on an involuntary treatment order following their index admission. Prevention and Recovery Care Services consumers also experienced positive clinical outcomes over the course of their index admission, but the magnitude of this improvement was not as great as for inpatient-only consumers. This type of clinical improvement is important for Prevention and Recovery Care Services, but they may place greater emphasis on personal recovery as an outcome.

Conclusion:

Prevention and Recovery Care Services can provide an alternative, less restrictive care option for eligible consumers who might otherwise be admitted to an acute inpatient unit and do so at no greater cost.

Keywords

Prevention and Recovery Care Services mental health sub-acute residential community

In Australia and overseas, various sub-acute residential alternatives to acute inpatient units have been developed (Thomas and Rickwood, 2013). In Victoria, these take the form of Prevention and Recovery Care Services (PARCS) (Harvey et al., 2019). PARCS aim to provide a partial alternative to inpatient admissions through short-term residential mental health care in the community. Admissions may be a ‘step-up’ from non-residential community care or a ‘step-down’ (typically an early discharge) from an acute inpatient unit. They are provided by the public sector clinical mental health service in partnership with a non-government organisation (Department of Health and Human Services, 2016). The study reported here was part of a large-scale evaluation of PARCS. When it commenced in 2016 there were 19 adult sub-acute PARCS operating in Victoria, as well as one extended Prevention and Recovery Care (PARC) and three youth PARCS.

In part, PARCS were established in response to constraints around inpatient care. The number of psychiatric beds per capita in Australia is low (Allison et al., 2018) and Victoria has the lowest rate of any state (Department of Health and Human Services, 2019). Inpatient services often operate at full capacity leading to waiting times for admissions, relatively brief stays and pressure for early discharge (Allison and Bastiampillai, 2015; Select Committee on Mental Health, 2006). Acute psychiatric inpatient care is costly (Australian Institute of Health and Welfare, 2020), and the inpatient environment can be stressful for consumers and staff (Sweeney et al., 2014).

The rationale for PARCS is not limited to relieving pressure on inpatient units. They are also designed to promote recovery-oriented care. The Victorian Government’s Framework for Recovery-oriented Practice distinguishes ‘clinical’ recovery, defined by reduction of symptoms and restoration of social functioning, and ‘personal’ recovery where the aim is ‘to support people to build and maintain a (self-defined and self-determined) meaningful and satisfying life and personal identity, regardless of whether there are ongoing symptoms of mental illness’ (Department of Health, 2011: 2). PARCS are designed to provide opportunities for personal recovery while also addressing clinical recovery. They do this by enabling consumers to make or retain connection with community life while in safe accommodation with on-site mental health treatment and support (Department of Health, 2010). In this way, they are consistent with the principle of provision of treatment and care in the least restrictive environment and they respond to consumers’ preferences for non-hospital care (Thomas and Rickwood, 2013; Trauer et al., 2001).

A number of evaluations of single PARC or PARC-like services have been conducted in Australia (Heyeres et al., 2018; Kinchin et al., 2019; Lee et al., 2014; Ngo et al., 2020; Siskind et al., 2013; Thomas et al., 2017b). Collectively, these have found that consumers are positive about PARCS and experience improvements in functioning and reductions in symptoms following a PARCS stay, that the cost of a PARCS admission is lower than that of an inpatient admission and that PARCS use is associated with a reduction in the number and duration of subsequent inpatient admissions. A broader evaluation of all Victorian PARCS also suggested that PARCS consumers’ mental health improved during an admission, although it indicated that the gains were not as substantial as they were for those who were admitted to an acute inpatient unit (Department of Health and Human Services, 2016).

These evaluations are limited by the fact that they are, for the most part, small and uncontrolled, relying on before-and-after designs. Only one evaluation has involved a matched comparison group (Siskind et al., 2013), but that considered a single service in isolation. Only one more substantial evaluation of multiple PARCS has been conducted (Department of Health and Human Services, 2016), but although this compared impacts for PARCS consumers and consumers in acute inpatient units, it did not involve a ‘like-with-like’ comparison. In addition, previous evaluations have focused on a narrow range of outcomes that do not take into account the manifold objectives of PARCS.

This study was designed to respond to the clear need for a large-scale comparison of a range of outcomes for appropriately matched groups of PARCS consumers and inpatients. As noted, it was part of a broader evaluation of Victorian PARCS by our group that involved multiple studies (Fletcher et al., 2019a; Harvey et al., 2019; Sutherland et al., 2020). It aimed to determine whether Victorian PARCS are achieving their objectives in relation to delivering services to people who might otherwise have required acute inpatient care. More specifically, it addressed the following research questions:

(a) Are PARCS associated with reduced service use and costs?

(b) Do PARCS foster least restrictive care?

Method

Study design and participants

This study employed a matched pairs design. This was considered appropriate and feasible because we had previously established substantial overlap between those accessing PARCS and those admitted to inpatient units (Sutherland et al., 2020) in terms of their characteristics and service use.

We used statewide routinely collected service use data to identify consumers whose index admission in 2014 was to one of the 19 adult PARCS that were operational in Victoria at the time. We matched these consumers to inpatient consumers whose index admission in 2014 was to an acute inpatient unit, drawing these from a pool who had not had a PARCS admission between 2012 and 2016. We refer to the former group as ‘PARCS consumers’ and the latter as ‘inpatient-only consumers’. It is worth noting here that our 2014 index admission dataset was not identical to the 5-year dataset reported in Sutherland et al. (2020), so any comparisons need to take this into account.

To maximise accuracy and the number of matched pairs, we used propensity score (PS) matching (Thoemmes et al., 2009) to match consumers on demographic, diagnostic and clinical variables. We then compared PARCS consumers and inpatient-only consumers on a range of outcomes related to our research questions.

Data source

We used data from the Client Management Interface/Operational Data Store (CMI/ODS) which is maintained by the Victorian Department of Health and Human Services. The CMI/ODS captures person-level and episode-level information for consumers who receive mental health care in Victoria, enabling an examination of mental health service utilisation and outcomes. Each consumer has a unique identifier that they carry across services and over time.

For any given episode of care, consumers can be described in terms of their demographic details (e.g. age group, sex, location of residence, main language spoken), clinical characteristics (e.g. International Classification of Diseases, 10th Revision [ICD-10] diagnoses) and legal status as per the Mental Health Act (e.g. inpatient treatment orders [ITOs], community treatment orders [CTOs]). The service utilisation associated with the episode is also captured (e.g. length of PARCS or inpatient stay, number of community contacts).

Clinical outcomes of admission to PARCS and inpatient services can also be tracked via the CMI/ODS. Outcomes are measured by the 12-item Health of the Nation Outcome Scales (HoNOS) (Wing et al., 1998). Each item is rated from 0 (no problem) to 4 (very severe problem), yielding individual item scores, scores on four sub-scales (1. Behaviour; 2. Impairment; 3. Symptoms; 4. Social) and a total score.

Data management

Following ethics approval from the University of Melbourne Human Research Ethics Committee (1648205), we received deidentified CMI/ODS data from the Department of Health and Human Services covering admissions to PARCS and inpatient units for the period 1 January 2012 to 31 December 2016. In total, this covered 44,202 individuals and 102,264 admissions. We excluded records from youth and extended PARCS (2049 records; 2%). We also excluded records where the admission had not ended at the end of the study period (70 records; 0.07%), duplicate records (3 records; <0.01%), and records where data were missing on age, sex and/or length of stay (7 records; <0.01%). We then identified 9538 individuals who were admitted to a PARC or an inpatient unit in 2014 and applied further restrictions around dates of admission and discharge as well as the dates of recorded HoNOS. This resulted in our excluding 4060 (42%) of remaining records. Ultimately, our analysis dataset was restricted to 5478 individuals. In total, 700 of these consumers had an index PARCS admission in 2014.

In order to be retained in the dataset, key variables had to be available at the index admission for matching (see below). In particular, the initial ‘primary’ diagnoses and HoNOS total score were important. The CMI/ODS permits multiple ‘primary’ diagnoses to be assigned on one date. We retained records where at least one of seven groups of primary ICD-10 diagnoses (psychoactive substance use; schizophrenia, schizotypal and delusional disorders; mood disorders; neurotic, stress-related and somatoform disorders; behavioural syndromes associated with physiological disturbance; personality disorders; developmental disorders) was recorded at the point of first admission to the PARCS or inpatient unit in 2014. The CMI/ODS also records HoNOS scores at key points during episodes of care. We identified the HoNOS that was recorded at the closest point in time to the index admission, noting that in some cases this may have been recorded towards the end of the previous episode. To operationalise this, we took the baseline HoNOS score as being recorded between 7 days prior to and 1 day following the index admission date and used this for matching. We did the equivalent for the discharge HoNOS score (using those recorded 1 day prior to and 7 days following the discharge date). Consumers without the relevant diagnoses at the index admission and/or who did not have HoNOS scores recorded for the specified periods in the admission were excluded, which reduced the sample for matching to 5478 consumers, 700 of whom had an index PARCS admission in 2014.

PS matching

PS matching was performed using the SPSS custom dialogue PS Matching provided by Thoemmes (n.d.). Logistic regression was used as the estimation algorithm along with nearest neighbour matching. No units outside of common support were discarded. A one-to-one matching ratio was used with a calliper value of 0.1.

The variables used for matching were age group (i.e. 11–20, 21–30, 31–40, 41–50, 51–60, 61+), sex (i.e. male vs female), legal status (i.e. not on an ITO or CTO during the admission, on an ITO or CTO for the entire admission, on an ITO or CTO for part of the admission), main language spoken (i.e. English vs other), the four sub-scale scores of the HoNOS, location (i.e. urban or rural) and separate binary variables for the presence of each of the seven subgroups of ICD-10 diagnoses listed above.

Operationalising outcomes

We made comparisons between PARCS consumers and inpatient-only consumers on the following outcomes:

(a) The number of admissions to PARCS and acute inpatient services in the 365 days post discharge from the index admission, and the total residential care days during these admissions;

(b) The number of community mental health contacts in the 365 days post discharge from the index admission;

(d) The amount of time spent on an ITO or CTO in the 365 days post discharge from the index admission;

(e) The change in HoNOS total score from admission to discharge in the index admission (Burgess et al., 2015).

We selected these outcomes as key because of their relevance to the overarching objectives of PARCS and, consequently, because they could inform our research questions.

The way in which we operationalised most of these outcomes is self-explanatory, but our ascertainment of costs requires additional explanation. The costs of services used by each consumer in the 365 days after their index admission were calculated by multiplying the number of PARCS and inpatient residential care days and the number of community contacts by the Victorian standard unit costs in 2014–2015 Australian dollars ($AUD). The standard unit costs were sourced from departmental policy and funding guidelines (Department of Health, 2014). The cost of an acute inpatient admission bed day was $687.50 and the cost of a PARCS residential care day was $451.00. The cost per contact with a community mental health service was $80.01. The Victorian Auditor-General’s Office (VAGO, 2019) reports that the published inpatient unit bed day rate ‘… meets only around 62 per cent of full costs to AMHSs’ (p. 41), whereas due to the tendering process, the published rates for PARCS are likely to be close to accurate. For this reason, we conducted a sensitivity analysis using the VAGO’s estimated unit cost for the inpatient bed days ($1108.87).

Data analysis

Differences in the percentage of PARCS consumers and inpatient-only consumers who had admissions and community mental health contacts in the 365 days post discharge from the index admission were assessed with Pearson’s chi-square test. Differences in the mean number and duration of subsequent admission, community contacts and involuntary treatment orders for PARCS consumers and inpatient-only consumers required the use of non-parametric procedures, specifically Mann–Whitney U tests, due to assumption violations. Despite the use of non-parametric methods, Cohen’s d is reported for these outcomes to allow meaningful comparison across outcomes.

General linear models (GLMs) were used to evaluate differences between PARCS consumers and inpatient-only consumers on total service costs during the year following discharge. Because of the large proportion of zero costs, two-part models were used to evaluate the difference in components of the total costs including admission (to both PARCS and inpatient services) and community mental health service costs between the two groups (Glick et al., 2014). In the two-part model, we first modelled the probability that a person has any health care expenditures with a logit model using the full sample. Then we estimated a GLM on the subset of people for whom there were any expenditures. The two-part model allowed for separate investigation of the effect of covariates on the extensive margin (logit model, if any expenditures) and on the intensive margin (GLM, amount of expenditures if any) (Deb and Norton, 2018). GLM using log link and gamma family was used for cost variables. We took this approach because cost data are usually right-skewed because it is impossible to incur costs less than zero and there are typically small numbers of high-resource-use consumers. The International Society for Pharmacoeconomics and Outcome Research (ISPOR) guidelines (Ramsey et al., 2015) recommend that generalised linear models using a log link and gamma family should be considered for multivariable analysis of costs for this reason.

For the HoNOS, single-factor independent-groups analysis of covariance (ANCOVA) was used to test for differences between the PARCS consumers and inpatient-only consumers at discharge, controlling for variability in scores at admission. A Cohen’s d effect size based on the covariate adjusted means is also provided. As a supplementary analysis, admission-to-discharge effect sizes were calculated separately for the PARCS and inpatient-only consumer groups. These effect sizes were compared using z-scores to establish whether any differences between them were statistically significant.

The per comparison alpha level for all tests was adjusted to 0.01 to account for multiple comparisons. The cost analyses were conducted in Stata version 15 SE. All other analyses were done using SPSS version 26.

Results

Case matching

Our tight matching criteria meant that of the 700 consumers with a PARCS index admission in 2014, 621 (89%) could be matched with an inpatient-only consumer (defined, as noted above, as a consumer whose index admission in 2014 was to an acute inpatient unit and who did not have a PARCS admission during the 5-year study period). The matched sample of 621 inpatient-only consumers represented 13% of the total inpatient sample of 4778. The characteristics of the matched PARCS and inpatient-only consumers are summarised in Table 1. Independent samples t-tests and chi-square analysis (not reported here) found no significant differences or associations between the matched PARCS consumers and inpatient-only consumers on any of the matching variables.

Table 1.

Characteristics of matched PARCS consumers and inpatient-only consumers.

Matching variables		PARCS consumers (N = 621)	Inpatient-only consumers (N = 621)
Age group, n (%)	11–20	19 (3)	15 (2)
	21–30	113 (18)	107 (17)
	31–40	189 (30)	210 (34)
	41–50	155 (25)	142 (23)
	51–60	112 (18)	114 (18)
	61+	33 (5)	33 (5)
Sex, n (%)	Female	362 (58)	363 (58)
	Male	259 (42)	258 (42)
Location, n (%)	Urban	453 (73)	447 (72)
	Rural	168 (27)	174 (28)
Length of stay of index admission	Days	17.66 (9.87)	13.73 (13.94)
Legal status at index admission, n (%)	Not on ITO or CTO during the admission	608 (98)	601 (97)
	On ITO or CTO for the entire admission	9 (1)	8 (1)
	On ITO or CTO for part of the admission	4 (1)	12 (2)
Main language spoken, n (%)	English	609 (98)	606 (98)
	Other	12 (2)	15 (2)
Diagnosis, n (%)	Psychoactive substance use	47 (8)	58 (9)
	Schizophrenia, schizotypal and delusional disorders	222 (36)	225 (36)
	Mood disorders	238 (38)	223 (36)
	Neurotic, stress-related and somatoform disorders	87 (14)	96 (16)
	Behavioural syndromes associated with physiological disturbance	10 (2)	3 (1)
	Personality disorders	87 (14)	85 (14)
	Developmental disorders	6 (1)	6 (1)
HoNOS sub-scale, M (SD)	Sub-scale A: Behaviour	2.27 (2.05)	2.48 (2.30)
	Sub-scale B: Impairment	1.71 (1.64)	1.72 (1.79)
	Sub-scale C: Symptoms	4.73 (2.03)	4.71 (2.14)
	Sub-scale D: Social	4.90 (3.18)	5.01 (3.40)

PARCS: Prevention and Recovery Care Services; ITO: inpatient treatment order; CTO: community treatment order; SD: standard deviation; HoNOS: Health of the Nation Outcome Scales.

We deliberately did not match on length of index admission. The mean length of index admission for PARCS consumers was 17.66 days (standard deviation [SD] = 9.87), longer than the mean length of index admission for inpatient-only consumers at 13.73 days (SD = 13.94). Had we matched on length of index admission, we would have created an inpatient-only comparison group with an artificially inflated length of stay, rather than a group that was similar in terms of the above person-based characteristics and received ‘usual care’ in the inpatient setting based on these characteristics. Matching on length of index admission would have had implications for our comparison of outcomes, because it would have created the opportunity for the inpatient-only group to receive a greater ‘dose’ of treatment than ‘usual care’. Another way to think of this is that the PS matching exercise was done to mimic what would have happened in a randomised controlled trial. In such a trial, the total pool of consumers would have been selected by virtue of their being eligible for a PARCS stay (on the basis of key person-based characteristics), and half of them would have been randomised to a PARCS admission and the other half would have been randomised to an inpatient admission. The average duration of these admissions would have been determined after the randomisation as each group received their allocated care.

Analysis of outcomes

Table 2 shows that PARCS consumers were significantly less likely to have an inpatient admission in the 365 days following discharge from the index admission than their inpatient-only counterparts (24.0% vs 34.5%). In addition, Table 2 shows that the mean number of inpatient admissions was lower for PARCS consumers (0.45 vs 0.62), as was the mean number of inpatient bed days (4.99 vs 10.00). By design, the way we selected inpatient-only consumers on the basis of their not having a admission during the study period meant that PARCS consumers were significantly more likely to have a subsequent PARCS admission in the year after their index admission (29.3% vs 0.0%). When PARCS and inpatient stays were considered together, PARCS consumers had greater overall levels of service use in terms of both the overall number of admissions (0.89 vs 0.62) and duration of admissions (12.24 vs 10.00). The two groups did not differ in terms of their average number of community mental health care contacts.

Table 2.

Inpatient and PARCS admissions and community mental health contacts in the 365 days post discharge from the index admission.

Percentage of consumers with inpatient and PARCS admissions and community mental health contacts
	PARCS consumers (N = 621)			Inpatient-only consumers (N = 621)
	%	n	95% CI	%	n	95% CI	χ² (degree of freedom)	p
Inpatient admissions	24.0	149	[20.8, 27.2]	34.5	214	[30.8, 38.3]	16.4 (1)	<0.001
PARC admissions	29.3	182	[25.9, 33.0]	0	0	–	213.2 (1)	<0.001
Inpatient + PARC admission	40.3	250	[36.5, 44.1]	34.5	214	[30.8, 38.3]	4.5 (1)	0.035
Community contacts	80.2	498	[76.9, 83.2]	82.9	515	[79.8, 85.7]	1.5 (1)	0.214
Any services	80.2	498	[76.9, 83.2]	82.9	515	[79.8, 85.7]	1.5 (1)	0.214
Mean number and duration (days) of inpatient and PARCS admissions and community mental health contacts
	PARCS consumers (N = 621)		Inpatient-only consumers (N = 621)
	M	SD	M	SD	U	z	p	d
Inpatient admissions	0.45	1.13	0.62	1.16	173,189	3.87	<0.001	0.15
PARC admissions	0.44	0.79	−	−	−	−	−	−
Inpatient + PARC admissions	0.89	1.58	0.62	1.16	177,047	2.88	0.004	0.20
Inpatient bed days	4.99	14.55	10.00	30.61	171,257	4.25	<0.001	0.21
PARC residential care days	7.25	14.51	−	−	−	−	−	−
Inpatient + PARC bed days/residential care days	12.24	22.38	10.00	30.61	176,413	2.99	0.003	0.08
Community contacts	19.22	32.94	19.10	29.96	185,250	1.20	0.23	<0.01

PARCS: Prevention and Recovery Care Services; CI: confidence interval; SD: standard deviation.

By definition, inpatient-only consumers could only have inpatient admissions in the 365 days post discharge from the index admission, not PARC admissions. By contrast, PARC consumers could have inpatient admissions and PARC admissions during this period.

Table 3 quantifies the service use–related outcomes in terms of costs. Like Table 2, Table 3 considers these outcomes for the 365 days following discharge from the index admission. However, it also presents the costs of the index admission itself, for completeness. PARCS consumers’ index admissions were significantly less costly than inpatient-only consumers index admissions ($7966 vs $9439). As a total group, PARCS consumers also incurred significantly lower costs in terms of inpatient admissions in the subsequent year ($3429 vs $6872), but this comparative saving was lost when the cost of their PARCS admissions was added ($6697 vs 6872). Overall, there was no significant difference in cost per person of bed-based and community services between the two groups ($8235 vs $8400). When only those consumers who had used a given service type were considered, no significant differences in residential care costs or total service costs were found between the two groups. When considering the entire care pathway costs (including the index admission and 365-day follow-up), PARCS consumers had lower total costs than inpatient-only consumers ($16,201 vs $17,839); however, this result did not reach statistical significance. The sensitivity analysis using the Auditor-General’s cost estimates increased the inpatient bed day rate by around 40%. This alternative costing showed no significant difference between the two groups in residential care costs during the 365 days following the index admission; however, PARCS were associated with significantly lower costs than inpatient units at the index admission. Combining both the costs at index admission and total costs over the 1-year follow-up led PARCS to be associated with significantly lower mean costs for the full care pathway compared to inpatient units ($18,302 vs $27,836).

Table 3.

Costs of index admission and costs of inpatient and community care in the 365 days post discharge from the index admission.

Primary analysis
		PARCS consumers (N = 621)			Inpatient-only consumers (N = 621)
	Costs	M ($)	n	95% CI	M ($)	n	95% CI	z	p
Index admission		7966	621	[7616, 8317]	9439	621	[8684, 10,193]	−3.65	<0.001
The 365 days following index admission (all consumers)	Inpatient admissions	3429	621	[2641, 4216]	6872	621	[5215, 8529]	−4.1	<0.001
The 365 days following index admission (all consumers)	PARC admissions	3269	621	[2754, 3784]	0	0	N/A	N/A	N/A
	Inpatient + PARC admission	6697	621	[5680, 7715]	6872	621	[5215, 8529]	−0.18	0.86
	Community contacts	1537	621	[1330, 1745]	1528	621	[1359, 1698]	0.07	0.947
	Total costs (any service)	8235	621	[7127, 9343]	8400	621	[6691, 10,109]	−0.16	0.873
The 365 days following index admission (those who used each type of service in this period)	Inpatient admissions	14,290	149	[11,679, 16,901]	19,941	214	[15,631, 24,250]	−2.18	0.029
	PARC admissions	11,154	182	[10,037, 12,270]	0	0	N/A	N/A	N/A
	Inpatient + PARC admission	16,637	250	[14,670, 18,603]	19,941	214	[15,631, 24,250]	−1.5	0.133
	Community contacts	1917	498	[1670, 2165]	1843	515	[1649, 2037]	0.47	0.641
	Total costs – any services	10,269	498	[8946, 11,591]	10,129	515	[8100, 12,158]	0.11	0.911
	Total cost – index admission + follow-up	16,201	621	[15,032, 17,370]	17,839	621	[15,836, 19,843]	−1.42	0.157
Sensitivity analysis
		PARCS consumers (N = 621)			Inpatient-only consumers (N = 621)
	Costs	M ($)	n	95% CI	M ($)	n	95% CI	z	p
Index admission		7966	621	[7616, 8317]	15,224	621	[14,007, 16,441]	−11.30	<0.001
The 365 days following index admission (all consumers)	Inpatient admission	5530	621	[4260, 6780]	11,083	621	[8411, 13,756]	−3.69	<0.001
The 365 days following index admission (all consumers)	Inpatient + PARC admission	8799	621	[7342, 10,255]	11,083	621	[8411, 13,756]	−1.48	0.140
	Total costs (any service)	10,336	621	[8798, 11,875]	12,612	621	[9890, 15,333]	−1.43	0.153
	Total cost – index admission + follow up	18,303	621	[16,720, 19,885]	27,836	621	[24,646, 31,026]	−5.25	<0.001

PARCS: Prevention and Recovery Care Services; CI: confidence interval.

Table 4 shows that PARCS consumers were significantly less likely than inpatient-only consumers to experience an ITO (8.5% vs 17.4%) or a CTO (4.3% vs 11.4%) in the 365-day period following discharge from the index admission. It also suggests that PARCS consumers spent fewer days on ITOs and CTOs when they were on them, although the very high level of variability on these measures prevented these differences from reaching statistical significance.

Table 4.

Inpatient and community treatment orders in the 365 days post discharge from the index admission.

Percentage of consumers with ITOs and CTOs
	PARCS consumers (N = 621)			Inpatient-only consumers (N = 621)
	%	n	95% CI	%	n		95% CI	χ² (df)			p
ITO	8.5	53	[6.5, 11.0]	17.4	108		[14.5, 20.6]	21.6 (1)			<0.001
CTO	4.3	27	[2.9, 6.3]	11.4	71		[9.0, 14.2]	21.45 (1)			<0.001
Mean duration (days) on ITOs and CTOs
	PARCS consumers (N = 621)			Inpatient-only consumers (N = 621)
	M	n	SD	M		n	SD		U	z	p	d
ITO	61.08	53	165.47	104.51		108	226.84		2523	1.22	0.22	0.21
CTO	219.78	27	232.54	299.48		71	287.16		784	1.39	0.17	0.29

PARCS: Prevention and Recovery Care Services; CI: confidence interval; ITO: inpatient treatment order; CTO: community treatment order; SD: standard deviation.

Table 5 shows the results for clinical change during the index admissions as measured by the HoNOS total and sub-scale scores. These results provide an interesting contrast with the previous outcomes. Here, the inpatient-only consumers had better outcomes than the PARCS consumers. Both groups demonstrated notable admission to discharge reductions in HoNOS total score, but the inpatient-only consumers had a significantly larger reduction as measured by both ANCOVA and effect size comparisons. A similar pattern was evident for the four HoNOS sub-scale scores. For all four sub-scales, both groups reported reductions in scores with inpatient-only consumers revealing significantly larger reductions for three of four sub-scales; no significant effect size difference in reduction between the groups was found for Sub-scale A – Behavioural.

Table 5.

HoNOS scores at admission to discharge in the index admission.

HoNOS score		PARCS consumers (N = 621)		Inpatient-only consumers (N = 621)
HoNOS score		M	SD	M	SD	F	df	p	d
Total	Admission	13.61	5.64	13.92	6.43
	Discharge	10.03	5.73	8.26	5.39	37.61	1, 1239	<0.001	0.35
	d*	−0.64		−0.88
Sub-scale A: Behavioural	Admission	2.27	2.05	2.48	2.30
Sub-scale A: Behavioural	Discharge	1.58	1.84	1.48	1.81	3.42	1, 1239	0.07	0.11
	d	−0.34		−0.44
Sub-scale B: Impairment	Admission	1.71	1.64	1.72	1.70
Sub-scale B: Impairment	Discharge	1.47	1.62	1.08	1.38	24.50	1, 1239	<0.001	0.31
	d*	−0.15		−0.38
Sub-scale C: Symptoms	Admission	4.73	2.03	4.71	2.14
Sub-scale C: Symptoms	Discharge	3.27	1.90	2.78	2.03	20.73	1, 1239	<0.001	0.26
	d*	−0.72		−0.90
Sub-scale D: Social	Admission	4.90	3.18	5.01	3.40
	Discharge	3.71	2.98	2.93	2.68	27.76	1, 1239	<0.001	0.30
	d*	−0.37		−0.61

PARCS: Prevention and Recovery Care Services; SD: standard deviation; HoNOS: Health of the Nation Outcome Scales.

Inferential results are for analysis of covariance (ANCOVA) with associated effect size comparing groups at discharge, controlling for between-group variance at admission. d in the body of the table relates to the admission to discharge effect size calculated for each group separately. d* indicates that the two effect sizes were significantly different at p < 0.01 (adjusted for multiple comparisons).

Discussion

This study compared outcomes for consumers whose index admission in 2014 was to a PARC with consumers who were similar to them in many ways but whose index admission in the same year was to an acute inpatient unit and who had no PARC stays for the study period. The outcomes were specifically selected to allow us to answer three key research questions, reiterated below.

Are PARCS associated with reduced service use and costs?

PARCS consumers were significantly less likely than inpatient-only consumers to have an inpatient admission in the 365 days following discharge from the index admission. Their mean number of inpatient admissions was lower, as was their mean number of inpatient bed days. However, once their subsequent PARCS stays were added to the equation, their mean overall number of admissions and mean overall number of residential care days were significantly higher. As a consequence, although PARCS consumers had lower acute inpatient costs, their total residential care costs were similar to those for inpatient-only consumers. However, this finding was different and favoured PARCS consumers when a unit cost of inpatient care that was more in line with the VAGO’s (2019) estimate was used. A safe conclusion would be that the total residential and community service costs in the year post index admission and for the full pathway of care were no more for PARCS consumers than inpatient-only consumers and likely to be less.

The finding that PARCS consumers had fewer acute inpatient unit bed days than inpatient-only consumers suggests that PARCS are achieving their objective of reducing inpatient admissions. The finding that this comparatively lesser use of inpatient admissions by PARCS consumers was accompanied by additional use of PARCS and more residential care days overall is consistent with an evaluation of the comparable Queensland crisis house model by Siskind et al. (2013).

Do PARCS foster least restrictive care?

The fact that PARCS consumers were less likely to have subsequent inpatient admissions is consistent with the provision of least restrictive care. PARCS are purpose-built or renovated with attention to resident amenity, and they have a more domestic feel than inpatient units which often have features like seclusion rooms. Most PARCS have 10 beds or fewer, in contrast with inpatient units which usually have 25–29 beds. Consumers also frequently experience inpatient units as custodial and unsafe (Bower et al., 2003; Brophy et al., 2016; Fletcher et al., 2019b). For these reasons, PARCS stays are likely to be viewed as a less restrictive, more acceptable option than inpatient admissions by consumers and, potentially, clinicians and families. Any ‘substitution’ of PARCS admissions for inpatient admissions may provide an opportunity for more substantive support for personal recovery in an environment that generally receives high rates of satisfaction (Lee et al., 2014; Ngo et al., 2020; Thomas et al., 2017b). Having said this, since the availability of both PARCS and inpatient beds is limited, it is not possible to determine whether the subsequent patterns of service use by the PARCS and inpatient-only consumers in our study reflected consumer choice, clinical decision-making or availability.

The other variables that relate to the provision of least restrictive care are those concerning involuntary treatment orders. PARCS consumers were significantly less likely to experience an ITO or a CTO in the 365 days following their index admission than were matched inpatient-only consumers. They also tended to spend less time on each type of order, although the latter differences did not reach statistical significance. Taken together, these findings suggest that offering services that are more attractive, more likely to provide opportunities for recovery-oriented care and more likely to be received with high levels of satisfaction than traditional inpatient care may have the long-term impact of minimising compulsory treatment (Brophy and McDermott, 2003).

Are PARCS associated with positive clinical outcomes?

PARCS admissions were associated with positive clinical outcomes, with admission-to-discharge HoNOS total scores moving from the ‘moderate’ to ‘mild’ range (Parabiaghi et al., 2014). However, the improvements for PARCS consumers were smaller in magnitude than those for inpatient-only consumers, and inpatient-only consumers were discharged with lower HoNOS total scores. The average improvement of more than five points for the inpatient-only group exceeded the criterion of at least four points for a clinically relevant change in total score (Egger et al., 2015; Parabiaghi et al., 2014), whereas the mean change for the PARCS group fell slightly short of this criterion. The relatively modest degree of HoNOS change for PARCS consumers and the residual mild level of severity at discharge are consistent with other studies (Department of Health and Human Services, 2019; Lee et al., 2014; Thomas et al., 2017a).

One interpretation of these findings is that although PARCS lead to positive clinical outcomes for consumers, inpatient units yield even greater improvements in this regard. Others have suggested that this may be the result of ‘more intensive treatment’ in the inpatient setting (Department of Health and Human Services, 2016: 12). Our analysis of costs lends some support to this, at least to the extent that resource inputs might be considered to be a marker of intensity of treatment; on average, index inpatient admissions were more expensive than index PARCS admissions ($9439 vs $7966). Another explanation may be that although clinical improvement is important for PARCS, they have an even stronger emphasis on personal recovery.

Strengths and limitations

This study provides the strongest evidence to date on the outcomes of PARCS. It drew on data from all 19 adult PARCS that were operational in Victoria during the 5-year study period. It considered a range of outcomes that were selected to assess the objectives that PARCS are designed to achieve. It used sophisticated techniques to match PARCS consumers to an appropriate comparison group on the basis of demographics, diagnosis and clinical severity, allowing us to be confident that we were making comparisons between substantively equivalent groups.

The study had some limitations, however. Our choice of a matched pairs design allowed us to make ‘like-with-like’ comparisons but meant that our inpatient-only sample was not representative of all inpatients, so inferences about inpatients as a whole cannot be made. For that matter, our PARCS sample only included consumers whose index admission in 2014 was to a PARC; there may have been differences between this group and other PARCS users.

PS matching has been criticised by King and Neilson (2019) as introducing various biases to matched samples, although we would note that our approach adopted several of King and Neilson’s recommendations (e.g. a small calliper value). Additionally, our PARCS and inpatient-only consumers were similar on the key domains we used for matching but they may have differed on other variables (e.g. level of acuity or risk at first admission, morbidity history). It may be that those who were perceived to have less acute symptoms or be at lower levels of risk were appropriately given the option of a PARCS admission and that their subsequent patterns of care – particularly with respect to ITOs and CTOs – may have had more to do with their typical pattern of clinical risk than their initial PARC stay. Similarly, if the PARC consumers had a history of a shorter duration of illness, less comorbidity or less severe diagnoses, this may have contributed to their lower levels of inpatient service use in the 365-day follow-up period.

We purposely selected outcome measures that allowed us to test the extent to which PARCS were meeting their objectives. However, we were constrained by the data that were available in the CMI/ODS. The data that allowed us to look at service use (and consequently costs) and least restrictive care were fit for purpose, but the data on clinical outcomes may not have been ideal for several reasons. The HoNOS is a valid and reliable measure of clinical change (Pirkis et al., 2005; Trauer et al., 1999), but it is not always optimally completed and there may have been biases in the way in which it was administered in PARCS compared with inpatient settings. Previous studies have indicated that there are systematic differences across other service settings, with significantly higher ratings recorded by community teams at intake compared with ratings made by inpatient staff at discharge less than 4 days earlier (Luo et al., 2016). In addition, and as alluded to above, assessing consumer outcomes on the basis of clinical change alone was imperfect, given the emphasis of PARCS on supporting personal recovery. The HoNOS social sub-scale came closest to assessing this, but even this does not purport to be a measure of personal recovery. We are currently conducting a longitudinal outcomes study and an in-depth qualitative study which will shed further light on the extent to which PARCS lead to personal recovery benefits.

Conclusion

Victorian PARCS appear to be largely achieving their stated objectives. Consumers who have an initial PARCS stay make less subsequent use of acute inpatient services and, on balance, incur costs that are similar to or lower than consumers who only use inpatient services. They are also less likely to spend time on an involuntary treatment order following their index admission than their inpatient-only counterparts. PARCS consumers also experience positive clinical outcomes over the course of their index admission, although the magnitude of this improvement is not as great as for inpatient-only consumers. Although we cannot completely rule out the possibility that PARCS consumers and inpatient-only consumers have different levels of acuity, risk and morbidity history, it seems reasonable to conclude that PARCS can provide an alternative, less restrictive care option for eligible consumers who might otherwise be admitted to an acute inpatient unit and do so at no greater cost.

Footnotes

Author contributions

J.Fa. contributed to the grant application, study design and analyses and co-wrote the manuscript. L.B. conceived the study, obtained funding, participated in the analysis and interpretation of data, and co-wrote the manuscript. J.R. led the analysis of service data and co-wrote the manuscript. H.T. contributed to the acquisition of data, led initial data analyses and contributed to the manuscript. L.K.-D.L. conducted and wrote the analysis of costs. C.M. designed the costs analyses and contributed to critical revision of the manuscript. J.Fl. contributed to the study concept and design, and participated in the analysis and interpretation of data. C.H. contributed to the grant application and study design, participated in the interpretation of data and critical revision of the manuscript. E.M. contributed to the acquisition of data, critical revision of the manuscript and administrative tasks. R.N. contributed to the study design, interpretation of data and critical revision of the manuscript. G.S. obtained the data and participated in planning the analyses. M.J.S. contributed to the study design and critical revision of the manuscript. G.M. contributed to the study design and the critical revision of the manuscript. R.V. contributed to interpretation of data and critical revision of the manuscript. J.P. provided overall leadership of the study, contributed to the study design, participated in critical revision of the manuscript and wrote the final draft. All authors read and approved the final manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: The project was funded by an NHMRC partnership grant (APP1115907) and is a partnership between academic institutions, Mental Health Community Support Services (MHCSS), clinical mental health service providers and the Victorian Government. M.J.S. is a recipient of an Australian Research Council Future Fellowship (project number FT180100075) funded by the Australian Government.

ORCID iDs

John Farhall

Lisa Brophy

Long Khanh-Dao Le

Cathrine Mihalopoulos

Georgina Sutherland

Matthew J Spittal

Graham Meadows

Jane Pirkis

References

Allison

Bastiampillai

(2015) Mental health services reach the tipping point in Australian acute hospitals. Medical Journal of Australia 203: 432–434.

Allison

Bastiampillai

Licinio

, et al. (2018) When should governments increase the supply of psychiatric beds? Molecular Psychiatry 23: 796–800.

Australian Institute of Health and Welfare (2020) Mental Health Services in Australia. Available at: www.aihw.gov.au/reports/mental-health-services/mental-health-services-in-australia/report-contents/expenditure-on-mental-health-related-services/specialised-mental-health-services-expenditure (accessed 21 December 2020).

Bower

McCullough

Timmons

(2003) A synthesis of what we know about the use of physical restraints and seclusion with patients in psychiatric and acute care settings: 2003 update. Online Journal of Knowledge Synthesis for Nursing 10: 1–29.

Brophy

McDermott

(2003) What’s driving involuntary treatment in the community? The social, policy, legal and ethical context. Australasian Psychiatry 11: S84–S88.

Brophy

Roper

Hamilton

, et al. (2016) Consumers and their supporters’ perspectives on poor practice and the use of seclusion and restraint in mental health settings: Results from Australian focus groups. International Journal of Mental Health Systems 10: 6.

Burgess

Pirkis

Coombs

(2015) Routine outcome measurement in Australia. International Review of Psychiatry 27: 264–275.

Deb

Norton

(2018) Modeling health care expenditures and use. Annual Review of Public Health 39: 489–505.

Department of Health (2010) Adult Prevention and Recovery Care (PARC) Services Framework and Operational Guidelines. Melbourne, VIC, Australia: Victorian Government.

10.

Department of Health (2011) Framework for Recovery-oriented Practice. Melbourne, VIC, Australia: Victorian Government.

11.

Department of Health (2014) Health Policy and Funding Guidelines 2014-15. Part 2: Pricing and Funding Arrangements for Victoria’s Health System. Melbourne, VIC, Australia: Victorian Government.

12.

Department of Health and Human Services (2016) Mental Health Prevention and Recovery Care: A Clinical and Community Partnership Model of Sub-acute Mental Health Care. Melbourne, VIC, Australia: Victorian Government.

13.

Department of Health and Human Services (2019) Adult Mental Health Quarterly KPI Report (Revised). Melbourne, VIC, Australia: Victorian Government.

14.

Egger

Weniger

Prinz

, et al. (2015) Health of the Nation Outcome Scales in a psychiatric inpatient setting: Assessing clinical change. Journal of Evaluation in Clinical Practice 21: 236–241.

15.

Fletcher

Brophy

Killaspy

, et al. (2019a) Prevention and Recovery Care services in Australia: Describing the role and function of sub-acute recovery-based residential mental health services in Victoria. Frontiers in Psychiatry 10: 735.

16.

Fletcher

Buchanan-Hagen

Brophy

, et al. (2019b) Consumer perspectives of safewards impact in acute inpatient mental health wards in Victoria, Australia. Frontiers in Psychiatry 10: 461.

17.

Glick

Doshi

Sonnad

, et al. (2014) Economic Evaluation in Clinical Trials. Oxford: Oxford University Press.

18.

Harvey

Brophy

Tibble

, et al. (2019) Prevention and Recovery Care Services in Australia: Developing a state-wide typology of a sub-acute residential mental health service model. Frontiers in Psychiatry 10: 383.

19.

Heyeres

Kinchin

Whatley

, et al. (2018) Evaluation of a residential mental health recovery service in North Queensland. Frontiers in Public Health 6: 123.

20.

Kinchin

Russell

AMT

Tsey

, et al. (2019) Psychiatric inpatient cost of care before and after admission at a residential subacute step-up/step-down mental health facility. Journal of Medical Economics 22: 491–498.

21.

King

Neilson

(2019) Why propensity scores should not be used for matching. Political Analysis 27: 435–454.

22.

Lee

Collister

Stafrace

, et al. (2014) Promoting recovery via an integrated model of care to deliver a bed-based, mental health prevention and recovery centre. Australasian Psychiatry 22: 481–488.

23.

Luo

Harvey

Tran

, et al. (2016) Consistency of the Health of the Nation Outcome Scales (HoNOS) at inpatient-to-community transition. BMJ Open 6: e010732.

24.

Ngo

Ennals

Turut

, et al. (2020) Step-up, step-down mental health care service: Evidence from Western Australia’s first: A mixed-method cohort study. BMC Psychiatry 20: 214.

25.

Parabiaghi

Kortrijk

Mulder

(2014) Defining multiple criteria for meaningful outcome in routine outcome measurement using the Health of the Nation Outcome Scales. Social Psychiatry and Psychiatric Epidemiology 49: 291–305.

26.

Pirkis

Burgess

Kirk

, et al. (2005) A review of the psychometric properties of the Health of the Nation Outcome Scales (HoNOS) family of measures. Health and Quality of Life Outcomes 3: 76.

27.

Ramsey

Willke

Glick

, et al. (2015) Cost-effectiveness analysis alongside clinical trials II – An ISPOR Good Research Practices Task Force report. Value in Health 18: 161–172.

28.

Select Committee on Mental Health (2006) A National Approach to Mental Health: From Crisis to Community. Canberra, ACT, Australia: Commonwealth of Australia.

29.

Siskind

Harris

Kisely

, et al. (2013) A retrospective quasi-experimental study of a community crisis house for patients with severe and persistent mental illness. Australian and New Zealand Journal of Psychiatry 47: 667–675.

30.

Sutherland

Harvey

Tibble

, et al. (2020) Similarities and differences in people accessing prevention and recovery care services and inpatient units in Victoria, Australia. BMC Health Services Research 20: 542.

31.

Sweeney

Fahmy

Nolan

, et al. (2014) The relationship between therapeutic alliance and service user satisfaction in mental health inpatient wards and crisis house alternatives: A cross-sectional study. PLoS ONE 9: e100153.

32.

Thoemmes

(n.d.) Propensity Score Matching in SPSS. Tübingen: University of Tübingen.

33.

Thoemmes

West

Hill

(2009) Propensity score matching in a meta-analysis comparing randomized and non-randomized studies. Multivariate Behavioral Research 44: 854.

34.

Thomas

Rickwood

(2013) Clinical and cost-effectiveness of acute and subacute residential mental health services: A systematic review. Psychiatric Services 64: 1140–1149.

35.

Thomas

Rickwood

Brown

(2017a) Symptoms, functioning and quality of life after treatment in a residential sub-acute mental health service in Australia. Health and Social Care in the Community 25: 243–254.

36.

Thomas

Rickwood

Brown

(2017b) Symptoms, functioning and quality of life after treatment in a residential sub-acute mental health service in Australia. Health and Social Care in the Community 25: 243–254.

37.

Trauer

Callaly

Hantz

, et al. (1999) Health of the Nation Outcome Scales. British Journal of Psychiatry 174: 380–388.

38.

Trauer

Farhall

Newton

, et al. (2001) From long-stay psychiatric hospital to Community Care Unit: Evaluation at 1 year. Social Psychiatry and Psychiatric Epidemiology 36: 416–419.

39.

Victorian Auditor-General’s Office (VAGO) (2019) Access to Mental Health Services (Independent Assurance Report to Parliament 2018–19). Melbourne, VIC, Australia: VAGO.

40.

Wing

Beevor

Curtis

, et al. (1998) Health of the Nation Outcome Scales (HoNOS). Research and development. British Journal of Psychiatry 172: 11–18.