Personalization in the health care system: do personal health budgets have an impact on outcomes and cost?

Abstract

Objectives

In England’s National Health Service, personal health budgets are part of a growing trend to give patients more choice and control over how health care services are managed and delivered. The personal health budget programme was launched by the Department of Health in 2009, and a three-year independent evaluation was commissioned with the aim of identifying whether the initiative ensured better health- and care-related outcomes and at what cost when compared to conventional service delivery.

Methods

The evaluation used a pragmatic controlled trial design to compare the outcomes and costs of patients selected to receive a personal health budget with those continuing with conventional support arrangements (control group). Just over 1000 individuals were recruited into the personal health budget group and 1000 into the control group in order to ensure sufficient statistical power, and followed for 12 months.

Results

The use of personal health budgets was associated with significant improvement in patients’ care-related quality of life and psychological wellbeing at 12 months. Personal health budgets did not appear to have an impact on health status, mortality rates, health-related quality of life or costs over the same period. With net benefits measured in terms of care-related quality of life on the adult social care outcome toolkit measure, personal health budgets were cost-effective: that is, budget holders experienced greater benefits than people receiving conventional services, and the budgets were worth the cost.

Conclusion

The evaluation provides support for the planned wider roll-out of personal health budgets in the English NHS after 2014 in so far as the localities in the pilot sample are representative of the whole country.

Keywords

quality of life quantitative methods research policy

Introduction

The personal health budget initiative was proposed in the 2008 National Health Service (NHS) Next Stage Review as a way to encourage the NHS to become more responsive to the needs of patients. It was argued that the provision of greater choice to patients around the type and extent of health care would ultimately result in improved system efficiencies.¹ The 2010 White Paper Equity and Excellence: Liberating the NHS re-affirmed the importance of ensuring that patients are involved in all decision making.¹ Subsequently, the Department of Health published two consultations outlining proposals to secure shared decision-making over care and treatment.^2,3

The principles underlying the personal health budgets initiative⁴ were drawn from the experience in social care with personal (individual) budgets⁵ and include the following:

Recipients know the resource level of the available budget.

Patients are encouraged to develop a support/care plan that details how the resource will be used to meet their identified needs.

Patients decide how they would like the budget to be managed. There are three options: notionally, where the budget is held by the commissioner, but the budget holder is aware of the service options and their financial implications; managed by a third party; or as a direct payment.

Personal health budget pilot programme

The personal health budget pilot programme was launched by the Department of Health in 2009.⁴ An independent evaluation was commissioned to run alongside the pilot programme. The overarching aim was to identify whether personal health budgets ensured better health and care outcomes when compared to conventional service delivery and, if so, the best way for the initiative to be implemented. Of the 64 pilot sites involved in piloting personal health budgets, 20 sites were selected to be in-depth evaluation sites, with the remainder being wider-cohort sites. The in-depth sites offered personal health budgets to individuals with the following health characteristics: long-term conditions (including chronic obstructive pulmonary disease (COPD), diabetes and long-term neurological conditions; mental health issues; NHS Continuing Healthcare; and stroke).

In this paper, we focus on the evaluation of the effectiveness and cost-effectiveness of personal health budgets.

Method

The evaluation used a controlled trial with a pragmatic design to compare the experiences of people selected to receive personal health budgets with those continuing with conventional support arrangements during the study period. Participants were recruited to the study in one of two ways, based on how personal health budgets were being implemented within the pilot sites. In some pilot sites the personal health budget group was recruited by those health care professionals offering budgets, while a control group was recruited by non-participating health care professionals in the same site. Other sites randomized patients into either the personal health budget group or the control group (25% of participants were randomly assigned).

Data about respondents were collected at baseline (between April 2010 and June 2011, after gaining informed consent from participants) and again 12 months later (April 2011 to June 2012).

The National Research Ethics Service conferred a favourable ethical opinion for the evaluation. Subsequently, the research was given Research Governance management authorization to commence the study in each pilot site.

Data collection

Figure 1 shows the sequencing of the main quantitative data collection within the in-depth pilot sites.

Figure 1

Main quantitative data collection.

A number of validated outcome measures were used in the structured interviews, including the following:

Health-related quality of life was measured using EQ-5D (Euro-Qol).⁶

An early version of the Adult Social Care Outcome Toolkit (ASCOT) measure (Netten et al.⁷) that reflected level of need along nine care-related dimensions: personal care/comfort; social participation and involvement; control over daily life; meals and nutrition; safety; accommodation cleanliness and comfort; occupation and usual activities; anxiety; and dignity and respect.

Subjective global scale based on the measure used by the Office for National Statistics (ONS) in the Integrated Household Survey (IHS) that captured general life happiness and satisfaction.⁶

Psychological wellbeing as measured by the 12-item version of the General Health Questionnaire.⁸

In addition, demographic and socio-economic information was collected within the interviews, as well as information about current circumstances.

Participants’ health condition, clinical indicators and their use of primary health care services were gathered from GP records, whilst their use of secondary care was extracted from the Hospital Episodes Statistics database (Health and Social Care Information Centre). Both sets of data were collected at two time-points during the study period: first, around the time of consent to explore the previous 12 months’ activity; second, around 12 months after participants agreed to take part to gather information for the year following consent. The data collection allowed the evaluation team to explore whether personal health budgets had an impact on the use of either primary or secondary care compared to receiving conventional services.

For personal health budget holders, the support/care plan was used to extract information about the size of the budget, the deployment of the budget, and the support/services that the budget was spent on. Personal health budgets were used to secure an array of services and support, such as home care services, transport, complementary therapies, talking therapies, physiotherapy, chiropody and psychiatric appointments.

Statistical analyses

Study consent was initially gained from 2700 people. Some 302 people were excluded from the study because: they had not taken part in the baseline (or follow-up) interviews and had in effect withdrawn consent before baseline; they were in residential care at baseline; or they had died before baseline. The remaining 2398 cases were suitable for multiple imputation. Of these, 158 people died before follow-up and a further five cases were excluded because they were aged under 18. This left an active sample of 2235 cases, with 1171 in the personal health budget group and 1064 in the control group.

To identify the effects of using personal health budgets, it was necessary for the evaluation to address four key issues. First, a method was needed to attribute any relevant differences between the personal health budgets group and the control group as being due to the use of the budget and not any confounding variable (e.g. service restructure). Secondly, we needed to be confident that the differences observed were not just due to chance owing to how participants in each group had been sampled. Thirdly, specific statistical techniques would be needed to compensate for a dataset that inevitably contained some missing values. Finally, in comparing the costs and benefits of complex interventions such as personal health budgets, it was necessary to recognize that we could not collect data on every possible impact or detail of implementation.

As regard the first issue, the data collection was designed to allow a difference-in-difference method to be used. This approach helps to remove any differences between the personal health budget and control groups in the level of the outcome or benefit and costs indicator at baseline. We compared our outcome indicators at follow-up between the two groups after subtracting any difference between the groups in the relevant indicator at baseline. The impact measure is therefore the follow-up difference net of any baseline difference in the indicator in question. This approach assumes that, without the intervention, the situation of the intervention group would change through time (on average) by the same amount as the control group. As an additional safeguard against selection bias, we also used multiple regression to account for any differences between groups in the change in costs and outcomes due to confounding baselines factors (such as sociodemographic and socioeconomic factors and health status: for example, health condition and comorbidities), not use of personal health budgets.

Regarding the second problem, we collected data from samples of people using personal health budgets and conventional service delivery. The value of collected indicator variables for each group therefore only imperfectly reflects the true value: they are subject to statistical noise. For this reason we calculated both parametric (i.e. assumed normally distributed) and non-parametric (bootstrapped) statistical error margins. We thus report the statistical significance of the results using the following significance levels for the interpretation: either a 10% (P > 0.01), 5% (P > 0.05) or 1% (P < 0.01) probability that an observed effect may have occurred by chance, as reported below.

A multiple imputation approach was used to tackle missing data. This technique uses information inherent in the whole dataset to predict what the random missing values would have been. It requires that the reasons for the data being missing must be accounted for by factors that do not have missing values.⁹ The pattern of missing data in the sample was as follows. Regarding EQ-5D, ASCOT, and General Health Questionnaire (GHQ) scores, at least some follow-up outcomes data were available in 1656 cases (74% of the active sample of 2235 cases). There were 2104 cases (94%) with at least some service data at follow-up and 2133 cases (95%) with either some follow-up outcomes data or some service data.

To tackle the fourth issue, we needed to make assumptions, albeit based on discussion with study participants about some of the detail. A key assumption in this regard was the identification of personal health budgets provided in addition to or as a substitute for conventional services.

Cost-effectiveness was assessed by estimating whether the personal health budget group experienced better quality of life than the control group who received conventional service delivery, after netting-off the difference in service and support costs between the groups. Quality of life (both EQ-5D and ASCOT) was expressed in monetary terms – by applying willingness-to-pay (WTP) thresholds to the quality of life indicators – to allow the netting-off of costs. At the time of writing, the National Institute for Health and Clinical Excellence (NICE) operates with willingness-to-pay thresholds of between £20,000 and £30,000: that is, if an intervention improves a person’s quality of life by an amount corresponding to being in full health compared to a state that is no better than being dead over the course of a year, the value of this improvement in monetary terms is between £20,000 and £30,000.

Net monetary quality of life at any time is therefore equal to: quality of life level, times WTP threshold, minus costs of services used. The estimated probability of this value being greater for the personal health budget group at follow-up compared to the control group (after subtracting the respective baseline values to account for any baseline differences between the groups), can be interpreted as the probability that personal health budgets were (more) cost-effective than conventional service delivery.

Sensitivity analysis was conducted regarding assumptions about the costing of services used by the personal health budgets group, and about the assumptions used in the multiple imputation procedure. We tested the sensitivity of three types of assumptions:

Statistical assumptions. For key analyses such as the cost-effectiveness estimates we used both parametric and non-parametric (bootstrapping) methods. We found very little difference in the results.

Costing assumption. On testing the sensitivity of the main results to this assumption, we did not find any qualitative impact on the results until quite unrealistic assumptions were tried.

Multiple imputation. To test the sensitivity of the main results, we first added a further five imputations to our main dataset with a different randomly selected seed value, and second, used a variant imputation model. The main results in both alternative cases were very similar to the original estimates, with, if anything, slightly better statistical significance. In particular, with both the alternative dataset and the alternative imputation model, the results for the whole sample analysis indicated that personal health budgets were cost-effective on the ASCOT scale at the 5% significance level rather than at 10%.

Results

Table 1 shows the primary health condition breakdown by personal health budget group and control group at baseline and at 12 months within the active sample.

Table 1.

Completed main outcome questionnaires by health condition.

Baseline	Main follow-up
Personal health budget group	1141	663
NHS continuing healthcare	153	94
Diabetes	170	97
Stroke	116	71
Mental health	228	105
COPD	192	140
Neurological	284	159
Control group	1027	678
NHS continuing healthcare	86	61
Diabetes	235	165
Stroke	116	83
Mental health	184	92
COPD	152	111
Neurological	262	173

Variation in clinical outcomes

In the sample, the mortality rate overall at follow-up was 7.7%. Rates in the sample were slightly higher for the personal health budget group, but this difference was not statistically significant (P = 0.109). Mortality rates were higher overall for the NHS Continuing Healthcare sub-group than the rest of the sample, at 15.43% overall (12.42% control and 16.97% personal health budget group sample), but again this difference was not statistically significant, suggesting that the use of personal health budgets was not associated with differential mortality.

Exploring clinical outcomes for specific conditions, Table 2 shows that personal health budgets did not a have significant impact for the diabetes sub-group (HbA1c) or for the COPD sub-group (lung-function FEV1) when compared to conventional service delivery.

Table 2.

Change in clinical outcomes between baseline and follow-up.

Coeff	P > t
HbA1c – Diabetes health cohort	−0.481	0.449
FEV1 – COPD health cohort	0.069	0.755

Significance levels: *P < 0.10, **P < 0.05, ***P < 0.001.

Variations in subjective outcomes

Individuals in the personal health budget group reported statistically significantly improved care-related quality of life (ASCOT) and psychological wellbeing (GHQ-12) compared with people in the control group (see Tables 3 and 4). However, there was no significant difference for health care-related quality of life (EQ-5D) or subjective wellbeing.

Table 3.

ASCOT and EQ-5D outcome difference-in-difference, personal health budget group, with control factors.

	Care-related quality of life (ASCOT)		Health-related quality of life (EQ-5D)
	Coeff	Prob	Coeff	Prob
PHB group	0.028	0.047**	−0.018	0.167
Age	−0.002	<0.001***	−0.001	0.023**
Male	−0.004	0.741	0.011	0.432
ADL score	2.11E-04	0.813	−0.004	<0.001***
Receives benefits	−0.014	0.420	0.011	0.427
Uni/college educ.	0.010	0.701	0.019	0.175
Intermediate educ.	−0.004	0.840	0.022	0.198
Health condition
Continuing healthcare	0.009	0.656	−0.074	0.001**
Stroke	−0.004	0.873	−0.001	0.977
Diabetes	0.044	0.146	−3.18E-04	0.988
Mental health	0.042	0.176	−0.012	0.635
COPD	0.040	0.140	0.016	0.514
Neurological	0.043	0.215	−0.022	0.298
Follow-up period	2.90E-04	0.319	2.95E-05	0.889
Consent date	−2.71E-05	0.810	7.55E-05	0.473
Area cost adjust	0.079	0.564	0.193	0.186
Area
Town & fringe	0.026	0.310	0.014	0.639
Rural	0.019	0.578	0.036	0.114
Constant	0.385	0.858	−1.501	0.456
N	2235	2235
Model F	2.010	0.011**	2.000	0.011**
Controls - Joint sig	1.670	0.052*	2.110	0.008**

Significance levels: *P < 0.10, **P < 0.05, ***P < 0.001.

Table 4.

GHQ-12 and subjective wellbeing difference-in-difference, personal health budget group, with control factors.

	Psychological wellbeing (GHQ-12)		Subjective wellbeing
	Coeff	Prob	Coeff	Prob
PHB group	−0.852	0.096*	0.762	0.213
Age	0.027	0.028**	−0.042	0.022**
Male	1.030	0.059*	−0.669	0.110
ADL score	0.113	0.013**	−0.041	0.306
Receives benefits	−0.291	0.604	0.132	0.865
Uni/college educ.	−0.334	0.561	0.446	0.457
Intermediate educ.	0.288	0.648	−0.755	0.266
Health condition
Continuing healthcare	1.423	0.060*	−1.391	0.165
Stroke	−1.801	0.033**	0.569	0.633
Diabetes	−1.891	0.047*	1.563	0.101
Mental health	−0.459	0.653	2.233	0.066*
COPD	−1.278	0.136	1.141	0.350
Neurological	−1.119	0.153	1.015	0.410
Follow-up period	−0.003	0.663	0.014	0.062*
Consent date	−1.57E-04	0.954	0.002	0.611
Area cost adjust	1.016	0.842	−0.141	0.981
Area
Town & fringe	−0.549	0.415	0.947	0.295
Rural	−1.048	0.270	1.305	0.166
Constant	−1.188	0.982	−45.595	0.605
N	2235	2235
Model F	2.220	0.004**	1.790	0.025**
Controls – Joint sig	1.880	0.020**	1.590	0.064*

Significance levels: *P < 0.10, **P < 0.05, ***P < 0.001.

Were personal health budgets cost-effective?

In terms of care-related quality of life (ASCOT), the personal health budget group showed greater benefit (quality of life) at less cost, on average, than the control group. As shown in Table 5, net quality of life benefit was between £1520 and £2690 greater for the personal health budget group than the control group, after subtracting baseline differences. For example, at the £30,000 threshold, the extra net benefit averaged £2300 (£1180 minus £1120) more for the personal health budget group than the control group. The improvement in net benefit was statistically significant at the £30,000 WTP threshold and above. There was no statistically significant difference in net benefit between the groups when using the EQ-5D quality of life measure (Table 6). Sensitivity analysis supported these results: if anything, it showed personal health budgets to be cost-effective using ASCOT at higher significance levels.

Table 5.

Difference in mean NMB-change for ASCOT, whole sample, various cost effectiveness thresholds.

PHB	Control	Difference	Sig prob $(p : \neq 0)$	90% CI−	90% CI+
Benefits
ASCOT change	0.057	0.018	0.039
£-value of ASCOT change:
WTP £40,000	2290	720	1570
WTP £30,000	1720	540	1180
WTP £20,000	1150	360	790
WTP £10,000	580	180	400
Costs
Cost change	800	1920	−1120
Net benefit
NMB change:
WTP £40,000	1490	−1200	2690	0.057*	410	4970
WTP £30,000	920	−1380	2300	0.082*	140	4460
WTP £20,000	350	−1560	1910	0.124	−150	3960
WTP £10,000	−220	−1740	1520	0.198	−450	3490

Significance levels: *P < 0.10, **P < 0.05, ***P < 0.001.

WTP = Willingness-to-Pay.

NMB = Net Monetary Benefit.

Table 6.

Difference in mean NMB-change for EQ-5D, whole sample, various CE thresholds.

PHB	Control	Difference	Sig prob $(p : \neq 0)$	90% CI−	90% CI+
Benefits
EQ-5D change	−0.011	0.000	−0.011
£-Value of EQ-5D change:
£40,000	−420	0	−420
£30,000	−310	0	−310
£20,000	−210	0	−210
£10,000	−100	0	−100
Costs
Cost change	800	1920	−1120
Net benefit
NMB change:
£40,000	−1220	−1920	700	0.613	−1710	3110
£30,000	−1110	−1920	810	0.536	−1450	3060
£20,000	−1010	−1920	910	0.459	−1200	3030
£10,000	−900	−1920	1020	0.386	−980	3020

Significance levels: *P < 0.10, **P < 0.05, ***P < 0.001.

WTP = Willingness-to-Pay.

NMB = Net Monetary Benefit.

Discussion

In as far as the localities in the sample were representative of the whole country, and notwithstanding the methodological challenges in the study, as summarized below, the results provide support for the planned national roll-out of personal health budgets. The study suggested that personal health budgets had a positive impact on care-related quality of life and psychological wellbeing.

Health and clinical outcomes (other than psychological health) appeared not to be affected by the use of personal health budgets. Due to the relatively short follow-up period used in this study (one year), it is unsurprising that the underlying health status of patients was unaffected. Furthermore, personal health budgets could be seen as a vehicle to effectively manage the health condition rather than improve clinical health status.

A largely neutral impact on (recurrent) costs was also found. Overall, the results suggested that personal health budgets could cost-effectively improve care-related quality of life results, without negative effects on health status.

With the evaluation results available to help inform policy decisions, the Government announced (25 September 2012) that £1.5 million will become available to support the roll-out of the initiative beyond the pilot programme. Previously, the Secretary of State for Health had announced that, subject to the evaluation, by April 2014 everyone in receipt of NHS Continuing Healthcare (NHS CHC) will have the right to ask for a personal health budget, including a direct payment. Furthermore, it is planned that the new Clinical Commissioning Groups (CCGs) will be able to offer personal health budgets more widely.¹⁰

Nonetheless, there are a number of limitations that should be acknowledged when interpreting these results. Rather than a single intervention, personal health budgets were variously implemented, with different models operating in the twenty in-depth pilot sites across six patient groups. Personal health budgets were entirely new and a radical departure in some areas, and the overall structure and processes developed and changed during the course of the evaluation. Designing the study selection criteria in advance was not always possible. A particular difficulty was in establishing what the personal health budget was for and which services could be purchased.

Another tension in the design of the evaluation was between allowing sufficient elapsed time after baseline for the effects of personal health budgets to be felt on the one hand, and minimizing loss to follow-up on the other. The experience from the evaluation of the individual budgets pilots in social care (Glendinning et al.⁵) was that a six-month follow-up period was unlikely to be sufficient, and so we opted for a main follow-up period of one year. As a consequence, although final recruitment rates were good, drop-out rates were an issue and potentially impacted on the robustness of the evaluation findings and the extent to which results can be extrapolated. The study population was also in the most part very frail, with much lower than population-average health status and wellbeing scores. We expected drop-out rates to be higher for this study population as a result, but we could argue that the reasons for drop-out are due to baseline factors to a significant extent, and are therefore not a source of bias in the evaluation.

The quality of the structured interview outcome data was good, particularly the main subjective instruments. As we had to rely on local site tracking and records regarding mortality data, we were less able to rate their quality. Service data were drawn from a number of sources. Where possible, we did not rely on self-reported use; instead, we interrogated care plans, medical records and hospital episode statistics. Another issue was the sheer range of services and support that could be purchased, which resulted in a number of assumptions being made to produce like-with-like cost estimates between personal health budget and control groups.

A final consideration is that such a complex intervention presents methodological challenges necessarily resulting in the development of appropriate underpinning assumptions. We explored the sensitivity of the main findings by re-estimating net benefit differences with changes in the following:

Imputation dataset (created by adding further imputations);

Imputation models;

Budget level that constitutes personal health budgets substituting for, rather than being provided in addition to, conventional services.

In the main, the sensitivity analysis demonstrated a higher degree of statistical significance of the key results. Systemic interventions such as personal health budgets preclude the use of fully double-blinded randomized controlled trials (RCTs) and, although we used a range of methods to tackle the consequences of a more pragmatic design, it is important to be aware of the discussed associated limitations.

Conclusion

In conclusion, despite the study limitations, the evaluation did find that personal health budgets were cost-effective and had a positive impact on subjective outcomes. Generally, the findings provide support for the further implementation of personal health budgets after 2014.

Footnotes

Acknowledgements

The research is funded by the Department of Health. However, any views expressed in the report are those of the research team alone. We are grateful to the other team members within the evaluation of personal health budgets for comments on earlier drafts of the paper including: Kate Baxter, Jacqueline Davidson, Paul Dolan, Annie Irvine and Dominic King.

Declaration of conflicting interests

None declared.

Funding

This is an independent report commissioned and funded by the Policy and Strategy Directorate in the Department of Health. The views expressed are not necessarily those of the Department.

References

HM Government. Equity and excellence: liberating the NHS, London: HM Government, 2010.

Department of Health. Liberating the NHS: no decision about me, without me - further consultation on proposals to secure shared decision-making, London: Department of Health, 2012.

Department of Health. Liberating the NHS: greater choice and control. a consultation on proposals, London: Department of Health, 2010.

Department of Health. Personal health budgets: first steps, London: Department of Health, 2009.

Glendinning

Challis

Fernandez

. Evaluation of the individual budgets pilot programme. Final Report, York: Social Policy Research Unit, University of York, 2008.

Dolan

Layard

Metcalfe

. Measuring subjective wellbeing for public policy: recommendations on measures, London: The London School of Economics and Polical Science, 2010.

Netten A, Burge P, Malley J, et al. Outcomes of social care for adults: developing a preference-weighted measure. Health Assess 2012; 16: 1–165.

Goldberg

. General health questionnaire, Windsor: NFER Nelson, 1992.

Rubin

. Multiple imputation for nonresponses in surveys, New Jersey: Wiley, 1987.

10.

Department of Health. Personal health budgets update. London: Department of Health, 2012.