Abstract
Objectives
There is a need to develop a payment system for services for children with mental health problems that allows more targeted commissioning based on fairness and need. This is currently constrained by lack of clinical consensus on the best way forward, wide variation in practice, and lack of data about activity and outcomes. In the context of a national initiative in England our aim was to develop a basis for an improved payment system.
Methods
Three inter-related studies: a qualitative consultation with child and adolescent mental health services (CAMHS) stakeholders on what the key principles for establishing a payment system should be, via online survey (n = 180) and two participatory workshops (n = 91); review of relevant national clinical guidelines (n = 15); and a quantitative study of the relationship between disorders and resource use (n = 1774 children from 23 teams).
Results
CAMHS stakeholders stressed the need for a broader definition of need than only diagnosis, including the measurement of indirect service activities and appropriate outcome measurement. National clinical guidance suggested key aspects of best practice for care packages but did not include consideration of contextual factors such as complexity. Modelling data on cases found that problem type and degree of impairment independently predicted resource use, alongside evidence for substantial service variation in the allocation of resources for similar problems.
Conclusions
A framework for an episode-based payment system for CAMHS should include consideration of: complexity and indirect service activities; evidence-based care packages; different needs in terms of impairment and symptoms; and outcome measurement as a core component.
Introduction
There is international interest in the development of ‘episode of care’ units for use in payment systems for secondary health services. A unit of activity of this sort is a series of temporally contiguous services delivered by a single or multiple providers, to a particular type of patient defined by case-mix group.1,2 Episodes include both periods between admission to and discharge from a service, as well as longer periods in receipt of ongoing care.2,3 Application of episodes of care to reimbursement has spread worldwide since the introduction in the 1980s of Medicare’s Prospective Payment System, which uses Diagnosis-Related Groups. Structuring payment in this manner is appealing to policy makers and purchasers not only for physical health but also for mental health. 4
Quantity and cost of health services
To encourage the supply of appropriate health care, it is helpful to consider the rationale for introducing episode-based payment in the context of existing funding arrangements. If providers’ costs are reimbursed according to individual items or components (such as diagnostic procedure or out-patient appointment), aggregating these into episodes with prospectively set prices may improve productivity and control costs. 5 A prospective price assigned to an episode of care creates a financial incentive to reduce the quantity of items thereby countering the tendency for unnecessary components. There is also an incentive to cheaper components, where feasible. 6
In systems in which providers are funded through block contracts, moving to episode-based payment creates a closer linkage between the demanded and supplied quantity of health care. This may help to reduce waiting times by increasing throughput, while also providing a safeguard against the potential problem of over-provision of items within episodes.7,8 However, although there are incentives to control quantity and costs within episodes, there is also an incentive to increase the number of episodes,2,9 which may be undesirable. 7
The effect of introducing episode-based payment with prospectively set prices on hospital productivity has been explored in several countries by examining the average length of stay (LOS), a proxy for the quantity of inputs or costs required to produce an episode. Studies have generally found this approach is associated with modest or no reductions in average LOS,5,8 though caution is warranted due to the difficulty of isolating the effects of payment system reform from concomitant policies and trends in hospital productivity. 8
Quality of care
The contribution of episode-based units of activity to improving the safety or effectiveness of care is expected through several mechanisms.2,10–12 Categorizing episodes according to their complexity or severity identifies outputs of a provider that bear greater relation to patients’ health problem(s) than traditional conceptualizations of outputs such as the number of discharges or out-patient attendances. 10 This permits inter-provider comparison of the average costs/LOS of particular episode-based outputs, or identification of episodes with unexpectedly high or low costs/LOS within a given output category, which may indicate where quality deficiencies lie.10,11 This kind of analysis can be undertaken independently of a payment system, although an overarching financial agenda may encourage it, and may stimulate improvements in the accuracy and completeness of the coded data it relies on.6,11 However, some commentators argue that case-mix classifications designed to predict episode costs for reimbursement purposes have only limited use for quality improvement, either due to the high residual diversity of patients within particular categories, 13 or their inability to provide satisfactory information on predicted health outcomes. 14
Another potential path to quality improvement applies in health systems that use block contracts. By allowing patients to choose their provider and linking providers’ revenue to the number of episodes delivered, it has been proposed that providers would be encouraged to compete for patients on the basis of quality.12,15 This relies on patients receiving and using accurate and interpretable information on the quality of care of different providers. 9
Conversely, reimbursement structured by episode-based units of activity could have a negative impact on quality, due to the incentive to limit the costs of episodes.5,9 This raises the possibility of providers reducing necessary services or discharging patients too early, particularly if the severity or complexity of certain conditions is inadequately accounted for by the payment system. 16 Empirical study of effects on quality has been limited by the lack of availability of outcome data. 5 The limited findings do, however, broadly suggest that in the acute hospital sector, such payment systems have not led to a dramatic effect on outcomes. 16
Fairness and need
If prices assigned to episode categories accurately reflect their relative average costs,1,10,13 then, all else being equal, providers treating a more costly case-mix of patients should receive a higher level of reimbursement. Fairness therefore rests on how well the grouping methodology accounts for variation in the costs of episodes attributable to patient characteristics that are unevenly distributed among providers.13,17
In mental health, fairness has been considered from the perspective of the needs of patients.4,18–20 It is hoped that by categorizing them according to a particular logic applied to their attributes at the start of episodes (as opposed to the interventions they received during episodes), the resultant episode categorization will provide an estimate of their relative health care needs. A reimbursement system with episode prices weighted according to need could, therefore, have positive implications for equity of use, especially if the preceding payment arrangements take minimal or no account of need. However, it is useful to note the concerns that have arisen from acute hospital case-based payment in which a ‘residual’ range of complexity or severity of cases within a payment category may lead providers to develop strategies to avoid treating the more complex and severe cases within that category, for fear that payment will be inadequate.6,9 If these ‘above average’ cases cannot be identified early enough to be avoided, when they become apparent through higher than average resource use, the quantity of their care might be reduced. In designing needs-based episode categories for mental health, it is conceivable that the more precise the weighting of the episode categories for ‘actual’ need, the lower the risk of decreased access to care for the ‘higher need’ patients within episode categories. Rewards or sanctions linked to the quality of care and additional payment for high-cost outliers may also assist with mitigating this risk.6,9
Consequently, a major challenge in designing classifications of this type is the quantification of need. Need can vary depending on different perspectives and criteria for its nature and extent. 21 Thus, it is unsurprising that there have been significant differences in the approaches to designing such classifications, with developers drawing to varying degrees on clinical judgement (normative need) and statistical analysis of the relationship between patient characteristics and resource use (as a proxy for need).18,19,22 In regards to applications of methodology incorporating analysis of the relationship between patients and resource use, reported constraints have been the provider-level variation in treatment activity, and how little the information collected on patient characteristics at the start of an episode can predict variance of treatment activity – the ‘weak signal-loud noise’ problem.3,18 For adult mental health services in England, a combined data analysis and clinical judgement approach for determining case-mix categories has been used, employing unsupervised cluster analysis (which does not relate clusters to a dependent variable such as resource use) and multidisciplinary panel review of data. 22
Developing a payment system for CAMHS in England
Our aim was to develop a basis for a set of episode of care units to underpin a payment system for child and adolescent mental health services (CAMHS). It arose from a policy commitment to extend the move away from block contracts into more controlled and transparent ‘units of commissioning’ beyond acute physical health services. 23 Currently out-patient CAMHS are commissioned mainly by block contracts, or contracts based on itemized units of activity such as face-to-face contacts. 24 In part the wish to move from block contracts was fuelled by concerns that these arrangements do not appropriately take into account numbers of patients or their needs. 25 In addition, it was hoped that since the move from block contracts would involve agreeing a national way of categorising mental health care into ‘units of purchase’ that are clinically similar and have similar resource needs, this would also facilitate greater understanding of the quantity and quality of care delivered, since it was recognized that currently data are largely missing in regards to these crucial dimensions. 20 Moreover, it was hoped that the new payment system would support more productive discussions between purchasers and providers, and comparison of providers’ performance.4,20
Methods
An initial step in designing such a prospective payment system is to develop a framework for classification to differentiate groups of patients with different levels of need. Aspects of CAMHS likely to pose particular difficulties are: limited evidence on the effective interventions particularly for children with complex problems; 26 limited validation of assessment and outcome measures; 27 and lack of data on indirect activities (related to a patient’s care but where s/he is absent). In light of these and other challenges with quantifying need, the method aimed to consider and integrate both normative and empirical knowledge, by drawing on three different sources.
Consultation with CAMHS stakeholders
The aim was to establish views on: should payment case-mix groups be defined by common diagnostic characteristics, broader needs, or the level of resources used irrespective of the former; whether and how complexity factors should be taken into consideration; and should indirect activities be included? A sample of stakeholders was obtained through a nationally promoted online survey and two participatory workshops. The questionnaire survey was promoted through the project website and CAMHS networks in England. The 180 participants from across England consisted of clinicians (66%); service managers and finance officers (19%); voluntary sector (7%); commissioners (6%); social care providers (2%); and patients (2%). The questionnaire included open questions, which were subjected to content analysis by two researchers (MJ and PV). The findings were presented at two participatory workshops. Attendants were invited through the same procedure. Of the 91 participants (57 and 34 at each event), 65% were clinicians. The group facilitators collected individual and group recommendations, which were subjected to the same process of content analysis.
Contextual review of clinical guidelines
Clinical guidelines from the National Institute of Health and Care Excellence (NICE) were reviewed with the aim of identifying whether interventions should be adapted in terms of resource allocation to address complex needs. 28 Eleven guidelines were specific to children (depression, PTSD, obsessive-compulsive, self-harm, conduct, attention deficit hyperactivity disorder (ADHD), autism, eating disorders, alcohol dependence, looked after children and wellbeing in education), and four guidelines for adults referred to children (anxiety, bipolar, psychosis, borderline personality disorders). These were reviewed in terms of key treatment recommendations, how/where interventions should be implemented, length, skill mix, costs, contextual factors (risk, engagement, complexity) and expected outcomes. Complexity was defined as a multitude of mental health and social care factors involved, distinct from symptom severity. Guidelines were independently reviewed according to these criteria on a rating template, initially by one reviewer (RS), and subsequently by two reviewers (PV and RD).
Association between disorders and resource use
We analysed CAMHS clinical records to establish the extent to which resource use could be predicted from information on problem type and severity. Anonymised clinical records of closed cases were provided by an English inner-city CAMHS provider. All children were seen in the community, although a small number may have also had episodes of in-patient treatment. Interventions were delivered by 23 CAMHS teams across three localities. The sample (n = 1774) was constructed by selecting children for whom valid information was available on sex, age, presenting problem, impairment (measured by the Children’s Global Assessment Scale – CGAS 29 ) and number of appointments attended. The sample sizes in the three localities were 500, 590 and 684, respectively. The number of children in the 23 teams ranged from 1 to 350 (median=38). Of those, 961 (54%) were male and 813 (46%) female. The age distribution was: under 5 years 144 (8%), 6–12 years 789 (44%) and 13–18 years 841 (47%).
Results
Consultation with CAMHS stakeholders
The key emerging categories were that payment systems should adopt the following principles: needs are broader than diagnoses, taking complexity factors into consideration; needs should be linked with resource use and outcomes; indirect activities should be taken into account and costed; outcome measures should support clinical practice but not determine payment; allowance for migration across case-mix groups over time, including mapping to adult groups at 18th birthday; validation of non-burdensome assessment tools; and anticipation of varying degrees of joint health and social care commissioning.
Contextual review of clinical guidelines
All guidelines specified treatment modalities and their duration based on the available treatment in the area. There was variation in the definition of practitioners’ skills and care levels (referred to in eight guidelines). Contextual factors were usually mentioned in terms of comorbidity, with some reference to environmental risks. Six guidelines defined desired outcomes in terms of symptoms and impairment but not specifically to care packages. Seven guidelines referred to costing, with three (depression, conduct, ADHD) providing details on care packages, including indirect costs. Overall, identified conditions were predominantly diagnostic-driven, but usually not contextualized (framed in relation to real practice and settings) in planning appropriate interventions. The limited research evidence meant findings needed to be complemented by expert consensus. Indirect service activities such as consultation and liaison were referred to but not specified.
Association between disorders and resource use
Presenting problems were recorded by a clinician at assessment in accordance with data specification of the Child Outcomes Research Consortium. 30 It was possible to record the same child as having more than one problem. We initially considered 22 different problems and problem combinations, which all occurred in at least 0.4% of the sample, plus one residual category for less frequent types. Problems and problem combinations were then categorized into four groups, based on significance tests of coefficients in the statistical model and similarity of problems: high resource use (psychosis, eating disorder) 42 (2%); moderate resource use related to emotional problems (emotional problems only or associated with conduct, eating disorders, self-harm or learning disability) 764 (43%); moderate resource use related to conduct problems (conduct problems only or with other problems except emotional ones) 104 (6%); and low resource use (all other types of problems and problem combinations).
The dependent variable was the number of appointments attended after assessment. This was a discrete variable, with large variance relative to the mean. Its distribution approximated the negative binomial probability density function. Data were analysed using mixed negative binomial regression. The effects of sex, age and CGAS rating (higher scores indicate better psychosocial functioning) on number of appointments were modelled as fixed effects. To take account of the nested structure of the data (children were treated within clinical teams, which were located within one of three localities), a random intercept term was fitted for the three localities, as well as for the 23 teams. We suggest the three localities can stand as examples for CAMHS. Generic clinical teams were similar across localities but specialist teams (e.g. for neurodevelopmental disorders) varied within each locality. The average number of appointments differed considerably between localities.
The influence of patient characteristics on the number of appointments attended. Coefficient estimates from mixed negative binomial regression.
N = 1774. The log-link was used for the generalized linear mixed model.
Random effect variances: Region: 0.1996; Team: 0.2016.
Negative Binomial dispersion parameter (NB2 parameterization): 1.0921 (SE: 0.0455).
The age groups followed usual criteria of pre-, primary and secondary school age years.
The negative coefficient of CGAS rating shows that the higher a patient’s functioning at assessment, the lower the expected number of appointments (controlling for all other variables). Thus CGAS rating and problem type each make an independent contribution to the prediction of number of appointments. Figure 1 shows the number of appointments by CGAS rating, as well as the predicted number of appointments from the regression model, for teenage girls (plots for girls of other ages or for boys would look similar, except that predicted values would be slightly lower). It illustrates how poor the model fits overall, since the individual values are scattered widely around the lines of best fit.
Number of appointments by impairment (CGAS score) and problem type (results for girls aged 13–18, n = 484).
Finally, we tested whether random effects were necessary in the model. Removing the random effects led to a significantly worse fit to the data (likelihood ratio test statistic Λ = 400.9, df = 2, p < 0.001). Both the locality and the team effects were also individually statistically significant. This means that differences between teams and localities cannot be explained by case-mix differences, at least not insofar as such differences are reflected in the children’s sex, age, presenting problems and CGAS ratings. Clinical assessment can provide some information that is relevant for the prediction of resource utilization. However, a large amount of variation in resource use is not predictable using the data at our disposal.
Discussion
Key findings and methodological issues of studies 1–3.
Despite the diagnostic focus of the clinical guidelines, they covered the majority of child mental health problems and included recommendation of care packages, including suggested numbers of appointments, which will make a useful contribution to estimating relative needs and to understanding gaps between best practice and current resource use. The latter point is important when considering the potential effect on clinical practice of an episode-based payment system priced according to average costs as ‘small variation around the mean (cost) cannot necessarily be seen to be any better than large variation, if we have no idea what efficient practice is’. 5 A major challenge exists in defining efficient care for patients with multiple problems and complex factors, which the consultation suggested form a sizeable proportion. It may be productive to explore the feasibility of identifying common treatment components recommended in the guidelines and recombining them theoretically for cases with co-occurring problems. 26 Such a process would likely benefit from additional input from clinicians, particularly with regard to obtaining views on how interaction between particular combinations of problems might either potentiate or lessen resource use compared with single problem episodes. 1
Modest associations were found between particular patient attributes and resource use, and the analysis suggests that there is substantial variation in resource use between services that cannot be explained with available case-mix information. This is consistent with the results of other case-mix classification studies in mental health that analysed out-patient data.3,18,19 Aside from random error, several possible reasons for the inter-service variance can be hypothesized. These include systematic differences between services in unmeasured case-mix, influence of parents/carers on access to care, contributions of other providers (e.g. social care, schools), other external factors (e.g. living conditions), technical efficiency and clinical practice. It is also likely that some of the variation is due to differences in recording activity between teams and/or localities. This leads us to propose that a nationally agreed framework of more detailed and validated assessment tools and costing would enhance the data available for further empirical work.
Notwithstanding the importance of future research to establish whether prediction of resource use can be improved by taking into account complexity and context alongside problem types and severity, these preliminary findings suggest that the nature of CAMHS presents a significant challenge to defining episode-based outputs homogeneous enough for relative costs (and subsequently prices) to reflect the relative needs of patients. Even where case-mix classifications arguably deliver a better prediction of resource use, as in in-patient care for physical health problems, the extent of within-group variation has raised concern about potential detrimental effects on equity of use and provider financial stability.9,16
Gradual implementation, comprehensive monitoring of processes and outcomes, and refinement are, therefore, likely to be crucial to CAMHS payment reform.4,9 This will be vital for managing the complexities of the desirable and undesirable incentives that accompany an episode-based payment system, 6 including consideration of repeat episodes of care in conditions such as depression, when chronicity is predicted by repeat remission.
Footnotes
Acknowledgements
This project is part of the development of a payment system for child mental health services, which is funded by the Department of Health and NHS England. The authors’ views do not necessarily reflect those of the policy department. We are grateful to the CAMHS Payment System Project Group, the Expert Advisory Group, the participating services and a range of stakeholders for their valuable contributions.
