Improving community support for older people’s needs through commissioning third sector services: a qualitative study

Abstract

Aim

This exploratory study of commissioning third sector services for older people aimed to explore whether service data was fed back to commissioners and whether this could improve intelligence about the population and hence inform future commissioning decisions.

Background

Third sector services are provided through charities and non-profit community organizations, and over recent years services have developed that assess and advise people for self-management or provide wellbeing support in the community. Third sector services have an opportunity to reach vulnerable populations and to provide intelligence about them. Some third sector services are state funded (commissioned) in the United Kingdom. While evidence is available about the commissioning of statutory health and social care, as well as private providers, there is limited evidence about how third sector health services are funded.

Methods

Participants were recruited from commissioner organizations and third sector organizations, both with an interest in supporting the independence, self-management and wellbeing of older people. Organizations were recruited from five purposively selected sites within one region of England (East Midlands). Semi-structured interviews explored the relationships between commissioners and providers and the nature of funding arrangements, including co-production. Interviews also explored collection of data within the service and how data were fed back to commissioners. Focus groups were held with older people with the potential to benefit from wellbeing services.

Results

Commissioning arrangements were varied, sometimes complex, and often involved co-production with the third sector. Commissioners valued third sector organizations for their engagement with the local community, value for money, outreach services and ability to provide information about the community. Assessing the needs and outcomes of individuals was integral to delivery of support and advice to older people. Diverse approaches were used to assess an individual’s needs and outcomes, although there were concerns that some assessment questionnaires may be too complex for this vulnerable group. Assessment and outcomes data were also used to monitor the service contract and there was potential for the data to be summarized to inform commissioning strategies, but commissioners did not report using assessment data in this way, in practice. While the policy context encouraged partnerships with third sector organizations and their involvement in decision making, the relationship with third sector organizations was not valued within contract arrangements, and may have been made more difficult by the tendering process and the lack of analysis of service data.

Conclusion

This exploratory study has demonstrated a diversity of commissioning arrangements for third sector services across one region of England. Most commissioners invited co-production; that is, the commissioners sought input from the third sector while specifying details of the service. Service data, including assessments of needs and outcomes, were reported to commissioners, however commissioners did not appear to use this to full advantage to inform future commissioning decisions. This may indicate a need to improve measurement of needs and outcomes in order to improve the credibility of the commissioning process.

Keywords

commissioning community health discipline: public health older people third sector organizations

Introduction

The costs of maintaining comprehensive health and social care provision in England present a major challenge – and these costs are projected to increase in the future. Rising life expectancy, the growing population of older people, and technological advances, which improve diagnosis and increase treatment options, contribute to the pressures on shrinking public sector budgets.¹ This has been the situation for at least a generation, and many solutions have been proposed and tested to improve service delivery and reduce costs.²

The most recent systemic intervention in England is audacious in its scale and scope. Forty-four regions have been required to produce Sustainability and Transformation Plans (STPs) setting out in detail how they will bridge the funding deficit for the coming decade.³ Following these proposed reforms, new organizational structures are being planned in England, inspired by systems in United States⁴ and Europe.⁵ The aim is a partnership approach to health and social care systems which includes statutory (commissioners and providers) as well as non-statutory organizations (providers). For the third sector – non-profit-making charities and voluntary groups – to have a meaningful role in the proposed multiagency partnerships, there needs to be an understanding of the value of third sector organizations: of the intelligence they can bring to the partnership, as well as the services that they provide.

Services and interventions for older people delivered by the third sector aim to help maintain their health and well-being. Third sector interventions such as those to reduce loneliness, provide household help or give financial advice can be seen as helping to maintain the assets or resources of individual older people, thereby promoting greater resilience to the challenges of ageing and age-associated disease.⁶^,⁷ Services may accept referrals from health or social care professionals (as occurs, e.g. with social prescribing), or may have an outreach and self-referral approach.⁸ Assessment may be followed by advocacy and signposting to appropriate additional services, or a service may be delivered directly, for example home adaptations.

The increasing deployment of community services for older people in the UK led to policy context of improving integration with statutory services. For example, the National Service Framework for Older People (an English National Health Service (NHS) policy running from 2000 to 2010) promoted a ‘Single Assessment Process’ to avoid wasteful duplication, enable integrated working, and provide data for needs assessment. Assessment tools were developed to support this process, but in practice no single national assessment tool emerged, and most localities interpreted the Single Assessment Process as establishing a means to share assessments, rather than using standardized tools for the assessment.⁹^,¹⁰ The extent to which third sector services have applied the principles of sharing assessments between agencies, avoiding duplication, or using standardized tools to gather information is not known.

Commissioning in health, public health and social care refers to the process of assessing population needs, setting priorities, allocating resources (which involves assessing cost, efficacy, efficiency and risk), and managing the relationship with service providers – whilst involving patient and public representatives in the process.¹¹ The commissioning process begins with an assessment of the needs of the population of the community, leading to the planning and specification of services to meet those needs. Commissioners seek information about needs of communities through Joint Strategic Needs Assessment which are used to develop commissioning plans, however the quality of evidence gathered has been reported to be low.¹² Contracting and procurement of those services follow (e.g. by tender), with services monitored throughout the funded period and finally undergoing a review resulting in a reassessment of the needs of the community and a further repeat of the process. The commissioning cycle model has been developed from studies of the relationship between commissioners and statutory service providers (see Figure 1) (e.g. hospitals).¹³^,¹⁴ Evidence from a study of commissioning third sector social care services indicated that the process was consistent with the commissioning cycle model; however, the focus on outcomes was found to be a challenge in this context.¹⁵

Figure 1.

Commissioning cycle. Source: adapted from Shaw et al.¹³

It is unclear whether the commissioning cycle model is the most appropriate for the third sector. There is evidence that charities prefer the more traditional funding of block contracts and grants rather than the use of conditional contracts such as ‘payment by results’. In addition, there are concerns that many third sector organizations are not ‘commissioning ready’ because they do not have the necessary skills and experience needed to win contracts for services.¹⁶^,¹⁷

Given the potential value of commissioning third sector services for older people, and the challenges involved, this study (Supporting Older People’s Resilience through Assessing Needs and Outcomes – SOPRANO) aimed to better understand, and hence improve, the processes and relationships involved in commissioning third sector services for older people. Its purpose was therefore exploratory, and it could be seen as a case study of local commissioning for older people.

Methods

Study setting and design

The study involved qualitative interviews and focus groups. Participants were NHS and local authority commissioners, third sector managers and keyworkers, and representatives of older people in the community. The study was based within the East Midlands of England, UK. Full details of the protocol have been published elsewhere.¹⁸

Recruitment of participants

The sampling frame chosen for the study was the geographical area of the East Midlands, one of nine Government Office regions of England. At the time of the study, the East Midlands area had 22 health commissioning organizations (NHS Clinical Commissioning Groups) and nine local authorities which commission for public health and social care. Through stakeholder engagement, as well as a survey of commissioning organizations, study sites were purposively selected to represent a range of rural and urban geographies across the East Midlands. We sought to recruit managers from commissioning organizations and the third sector who knew each other through commissioning arrangements. We aimed to include a mixture of health, public health and social care commissioners and diverse third sector providers.

The original intention was to invite users of these services to participate in focus groups. However, after consultation with third sector providers, members of existing organizations that represent the public (Leicester Older People’s Forum, Advisory Group of Talk Eat and Drink programme in Lincolnshire, Nottingham Pensioners Action Group, East Midlands Later Life Forum) were invited to participate in focus groups.

The aim was to recruit a total of at least 30 commissioners and third sector informants, and to conduct three focus groups of public representative participants, with the aim of including a total of at least 10 public representative participants.

Interviews and focus groups

Semi-structured interviews were used for the commissioner and third sector participants. A literature review⁶ informed development of the interview schedule to explore the following themes: aims of services; how they were funded; how service users were identified; how service users’ needs were assessed; and what outcomes were measured. The focus groups explored similar themes. Two researchers developed and piloted the interview schedule (GS and NC); subsequently interviews and focus groups were conducted by one researcher (NC). Researchers had previous experience and training in interviewing participants. Interviews took place between July 2015 and July 2016, and focus groups during August 2016. Interview data were collected using digital audio recording. The focus groups were audio-recorded and researchers made field notes of the sessions. All participants were given a summary of the study aims and gave written consent to participate.

Analysis

Data were transcribed, anonymized, and then uploaded into the qualitative data analysis software NVivo V.11 (QSR International). The framework analysis method was used.¹⁹ A priori themes of the framework were drawn from the literature, following the interview schedule, and subthemes were added following a review of interview transcripts by members of the research team. Transcribed texts were coded by one researcher (CC) using NVivo V.11 and checked by another researcher (NC) and revised after discussion.

The framework approach facilitated the collation and comparison of data on similar topics elicited from the three stakeholder groups (commissioners, third sector providers and the public). Emergent findings were discussed with the wider study team and the study’s External and Public Advisory Group (see below). Preliminary findings were presented to the interviewees for validation. In drawing the final inferences, findings were triangulated between the sampled groups (commissioners, third sector providers and public representatives).

Ethics and oversight

Ethical approval was obtained from the University of Nottingham, School of Medicine ethical review board on 20 May 2015 (ref: Pa14052015 SoM RHA SOPRANO), and governance approval from the relevant research and development departments.

Public and stakeholder involvement

Prior to designing the study, a regional body which represents older people was consulted (East Midlands Later Life Forum). Three members of the regional body were invited to join an External and Public Advisory Group, which also included representatives of the health sector, third sector and academics (two from the host institution, and one international). The External and Public Advisory Group was consulted regularly on the design and management of the study.

Results

Five sites were purposively selected from rural and urban areas across the East Midlands. Thirty-three interviews were conducted, and 17 public representatives contributed to the three focus groups. Nine interviews were with commissioners (six from NHS Clinical Commissioning Groups, labelled ‘CCG’ in following quotes, and three from the local authority, labelled ‘LA’), 12 with third sector managers, and 12 with third sector keyworkers. The main third sector services with which staff were associated were signposting services, services to address loneliness and community hubs.

The analysis led to the elaboration of the following three themes: (a) aims of commissioning the third sector; (b) co-production and use of service data to inform commissioning; and (c) uncertainty of how to assess needs and outcomes.

Aims of commissioning the third sector

Commissioners reported that third sector services were popular with community members, which may indicate a view that the public relate to third sector services in a less formal way than the institutional services of health and social care. When engaging with citizens, commissioners felt that third sector keyworkers were better at ‘finding things that professionals hadn’t come across because they might have got over the threshold in a different way and were looking with different eyes’ (Commissioner 8, CCG). When engaging with citizens whose first language was not English, in an economically deprived urban area, keyworkers believed the key to connecting with these citizens was ‘not just translation, it’s a cultural thing as well, if you’ve got a local person who understands whatever, that can be really helpful’ (Third Sector keyworker 2). Having a shared cultural identity and local community connections were factors deemed more important than speaking the same language, an advantage they had over other mainstream services.

Factors such as this suggest the staff working in the third sector are best placed to ‘build the community to be responsible for the people that are isolated and lonely, so they can actually take action within their own community’ (Third Sector Manager 14). This may be done through low-level psycho-social care which is described as ‘self-management’ or ‘self-care support.’²

An additional reason why commissioners regarded self-management as important was its potential to reduce the need for statutory services and hence help efforts to control costs in a context of reduced public sector expenditure: ‘…the reality is currently now with Social Care neither have the capacity or the funding to deliver all the services that they need to deliver’ (Commissioner 2, CCG). Many third sector participants recognized the need for austerity, and stated that third sector providers were able to provide a cost-efficient response.

Commissioning managers noted that the community engagement of the third sector may offer two further benefits to the health sector. First, the local knowledge of communities can make an important contribution to strategy planning for commissioners. Second, third sector relationships with local communities may enable contact with isolated individuals and promote their access to health and social care services.

From the third sector perspective, funding from contract services has become more important due to ‘core funding’ being reduced. One manager perceived an increase in demand for services, and therefore a need to build capacity, yet funding was constrained. Consistent with the views of commissioners acting to develop a market for services, one third sector manager stated that local authorities should clarify their position as commissioners and regulators and not providers. This manager felt that it did not matter whether the provider was private or third sector, as long as individuals receive good quality services.

Coproduction and use of service data to inform commissioning

The research team observed that a contracting and tendering process was in place rather than the historical practice of awarding grants to third sector agencies. Although this facilitated coproduction of services which reassured service providers that ‘you’re absolutely part of our direction of travel’ (Commissioner 8, CCG), at the same time ‘we give out slightly mixed messages…. I mean you can have co-production, you just have to be careful about how you manage it so you don’t raise expectations’ (Commissioner 1, LA). This suggests that third sector services may lobby for their existing services rather than catering to the needs of the population, or overestimate the impact of their input in to the eventual contract offered to them.

This created a perceived mis-match of expectations for proposed services. While commissioners were enthusiastic about short-term pilot services, Third sector providers were less enthusiastic due to the temporary nature of the service in an environment where they wanted more secure, long-term funding.

So we did the first year, we drew in extra funding. At the end of that, towards the end of that first year, obviously we're saying ‘well make your mind up, are you going to have us for year two?’ It dragged on and they couldn’t decide yes or no and they thought, well alright you can have another six months and we said six months is no good, you might as well shut it. (Third sector manager 11)

Both commissioners and third sector managers expressed concerns that some commissioners were seeking to develop complex commissioning structures, including alliances of third sector organizations, and even creating a legal entity called a ‘special purpose vehicle’. The aim appeared to be to pool resources and expertise of third sector staff. Third sector managers noted that smaller organizations may be disadvantaged by such complex arrangements, because they did not have the capacity to respond effectively.

The majority of descriptions of the commissioning process for the third sector were consistent with the processes of the commissioning cycle model (Figure 1). For example, one respondent described the process of commissioning dementia services – starting with national strategy, followed by local consultation and service specification – and noted that third sector agencies were seen as an integral part of the dementia ‘pathway’. The exception to this commissioning cycle process may be the funding for projects or pilots, which some commissioners described as distinct from the full commissioning process. Due to the developmental nature of pilot projects, processes or outcomes may not be prespecified, but allow some flexibility during implementation.

The commissioning cycle includes the process of monitoring existing services, determining population needs, and planning services to meet those needs, whilst also taking public and user opinions into account. Commissioners greatly appreciated the ‘feedback from the patient to these respective groups … it’s almost like real-time engagement’ (Commissioner 6, CCG), highlighting the potential for constructive working relationships where service providers provided commissioners with intelligence to inform future service delivery. However, respondents from the third sector perceived that commissioners did not have the time or resources to sufficiently analyse the data being generated by the service, and that data collection served the purpose of bureaucracy rather than community intelligence:

I know half the time it’s lip-service and they don't actually read everything that we give them. And it’s a shame because actually we give a really good snapshot of what we do and the value of our service. (Third sector keyworker 5)

It is therefore not surprising that several third sector services reported that they were asked to collect too much monitoring information, or that they felt that the commissioners could make more use of the contract monitoring information they provided, for strategic purposes.

Uncertainty of how to assess needs and outcomes

There was considerable uncertainty about what assessment and outcome data could and should be collected by third sector providers. One third sector manager recognized that a comprehensive needs assessment should include ‘emotional wellbeing, financial wellbeing, independence, social inclusion and health, and physical health’. Respondents were asked about the use of the ‘Single Assessment Process’, intended as a comprehensive approach whereby health and social care organizations work together to ensure that assessment and subsequent care planning for older people are person-centred, effective and co-ordinated.⁶ However, despite agreement that the collection of routine information can be valuable for the commissioning process, participants did not believe that single assessment was helpful in the individual management of service users. One commissioner’s view was that it was not helpful to have a single assessment, because assessment is shaped by each practitioner’s specialism, even if they applied the same standard questionnaire.

On one hand I see the value of single assessments, on the other hand I think each professional goes to an assessment with a different perspective and asks the same questions, but gets, deduces different outcomes from the same answers, so sometimes it is important to ask the same questions. (Commissioner 1, CCG)

Both commissioners and providers noted that further drawbacks of a single assessment included the time required and the burden of the assessment process. Despite these concerns, there was evidence of some standardized tools being used successfully. Two respondents referred to using published tools as part of an assessment: the Health and Wellbeing Outcomes Star²⁰ and the Social Insight Tool.²¹ Although the Outcomes Star is an outcomes measure, the participant described how the multiple domains of the tool were used to prompt a conversation about needs. This assessment process had the advantage of leading directly to action planning as well as outcomes measurement.

We do a little bit of an assessment by taking people’s details and having a conversation, but a more in depth one, to find a broad idea of things, is something called Outcomes Star… Which is the eight different categories of a person’s life, really. So it’s covering the money, where they live, their friends and family, how positive they’re feeling and they’re managing their health and their lifestyle. … Then afterwards if we want to make a few little goals and action plan for them we can do that as well. (Third sector keyworker 9)

An alternative view about assessing needs, stated by one commissioner, was that health professionals who refer into third sector services should carry out the assessment and this information should be included within the referral. However, this may not be suitable for services accessed through self-referral. Other participants felt that the assessment process should be owned by the service user and that it should be informal. Consistent with these views, at one site a qualitative approach was used. The Social Insight Tool explores the challenges that an individual faces in daily life,²¹ and was used with a small sample of beneficiaries to understand complex needs and assess outcomes.

Respondents reported that not only were needs difficult to assess and measure, but so were outcomes, particularly for services delivering interventions intended to prevent ill health and maintain resilience:

What is harder to measure, but is the crux of it for me, is lifestyle, exercise, the self-care elements. But the emphasis, quite rightly, is very much on you have to be looking at these elements of the social model and getting a return on those as well. It’s not just about numbers. (Commissioner 3, CCG)

Practical issues with measurement were also reported, such as problems using standardized tools written in English for people with English as a second language, or assessments of a sensitive nature which were perceived as unhelpfully drawing attention to problems, such as a loneliness questionnaire.

Discussion

This qualitative research has explored commissioning of third sector services focused on prevention and providing support for the self-management of older people’s health needs. Commissioners anticipate that an increase in commissioning could be an affordable way of managing growing population-level health and social needs.

Commissioners attempted to incorporate third sector views and knowledge of local populations through coproduction. However, commissioning processes were varied and could be complex. This may indicate that the standard model of commissioning used within health services is too constrained for commissioning third sector services and has difficulty incorporating the sector’s specific characteristics. Additionally, the disparity between the bureaucratic processes of commissioners and third sector organizations may be a barrier to effective commissioning relationships.

Commissioners and third sector providers recognized that routine data from the services (e.g. for contract monitoring) has a potential value for strategic purposes, but these data were not used in practice. There was considerable uncertainty about what should be measured. Despite this, there was little appetite to impose a standardized approach to assessing needs or outcomes. The issue of the use of standardized assessments of beneficiaries, the collation of these assessments, and the use of the data to inform future service provision deserves further enquiry. The paradox is that whilst commissioners and third sector providers are aware of the potential value of such information, neither party has an appetite to use or contractually impose standardized assessment. Tools for this purpose have been developed (such as InterRAI–home care²² or EASY-Care²³). It is unclear if such tools will be used unless mandated, and also unclear whether the benefits of doing so (either to service users or to service planners) would outweigh the drawbacks in terms of time and resources to administer the tool.

Third sector agencies may also be seen, by commissioners, as the ‘voice of the community’. The third sector often occupies a position of trust and have close relationships with community members. An implication of this study is that, through the services they offer to members of the community, third sector organizations have insights into needs at a neighbourhood level that may be valuable to commissioners. Our findings indicate that community level needs could be communicated through better partnerships between commissioner and third sector organizations, rather than through technical methods, such as standardized tools. Whereas technical tools are in keeping with the contractual approach to commissioning, the former is consistent with relational commissioning.¹³^,¹⁴ The relational aspect of commissioning is often implicit and this should be studied in future, particularly in order to inform partnership working in health system reforms (Sustainability and Transformation Systems in England, and also Accountable Care Systems in England and elsewhere).

Although this research was carried out in one region in England, the sampling included diverse geographies, organizations and demographic characteristics. Coproduction is a broad policy goal which has been implemented in public-sector commissioning internationally; our findings contribute to the international discourse and policy development regarding commissioning relationships with third sector or non-profit organizations in, for example, Australia,²⁴ the United States,⁴ Europe²⁵ and Japan.²⁶

These findings confirm earlier observations that commissioners and the third sector are learning their way through the commissioning process.¹⁶^,¹⁷ They help to show that not being ‘commissioning ready’ is partly explained by third sector services not always being in a financial and organizational position to respond to business pressures by boosting or cutting investment and their workforce. It may not be lack of knowledge and experience, but might simply be an inherent consequence of how some volunteer third sector organizations operate.

Future directions for commissioning third sector services depend on how they are viewed by commissioners. Our findings indicate that some see third sector services as part of a continuum of comprehensive health and social care provision: from acute hospital services to community care. From this perspective the third sector may provide a resource of knowledge and advocacy to enable people to access mainstream health and care services, including early intervention (statutory or private sector). In addition, the third sector may, to some extent, be commissioned to replace low-level support that was previously provided by the statutory sector; for example, the informal advice provided by GPs and district nurses. Low-level support or early interventions are two approaches in which commissioners may invest with a view to creating efficiencies or future savings.

Policy makers, including those who set the rules and advice for commissioners and funders of care may wish to consider the degree to which a simple transactional market process is encouraged at the cost of long-term relationships between funders and third sector providers. Not all third sector organizations are robust business organizations. If the commissioning process ends up preventing third sector commissioning, then the potential benefits will be lost.

Footnotes

Acknowledgements

The authors would like to thank the SOPRANO study advisory group for their assistance in developing the study protocol and the wider NIHR CLAHRC EM team for their continued support of this research.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Disclaimer

The views expressed are those of the authors and not those of the NHS, NIHR, Department of Health, Office for National Statistics, Health Survey England or the Health and Social Care Information Centre.

Ethics approval

University of Nottingham, School of Medicine ethical review board on 20 May 2015 (ref: Pa14052015 SoM RHA SOPRANO).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This paper presents independent research commissioned by the National Institute for Health Research (NIHR) as part of the Collaboration for Leadership in Applied Health Research and Care East Midlands (CLAHRC EM).

References

Licchetta

Stelmach

Fiscal sustainability analytical paper: fiscal sustainability and public spending on health, https://obr.uk/docs/dlm_uploads/Health-FSAP.pdf (2016, accessed 6 December 2018).

Rechel

Grundy

Robine

J-M

et al . Ageing in the European Union. Lancet 2013; 381: 1312–1322.

Edwards

Sustainability and transformation plans: what we know so far. London: Nuffield Trust, www.nuffieldtrust.org.uk/resource/sustainability-and-transformation-plans-what-we-know-so-far (2016, accessed 10 January 2018).

Vaughan

SK.

The importance of performance assessment in local government decisions to fund health and human services nonprofit organizations. J Health Hum Serv Admin 2010; 32: 486–512.

Pestoff

Osborne

Brandsen

Patterns of co-production in public services. Public Manag Rev 2006; 8: 591–595.

Sands

Gladman

Chadborn

et al.

Commissioning to promote resilience in older people via third sector services. East Midlands Research into Ageing Network (EMRAN) Discussion Paper Series: Nottingham, 2016.

Windle

What is resilience? A review and concept analysis. Rev Clin Gerontol 2011; 21: 152–169.

Mossabir

Morris

Kennedy

et al . A scoping review to understand the effectiveness of linking schemes from healthcare providers to community resources to improve the health and well-being of people with long-term conditions. Health Soc Care Commun 2015; 23: 467–484.

Clarkson

Abendstern

Sutcliffe

et al . Reliability of needs assessments in the community care of older people: impact of the single assessment process in England. J Public Health (Oxf) 2009; 31: 521–529.

10.

Dickinson

Implementing the single assessment process: opportunities and challenges. J Interprof Care 2006; 20: 365–379.

11.

Newman

Bangpan

Kalra

et al. Commissioning in health, education and social care : Models, research bibliography and in-depth review of joint commissioning between health and social care agencies. Technical report. London, Institute of Education, http://eprints.ioe.ac.uk/16354/ (2012, accessed 10 January 2018).

12.

Beenstock

Sowden

Hunter

et al . Are health and well-being strategies in England fit for purpose? A thematic content analysis. J Public Health ( Health) 2015; 37: 461–469.

13.

Shaw

Smith

Porter

et al . The work of commissioning: a multisite case study of healthcare commissioning in England’s NHS. BMJ Open 2013; 3: e003341.

14.

Porter

Mays

Shaw

et al . Commissioning healthcare for people with long term conditions: the persistence of relational contracting in England’s NHS quasi-market. BMC Health Serv Res 2013; 13(Suppl. 1): S2.

15.

Miller

Allen

Mangan

et al . Singing from the same hymn sheet? Commissioning of preventative services from the third sector. J Integr Care 2013; 21: 288–299.

16.

Hedley

Joy

When the going gets tough: charities’ experience of public service commissioning. London: New Philanthropy Capital, 2012.

17.

Rees

Public sector commissioning and the third sector: old wine in new bottles?

Public Policy Adm 2014; 29: 45–63.

18.

Sands

Chadborn

Craig

et al . Qualitative study investigating the commissioning process for older people’s services provided by third sector organisations: SOPRANO study protocol. BMJ Open 2016; 6: e010724.

19.

Gale

Heath

Cameron

et al . Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13: 117.

20.

MacKeith

The development of the Outcomes Star: a participatory approach to assessment and outcome measurement. Housing, Care Support 2011; 14: 98–106.

21.

Wherton

Sugarhood

Procter

et al . Designing assisted living technologies ‘in the wild’: preliminary experiences with cultural probe methodology. BMC Med Res Methodol 2012; 12: 188.

22.

Bernabei

Landi

Onder

et al . Second and third generation assessment instruments: the birth of standardization in geriatric care. J Gerontol A Biol Sci Med Sci 2008; 63: 308–313.

23.

Craig

Chadborn

Sands

et al . Systematic review of EASY-care needs assessment for community-dwelling older people. Age Ageing 2015; 44: 559–565.

24.

Ryan

Co-production: option or obligation?

Aust J Public Adm 2012; 71: 314–324.

25.

Pestoff

Towards a paradigm of democratic participation: citizen participation and co-production of personal social services in Sweden. Ann Public Coop Econ 2009; 80: 197–224.

26.

Tsukamoto

Nishimura

The emergence of local non-profit – government partnerships and the role of intermediary organizations in Japan. Public Manag Rev 2006; 8: 567–581.