Medical leadership arrangements in English healthcare organisations: Findings from a national survey and case studies of NHS trusts

Abstract

This project sought to describe the involvement of doctors in leadership roles in the NHS and the organisational structures and management processes in use in NHS trusts. A mixed methods approach was adopted combining a questionnaire survey of English NHS trusts and in-depth case studies of nine organisations who responded to the survey. Respondents identified a number of challenges in the development of medical leadership, and there was often perceived to be an engagement gap between medical leaders and doctors in clinical roles. While some progress has been made in the development of medical leadership in the NHS in England, much remains to be done to complete the journey that started with the Griffiths Report in 1983. We conclude that a greater degree of professionalism needs to be brought to bear in the development of medical leadership. This includes developing career structures to make it easier for doctors to take on leadership roles; providing training, development and support in management and leadership at different stages of doctors’ careers; and ensuring that pay and other rewards are commensurate with the responsibilities of medical leaders. The time commitment of medical leaders and the proportion of doctors in leadership roles both need to increase. The paper concludes considering the implications of these findings for other health systems.

Keywords

clinical management leadership medical profession

Introduction

Involving doctors in leadership roles in the NHS has been an explicit aim since the Griffiths Report into NHS management was published in 1983.¹ The importance of medical leadership has been reiterated by a succession of national governments.^2,3 The risks of doctors not being involved in leadership roles were highlighted in a recent Inquiry report into failures of patient care at an NHS trust which described a hospital in which most medical staff felt disengaged and undervalued.⁴ The report argued that much more needed to be done to support doctors and other clinicians to become leaders in future. The English NHS is not alone in the aspiration to improve medical leadership, and a number of other health systems have also expressed similar aspirations and experienced reform processes around this agenda.⁵

Previous studies have described the impact of successive reforms and the steps taken to involve doctors in leadership roles in the 1980s and 1990s.^6–10 These studies noted the emergence of a more active management style in which managers were increasingly involved in questioning medical priorities.¹¹ They also showed how some doctors were appointed to lead clinical directorates with the support of general managers and nurse managers.¹² Despite these developments, it was clear that doctors retained a large measure of clinical autonomy and that the frontier of control between doctors and managers had shifted only a little.¹³ It was also apparent that doctors who moved into leadership roles found themselves in hybrid positions and occupied a precarious ‘no man’s land’ between the managerial and clinical communities.¹⁴ This gave rise to a perception that medical leaders had gone over to ‘the dark side’ among some of their colleagues.¹⁵

Most of the empirical studies of medical leadership in the NHS were conducted a decade or more ago. Since then, arrangements for involving doctors in leadership roles have evolved to include the use of divisional structures alongside clinical directorates and the introduction of service-line management approaches.¹⁶ Our research was designed to discover what kinds of arrangements are currently in use and how effective these are perceived to be. We did so by undertaking a survey of NHS trusts in 2010, supplemented by case studies of nine of the organisations that responded to the survey. The main aims were to provide a picture of the medical leadership structures in place in England and the processes at work within these structures. We also sought to identify the factors that were perceived to be important in facilitating and inhibiting doctors to take on leadership roles.

Our research was undertaken with the objective of supporting the NHS to make further and faster progress in improving medical leadership. To this end, we use the results to identify what now needs to be done to develop medical leadership within the NHS in England and to support doctors who wish to move into leadership roles to do so. In the final part of this paper, we put forward a series of recommendations that in our view need to be taken up with urgency and consider what the implications might be for other health systems.

Methods

Our study utilised a questionnaire survey of NHS trusts in England and case studies in which medical leaders, managers and nurse leaders were interviewed about the leadership arrangements in place in their trusts. Of the 245 trusts approached to take part in the survey, research approval was gained from 179. From this sample, 73 returned the questionnaire following reminders where appropriate, representing a response rate of around 40%. Questionnaires were completed either by trust medical directors or chief executives during the period March to December 2010. Data gathered from the survey included the types of organisational structures used by trusts, the number of doctors involved in leadership roles at different levels and the perceived effectiveness of these arrangements (see Box 1 for details).

Box 1.

Themes covered in questionnaire.

• Facts relating to size of trust, budget, staff number, number of consultants. • Number of doctors on different trust boards. • Doctor representation on trust committees. • Organisation of the trust below the management board, names and numbers of organisational units, what budgets they hold. • Roles that doctors play in organisational units, job titles and pay and reward details. • Proportion of doctors involved in formal leadership roles. • Where accountability lies in organisational units, which professionals lead. • What development programmes exist for doctors. • What changes have been made to organisational structure recently and what planned. • Perceptions of effectiveness of medical leadership arrangements and degree of responsibility doctors in trust feel in terms of finance, trust performance, quality of services, patient experience. • Challenges that trusts face in developing medical leadership.

Nine case study sites were selected from the trusts that responded to the survey. They were chosen following purposive sampling to ensure inclusion of trusts with a range of relevant characteristics. These included the size of trusts measured by their budgets and number of staff employed; the region in which they were located; the services they provided, such as acute, mental health and specialist services; and their status as foundation trusts or NHS trusts (see Table 1). Interviews were then undertaken during 2011 and the early part of 2012 with the boards of these organisations (n = 46) and a cross section doctors, nurses and managers in each site using a semi-structured approach (n = 105). This was designed to gather a common set of data from all sites while also allowing an opportunity for respondents to add information they felt to be important in relation to the issues we were investigating. All interviews were recorded, and then the interviews notes of the conversations were returned to the interviewee to verify. These notes were coded thematically according to the meta codes of structures, processes and outcomes, employing a range of themes that fell within these categories (see, Dickinson et al.¹⁷ for more details on methodology).

Table 1.

Overview of key features of case study sites.

Case study	Trust type	Foundation trust or non- foundation trust		Total budget (£million)	Total number of staff (total head count, not FTE)	Number of medical consultants	Number of doctors on trust board of directors	Number of doctors on trust’s management board	Principal organisational structure	Number of principal organisational units
A	Acute	FT	South West	193	3300	140	1	5	Directorate	5 directorates
B	Acute	FT	North East	450	6582	358	1	12	Division	36 directorates
C	Acute	Non-FT	Yorkshire and Humber	950	13000	714	1	1	All	5 divisions
D	Acute	Non-FT	South Coast	507	8743	504	2	8	All	5 divisions
E	Acute	FT	Yorkshire and Humber	178	4300	223	1	14	Directorate	12 directorates
F	Specialist	Non-FT	London	323	3594	249	3	6	Division	5 divisions
G	Mental health	FT	East of England	150	2000	70	2	5	Division	5 divisions
H	Mental health	Non-FT	West Midlands	140	3200	64	3	5	Service line	5 Directorates
I	Mental health	FT	North West	131.5	2808	86	2	5	Service line	4 service units

SHA: strategic health authority; FT: foundation trust.

The main purpose of the case studies was to gain a more detailed understanding of the leadership arrangements in place in these nine trusts and their effectiveness than was possible in the questionnaire responses. Issues explored included relationships between medical leaders, managers and nurse leaders; the process by which medical leaders were appointed and supported in their roles; and the relationship between medical leaders and their colleagues in clinical roles. The research team met regularly over the course of the fieldwork to discuss emerging findings from the case studies and to compare data gathered from different sites in order to build up a picture of how medical leadership had developed in the nine sites. This enabled us to identify differences between sites as well as a number of common themes which we now go on to discuss.

Results

The questionnaire survey found that there was wide variation in how trusts organised their services and involved doctors in leadership roles. Clinical directorates, divisions and service lines were the three main types of units used, sometimes in combination. It was not unusual for trusts to report that they had changed their organisational structures or were planning to do so following trust mergers, the appointment of new chief executives or changes to how services were provided. The variety of arrangements we found means that it is difficult to generalise about the types of structures that are currently used in the NHS.

The number of doctors on trust boards of directors (the board led by the trust chair) varied from one to four with most having only one, the medical director. The number of doctors on trust management boards (the board led by the chief executive) ranged from one to 17, the most common number again being one. Most trusts reported that between 10% and 20% of consultants were involved in formal leadership roles with medical leaders in clinical directorates, divisions or service lines seen as occupying the key positions in the case study sites. Other consultants took on trust wide leadership roles, for example in relation to quality and safety, clinical governance and research and development.

Medical leaders were most frequently identified as the accountable officers in clinical directorates, divisions and service lines. As a Director at Site A described,

the AMDs [Associate Medical Directors] are not some figureheads we wheel out when we need a doctor. They are genuinely the managers of that directorate and the general managers are junior to them. We were quite clear that we wanted the most senior person in the directorate to be a doctor.

Doctors usually worked closely with general managers and nurse leaders, although the latter did not have such a prominent role as general managers. This meant that the triumvirate of medical leader, general manager and nurse leader found in previous studies had been replaced by the duality of medical leader and general manager. As a Director at Site G explained,

The structure completely relies on the clinical director and general manager relationship working well. If we got a pairing who couldn’t work together then this would be unworkable. All our parings work, although some work much better than others. Where it works well the individuals have worked out which are their roles and responsibilities.

The need for medical leaders to retain some clinical commitments to remain credible with their colleagues was seen as an important factor influencing how much time they were able to spend on leadership activities. Trust medical directors were reported to commit at least half of their time to leadership roles whereas clinical directors committed around 20% of their time. In the case studies, we found that both medical directors and clinical directors often reported spending more of their time on leadership activities than allowed for in their contracts because of the demands of these roles.

The appointment of medical leaders was reported to have become more formalised with increased competition for roles and more rigorous selection processes being used. Despite this, it was not always easy to find more than one person willing and able to take on these roles, especially as the quality of the individuals appointed was seen, not surprisingly, as critical to their effectiveness. It was explained that doctors are not necessarily ‘queuing up for these roles’. For the most part, trusts were trying hard to make sure that they did manage to generate more competition in future and they were being ‘ruthless in getting the right people’ (Director, Site D). Most trusts had put in place training programmes to develop medical leaders, and these were delivered by external providers such as universities in some cases and by the trusts themselves in others.

The questionnaire survey asked respondents to assess the effectiveness of medical leadership arrangements in their trusts and most gave a positive response as shown in Figure 1. They were also asked about the degree of accountability doctors felt for service quality and finance. The results, displayed in Figure 2, show that doctors were reported to feel a much stronger sense of responsibility for quality of care than for finance.

Figure 1.

Self-rated responses on how well medical leadership arrangements work.

Figure 2.

Self-rating of doctor responsibility for quality and finance.

Both the questionnaire survey and the case studies identified a number of challenges in the development of medical leadership. These challenges included lack of support from general managers in some organisations and variations in the willingness of medical leaders to deal with difficult issues, for example with their medical colleagues. The absence of clear career structures for doctors going into leadership roles and lack of financial incentives to take on these roles were also mentioned as barriers. All of these factors help to explain why competition for leadership roles is often limited.

There was reported to be an ‘engagement gap’ in some sites between medical leaders and doctors in clinical roles as the former came to be identified more closely with trust managers. As a Director at Site B explained,

If you see clinician engagement as a triangle, we work well with the 13 divisional directors, and probably 80% well with directorates. It is when you get to the bottom of the triangle that there is always a problem, and we are looking at different ways now of trying to engage with those – who I call the backbenchers

At the same time, examples were given of the positive impact made by medical leaders in bringing about service improvements and taking responsibility for quality of care and budgets. We also identified organisations among the nine case studies where medical leadership seemed to be further developed than in most of the trusts who responded to the survey. Often we heard the argument that trusts were on a journey in developing effective medical leadership arrangements with much more work yet to do.

Discussion

The variations in structures and processes associated with medical leadership in NHS trusts reflect the fact that successive governments have not been prescriptive about the precise arrangements that should be adopted, even though politicians of all parties have emphasised the need for doctors and other clinicians to become more involved in leadership roles. The arrangements that exist result from the decisions of local NHS leaders on what is needed in their organisations, leading to the various permutations we have described. Our survey suggests that these arrangements continue to evolve in response to changes in trust leadership and organisational mergers and changes in how services are provided, pointing to the fluidity of organisational structures.

Research in Scotland in the 1990s identified three main types of clinical directorates, described as traditionalist, managerialist and power sharing.¹⁰ Most of our case study sites reported that they were medically or clinically led or had aligned structures in which doctors shared power with managers, rather than being managerially led. The sites also provided some evidence of their structures and processes leading to innovation and service change of a different order to that found in traditionalist clinical directorates, as described more fully in our research report.¹⁷ For these reasons, we would argue that the organisations we studied can best be described as power sharing because of the progress made in developing the duality of medical leader and general manager and the impact this was having in enabling improvements in care to occur.

Despite this, there is no reason to question the argument that established relationships and dynamics between doctors, nurses and managers have largely persisted, and there has been no fundamental change in how NHS organisations are run, a view endorsed by other studies.¹⁸ Taking the longer term perspective, it is clear that much more has been done to take medical leadership to the level aspired of it by national government. Notwithstanding recent criticisms about the growth in the number of managers and the costs of administration within the NHS, successive governments have relied on general managers to implement their reforms. They have also put in place career structures, training programmes and compensation arrangements to ensure that sufficient managers of the right calibre are available.

Much less effort has gone into the development of medical leadership, notwithstanding periodic rhetoric from government ministers and senior officials about its importance. The consequence is seen in the way in which medical leadership roles are perceived by senior doctors and the difficulties in finding well-qualified and credible candidates for these roles. Although respondents to our survey reported that medical leadership arrangements in their trusts worked well, the reality as revealed by interviews in the case study sites revealed a much more varied and nuanced picture. While some organisations have given priority to the development of medical leadership and report seeing benefits in terms of their performance, in most there continue to be challenges in attracting senior doctors into leadership roles and providing them with the time and other resources to carry out their roles effectively.

It is also clear from our research that the emergence of an ‘administrative elite’ of medical leaders has resulted in increasing differentiation between these doctors and the ‘rank and file’ whose main focus remains their clinical work.¹⁹ This helps to explain the engagement gap we identified and the sometimes uncomfortable position of medical leaders in the middle ground between general managers and their clinical colleagues. Compared with organisations in other systems that have given high priority to the development of medical leaders, the NHS in England has much more to do to make leadership roles attractive for doctors and to support them in taking on these roles.⁵

What then needs to be done to strengthen medical leadership? A number of other recent studies^20,21 concur with our findings about the kinds of barriers faced by doctors seeking to take on leadership roles. It is essential that a greater degree of professionalism is brought to bear in the development of medical leadership. This includes developing career structures to make it easier for doctors to take on leadership roles; providing training, development and support in management and leadership at different stages of doctors’ careers; and ensuring that pay and other rewards are commensurate with the responsibilities of medical leaders and compare favourably with those available to doctors who remain in clinical roles. Equally important is the need to value doctors who become leaders and to avoid the perception that in doing so they are going over to ‘the dark side’. Alongside these initiatives, every NHS trust must redouble and renew its efforts to develop medical leadership, learning from best practice in other health care systems where more progress has been made. This includes ensuring that medical leaders are able to commit sufficient time to carry out their responsibilities effectively. As our research found, medical directors typically spend around 50% of their time to leadership roles, while clinical directors commit only 20%. Our findings also show that only 10% to 20% of consultants are involved in leadership roles, meaning that this remains a minority activity and accounts for approximately 5% of the time of senior doctors working in NHS trusts.

We believe that the NHS will only make the step change that is needed if the time commitment of medical leaders increases substantially and if the proportion of doctors in leadership roles also increases. Equally important is to attract more doctors into leadership roles who are credible with their peers and for them to become role models for their colleagues. In organisations like Mayo Clinic and Kaiser Permanente in the United States, it is common for around one-quarter of doctors to hold leadership roles and for the majority of their time to be committed to these roles.²² This is especially important at the clinical directorate/divisional/service-line level where a 20% commitment is unlikely to be sufficient as financial and service pressures in the NHS increase.

The argument for a step change in approach is reinforced by research into the relationship between medical engagement on the one hand and organisational performance on the other. A study carried out in the NHS in England found a positive relationship between medical engagement and performance.²³ There is also evidence from other countries that shows a link between chief executives who come from medical backgrounds and organisational performance, and organisations with clinically qualified managers and organisational performance.^24,25 Although many other factors also contribute to how well organisations perform, this evidence indicates that medical engagement and leadership by doctors are critical ingredients.

The kinds of actions we have outlined here are crucial not only for the NHS to fulfil its medical leadership aspirations but these also apply to other health systems to a greater or lesser degree depending on the specific context. Without medical leadership being taken seriously, there is a risk that medical leaders will experience increasing frustration in not having the time and support to carry out their responsibilities effectively. This may result in them feeling they are being set up to fail, if not deliberately then at least through neglect of the resources and commitment required to make a reality of medical leadership. As a number of international commentators have noted,²⁶ medical leadership is not something that can be quickly brought about through a change in structure or just exhorting clinical and managerial colleagues to change. We need to think about how we value and treat those in medical leadership positions and ensure that we effectively support them so that they are enabled to deliver high-quality care.

Footnotes

Funding

This project was funded by the National Institute for Health Research Service Delivery and Organisation programme (project number 08/1808/236). The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR SDO programme or the Department of Health.

Declaration of conflicting interests

The authors declare that there is no conflict of interest.

References

Griffiths Report. NHS management inquiry, London: Department of Health and Social Security, 1983.

Department of Health. Next stage review: high quality care for all, London: Department of Health, 2008.

Secretary of State for Health. Equity and excellence: liberating the NHS, London: HSMO, 2010.

Francis

. Report of the Mid Staffordshire NHS foundation trust public enquiry, London: The Stationery Office, 2013.

Ham

Dickinson

. Engaging doctors in leadership: what can we learn from international experience and research evidence?, Warwick: NHS Institute for Innovation and Improvement, 2008.

Harrison

Pollitt

. Controlling the health professionals: the future of work and organization in the NHS, Buckingham: Open University Press, 1994.

Kitchener

‘All fur coat and no knickers’: contemporary organisational change in United Kingdom hospitals. In: Brock

Powell

Hinings

(eds). Restructuring the professional organisation, London: Routledge, 1999.

Bate

. Changing the culture of a hospital: from hierarchy to networked community. Public Adm 2000; 78: 485–512.

Marnoch

McKee

Dinnie

. Between organizatiuons and institutions. Legitimacy and medical managers. Public Adm 2000; 78: 967–987.

10.

McKee

Marnoch

Dinnie

Medical managers: puppetmasters or puppets? Sources of power and influence in clinical directorates. In: Mark

Dopson

(eds). Organisational behaviour in health care: the research agenda, Basingstoke: Macmillan Press Ltd, 1999, pp. 89–116.

11.

Strong

Robinson

. The NHS under new management, Milton Keynes: Open University Press, 1990.

12.

Chantler

. The role and education of doctors in the delivery of health care. Lancet 1999; 353: 1178–1181.

13.

Harrison

. Managing the national health service: shifting the frontier?, London: Chapman and Hall, 1988.

14.

Fitzgerald

Lilley

Ferlie

. Managing change and role enactment in the professionalised organisation, London: Report to the National Coordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO), 2006.

15.

Spurgeon

Clark

Ham

. Medical leadership: from the dark side to centre stage, London: Radcliffe Publishing Limited, 2011.

16.

Foot

Sonolo

Maybin

. Service-line management. Can it improve quality and efficiency?, London: The King's Fund, 2012.

17.

Dickinson H, Ham C, Snelling I, et al. Are we there yet? Models of medical leadership and their effectiveness: an exploratory study. Final Report, NIHR Service Delivery and Organisation programme, 2013.

18.

Greener

Harrington

Hunter

. A realistic review of clinico-managerial relationships in the NHS: 1991–2010, Southampton: National Institute for Health Research Evaluations, Trials and Studies Coordinating Centre, 2011.

19.

Friedson

. Professional powers: a study of the institutionalization of formal knowledge, London: The University of Chicago Press, 1986.

20.

British Medical Association . Doctors' perspectives on clinical leadership, London: BMA, 2012.

21.

Ham

Clark

Spurgeon

. Doctors who become chief executives in the NHS: from keen amateurs to skilled professionals. J R Soc Med 2011; 104: 113–119.

22.

Ham

. Doctors in leadership: learning from international experience. Int J Clin Leadersh 2008; 16: 11–16.

23.

Spurgeon

Mazelan

Barwell

. Medical engagement: a crucial underpinning to organizational performance. Health Ser Manage Res 2011; 24: 114–120.

24.

Goodall AH. Physician-leaders and hospital peformance: is there an association? www.amandagoodall.com/SS&MarticletJuly2011.pdf (2011, accessed 22 June 2012).

25.

Dorgan

Layton

Bloom

. Management in healthcare: why good practice really matters, London: McKinsey & Co and Centre for Economic Performance, 2010.

26.

Braithwaite

Westbrook

. Rethinking clinical organisational structures: an attitude survey of doctors, nurses and allied health staff in clinical directorates. J Health Ser Res Policy 2005; 10: 10–17.