Abstract
The aim of this article is to explore the effect of the joint procurement model adopted during the English National Programme for Information Technology (NPfIT) on the customisation, design and usability of a hospital ePrescribing system. Drawing on qualitative data collected at two case study sites deploying an ePrescribing system jointly procured within one of the NPfIT’s geographical clusters, we explain how procurement decisions, difficult relationships with the supplier and strict contractual arrangements contributed to usability issues and difficulties in the customisation process. While some limited change requests made by users were taken up by the developers, these were seen by users as insufficient to meet local clinical needs and practices. A joint procurement approach, such as the NPfIT, thus limited the opportunity and scope of the changes to the ePrescribing system, which impinged not only on the perceived success of the implementation but also on the system’s usability.
Keywords
Background and significance
England’s National Programme for Information Technology (NPfIT), launched over a decade ago with the aim of having an electronic health record (EHR) for every patient in the National Health Service (NHS), was widely seen at the time as the biggest and most complex public sector health information technology (IT) programme ever undertaken.1 –5 By 2011, the Programme had made little progress in achieving its aims, and as a result the English government declared it to have ‘failed’ 6 and initiated its dismantlement. Despite this, the Programme did not vanish altogether, nor did many of the ideas behind it. A number of its components and contracts will remain in place until later this year, 6 with doubtless long-term influences on product choice and availability. The overall aim of moving towards hospital EHRs has also been maintained in the form of the English government’s ambition to make the NHS paperless by 2018. 7 Moreover, there is continued interest from government and publicly funded bodies such as the NHS in the principle of joint procurement with repeated calls being made for improved cohesion in national procurement, 8 to achieve efficiency savings and financial benefits and to stimulate innovation. 9 Indeed, the inefficiencies of localised uncoordinated procurements have been well documented 10 and have underpinned arguments for more collaborative approaches. This raises the possibility therefore of large, multi-site contracts being signed once again by England’s NHS.
Joint procurement is complex, however, and may not necessarily achieve the desired results. Indeed, while procurement of IT through the National Programme was intended to provide consistency in patient care, standardise user interfaces and improve clinical governance, it may, paradoxically, have in fact reduced the pace of adoption of ePrescribing because many hospitals deferred their planned roll-outs11,12 in an effort to benefit from a Programme which was, however, later to be abandoned. A recent IT benchmarking exercise by two London NHS Hospital Trusts with decision-making powers devolved from central government, who took part in a local procurement partnership scheme, further illustrates the adverse impacts of joint procurement. When reporting on the outcome of the exercise, it was concluded that despite using the same EHR system, ‘there are major issues in how the two Trusts’ IT systems operate’, 13 such that the level of investment required negated the benefits of joint procurement. This in itself highlights one of the main predicaments of the National Programme: a joined-up approach seeking to improve the integration of information within a highly politicised efficiency framework which calls for better management of resources, 14 while seemingly ignoring the realities of technological adoption in diverse local settings. Too large and overly centralised,15,16 top-down, 17 and bureaucratic,18 –20 providing limited opportunities for customisation and simplistic assumptions about the use of ‘generic solutions’, 21 the Programme failed to recognise the different stages in the adoption of technology and the ‘lifecycle’ 22 or ‘biography’ 3 that marks the journey of an IT system’s implementation. There are now mounting concerns that the ‘techno-utopian vision’ 23 of a paperless NHS with its ‘one size fits all’ 16 undertones could lead to a repeat of the difficulties experienced by the NPfIT. With such concerns in mind, we designed a qualitative study to describe and understand the implications of this procurement model and the technological ambitions it pursued. More specifically, by exploring the experiences of those involved in the customisation and implementation of an ePrescribing system procured within the NPfIT, the study aimed to demonstrate how the system’s usability, and therefore users’ perception of it, were affected by a joint procurement model.
Materials and methods
Design
The design of the study was based on a comparative case study method, where two strategically selected ‘intrinsic’ 24 cases (hereafter referred to as Sites J and K) that hold particular characteristics, were chosen to allow the exploration of key themes and the acceptance or rejection of theoretical propositions 25 and in particular how the shortcomings of the NPfIT5,6,10,11,15,16 –20,23 and the implementation issues inherent in such a procurement model11,21,26 –31 may help explain the complexity of an IT system’s adoption within and across organisations. We chose a comparative case study approach to allow good internal and external validity, 25 by delineating the relation between particular events, and the crossover from the findings to the theory, thus helping elucidate causality. 32 Given that the aim of the study was to understand experiences associated with the customisation of a single system procured within the particular context of a multi-site contract, the cases selected were not intended to be representative of other hospitals implementing ePrescribing. Instead, they served to offer details of the real-life context of each case study, 23 thereby highlighting important considerations in the procurement, customisation and deployment of IT systems involving multiple public and private sector actors.
Sampling and recruitment
The two case study hospitals were drawn from one of the five NPfIT geographical clusters (North West/West Midlands, North East, East of England/East Midlands, London, Southern), which were served by a single contracted local service provider (LSP). The LSP’s role was to provide the technological infrastructure only, and a healthcare software supplier was thus subcontracted by the LSP. 11 The two sites were selected as they were both deploying the same ePrescribing system, working with the same contractor and subcontractor, and on similar timescales. These sites were also selected because of their differences, in terms of size (Site J = 1000 beds and Site K = 500 beds) and stages in their IT deployment, which was seen as indicative of their digital maturity. 7 Site J has over the last decade gradually deployed a number of IT modules in its efforts to move towards EHR, whereas Site K had no previous experience with an EHR system.
Individual interviewees were selected to represent a range of perspectives from IT and the interdisciplinary medical teams taking part in the customisation work at the hospitals. Participants were referred to the study initially via the ePrescribing clinical lead at each hospital and then from interviewees themselves. Follow-up invitations to take part were sent by the lead researcher directly.
Data collection
Qualitative data were collected between April 2013 and November 2013 by means of 39 semi-structured face-to-face interviews. During these interviews, 40 members of staff at the hospital sites and two managers from the system vendor company were interviewed. Staff interviewed included members of the IT and implementation teams, doctors, nurses, pharmacists and allied health professionals.
Interviews lasted between 15 and 90 minutes, were audio-recorded with the participants’ consent and then transcribed verbatim. A topic guide (see Table 1, Appendix 1) was used to ensure whether sufficient data could be obtained on key areas of interest, including the system’s customisation and delivery, usability and lessons learnt. Saturation in the data collection was seen to have been achieved when no new themes emerged from the interviews. Additionally, 45 observational fieldnotes were taken at the sites and at a series of three system customisation events in January, April and June 2013, organised by Site J at the supplier’s offices. These meetings involved interdisciplinary medical teams and implementation staff from both the hospital and the vendor. Together they discussed and sought to address design issues in the system and then reviewed and validated modifications. The observational notes made at the events were used to supplement the data collected during the interviews, and in particular to elucidate on important details of how vendors, implementation teams and end-users interacted in the context of the system’s customisation.
The data were collected by a social science researcher external to the sites and the purpose of the study was presented as an evaluation to inform future implementations of ePrescribing systems. It was hoped this would allow participants to have a frank and open discussion, which focused on the lived experience of the implementation.
Data analysis
Interview transcripts were fully anonymised, coded and analysed using the qualitative software QSR NVivo 9 (QSR International, Inc., Melbourne, Australia) to explore core themes relating to processes of adoption of technology, and how these are shaped by perceptions of the system, key actors and the context of the National Programme. Inductive approaches were employed to develop a coding framework which combined a thematic analysis of the data and the literature pertaining to implementation and adoption of EHR.2,5,11,16 –20,22,23,26 –31,33 The coding framework was used to foreground three key areas that defined the customisation, deployment and usability of the system: (1) end-users’ perception of their ability to influence decisions whether at the point of procurement or customisation, (2) a perceived divergence of goals between the supplier, hospital management and end-users, and (3) contractual arrangements. This allowed to unravel the complexity of customisation in a joint procurement model, and clarify where boundaries of control and autonomy lie for users in the development of an ePrescribing system.
Each case study was considered individually in the first instance to allow coding to be applied and then findings to be compared and contrasted. The observations and fieldnotes were similarly codified and also used as part of disconfirming data searches.
Ethics approval and permissions
Participation in the study was voluntary, following the email invitation sent by the lead researcher to potential participants. An information leaflet was provided to all interviewees, and a consent form subsequently completed and signed by each participant prior to the start of the interview. All records of individual names or contact details were removed from the dataset prior to analysis to ensure that all responses remained anonymous. The programme of work was reviewed by and received ethical approval from the Centre for Population Health Sciences Ethics Review Group at The University of Edinburgh.
Findings
There were significant challenges adapting the ePrescribing system and ensuring its readiness for go-live. Users perceived the customisation process as a mechanism they were entitled to pursue in order to make user-led design and usability decisions. In reality, however, choice and customisation were often limited and seemingly strictly controlled by suppliers whose inability to respond to user demands was justified by the legacy of the joint procurement of the NPfIT. The push and pull of control and autonomy in the local customisation of the system mirrors tensions noted elsewhere between standardisation and diversity 34 and standardisation and flexibility, 35 when attempting to design a system for a specific user and for a wider market. In the case studies, the tensions between stakeholders became especially apparent in relation to procurement decisions, goal alignment and contracts. We report on each of these below.
Procurement decisions
Overall, end-users were very positive about the move to an electronic environment. However, most interviewees voiced their dislike of the system selected. Originally designed as a billing system for the American market, according to respondents the system did not reflect local working practices, including the manner in which hospital doctors prescribe in England, the individual professional tasks of interdisciplinary medical teams, or national and local prescribing safety policies and guidelines. At a basic level, the system was described as ensuring that ‘the right drug and dose [are] given at the right time, to the right patient’ (Nurse Lead for IT #38, Site K). However, the role of individual clinicians and the manner in which drugs were ordered and dispensed, and on whose instructions, created a gulf in the applicability of the system’s functionality in an English hospital. There was a perception among respondents that in many cases the system was not fit for purpose, which left end-users bewildered of how such a system came to be procured nationally. The procurement process appeared to have created from the outset negative views of a system which end-users saw as ill-suited for the clinical setting in which it was being introduced. The centralised (and thus organisationally and temporally distant) selection process led to feelings of exclusion among the everyday users of the system. There was, as one interviewee made clear, frustration that the views of doctors appeared to have been ignored when contracts were signed between the English government and the supplier: I went to some of the inaugural meetings in 2004 and I said that I thought it was a very bad idea to have an existing American system supplied by a company which had shareholders and which needed to be adapted and I thought really we should just start our own department and design something from scratch and they said oh sorry the contract has already been signed … We’re stuck with it basically. (Consultant #31, Site K)
This frustration was compounded by a perception among end-users that influencing decisions for procurements after 2015 (when NPfIT contracts come to an end) was, as one respondent put it, ‘hugely prejudiced’, a situation he went on to describe as ‘like being handcuffed and having a gun to your head’ (Consultant #2, Site J), with decisions resting on financial rather than usability considerations. Some respondents suggested that the incumbent system had an unfair advantage over other systems, as the level of investment made to ensure the product’s readiness – such as the resources allocated to the customisation of the system, training, engagement as well as the accumulation of knowledge and expertise – was seen as considerable and difficult to replicate. The implementation teams at the sites have now confirmed that the existing supplier will continue provision beyond the 2015 end of the NPfIT arrangements.
Divergent goals
Commercial product within a health context
The financial pressure of the procurement process had favoured a climate which seemed to be at odds with clinical practice and patient safety. One participant explained how suppliers used ‘hard sell techniques like washing machine salesmen and … double glazing merchants’ (Consultant #31, Site K), to ‘make money’ (Consultant #32, Site K).
The commercial aspirations of a corporate organisation should be of little surprise. However, while rightly recognising this dimension of the supplier’s agenda, end-users perhaps wrongly assumed an inevitable divergence of goals between the hospital and the supplier. End-users directed their anger at what they saw as opportunistic behaviour by vendors and expressed their concern about the reasons they felt lay behind the delays in fixes to the system: And a lot of the things that were errors they have clearly sat on to wait, rather than fixing it now and then you can see they have sat on these errors because … when you go for a change request later on they get paid money for it … there is a point in the contract where they started getting paid for change requests. (Consultant #32, Clinical Engagement Team, Site K)
While such delays were likely the result of unrealistic expectations on the part of the end-users, a lack of specification in the contract itself or a combination of both, they did also suggest a perception that the goals of suppliers and end-users were misaligned. End-users did not recognise that there was much more to be gained for suppliers from a successful deployment in a site which could act as a showcase, and thus, help build future business.
Relationship management
Discussions with participants at the sites frequently suggested a gulf between the supplier and end-users which, as one participant explained, meant that ‘they don’t understand what we’re talking about, we don’t understand what they talk about’ (Consultant #8, Site J), leading to what others described as usability and safety issues because of a ‘disconnect’ between users and those designing the system: … poor design, poor IT as far as I’m concerned … So if you have a system that isn’t well designed from the ground up then … I can’t see that necessarily the system will improve over time … the answer to an awful lot of questions at these meetings from [the supplier] is, it’s not possible. (Consultant #9, Site J)
One nurse described his frustration when usability issues could not be rectified: I mean they’re a computer company but it’s something they said absolutely can’t be changed to just bring these two boxes closer together … it’s just one of these non-negotiable dead ends. (Nurse #5, Site J)
There were other instances, where end-users felt that basic changes, which would reflect more closely the paper charts they were both familiar with and valued, could not be included. It was also pointed out how change requests that would help maximise the benefits of having a computerised system, such as automatic calculations or self-populating fields, were turned down.
Such failures to respond adequately to end-users’ requests, often led to what one respondent called a ‘love–hate’ relationship, which made for an uncomfortable collaboration at times, in which the supplier’s reasons for not taking forward change requests were questioned: I know their product as well as they do, which means that I can work out when they’re maybe not telling the truth because it’s creating a certain amount of extra work for them. Obviously their aim is to deploy projects in a certain length of time to make sure they’re successful and they get it off their books and they get paid, they’re a corporation; that’s what they’re there to do, whereas I’m here to ensure that we do this project safely. I want to complete and preferably on time, if it slides a few weeks for patient safety reasons then I’m happy with that. (Pharmacist #29, Site J)
The divergent priorities of the stakeholders were visible in the mechanisms each employed to achieve their desired objective. In the case of dissatisfied end-users, this involved resistance and disengagement, for instance by failing to attend product review meetings or training sessions, or strongly and openly criticising the system. According to some end-users, the implementation team had included only a select group of clinicians who were unrepresentative of the wider user-base, arguably ensuring a tighter grip on the customisation. It was clear, however, that for the implementation team, as illustrated in the quote above, it was product expertise which became a formidable asset in the management of the relationship and the completion of the project. As noted earlier, the supplier for its part was keen to ensure delivery of the contract by completing the project to help it consolidate and expand its position in the UK’s ePrescribing market.
Contracts
The contract itself came to embody this divergence of goals because of the structure it imposed on both the supplier and users, and the distance it created between them. It was frequently described how making requests for amendments was extremely difficult to influence, which was significant given that the system was at the outset seen as ill-suited to the English context: The contract … limits what the company can respond to in terms of staffing, the product and the services that you can get, so you get a standard approach rather than a customised approach if you went directly. (Programme Delivery Manager, #23, Site K) … the contractual arrangements make it quite difficult between the end-user organisation and getting to the intellectual property owners of the system, [it] is quite hard work because you’ve got to go through [the contracted LSP], you’ve got to go through the National Programme for IT and various levels of administration who don’t actually hold the contract. (Director IT #17, Site J)
The contract was repeatedly described as an obstruction to the delivery of the project, rather than a consensus building instrument, symbolising a lack of choice and loss of autonomy in the deployment of a ‘deficient’ system which had been ‘inherited’. It was pointed out that improving the system’s usability was often difficult, not only because the contract limited the scope of the project, and therefore which changes could be made, but also because those involved did not know what would be needed to make the system fully functional. The supplier too seemed to have a poor understanding of what enhancements were required during the customisation. As a result, the product was seen as far from being up to speed with current clinical requirements. The contractual limitations meant that if a change could not be made to the system, a physical workaround had to be found, making the transition to electronic prescribing only a partial reality: … there will be manual workarounds and some of them will be, not necessarily elegant but will present problems but not insurmountable problems there will be physical workarounds, i.e. documenting on paper for some parts. (Change Manager for Clinical Systems #23, Site K)
The difficulties associated with the customisation and the additional time and associated costs these have created, meant that some areas of the project had to be scoped-out, including in the case of Site K fluids and intravenous medicines. At Site J, the system had only been rolled out to approximately 40 per cent of the hospital by the end of the contract. As a result of these partial-implementations, a dual system of paper and electronic prescribing had remained to greater and lesser extent in both hospitals. The 10-year timeframe of the contract also meant that by the time the system was ready to be rolled out, it was seemingly outdated and had locked the hospital Trusts into an older version of the system to the one currently available from the supplier. The timescales of the project meant that usability and arguably therefore safety issues were not appropriately addressed: … there are certain limitations into how you can manipulate the system and I think that is one of the biggest drawbacks … because we’re in Programme we have to implement it before 2015 when the Programme finishes so, you know, you’re sort of pressured into rushing certain elements of it. (Pharmacist #35, Site K) … the problem is we’re all working within a tight contract which wasn’t drawn up by the Trust so to some extent, you know, what we’re working with isn’t our choice and there may be things that we want to change that we can’t change. (ePrescribing Project Manager #3, Site J)
The national contracts were used to deflect away from the Trust problems with the system, defer accountability to the LSP and to justify its acceptability. Indeed, the system was frequently described as having been obtained virtually ‘for free’ as part of the National Programme, which did appear to soften some users’ perceptions of its shortcomings. For others, however, the system added too few clinical improvements: …. it fails what I would call the pen and paper test … it’s significantly more laborious or complex to do a task … and that has never really been factored in by some of the enthusiasts for the system who look at the gains from the system being about audit trails and about accountability. (Consultant #2, Site J)
Discussion and conclusion
This work has underlined how procurement is a critical aspect of the process of technological deployment and adoption, and that a joint procurement model may paradoxically make it difficult to fulfil the goals that the contract was intended to meet, undermining the claimed benefits of efficiency gains in terms of cost-savings and better use of expertise. In particular, this model appears to exacerbate usability issues and difficulties in the customisation process, by making it unclear where leadership and control of the project lie. User organisations find it difficult to specify in advance their requirements for such complex IT applications, in part because their requirements change as they learn how to implement and exploit them. This reflects the call made elsewhere to improve mechanisms to exchange knowledge in the field. 36
Evolving user requirements limit the scope for strict enforcement of procurement contracts, generating problems that economists describe as ‘incomplete contracting’ that may mandate in favour of long-term collaborative procurement arrangements. 34 Such incomplete contracting and the asymmetrical distribution of IT knowledge between suppliers and users give suppliers autonomy and leave users vulnerable to opportunistic behaviour. However, the reputational damage of failure in the health service is potentially very large, leading to efforts to attempt to customise, such as in the case studies where a wide range of changes were made – when this appeared feasible – to improve the functionality and usability of the system. This motivates the supplier’s continued efforts to successfully complete delivery of the project. The discontent consistently expressed by end-users would appear rather paradoxical given the importance of reputation to the supplier, which allows end-users to control not only the relationship within their current contract but also of future procurements within the Trust and beyond.
Arguably, such reputational control becomes especially marked in a joint procurement context because of the impact failure in one site could have on other members of this alliance, and of the scale of the projects associated with joint procurement. Yet it is this very joint procurement model which leads to end-users’ frustration as they are denied the control of a customisable system, resulting in a perception that the supplier is incompetent and the system weak. Fundamentally, there is a mismatch between the supplier’s generification of its products 3 and end-users’ desire to customise a system to meet local clinical needs and practices. Incomplete contracting flags the users’ difficulties in anticipating how best to translate these needs and practices in an electronic environment and how such difficulties need to be handled within the contract. Addressing the difficulties associated with procurement would certainly remove much uncertainty in the customisation and deployment of systems, which is why the NHS is now grappling with such issues in its ePrescribing Procurement Toolkit. 37
There are of course some unique features to the contractual arrangements currently in place at the two case study sites, which are by no means typical of all NHS hospitals deploying IT systems, let alone those rolling-out ePrescribing systems (whether these are customisable systems or not), or indeed in deployments in other parts of the world. That said, they do illustrate lessons that need to be learnt from the problems of the NPfIT beyond the ‘symbolic narrative of failure in innovation’. 38 The study demonstrates how procurement and contractual arrangements are intrinsic to a system’s ease of deployment and use, and goes some way to demystifying the benefits and pitfalls of large scale procurements. This understanding may lead to more realistic expectations of how ePrescribing and other systems can be meaningfully adopted to reflect organisational priorities as well as local practices and policies. More work is clearly needed to establish whether this may mark a shift towards procuring systems which can be configured rather than customising. A configured system may reduce the need to address issues at the level of the user interface and may allow a more localised bottom-up approach to IT deployment, which provides potentially a system that is more clearly aligned to the needs of individual medical specialties. However, interoperability issues may make the deployment of such systems difficult to achieve in practice at a technical level because of the lack of standardisation of data and data structures. 11 Indeed, despite the often cited success of picture archiving and communication systems (PACSs) within the NPfIT, the adoption rate should be attributed to PACS’ maturity and the standards that have enabled collaboration between suppliers, as well as the limited stakeholder perspectives that needed to be balanced. 39 It will be important, therefore, for future research to explore whether these tensions occur in customisations outside of such contractual arrangements, and indeed whether long-term usability and user adoption are affected by the difficult path to deployment we found in these two sites.
Footnotes
Appendix 1
Interview topic guide.
| • Interviewee’s background and involvement in ePrescribing deployment at site |
| • Details of the system (previous systems, decision to implement, choice of system and integration with other systems, including local and national systems) |
| • Views on system usability, implementation strategy, customisation process |
| • Engagement efforts, consultation (including changes that the user would like to see happening in the system and in relation to the implementation strategy); client/supplier relationship |
| • Perceived/anticipated consequences of the system on the quality of care, information flow, patient experience, roles and practices of healthcare professionals, the organisation, the local community |
| • Training received and ongoing support; IT literacy and skills |
| • Other impact/benefits/challenges/concerns |
Acknowledgements
We gratefully acknowledge the input from our Independent Programme Steering Committee and members of our Patient and Public Involvement (PPI) group. We thank the editor and two anonymous reviewers for their constructive comments which helped us to improve the manuscript. We gratefully acknowledge the input of our research secretary who transcribed the interviews. We also thank the two hospitals who took part in the study and all the study participants for their time and responses.
Collaborators
On behalf of the NIHR ePrescribing Programme Team: Dr Jamie Coleman (Senior Clinical Lecturer Clinical Pharmacology, University of Birmingham), Professor Tony Avery (Professor of Primary Health Care, University of Nottingham), Dr Laurence Blake (University of Birmingham), Dr Kathrin Cresswell (Senior Research Fellow, University of Edinburgh), Dr Sarah P Slight (Senior Lecturer in Pharmacy Practice, School of Medicine Pharmacy and Health, University of Durham and Visiting Research Scholar, Division of General Internal Medicine, Brigham and Women’s Hospital/Harvard Medical School, Boston), Dr Alan Girling (Senior Research Fellow, University of Birmingham), Professor Richard Lilford (Professor of Public Health, University of Warwick), Dr Lucy McCloughan (eHealth Research Manager, University of Edinburgh), Dr Zoe Morrison (Senior Lecturer, University of Aberdeen), Dr Hajar Mozaffar (Research Fellow, University of Edinburgh), Dr Ann Robertson (Research Fellow, University of Edinburgh), Dr Behnaz Schofield (Research Fellow, University of Edinburgh), Professor Jill Schofield (Head of The York Management School, University of York), Ms Ann Slee (Honorary Fellow, University of Edinburgh).
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Ethical approval
The programme of work has been reviewed and received institutional ethical approval.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a programme of independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research scheme (RP-PG-1209-10099).