Abstract
Objectives
To evaluate an accelerated form of experience-based co-design (EBCD), a type of participatory action research in which patients and staff work together to improve quality; to observe how acceleration affected the process and outcomes of the intervention.
Methods
An ethnographic process evaluation of an adapted form of EBCD was conducted, including observations, interviews, questionnaires and documentary analysis. Whilst retaining all components of EBCD, the adapted approach replaced local patient interviews with secondary analysis of a national archive of patient experience narratives to create national trigger films; shortened the timeframe; and employed local improvement facilitators. It was tested in intensive care and lung cancer in two English National Health Service (NHS) hospitals. A total of 96 clinical staff (primarily nursing and medical), and 63 patients and family members participated in co-design activities.
Results
The accelerated approach proved acceptable to staff and patients; using films of national rather than local narratives did not adversely affect local NHS staff engagement, and may have made the process less threatening or challenging. Local patients felt the national films generally reflected important themes although a minority felt they were more negative than their own experience. However, they served their purpose of ‘triggering’ discussion between patients and staff, and the resulting 48 co-design (improvement) activities across the four pathways were similar to those in EBCD, but achieved more quickly and at lower cost.
Conclusions
Accelerated EBCD offers a rigorous and relatively cost-effective patient-centered quality improvement approach.
Background
Measuring or improving patient experience?
Measuring, understanding and improving patient experience is a priority for healthcare systems worldwide. 1 Policy documents in several countries describe ‘patient experience’ as a core component of care quality, drawing on frameworks for patient-centered care developed by, for example, the Picker Institute (following Gerteis et al. 2 ) and the US Institute of Medicine. 3 The English NHS has led the way in measuring patient experience by introducing the first nationally mandated patient survey. 4
Justifications for seeking to improve patient experience include moral humanitarian concern; the need to maintain political legitimacy in state-funded systems; and the need to increase/maintain market share in more competitive systems. A growing international body of evidence suggests patient experience, safety and clinical effectiveness are linked, showing positive associations between patient experience measures and both self-rated and objectively measured health outcomes5,6 as well as positive organizational outcomes.7–13
Despite the strong policy focus, some patients continue to report unsatisfactory experience in even the best funded health systems. 14 In England, recent government policy has identified the lack of a genuinely patient-centered approach. Examples of poor care have dominated healthcare debate, the Francis Inquiry into Mid Staffordshire Hospital being the most widely reported. 15
Robert and Cornwell 4 argue that the focus on experience surveys and targets may paradoxically have ‘contributed to a tick-box or compliance mentality on the part of [hospital] management boards… [and] lulled them into thinking that they were paying attention to their patients' experiences’. Experience data presented to NHS hospital boards is often ‘noted for information’ rather than leading to action points and organizational learning. 16
Failure to attend to basic human needs and dignity, as documented in the Francis Report, has resulted in numerous recommendations to re-educate healthcare staff and initiatives to improve compassion. No one would dispute the importance of compassion and kindness, but the danger is that the energy behind this latest series of initiatives may produce more targets, tools, metrics and inspection, reinforcing the ‘compliance mentality’ as organizations work to measure how compassionate their staff are, rather than the more fundamental task of focusing on patients and changing the culture to enable this to happen.
A large, growing body of quantitative and qualitative research3,17–19 already identifies aspects of care that patients and families think are important. The challenge is to find meaningful ways to enable organizations to learn from this evidence and move beyond data-gathering to using patient experiences to improve care.20–22 Surveys provide useful information to make comparisons between sites and across time, but as a recent review of the impact of the UK National Inpatient Survey 23 concludes, ‘the inpatient survey is not in itself a quality improvement tool’ and ‘simply providing hospitals with patient feedback does not automatically have a positive effect on quality standards’.
There is increasing interest in the use of in-depth qualitative research to gain richer, meaningful accounts of what being a patient is like, yet healthcare providers struggle to use qualitative experiential evidence for improvement. 16
The value of narrative and experience-based co-design
Narrative persuasion is an established psychological theory 24 demonstrating that narrative is a powerful way of accessing human experience, transporting us to another world and enabling us to see things through another's eyes. Narratives can engage care providers, at a deep emotional level, in reflecting on how services could be improved.25,26 Narratives are not gathered because they are assumed to be objective, accurate or verifiable but because they are uniquely human and subjective, describing not a fact or a reality but a recalled experience or set of experiences.
Experience-based co-design (EBCD), a participatory action research approach, marks a significant contribution to using narratives and involving patients in quality improvement in healthcare. 26 The approach has been implemented in over 60 services internationally and independently evaluated in both England and Australia.27–29 It is effective in making specific improvements to particular services, as well as wider improvements within – and sometimes between – healthcare organizations.
EBCD projects typically last 12 months, beginning with a 6-month ‘discovery’ phase, in which local patients and staff are interviewed about their experiences of a service. Patient narratives are video-recorded, and used to develop a ‘trigger film’ to stimulate discussion between staff and patients about potential quality improvements. The discovery phase is characterized by rigorous, narrative-based research with a broad sample, rather than relying on a few anecdotes. Equally important is the subsequent co-design phase, in which patients, families and staff join as partners in small working groups to set priorities for quality improvement, and design and implement changes.
However, the discovery phase is felt by staff to be lengthy and costly, and has been reported as a barrier to adoption of the approach. 30 Undertaking 5–6 months of qualitative interviewing on each pathway in each hospital is seen as impractical.
‘Accelerated’ EBCD and our research questions
The Health Experiences Research Group (HERG) at the University of Oxford collects and analyses video/audio-recorded interviews with people about experiences of illness. It has an archive of around 3000 interviews, covering over 80 different conditions or topics. Selected interview extracts are disseminated for a lay audience on www.healthtalkonline.org. In this study we investigated whether this archive could replace the need for local discovery interviews, thus making EBCD faster and cheaper; we describe this as accelerated EBCD or ‘AEBCD’. By using national rather than local patient interviews, we aimed to halve the cycle from 12 to 6 months. Our research questions can be summarized as:
What improvement activities does the approach stimulate (for what cost) and how do these compare with traditional EBCD? How does using films of national rather than local narratives affect the level and quality of engagement with service improvement by local NHS staff? How well do local patients feel national narratives represent themes important to their own experience?
Methods
The intervention was an adapted form of EBCD: we used national trigger films, shortened the time frame and employed local improvement facilitators (senior nurses or service managers in the participating hospitals who were trained in EBCD). It was tested in intensive care and lung cancer in two English NHS hospitals. 96 staff (primarily nursing and medical), and 63 patients and family members participated.
The trigger films were derived from secondary analysis of the Oxford University collections of patient experience interviews. We re-analysed all the interviews in the archive with people with lung cancer, and individuals and family members with experiences of intensive care. The local facilitators interviewed staff to understand their experience of providing care. Thereafter, the process followed the usual EBCD pattern: the film was shown to local patients in a workshop; staff had a separate meeting to discuss the results of their own feedback. Staff and patients then came together in a further workshop to view the film, agree priorities for quality improvement and establish co-design working groups to take these priorities forward.
An ethnographic process evaluation using multiple data sources was conducted to observe implementation in both pathways in each hospital. Ethnography can contribute to process evaluation by providing rich accounts of activities, projects and programmes. 31 The ethnographer was in post throughout and was therefore able to observe all stages of the project. Data collection took place between November 2011 and December 2012 including 155 hours of observations (of facilitator training sessions, staff and patient workshops, joint events, co-design meetings and celebration events, project steering group and core group meetings); 30 interviews (designed around the following themes: involvement in the project and perceptions of the process; project contribution to service delivery; and project sustainability and legacy); two group interviews with four patient participants to discuss their involvement and perceptions of the process during each step of the intervention; cost analysis; and four service improvement logs (tables listing local improvement activities completed by the facilitators). Interviews, training sessions and events were audio-recorded and transcribed. Observations and brief conversations were recorded as field notes and 22 reflective diaries were completed (standard forms e-mailed to project team members on a monthly basis to gather direct information and reflections on issues and concerns regarding the day-to-day running of the project). 166 evaluation questionnaires (self-completion questionnaires, identical to those used in previous EBCD projects) were distributed to all participants at the end of each event and collected by the evaluator. Reports from 12 previous EBCD projects were analysed for comparison.
Transcripts, documents, fieldnotes and emails were entered into NVivo. Coding was based on our research questions (see above), as well as emerging themes (such as different styles of facilitation and implementation). Data were tabulated using framework analysis, a method of drawing up summarized data by theme and by case into a matrix. 32 Data analysis involved the following stages: familiarization with AEBCD and EBCD data; thematic analysis of qualitative data; tabulation and graphical representation of quantitative questionnaire and quality improvement data; indexing and developing a comparative framework based on key themes. Members of the project team refined the analysis at a two-day workshop.
Results
The accelerated approach proved readily acceptable to staff and patients; using films of national rather than local narratives did not adversely affect local staff engagement, and may have made the process less threatening or challenging. Local patients felt the national films generally reflected important themes although a minority felt they were more negative than their own experience. However, they served their purpose of ‘triggering’ discussion and the resulting improvement activities were similar to those in EBCD, but this was achieved more quickly and at reduced cost.
Improvement activities and cost
Comparison of improvement activities in accelerated experience-based co-design (AEBCD) and experience-based co-design (EBCD).
Note: Changes in the four AEBCD pathways were compared with those in two previous EBCD projects on breast cancer and lung cancer. The scale of the changes was similar. There were 28 activities across the two EBCD pathways, compared to 48 across the four AEBCD examples. There were similarities in terms of the co-design group topics. For example, two EBCD lung cancer groups focused on the same overall topics as the AEBCD lung cancer groups: information and diagnosis giving. There are also identical improvement actions in the EBCD and AEBCD lists. For example, name boards were introduced so that patients would know staff names and both groups identified the importance of a special, private room for diagnosis giving. Finally, there was a similar distribution of activities, with more small-scale changes and process redesign within teams than wider process redesign between services and between organizations. An overview of the number and nature of changes in EBCD and AEBCD is given in the box.
Although small-scale changes and process redesign within one service area are the most common result of both AEBCD and EBCD, it was observed that small-scale change is often remarkably complex to implement, and what looks like a small change can be immensely valuable to patients. Moore and Buchanan 33 have recently referred to this as ‘sweating the small stuff’. For example, ensuring an inpatient's belongings follow them as they move around the hospital seems a minor issue, but as the facilitator involved in this change noted, being without your false teeth when talking to the consultant is both practically difficult and humiliating.
Comparison of costs between accelerated experience-based co-design (AEBCD) and experience-based co-design (EBCD).
Acceptability and impact – staff participants
Previous evaluations have EBCD to be highly acceptable.27,28,35 Before the study, we speculated that removing local patient interviews might negatively affect local ownership and engagement. In fact, the project was positively received by both patients and staff; results from post-workshop questionnaires suggested broadly similar satisfaction levels to traditional EBCD. The interviews and comments recorded during observations featured many positive accounts. The opportunity to hear directly from patients and carers had a transforming effect on some staff. One described herself as ‘a better nurse because of it.’ A facilitator commented: The absolute core of it is the humanistic connections. Because of hearing patients', relatives' and staff experience it acts as a catalyst and gives you energy to keep going and make the change, and make sure it happens […] So many people have echoed that without me saying anything. I have not had to convert people into this methodology now. People just get it and wonder why we've never done it before. (Facilitator, interview) ‘I think [AEBCD] is phenomenal [… .] I don't have any idea, as a unit we don't have an idea of what is beyond what I'm seeing. This informs that process. So I can see that this person is not only a human being, but he is also a father, he is a son, he is a brother, he is a friend, he is a cousin, he's a plumber or an electrician, he is a sportsman, he has an interest in horse-riding, whatever it happens to be. He has a dog, he has a budgie, he has plans, he has expectations, he has regrets, he has feelings.’ (Consultant, intensive care, interview) ‘I have already changed the way I think and care for patients even though we haven't started implementing changes yet. I have a better understanding now of how things are from the patients' perspective.’ (Nurse, intensive care, fieldnote of conversation)
Critical aspects of the process included creating space to think and listen to others (both patients and colleagues): ‘I think the most important things were that staff really appreciated the time to think about the experience. It became apparent that they perhaps didn't have or make time to reflect on what they do in their daily workings, so I think they actually found it quite cathartic and therapeutic. […] I think the staff really appreciated that their point of view was being listened to because up until that point the political drive had always been patient experience and now all of a sudden we were interested in staff experience.’ (Facilitator, interview)
The observational data confirmed that the film achieved the goal of triggering thought, discussion and plans for action, and that using national narratives seemed to allow staff to engage comfortably with patient experiences.
One facilitator reflected that the use of national narratives may have helped staff feel less threatened by negative comments and able to externalize criticisms of care. It made me wonder that […] if the DVD had been produced locally, and we were all in a room together, and that was maybe the first time that healthcare professionals had had feedback from patients, as to how potentially devastating that could have been [… .] It did make me grateful that I didn't have to deal with a potential fallout if a patient had criticised a member of staff. […] I think that potentially could have quite catastrophic effects. That's why I wondered whether the Healthtalkonline actually minimised that risk.
Acceptability and impact – patient/carer participants
Intriguingly, patients and relatives sometimes felt the national films were more negative than their own experiences; this may be partly because people who volunteer for such projects feel positive about their local hospital and want to give something back for potentially life-saving care. However, the great majority of those present rated the film as ‘good’ (71%) or ‘excellent’ (14%) in answer to the questionnaire item ‘Seeing the film today – did you think it was a good representation of what it is like to be a [lung cancer/intensive care] patient at X NHS Trust?’ A few (8.5%) rated it as ‘average’ but no one as a ‘poor’ representation of their experience. Data were incomplete for 6.5%.
Workshop observations showed that after discussing the film, patients in both pathways gradually revealed more things they felt had not gone well in their care, and began to reflect on aspects of care which could be improved. It thus seems to have served a similar role to the film in other EBCD projects – triggering discussion. Overall reflections on the process focus on the value of meeting with other patients/relatives and staff and getting truly involved in change. ‘It was great to see you all at the “Patient Experience” event this week, what a wonderful thing it will be if we can contribute to and change what is already an incredible service that the intensive care department already provide - it makes me feel humble to be part of such a cause. Sorry to be gushing, but [facilitator] you truly are a guardian angel. I also feel that this experience should be shared with other hospitals as it seems they could learn a lot.’ (Patient, intensive care, email correspondence)
A lung cancer patient contrasted AEBCD with normal consultation in which ‘people say I'll do this, I'll do that, and nothing happens’. An intensive care participant said, ‘I feel I was really listened to’ and it was ‘much better than expected.’ Several expressed surprise that they had felt able to contribute as equal partners and that their views were taken seriously. ‘I wondered what, if anything, will be taken on board. To be honest with you, everything has been taken on board, and that in itself was a complete surprise.’ (Patient interview) ‘There was a comment from a patient within the event who said that everything that had been discussed had been taken forward into improvements… ‘our views were not dismissed, they were looked at and things changed’. ‘Thank you for looking at the little things.’ (Excerpt from ethnographer's field notes at celebration event)
Conclusions – the fundamental importance of co-design
This study was designed to answer specific questions about whether local patient discovery interviews in traditional EBCD could be replaced with a nationally derived trigger film, how this would affect patient and staff engagement, and whether this would affect the resulting quality improvement activities. For this study, we re-analysed just three of the interview collections in the archive. At the time of writing, it contains collections on over 80 different conditions, with more added each year. Sister projects have also been established in eight other countries under the auspices of the DIPEx International collaboration (see www.dipexinternational.org). This is an enormous potential resource for new films on a range of topics.
As the intervention progressed, it became increasingly apparent that agreement or identification with the film content was not central. The purpose of the films in both AEBCD and traditional EBCD is simply to ‘trigger’ discussion and enable people to share ideas and concerns; then joint work to redesign care can begin. If national trigger films are ‘good enough’ to initiate such conversations, local specifics can be brought into discussion along the way, and even disagreement with the content can generate productive discussion. But the element of the intervention that left both staff and patients feeling energized and empowered is the direct mutual encounter, the active partnership in co-design groups to achieve change, and the sense of tangible results.
A realist review of studies using participatory research 37 argues that the link between process and outcome in participatory research can best be explained by the middle range theory of ‘partnership synergy’; 38 that is to say, ‘multiple stakeholder collaboration creates or enhances research outcomes beyond what could be achieved by a single person or organization working under similar conditions’. 36 While on occasion the result can be conflict and negative outcomes, the review demonstrates that partnership synergy brings many benefits, including culturally and logistically appropriate research; better quality of outputs and outcomes over time; increased sustainability of project goals beyond funded timeframes; system changes; and new, unanticipated projects and activities. This resonates strongly with our findings, and is the antithesis of the problems underlying the Francis Report of failing to listen to patients or take their concerns seriously.
It is possible to implement improvements based on patients' experiences without involving them in the change process. Staff could be shown a trigger film or interview data previously collected from patients and then decide as a staff working group what needs doing. It is true that just seeing patient narratives on film can in itself have a powerful effect. But our observations suggest face-to-face encounters with patients have been even more transformative, inspiring and revelatory to staff in making them think differently about their values and practice. Having continued patient involvement helps ensure improvements really do address patient concerns and holds staff to account to see change through. Patients' physical presence constantly reminds everyone who change is for, and why it matters compared to other potentially overwhelming work pressures and demands. When staff report that this is the first time in 20 years that they have really talked to patients in this way or that it is the most rewarding thing in their careers, the potential of EBCD to reconnect staff with their fundamental values of care and compassion is striking. Patients, too, report a new level of appreciation for staff, a belief that they will be listened to and that change is possible, and renewed trust in their local NHS.
There is limited evidence on the costs and cost-effectiveness of patient-centered quality improvement. In comparing EBCD and AEBCD, this study has for the first time analysed the costs of EBCD. The next stage is to understand more about relative cost-effectiveness as no quality improvement activity is cost-free; it requires staff time, energy and commitment to make it happen. We believe there may still be scope for further economies in developing trigger films, whilst ensuring they remain based on high quality research into patient experience.
As ever with in-depth qualitative research, our findings relate to a small number of sites and may not transfer easily to other contexts; further evaluation would be helpful. What we have demonstrated, however, is one way to implement a proven technique faster and more cheaply without undermining its effectiveness. But, perhaps more importantly, we believe it is an approach that can revitalize staff. This is supported by work demonstrating that staff experience and patient experience are closely linked, 39 providing a new way of stimulating compassionate care.
Footnotes
Acknowledgements
The authors thank all the patients, family members and staff who took part in the project and the two trusts involved; members of our project advisory group; Alison Chapple and Joe Calabrese for leading the secondary analysis, Alison Chapple and Suman Prinjha for conducting the original interviews; the Intensive Care National Audit and Research Centre, the Department of Health and Macmillan Cancer Support for funding the original primary research; the DIPEx Charity for producing the trigger films; Thames Valley Health Innovation and Education Cluster; Farasat Bokhari and Graham Cookson for work on the cost analysis; and our peer reviewers for their helpful comments.
Ethics
This study was approved by the National Research Ethics Committee North West, reference 11/NW/0653
Funding
This project was funded by the National Institute for Health Research Health Services and Delivery Research Programme (project number 10/1009/14). Louise Locock was supported by the NIHR Oxford Biomedical Research Centre.
Department of Health Disclaimer
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HS&DR Programme, NIHR, NHS or the Department of Health.
