Changes in inpatients’ experiences of hospital care in England over a 12-year period: a secondary analysis of national survey data

Abstract

Objectives

Adult inpatient surveys generate approximately 70,000 responses per year about patients’ experiences of National Health Service hospital care in England. We examine historical data to assess change between 2002 and 2013 and consider the factors that may have stimulated any change.

Methods

Archived national data from National Health Service Inpatient Surveys between 2002 and 2013 (comprising 840,077 patient responders) were obtained. Questions were selected for inter-year analysis if they had been replicated for at least seven years. The percentage of responses in the most positive category was compared for each question’s earliest and most recent year. The statistical significance of differences was tested using chi-square. Also, since such large sample sizes mean that even 1% differences are statistically significant, effect sizes were used to assess the practical significance of those differences.

Results

There were statistically significant (p < .001) increases in positive responses to 35 questions, a significant deterioration for 8 questions and no change for 7 questions. There was one ‘moderate’ improvement (ϕ = 0.3), six ‘small’ improvements (ϕ > 0.1) and one ‘small’ decline, but differences were not meaningful for 42 questions. The greatest improvements were for patients receiving copies of doctors’ letters; single sex ward areas; clinicians’ hand washing; ward cleanliness and planned admission waiting times. The greatest decline was that fewer responders said their call bells were usually answered within 2 min.

Conclusions

More aspects of care have improved than have deteriorated. This study highlights the need for a consistent repeated survey programme to detect changes over the long term, since year-to-year changes tend to be small. The greatest improvements are in areas that can be influenced by organisation-wide interventions and many are associated with top–down government policies, targets or media campaigns. Patients’ evaluations of many aspects of their interactions with clinicians are unchanged or have declined. Further research could test whether ward-specific facilitated communication of survey results to clinicians could drive improvements in clinician–patient interactions.

Keywords

inpatient survey patient experience patient satisfaction quality improvement

Introduction

Measures of patients’ experiences have received increasing worldwide attention in the last two decades. National patient experience surveys now take place regularly in the US, the Netherlands, Norway, Scotland and England, and regional programmes exist in other countries.¹ With the publication of the National Health Service (NHS) Plan, England was the first country to mandate regular nation-wide surveys of hospital inpatients² and, since 2002, approximately 70,000 patients per year across England’s NHS hospitals have reported on their experiences by completing the mailed Adult Inpatient Questionnaire. All acute NHS hospitals participate in the survey, adhering to a standard method and submitting response data to a central body, which is currently the Care Quality Commission (CQC).³ The survey method⁴ and many of the questions have been consistent over time making it possible to consider trends in the national results.⁵

Few of the countries that conduct regular patient experience surveys have attempted to monitor national trends. In Australia, a study concluded that very high levels of patient satisfaction with general practice meant that the survey instrument was not useful for detecting changes.⁶ A Korean study found a substantial improvement in responses to a single question about satisfaction with ‘overall health services’ between 1989 and 2003.⁷ In England, one study noted generally ‘small improvements’ in inpatient survey results between 1998 and 2008,⁸ but another found ‘almost no change’ between 2002 and 2009.⁹ A third report found improvements in inpatient waiting times and clinicians’ hand cleaning but a decline in the availability of hospital staff between 2002 and 2007.¹⁰

Alongside the patient survey data, other evidence offers a mixed impression of the direction of change in patients’ experiences. Concerns have been raised about the quality of NHS nursing care¹¹ including the Francis Report into serious failings at Mid-Staffordshire NHS Foundation Trust which highlighted examples of unacceptably poor NHS nursing care.¹²

There are divergent views on the overall impact of nationwide NHS targets on the quality of care. Some argue that such targets have stimulated sustained improvements because they ‘rewarded success and penalized failure’.¹³ Following the publication of the NHS Plan Waiting Time targets, planned admission waiting times have declined.^14,15 Also, the target that 95% of patients should be admitted, transferred or discharged from emergency departments within 4 h of arrival was also broadly met until recently.¹⁶ However, others criticize the targets for distorting clinical priorities and being too arbitrary to promote meaningful improvements.¹⁷

Clinicians are sometimes sceptical about the relevance of survey data to their practices¹⁸ and they may be relatively less engaged than health care managers in quality improvement programmes.^19,20 Nurses are more likely than doctors to have the task of coordinating responses to NHS patient survey results, suggesting that doctors and nurses may differ in the extent to which they engage with patients’ evaluations of care.⁴ Most of the Inpatient Survey questions can broadly be grouped into categories according professional groups’ primary responsibility: doctors, nurses, health professionals in general or hospital managers.

Our aim was to analyse England’s Inpatient Survey data using explicit criteria to measure the statistical and practical significance of changes. The 12-year time frame of this study is longer than that of any other study. We include all of the questions that were used in at least seven annual surveys between 2002 and 2013 to assess how patients’ experiences have improved or deteriorated between the earliest and most recent year. We examine whether the magnitude or direction of change differs according to the occupational groups responsible for various aspects of care.

Methods

Data sources

The data for the 11 national Inpatient Surveys conducted annually between 2002 and 2013 inclusive were obtained from the UK Data Service.²¹ England’s NHS hospital care is organized into NHS Trusts, which constitute one or more hospitals. Annually, all NHS trusts in England are mandated to conduct a mailed survey of 850 consecutive discharged inpatients. The number of participating trusts has declined from 176 in 2002 to 156 in 2013 due to mergers. Since 2002, 840,077 patients have responded to the questions used in this analysis. In 2002, nearly 95,000 patients from 176 trusts returned usable questionnaires: a response rate of 63%. The response rate has declined steadily with 62,433 responders in 2013, representing a response rate of 49%.

Question selection

Questions were considered if they had been included in the national Inpatient Survey since 2006 or earlier, had been included for at least seven consecutive years between 2002 and 2013, and if they were asked in exactly the same way and offered the same response options. There were 50 such questions, 44 of which were included in the 2013 survey, while six were included most recently in 2011. Twenty-two of the 50 selected questions had been included since 2002, 11 since 2004, 13 since 2005 and 4 since 2006.

Summarising question responses

To summarize question responses, a national mean unweighted percentage of patients who gave the most positive response to each of the 50 questions at each survey interval was computed. Neutral responses such as ‘Don’t know’ were excluded from the denominator. This method of summarising responses differs from the CQC’s method, which computes a mean score by scoring response options (Yes, completely; Yes, to some extent; No) for each question at equal intervals between 0 and 10, based on an assumption of equal differences between options. Unlike the CQC’s approach, our method of summarising question response does not distinguish between less positive response options but, arguably, it is more transparent, more practical and does not rely on untested assumptions. The percentage of patients who gave a particular response is more meaningful than a score between 0 and 10. We examine changes in responses to 50 questions, so it is more practical to compare one figure for each question, rather than examining changes in response proportions for up to six different response options. Furthermore, the questions in the Inpatient Questionnaire are specifically designed to elicit reports of ‘what happened’,²² rather than to generate scores on a scale.

Statistical analysis

Where possible, comparisons were made between question responses to the first survey (2002) and the most recent survey year for which data were available (2013). If a question was not included in either of those years, the earliest and most recent year’s results for each question were used. Chi-square analyses tested the statistical significance of changes in question responses. However, the Inpatient Surveys’ large sample sizes mean that changes as small as 1% are statistically significant. Arguably, such small changes should not be judged to be of practical significance. Therefore, in addition, the effect sizes of inter-year differences are measured with phi-coefficients²³ which, unlike statistical significance, are not confounded by sample size. Repeated chi-square tests were carried out on the same data, so Bonferroni’s correction was applied, reducing the significance level from p < .05 to p < .001.

Results

Trends over time

Figures 1 –4 show the percentages of positive responses to questions over time. Each line represents one question and shows changes in the percentage of patients who gave the most positive response over the years the survey was conducted. The relative position of the lines on the y-axis is a function of the question wording and scoring; we are not concerned with making comparisons among different lines. What is of interest is whether each line has risen (or fallen) over the period that a question has been asked. The 50 questions are divided into four categories, which broadly correspond to the care given by four different occupational groups, although some questions fall more neatly into one occupational category than another.

Figure 1.

Positive responses to questions about hospital doctors.

Figure 2.

Positive responses to questions about care from ward nurses.

Figure 3.

Positive responses to questions about care given by unspecified health professionals.

Figure 4.

Positive responses to questions about hospital organisation and management.

Figure 1 illustrates that most of the 11 questions about care given by hospital doctors remained largely unchanged between 2002 and 2013. An exception is Doctors always cleaned hands, which improved by 12% between 2005 and 2011.

Figure 2 shows the nine questions about care given directly by ward nurses. Between 2005 and 2011, there was a 10% improvement in Nurses always cleaned hands. Between 2002 and 2013, there was a 6% improvement in Nurses always gave understandable answers to questions, and a 6% improvement in Always got help to eat meals. In contrast, there was a 10% decline in Call bells were usually answered within two minutes.

Figure 3 shows that the response to most questions about direct care received from unspecified health care staff have remained fairly stable over time, but there are 7% improvements in responses to three questions: Always given privacy when discussing condition, Family given information to care for patient at home and Given written information about medicines.

Figure 4 shows steady improvements in several of the aspects of care that are susceptible to the influence of managers. There is a 30% improvement in Received copies of doctors’ letters; a 17% improvement in Did not share a bathroom with opposite sex patients and 10% improvement in Did not share sleeping area with opposite sex patients. Two questions about cleanliness have also improved: Ward ‘very clean’ by 13% and Toilets and bathrooms ‘very clean’ by 12%. There was also a 9% increase in the proportion of patients who thought their Planned admission was ‘as soon as necessary’. The change in Waited 4 hours or less to be admitted from Emergency Department is nonlinear: it improved by 8% (from 66% to 74%) between 2002 and 2005, but subsequently declined to 70% in 2011.

Chi-square tests comparing the earliest and the most recent year’s results for each question indicate statistically significant improvements in responses to 35 questions, deterioration in responses to eight questions, and no change for seven questions (p < .001) (details appear in the Appendix, available online). Only one of the effect sizes for inter-year comparisons reached the conventional 0.3 for a ‘moderate’ effect (Received copies of doctors’ letters). There were six ‘small’ (ϕ > 0.1) improvements: for two questions about hand cleaning, two questions about same sex ward areas and two questions about ward cleanliness. There is one ‘small’ decline for Call bells were usually answered within two minutes, but no meaningful difference over time for 42 questions.

Discussion

Main findings

Many aspects of inpatients’ experiences have not changed substantially, but there have been some noteworthy improvements and declines. The most improved areas are those that are mainly the responsibility of managers or have been the focus of national policies, targets or campaigns. They are also in areas which are relatively easy to define, measure, record and count: copying letters to patients; ward area cleanliness, single sex ward areas, clinicians’ hand washing, inpatient waiting times and emergency department waiting times. The main areas of stasis and decline concern clinician–patient interactions. This may reflect difficulty of engaging clinicians in quality improvement or a lack of concerted efforts to involve them, or it could be due to the relative complexity of the interactive aspects of patients’ experiences, which make them less easy to measure and incentivize.

Comparison of nurses and doctors

Responses to questions about nursing care have changed more than those about care given by doctors. The greatest deterioration is in the time taken for nurses to answer call bells, but there were improvements in nurses giving understandable answers to patients’ questions and patients getting help to eat meals. Experiences of care from doctors have remained stable over time, except for an improvement in cleaning their hands, which also improved for nurses.

Success of hand hygiene campaigns

The improvements in patients’ perceptions of hand hygiene suggest that clinicians have engaged with quality improvement programmes. The National Patient Safety Agency’s ‘Clean Your Hands’ campaign sought to raise awareness about the importance of hand hygiene in reducing the incidence of hospital-acquired infections and the message has been enthusiastically embraced by the national media. It is likely that this progress is partly due to the relative ease with which the required action (hand cleaning) can be defined and measured. It may also reflect clinicians’ interest in practical clinical issues rather than abstract concepts.²⁴

Effects of targets

The results broadly support the use of targets, in that patients report better care in areas where the most high-profile financially incentivized targets were in place. The results for two questions clearly reflect the influence of two financial incentives: an 8% increase between 2002 and 2009 in patients who thought their planned admission was ‘as soon as necessary’; and 8% fewer patients reporting a wait of more than 4 h in emergency departments in 2005 compared to 2002. In contrast, the decline in time taken to answer call bells could reflect the absence of targets for this aspect of care, and could indicate that nurses’ attention was focused on ensuring that other targets were met at the expense of responding to the immediate needs of their patients.

Relationship of patients’ priorities to improvements in care

Some of the greatest improvements have been in ward cleanliness and hand washing, issues which previous research suggests are of high priority to patients.^25,26 On the other hand, the large improvement in patients receiving copies of their letters does reflect patients’ priorities. Other issues of relatively high priority, such as pain relief, information about medicines and being able to talk to staff about their concerns, are unchanged or have declined.

Strengths and limitations of this study

This study is the first to consider the annual Inpatient Survey data by matching successive years’ data for each question. In so doing, it offers the longest and broadest analysis of these data to date. While the survey method has remained constant over time, the decline in response rates could account for some change, but this seems unlikely since some aspects of care have declined while others have improved, and largest improvements are associated with national campaigns.

This analysis offers insights on national trends and helps us understand which areas of care are most susceptible to quality improvement efforts. However, these nonexperimental data cannot support inferences of cause and effect.

Future research

Few studies have attempted to measure experimentally the impact of improvement strategies on patients’ experiences. One such study provided preliminary evidence that ward discussions with nurses about their recent patient survey results improved nursing care.²⁷ This is a rare example of a small randomized trial to test a strategy to improve patients’ experiences in the interpersonal aspects of care which have so far been impervious to improvement efforts. Among the challenges of conducting such research is that interventions in hospitals are not always specific to individual patients or even to hospital wards. Therefore, in randomized trials, the risk of contamination of the control group by the experimental group is relatively high. A further difficulty, highlighted by this study, is that the wider NHS context of national policies may have a strong impact on the quality of care, and this could mask or exaggerate the impact of local quality improvement efforts in single-site studies. This underlines the importance of conducting randomized trials, especially to test ways of improving the interpersonal aspects of care, which are falling behind.

Implications

Since 2002, many national efforts to improve care in specific target areas (such as waiting times and ward cleanliness) have been successful. Where there has been progress, it has been incremental and year-to-year changes have been small. There is a need to continue consistent repeated surveys to detect changes over the long term.

Footnotes

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

References

The Evidence Centre. Measuring patient experience, London: The Health Foundation, 2013.

Department of Health. The NHS plan: a plan for investment a plan for reform, London: The Stationery Office, 2000.

The Co-ordination Centre for the NHS Patient Survey Programme. Guidance manual for the NHS adult inpatient survey, Oxford: Picker Institute Europe, 2014. http://www.nhssurveys.org/Filestore//Inpatient_2013/IP13_Guidance_v12.pdf (2013, accessed 5th March 2014).

Reeves

Seccombe

. Do patient surveys work? The influence of a national survey programme on local quality improvement initiatives. Qual Saf Health Care 2008; 17: 437–441.

DeCourcey

West

Barron

. The National Adult Inpatient Survey conducted in the English National Health Service from 2002 to 2009: how have the data been used and what do we know as a result? BMC Health Serv Res 2012; 12: 71–71.

Allan

Schattner

Stocks

. Does patient satisfaction of general practice change over a decade? BMC Family Pract 2009; 10: 13–13.

Cho

Kim

. Trends in patient satisfaction from 1989–2003: adjusted for patient characteristics. J Korean Acad Nurs 2007; 37: 171–178.

Leatherman

Sutherland

. The quest for quality: refining the NHS reforms, London: Nuffield Trust, 2008.

Thorlby

Maybin

. A high performing NHS?, London: Kings Fund, 2010.

10.

Richards

Coulter

. Is the NHS becoming more patient-centred?: Trends from the National Surveys of NHS Patients in England 2002–07, Oxford: Picker Institute Europe, 2007.

11.

Delamothe

. We need to talk about nursing. BMJ 2011; 342: 416–416.

12.

Francis

. The mid Staffordshire NHS foundation trust public inquiry, London: The Stationery Office, 2013.

13.

Bevan

. Have targets done more harm than good in the English NHS? No. BMJ 2009; 338: 442–443.

14.

Dimakou

Parkin

Devlin

. Identifying the impact of government targets on waiting times in the NHS. Health Care Manag Sci 2009; 12: 1–10.

15.

Appleby

Devlin

. Measuring success in the NHS, London: Dr Foster, Kings Fund and City University, 2004.

16.

Alberti

. Transforming emergency care in England, London: Department of Health, 2004.

17.

Gubb

. Have targets done more harm than good in the English NHS? Yes. BMJ 2009; 338: 442–443.

18.

Davies

Meterko

Charns

. Factors affecting the use of patient survey data for quality improvement in the Veterans Health Administration. BMC Health Serv Res 2011; 11: 428–432.

19.

Ham

. Improving the performance of health services: the role of clinical leadership. The Lancet 2003; 361: 1978–1980.

20.

Darzi

. High quality care for all, London: Department of Health, 2008.

21.

Care Quality Commission. Acute trusts: adult inpatients surveys 2002–2013, http://discover.ukdataservice.ac.uk/ (2013, accessed 8 December 2014).

22.

Bruster

Jarman

Bosanquet

. National survey of hospital patients. BMJ 1994; 309: 1542–1549.

23.

Cohen

. Statistical power analysis for the behavioral sciences, New York: Lawrence Erlbaum Associates, 1988.

24.

Siriwardena

. Engaging clinicians in quality improvement initiatives: art or science? Quality Prim Care 2009; 17: 303–305.

25.

Boyd

. The 2006 inpatients importance study, Oxford: Picker Institute Europe, Oxford. http://www.nhssurveys.org/Filestore/documents/Findings_and_development_of_the_2006_Inpatients_Importance_study_final.pdf (2007, accessed 5 March 2014).

26.

Reeves

. Building on the experiences of NHS Boards, Edinburgh: Scottish Government Social Research, Edinburgh. http://www.scotland.gov.uk/Resource/Doc/278973/0083963.pdf (2008, accessed 23 April 2014).

27.

Reeves

West

Barron

. Facilitated patient experience feedback can improve nursing care: a pilot study for a phase III cluster randomised controlled trial. BMC Health Serv Res 2013; 13: 259–259.