Impact of ‘high-profile’ public reporting on utilization and quality of maternity care in England: a difference-in-difference analysis

Abstract

Objectives

To evaluate the impact of ‘high-profile’ (with extensive media coverage) public reporting versus public reporting without ‘high profile’ coverage on utilization and perceived quality of maternity services in England’s National Health Service.

Methods

Analysis of national hospital administrative data using difference-in-difference models with propensity score matching, and of two maternity surveys from 2007 and 2010. Outcomes were counts of women admitted for delivery of a baby and the percentage of women rating their care positively in 2007 and 2010.

Results

Hospitals highly publicized as providing the best maternity care in England had fewer admissions annually and lower occupancy rates than the national comparison group (63.0% vs. 77.3%; p = 0.09). There was no statistically significant change in overall maternity admissions in the best hospitals (+2.2%, p = 0.40 at six months), or the worst hospitals (−2.8%, p = 0.49 at six months) during any period in the 36 months after public reporting relative to their matched comparison groups. Survey analyses found that compared to the national comparison group of hospitals without ‘high profile’ media coverage, the worst rated hospitals experienced greater improvements in perceived quality after public reporting but these findings were not maintained in the analysis of matched hospitals.

Conclusions

‘High-profile’ public reporting of maternity care in England was not associated with changes in the use of maternity services or improvements in patient-reported quality. These findings provide further evidence that public reporting is unlikely to drive major improvements in health system performance through the mechanism of patient choice.

Keywords

health policy patient choice public reporting

Introduction

Many health systems have instituted public reporting of performance data to improve quality, safety, responsiveness and accountability.¹ While public reporting has been postulated to improve health system performance via several mechanisms, most policy focus has centred on the selection pathway. This involves the market-like mechanism of ‘choice’, whereby patients (or advocates for them such as their family doctor) use publicly reported performance information to make informed decisions on where they receive care, and the associated ‘competition’ between health care providers to attract patients, in order to increase their market share.²

Choice of provider has recently become enshrined in health policy in England through the National Health Service (NHS) Constitution which states that providing information to support choice is a major priority for the NHS.^3,4 This commitment is supported by a series of reforms undertaken over the past decade including the expansion of public reporting, Payment by Results (a form of activity based funding for hospitals introduced in 2003/04 where the ‘money follows the patient’⁵) and Choose and Book (an electronic booking system introduced in 2005, which permits choice of any provider in the country at the point of referral^6,7).

Existing evidence provides little support for the selection pathway as a mechanism for health care system improvement. For example, a systematic review published in 2008 found that public reporting of performance data did not influence patient choice of hospitals, with physician recommendation and geographical proximity being more important factors.¹ It has been suggested that this absence of effect is due to lack of time or motivation among patients to seek out this information and difficulties in understanding performance data when they do.⁸ Although some research identifies a role for the publication of performance information in health care quality improvement, the mechanisms behind this are still debated.¹ Most of this evidence comes from examination of a few select schemes in the United States, mainly focused on cardiac surgery and there is a lack of research in other countries and clinical areas.⁹

We test the hypothesis that ‘high-profile’ public reporting of the quality of maternity care in England and associated media coverage will influence women’s choice of health care provider. We use the example of widespread media reporting of an English Healthcare Commission report in early 2008, which presented simple data showing the 10 best and 10 worst maternity care providers in the country. Although data were released on the quality of maternity care for all hospitals, hospitals in the comparison group were not subject to such media reporting. Our hypothesis is based on assumptions that pregnant women may be more proactive in seeking (and more responsive when receiving) publicly reported information and media coverage, and have greater scope to use this to inform their choice of provider.

Methods

Context

In July 2008, the English health care quality regulator published Towards Better Births, a major report which provided scored assessments of each NHS hospital trust in England providing maternity care.¹⁰ These assessments were based on: (1) a survey in June 2007 of 26,000 women who had recently given birth; (2) a web-based maternity questionnaire filled out by service managers on issues such as staffing arrangements and (3) a voluntary web-based survey of trust staff (completed by staff at half of hospitals and comprising 4950 responses).

The information from the patient survey was released on a website called Birth Choice UK in November 2007,¹¹ where patients could see how well individual hospitals performed. Hospitals were not ranked in the Healthcare Commission report, or on the Birth Choice UK website. In January 2008, the patient survey information attracted considerable media interest in many national outlets (such as the BBC, The Times, The Daily Telegraph), which focused on the 10 best and worst performers by name (for example, the BBC website included headlines such as ‘NHS maternity units falling short’¹²). There was substantial additional coverage in local newspapers in areas where maternity services were reported as doing particularly well or badly.

The best and worst groups are listed here (Box 1).

Box 1.

Best and worst hospitals.

Best hospitals

• Stockport NHS Foundation Trust

• Gateshead Health NHS Trust

• North Cumbria Acute Hospitals NHS Trust

• Northern Devon Healthcare NHS Trust

• Northumbria Healthcare NHS Foundation Trust

• County Durham and Darlington NHS Foundation Trust

• Royal Cornwall Hospitals NHS Trust

• Salford Royal NHS Foundation Trust

• Hull and East Yorkshire Hospitals NHS Trust

• Wirral University Teaching Hospital NHS Trust

Worst hospitals

• West Hertfordshire Hospitals NHS Trust

• Newham University Hospital NHS Trust

• Barking, Havering and Redbridge Hospitals NHS Trust

• Chelsea and Westminster Hospital NHS Foundation Trust

• Homerton University Hospital NHS Foundation Trust

• Mayday Healthcare NHS Trust

• Bromley hospitals NHS Trust

• Luton and Dunstable NHS Foundation Trust

• Bart’s and the London NHS Trust

• St. George’s Healthcare NHS Trust

Sample

There were 146 NHS hospital trusts providing maternity services in England for the duration of the study. We excluded 35 trusts with fewer than 100 maternity admissions or fewer than 100 Caesarean sections in any six-month period during the study. We excluded one trust (Bromley NHS hospitals trust) in the worst-performing group as it merged to become South London Healthcare NHS Trust during the study period, as well as two specialist women’s trusts. We grouped the sample into the 10 best and nine worst performing hospitals and used the remaining 89 trusts as a national comparison group.

Data

We used maternity data from Hospital Episode Statistics (HES), the national administrative database for hospital activity in England, for the financial year 2006/2007 to 2010/2011. HES maternity data captured 96% of all hospital births in England in 2008.¹³ Data on the number of maternity beds in each hospital was taken from the Health and Social Care Information Centre. Data on the demographic characteristics of the local areas were taken from 2006/2007 population projections from the Office for National Statistics.¹⁶ Information on the geographical position of NHS hospitals came from the NHS Organisation Data Service.

All analyses were at NHS trust level, which is a single or small group of hospitals in a defined geographical area operated by the same management team. We refer to trusts as hospitals hereafter. Geographical distance data were calculated from the population-weighted centroid of each patient’s Lower Super Output Area (LSOA) and the hospital where the birth occurred.

Data for the three measures of quality were from the NHS Maternity Survey, a postal survey conducted in 2007^{1
5} (of 26,042 women) and 2010^{1
6} (of 25,488 women).

Outcome measures

The main outcome measure was counts of women admitted for delivery of a baby by six-month period for each NHS hospital. Secondary outcome measures were: (1) percentage of women giving birth by caesarean section; (2) percentage of deliveries to mothers over the age of 35 years; (3) percentage of mothers from the 20% most deprived areas in England, based on Index of Multiple Deprivation (IMD) income scores and (4) percentage of mothers from the 20% least deprived areas in England, based on IMD income scores. These groups were examined separately in order to assess if different groups were more sensitive to public reporting due to demographic characteristics or clinical need (for C-Sections). For example, women in less deprived areas may be more likely to attend a non-local hospital which may lead to changes in their pattern of utilization when given information on quality of care. We also examined the percentage of women admitted to their nearest hospital (shortest geographical distance) to ascertain if public reporting changed the distance women were willing to travel to attend particular hospitals.

Outcomes from the patient surveys were the percentage of women reporting that the care they had received during: pregnancy, labour and birth, and after the birth of their child, was excellent, very good or good.

Analysis

We used a difference-in-difference study design, which is a quasi-experimental method commonly used for policy evaluation.¹⁷ We compared the numbers of maternity admissions in hospitals reported as the best or worst in the country with accompanying high profile media coverage relative to a comparison group which did not receive this media coverage to evaluate whether public reporting was associated with changes in admissions. Separate analyses were used for the best and worst groups, to compare each to the relevant comparison groups. We used panel data regression with fixed effects and used the log of the difference between our ‘treated’ and ‘untreated’ groups, so that results can be interpreted as relative changes from baseline. This was set as the period from September 2007 to March 2008, which incorporates both when the information was put on the Birth Choice UK website and the extensive media reporting. The rationale for using this baseline is that women close to their delivery date would have already chosen a provider and that any effects would be more likely to be found among women at early stages of pregnancy.

We first compared admissions in our best and worst group to the national comparison group. NHS hospitals within 10 km of the best or worst hospitals were excluded from the analysis, as their patient numbers may have been affected if patients moved from the best or worst hospitals, which would amplify any possible effects. We then constructed two additional comparison groups using propensity score matching, where the propensity score represented the probability of being in the worst or best group, using data from 2006/2007. We examined a large number of demand and supply variables potentially relevant to the probability of being in the worst (or best) hospital group including: number of maternity beds; number of GPs in the catchment area; percentage of catchment area classified as urban; and number of other NHS hospitals within 10 or 20 km. Standard balancing tests for matching suggest that the number of admissions and socioeconomic deprivation in the hospital catchment area were the most appropriate variables for matching. The matched group comprised the three most similar hospitals to each hospital in the best or worst group in terms of their propensity score.

Survey analysis

Changes in quality scores derived from the NHS Maternity Survey between 2007 and 2010 in the best and worst hospitals and two comparison groups were calculated using z-tests. The two comparison groups were the national comparison group and the 10 hospitals with the most similar quality scores to the mean score achieved by the best or worst hospitals in 2007. This second comparison group addressed the possibility of the best group being subject to a ceiling effect due to their high scores in 2007.

Sensitivity analyses

We conducted a number of sensitivity analyses to test the robustness of our findings: excluding hospitals within 20 km of the best or worst hospitals (rather than 10 km – see online Tables A4 and A5); using a different baseline period (online Tables A6 and A7); and using five comparison hospitals in the matched analysis rather than three (online Tables A8 and A9). The appendix also contains details of the average distances travelled for admission over the time period (online Table A10), as well as the percentage of women admitted to their closest hospital (online Table A11).

Results

In 2006/2007, the best hospital group had fewer maternity admissions (3283 vs. 4574, p = 0.02) and the worst hospitals had more admissions than the national comparison group (5309 vs. 4574, p = 0.21), although the latter difference was not statistically significant (Table 1). The best hospital group also had a lower occupancy rate of maternity beds than the national comparison group (63.0% vs. 77.3%, p = 0.09). Both the best and worst hospital groups had a smaller percentage of deliveries from women living in affluent areas than the national comparison group (10.5%, 9.4% and 17.0%, respectively, p = 0.09 and p = 0.06). Nearly half of admissions to the worst hospital group were in women living in the most deprived quintile in England (45.7% vs. 26.0% nationally, p < 0.01). The worst hospital group had more NHS hospitals within a 10 km radius than the best group (20 vs. 12), and a larger population aged 15–34 years in their catchment area which reflects the fact that most were based in London. Findings from propensity score matching are presented in online Table A1 and the t tests of differences before and after matching indicate that comparison groups generated by our matching are more similar to the best and worst hospitals than the national comparison group.

Table 1.

Summary statistics for best, worst rated and national comparison group hospitals in 2006/2007.

	Best	Worst	National comparison group
Hospitals	10	9	89
Annual number of maternity admissions (SD)	3,283 (1,328)**	5,309 (1,654)	4,574 (1,685)
Number of maternity beds (SD)	53.6 (25.2)	70.8 (21.8)	64.5 (27.4)
Occupancy of maternity beds% (SD)	63.0 (11.9)*	79.2 (25.0)	77.3 (26.6)
Number of Whole Time Equivalent GPs	5.8 (1.7)	4.9 (0.9)	5.4 (1.1)
Catchment area population aged 15–34	253,707	1,501,304**	477,915
% catchment area which is urban	83.0 (17.0)	96.9 (5.3)**	88.0 (10.5)
Number of NHS hospitals within 5 km	9	12	–
Number of NHS hospitals within 10 km	12	20	–
Mean income score for area from IMD 2004	0.176 (0.033)**	0.167 (0.044)*	0.136 (0.047)
% of deliveries by C-section	9.3 (0.9)	7.6 (2.8)**	9.9 (1.9)
% of deliveries to women>35 years old	17.5 (2.5)*	21.4 (6.2)	20.3 (5.0)
% of deliveries from deprived mothers^a	29.4 (13.2)	45.7 (33.5)**	26.0 (18.3)
% of deliveries from affluent mothers^a	10.5 (6.6)*	9.4 (9.5)*	17.0 (11.8)
% of deliveries for which hospital was closest	84.4 (17.5)	76.0 (14.5)	79.0 (19.3)

Defined as mothers living in the 20% most and least deprived areas nationally as defined by the Index of Multiple Deprivation.

Significantly different to national comparison group at p ≤ 0.10.

Significantly different to national comparison group at p ≤ 0.05.

IMD: index of multiple deprivation; NHS: National Health Service.

Impact of public reporting on maternity admissions in best and worst hospitals

Trends in overall admission numbers for the best and worst group are shown in Figure 1.

Figure 1.

Overall maternity admission numbers in “best” group and “worst” group, April 2006–March 2011.

In the best hospitals, there was no significant percentage change in the number of maternity admissions (+2.2% at six months post baseline; p = 0.40) after public reporting (Table 2). There were no significant changes in maternity admissions within subgroups, including women having a C-section, those living in deprived areas, or for whom this was the closest hospital after baseline. There were small but significant decreases (−1.1%; p = 0.01) in maternity admissions in women aged over 35 years in the best hospitals 19–30 months post baseline. There were no statistically significant changes in the percentage of women attending the best hospitals for which this was their closest hospital.

Table 2.

Changes in maternity admissions, relative to baseline in the best hospitals (matched analysis).

	Pre baseline				Post baseline
	13–18 months	7–12 months	0–6 months	Baseline	0–6 months	7–12 months	13–18 months	19–24 months	25–30 months	31–36 months
Admissions	3.6 (0.16)	2.0 (0.45)	−0.7 (0.79)	Baseline	2.2 (0.40)	2.3 (0.37)	2.9 (0.27)	−0.6 (0.82)	2.1 (0.41)	2.3 (0.38)
% births by C section	0.6 (0.16)	−0.4 (0.42)	0.8 (0.08)	Baseline	−0.4 (0.39)	0.1 (0.89)	−0.1 (0.80)	−0.5 (0.26)	−0.5 (0.28)	0.3 (0.46)
% births by mothers >35	−0.7 (0.10)	−0.3 (0.54)	0.0 (0.94)	Baseline	−0.1 (0.84)	−0.1 (0.79)	−0.5 (0.24)	−1.1 (0.01)	−1.1 (0.01)	−0.9 (0.04)
% deprived mothers	−0.7 (0.18)	−0.7 (0.18)	−1.5 (<0.01)	Baseline	−0.8 (0.11)	−0.5 (0.31)	0.0 (0.93)	−0.6 (0.27)	−0.6 (0.26)	−0.4 (0.48)
% affluent mothers	−0.2 (0.53)	−0.3 (0.40)	0.4 (0.22)	Baseline	−0.3 (0.49)	−0.5 (0.20)	0.0 (0.97)	0.0 (0.99)	−0.8 (0.03)	−0.5 (0.18)
% attending closest hospital	0.2 (0.62)	0.5 (0.25)	−0.5 (0.24)	Baseline	−0.3 (0.51)	−0.3 (0.49)	−0.3 (0.41)	0.1 (0.86)	0.5 (0.25)	−0.2 (0.64)

Note: Statistically significant findings at p≤0.05 in bold.

In the worst hospitals, there was no significant percentage change in the number of maternity admissions (−6.8% at 7–12 months post reporting; p = 0.09) after public reporting (Table 3). The worst performing hospitals did experience a significant increase in the percentage of maternity admissions in women over the age of 35 years (+1.6%, p < 0.01, 19–24 months post baseline) and the percentage of births by C-section which persisted until the end of the study period (+1.2%, p = 0.05 for 7–12 months post baseline). There was a decrease in the percentage of women for whom one of the worst hospitals was their closest hospital from 13 months post baseline until the end of the study period (e.g. −4.2%, p < 0.01 at 13–18 months).

Table 3.

Changes (%) in maternity admissions, relative to baseline in the worst hospitals (matched analysis).

	Pre baseline				Post baseline
	13–18 months	7–12 months	0–6 months	Baseline	0–6 months	7–12 months	13–18 months	19–24 months	25–30 months	31–36 months
Admissions	−0.6 (0.88)	4.0 (0.33)	1.9 (0.63)	Reference	−2.8 (0.49)	−6.8 (0.09)	−3.9 (0.34)	−1.9 (0.64)	−2.9 (0.48)	−1.5 (0.70)
% births by C section	−0.5 (0.39)	−0.7 (0.24)	−0.1 (0.86)	Reference	0.9 (0.16)	1.2 (0.05)	1.7 (0.01)	1.2 (0.05)	1.5 (0.02)	1.9 (<0.01)
% births by mothers >35	−0.2 (0.75)	0.5 (0.31)	0.3 (0.62)	Reference	0.8 (0.11)	0.9 (0.08)	1.3 (0.02)	1.6 (<0.01)	1.5 (0.01)	1.8 (<0.01)
% deprived mothers	−0.1 (0.79)	−0.3 (0.54)	0.2 (0.74)	Reference	−0.1 (0.81)	−0.5 (0.35)	−1.1 (0.05)	−0.4 (0.45)	−0.3 (0.64)	−0.6 (0.25)
% affluent mothers	−0.1 (0.82)	0.0 (0.97)	−0.2 (0.66)	Reference	0.0 (0.95)	0.2 (0.55)	0.4 (0.32)	0.1 (0.83)	0.5 (0.23)	0.6 (0.12)
% attending closest hospital	0.5 (0.44)	0.5 (0.44)	0.0 (0.99)	Reference	−0.3 (0.63)	−0.5 (0.40)	−4.2 (<0.01)	−4.4 (<0.01)	−5.4 (<0.01)	4.8 (<0.01)

Note: Statistically significant findings at p≤0.05 in bold.

Impact of public reporting on perceived quality of maternity care in best and worst hospitals

In a patient survey in 2007, the best hospital group was performing significantly better and the worst group less well than the national comparison group (Table 4). Between 2007 and 2010 the best group improved more slowly than nationally on all three outcomes (e.g. +5.8% for overall care after birth compared to +8.4% nationally, p < 0.01). However, when matched to hospitals with similar levels of satisfaction in 2007, the best group improved faster on one measure (care during labour, +4.4% vs. +1.0%, p < 0.01), less well on one (care after birth) and similarly on one (care during pregnancy). The worst group experienced greater improvement than the national comparison group between 2007 and 2010 on all three patient-reported outcomes. In the analysis matched to hospitals with similar levels of satisfaction in 2007, the worst hospitals performed less well on one outcome (care during pregnancy, +6.5% vs. +9.5%, p < 0.01), similarly on one outcome (care during labour) and better on one outcome (care after birth).

Table 4.

Percentage of women rating their care positively during pregnancy, labour and after birth before and after public reporting.

		Nationally	Best group (p value difference from national group)	Controls matched on 2007 score (p value difference from best group)	Worst group (p value difference from national group)	Controls matched on 2007 score (p value difference from worst group)
During pregnancy	2007	87.9	91.9	91.7	81.9	81.8
	2010	92.4	95.0	94.0	88.4	91.3
	Change in %	+4.5	+3.1 (<0.01)	+2.3 (0.06)	+6.5 (<0.01)	+9.5 (<0.01)
During labour	2007	89.0	92.9	92.5	81.7	82.3
	2010	93.4	95.7	93.5	89.4	90.0
	Change in %	+4.4	+2.8 (<0.01)	+1.0 (<0.01)	+7.7 (<0.01)	+7.7 (0.96)
After birth	2007	80	85.2	76.7	65.4	72.8
	2010	88.4	91.0	87.2	80.8	86.4
	Change in %	+8.4	+5.8 (<0.01)	+10.5 (<0.01)	+15.4 (<0.01)	+13.6 (0.04)

Note: Rating care positively defined as percentage of women stating that their care was ‘good’, ‘very good’, or ‘excellent.’

Sensitivity analyses

Results from our unmatched analysis (online Tables A2 and A3) and our sensitivity analyses were substantially similar to the matched analysis presented above (online Tables A4 to A9). Online Table A10 shows that the average distance travelled to attend the best hospitals was much further than for the worst hospitals (14.4 km vs. 4.7 km at baseline) and that this did not change over the study period. Online Table A11 shows the percentage of women who attended their closest hospital during the time period and suggests a small decline in the percentage of women at the best hospitals for whom this was their closest hospital (84.0% vs. 84.3% at baseline).

Discussion

Our findings indicate that public reporting of the quality of maternity services in England in 2008, which received widespread coverage in the media, had no significant impact on utilization or perceived quality of services. We found that the number of maternity admissions did not significantly decrease in the nine hospitals widely reported as providing the worst maternity care and did not increase in the 10 hospitals widely reported as providing the best maternity care. The percentage of women rating their hospital care positively improved more in the worst rated hospitals compared to a national comparison group, but this finding did not hold in a comparison with hospitals with similar satisfaction levels at baseline. This suggests that this effect may have been due to regression to the mean and that high-profile public reporting may not have stimulated improvements in quality within these hospitals. It should be noted, however, that there were national improvements in many outcomes over this period, which have been attributed primarily to increased financial resources and strong performance management.¹⁸

The effects of the release of these specific reports on patient utilization and quality of care have not been studied in detail previously. Anecdotal evidence recently presented to the Nuffield Trust in interviews with service managers suggests that the providers ranked as best performing in these surveys did experience a surge in demand for care, and that this caused problems in managing admissions within capacity constraints.²¹ It is possible that these effects were too modest to be picked up as statistically significant, and that even small changes caused capacity problems. However, our findings, are consistent with previous research which suggests that public reporting is unlikely to drive major improvements in health care system performance through the mechanism of patient selection.^1,20 It is also consistent with work on changes in hospital utilization in response to high profile reporting of negative events in the English NHS, which found that the effects were very modest and not sustained.²¹ Research appearing to show a link between the introduction of market-like competition in the NHS and reductions in hospital acute myocardial infarction mortality has concluded that this may be due to patients becoming more responsive to quality metrics.²² Yet, this research was based on a different clinical area and did not explicitly consider the quality of information available to patients.

Much of the impetus for the public release of performance information comes from a 2003 paper on the two pathways to improvement by Berwick et al.² The first is selection – that information will cause patients to preferentially attend good providers, thus placing pressure on providers to improve; and the second is change in care – that information will cause providers to better direct their efforts (the quality improvement pathway).² A third pathway, the reputation pathway, has also been proposed, whereby providers are encouraged to improve by their desire not to have the reputation of themselves or their workplace tarnished, rather than any specific concerns about patient numbers. Those who identified the reputation pathway set out three key elements for success – that a report is widely disseminated, that it is easily understandable, and that it will be followed by further reports on performance.²⁵ It should be noted that the media coverage associated with the Healthcare Commission report examined here satisfied the first two of these criteria, but not the third. It remains possible that improvements might have been experienced if this reporting had been designed explicitly in line with the recommendations of the reputation pathway.

The current findings provide only limited support for the change in care pathway, whereby hospitals are motivated by the release of data on their quality relative to peers. It also provides limited support for the selection pathway, as there were only weak effects on patient numbers, and the reputation pathway, as hospitals highly publicized as poorly performing did not improve conclusively more than other hospitals. Our findings concur with those from a randomized trial of releasing information to the public on the quality of obstetric care in the United States, which found no associated changes in market share. The trial did, however, find increases in obstetric quality after information release, although this was based on selected quantitative indicators, such as adverse events, as opposed to the patient-reported outcomes used here. Achieving improvements in quality may be constrained by a number of factors. For example, most of the worst-performing hospitals in our sample were based in London, where recruitment and retention of staff are challenging due centralized pay regulation in the NHS and the high cost of living.²⁴

Strengths and limitations

This is the first study to examine changes in utilization after high-profile reporting and correlated this reporting against patient experience measures. Although previous work has examined the impact of high-profile health care scandals using a similar framework,²¹ and some studies have linked market-like competition to changes in patient utilization,²² this study addresses a notable lack of evidence on the routine release of performance information in England. The study has a number of strengths and limitations. Measures of quality were derived from previously validated representative surveys. Also, findings from sensitivity analyses were broadly consistent with the main analyses. Weaknesses include that more detailed data covering patient flows, referrals and patient experience would be required to determine the causal links between information release and these outcomes. During the study period, there were other concurrent changes in the NHS in England such as reconfiguration of some maternity services and changes in guidelines for the management of patients. Nonetheless, given our efforts to match NHS hospitals to different comparison groups, it seems unlikely that such structural changes affected our results. The quality data used in this study come from only three measures of the patient experience of maternity care in England, although these were the three most closely linked to the overall rating of provider.²⁵ Although not a main focus of the study, it would have been preferable to use more detailed data on travel times for the analysis on changes in distance to maternity services, and also to have used the individual hospital site within a trust that patients were admitted to. These data were unfortunately not available.

A recent report commissioned by the Secretary of State for Health in England recommended prioritizing maternity services for the development of quality ratings as these are considered high risk due to the large number of negligence claims.¹⁹ Our finding that high profile public reporting of maternity care quality in England did not have a major impact on utilization or patient-reported quality of care raises a question about the likely impact of this approach. Internationally, the number of countries introducing public reporting schemes is increasing, including Denmark,²⁶ and Canada.²⁷ The impact of these schemes on market share and quality should be carefully evaluated. Proponents of choice in public services have argued that only small shifts in utilization, in the order of 5% to 10%, are needed in order to improve quality.²⁸ However, this assertion has not been empirically tested in health care settings and is not supported by our finding that the worst group experienced a small (6%), but not statistically significant, reduction in utilization without any accompanying improvement in quality. It should be remembered, however, that the control group in this case was subject to some public reporting insofar as its quality data were publicly available.

There are unanswered questions arising from this research. While this study focused on maternity care, there are major changes across the whole of the NHS. The English NHS recently started to publish information from the Family and Friends Test (whereby patients are asked how likely they are to recommend a service) in July 2013²⁹ with the explicit intention that patients can use these scores to make choices. A further area for research is whether certain groups of patients are more or less likely to respond to public reporting. We found some potentially important sub-group effects which could be explored in further research. Future work could also use other measures of quality of maternity care than patient-reported outcomes. The mediating impact of the media, family, friends and medical professionals in conveying quality information also deserves further investigation.

Conclusion

Despite the English NHS policy emphasis on public release of information as a key mechanism for quality improvement by shifting patients away from poor services and towards good services, we identified little evidence to support this in the context of maternity care. Our findings caution against simplistic assumptions that being publicized as a good or poor quality provider in both official and media reports is sufficient to drive improvements through market pressures alone.

Footnotes

Acknowledgements

This article presents independent research commissioned by the National Institute for Health Research (NIHR) under the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for North West London. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. The funders had no role in the study design, interpretation, or the decision to publish, and the interpretation and conclusions contained in this study are those of the authors alone.

References

Fung

Lim

Mattke

. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008; 148: 111–123.

Berwick

James

Coye

. Connections between quality measurement and improvement. Med Care 2003; 41: I30–I38.

Department of Health. Handbook to the NHS constitution 2013. http://www.nhs.uk/choiceinthenhs/rightsandpledges/nhsconstitution/documents/2013/handbook-to-the-nhs-constitution.pdf (2013, accessed 12 January 2015).

Department of Health. The NHS Constitution 2013. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/170656/NHS_Constitution.pdf (2013, accessed 12 January 2015).

Department of Health. Reforming NHS Financial Flows: Introducing payment by results 2002. http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/Consultations/Closedconsultations/DH_4016901 (2002, accessed 12 January 2015).

Department of Health. Choose and book: patients' choice of hospital and booked appointment 2004. http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4088353 (2004, accessed 12 January 2015).

Mannion

Goddard

. Public disclosure of comparative clinical performance data: lessons from the Scottish experience. J Eval Clin Pract 2003; 9: 277–286.

Dixon A, Appleby J, Robertson R, et al. Patient choice: how patients choose and how providers respond. London: The King's Fund, 2010. http://www.kingsfund.org.uk/publications/patient-choice (accessed 23 January 2015).

Shekelle

Public performance reporting on quality information. In: Smith

(ed.). Performance measurement for health system improvement: experiences, challenges and prospects, Cambridge: Cambridge University Press, 2009.

10.

Healthcare Commission. Towards better births 2008. http://image.guardian.co.uk/sys-files/Society/documents/2008/07/10/Towards_better_births.pdf (2008, accessed 12 Januray 2015).

11.

BirthChoiceUK 2007. http://www.birthchoiceuk.com/ (2007, accessed 12 January 2015).

12.

BBC News 2008. NHS maternity units falling short 2008 http://news.bbc.co.uk/1/hi/health/7207215.stm (2008, accessed 12 January 2015).

13.

Brocklehurst

. Perinatal and maternal outcomes by planned place of birth for healthy women with low risk pregnancies: the Birthplace in England national prospective cohort study. BMJ 2011; 343: d7400–d7400.

14.

Office for National Statistics Population Estimates. http://www.ons.gov.uk/ons/taxonomy/index.html?nscl=Population+Estimates+by+Age+and+Sex (accessed 12 January 2015).

15.

The Healthcare Commission. Women's experiences of maternity care in the NHS in England: key findings from a survey of NHS trusts carried out in 2007 http://www.nhssurveys.org/Filestore/CQC/2007_Maternity_services_survey_report.pdf (2007, accessed 12 January 2015).

16.

National NHS patient survey programme. Survey of women’s experiences of maternity services. London: Care Quality Commission, 2010.

17.

Blundell

Costa Dias

. Evaluation methods for non experimental data. Fiscal Studies 2000; 21: 427–468.

18.

Dixon A, Mays N and Jones L. Understanding New Labour's market reforms of the English NHS. London: The King's Fund, 2011. http://www.kingsfund.org.uk/publications/understanding-new-labours-market-reforms-english-nhs (2011, accessed 12 January 2015).

19.

The Nuffield Trust. Rating providers for quality: a policy worth pursuing? http://www.nuffieldtrust.org.uk/publications/rating-providers-quality (2013, accessed 12 January 2015).

20.

Ketelaar

Faber

Flottorp

. Public release of performance data in changing the behaviour of health care consumers, professionals or organisations. Cochrane Database Syst Rev 2011; 47: CD004538–CD004538.

21.

Laverty

Smith

Pape

. High-profile investigations into hospital safety problems in England did not prompt patients to switch providers. Health Aff (Millwood) 2012; 31: 593–601.

22.

Gaynor M, Moreno-Serra R and C P. Death by market power: reform, competition and patient outcomes in the National Health Service 2010. http://www.bristol.ac.uk/cmpo/publications/papers/2010/wp242.pdf (2010, accessed 12 January 2015).

23.

Hibbard

Stockard

Tusler

. Does publicizing hospital performance stimulate quality improvement efforts. Health Aff (Millwood) 2003; 22: 84–94.

24.

Hall E, Propper C and Reenen Jv. Can pay regulation kill? Panel data evidence on the effect of labor markets on hospital performance 2008. Centre for Economic Performance discussion paper no 843. London: London School of Economics and Political Science http://www.nber.org/papers/w13776 (2008, accessed 12 January 2015).

25.

Raleigh

Hussey

Seccombe

. Ethnic and social inequalities in women's experience of maternity care in England: results of a national survey. J R Soc Med 2010; 103: 188–198.

26.

http://www.esundhed.dk/Sider/Forside.aspx Danish patient information pages (accessed 12 January 2015).

27.

http://www.cbc.ca/news2/health/features/ratemyhospital/ CBC News Health. Rate my hospital (accessed 12 January 2015).

28.

LeGrand J. The other invisible hand. Princeton: Princeton University Press, 2007.

29.

Department of Health. The NHS friends and family test. Publication guidance 2013. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/141446/Friends-and-Family-Test-Publication-Guidance-v2-FOR-PUBLIC_E2_80_A6.pdf.pdf (2013, accessed 12 January 2015).