Measuring and enhancing elective service performance in NHS operating theatres: an overview

How should theatre performance be measured?

Any measure of theatre performance should fulfil two essential criteria. First, it must be appropriate for the intended use; for example, performance indicators required by theatre managers to facilitate operational decision-making may differ from those required by strategic or policy decision-makers. Second, it must be adaptable and/or adjustable so as to be useful across the broad context of the entire surgical service. Healthcare provision within the NHS incorporates several imposed components, such as central government targets, an obligation to train junior medical personnel and the prioritization of emergency care over elective care, all of which can conflict with service efficiency. It should therefore be recognized that what is considered an appropriate performance benchmark for a particular operating theatre setting may differ significantly from achievable targets in others, either within the same trust (day surgery versus main theatres), between trusts (tertiary referral trusts versus district general hospitals) or indeed comparing public with independent sector settings.

To date, the most widely used measure of theatre performance has been operating theatre utilization rates. Reference to the concept of theatre utilization can be found in the medical literature as early as the late 1970s, 8, 9, 10 where its origins can be traced back to foreign, often privatized, healthcare systems. Accordingly, it has become the principal means of assessing theatre performance in the NHS. Key audits carried out by the Audit Commission 11 and the Healthcare Commission 12 have served to strengthen its status as the principle performance indicator.

There are, however, several limitations to its use; first, in its current form measured utilization simply reflects the proportion of an allocated session time that is consumed by anaesthetic and surgical procedures – that is, it only offers a quantitative measure of how well session time has been used. It fails to consider the operative workload achieved during the session time (case-mix adjusted utilization). Taken to an extreme, a hernia that takes four hours to perform on a four-hour session represents 100% utilization of session time. The case-mix adjusted utilization of this operating session could be considered poor, although factors surrounding the circumstances of the operation need to be considered, such as surgeon seniority, emergency/elective procedure, complexity of the hernia repair. Measurement and reporting of simple ‘unadjusted’ utilization rates can therefore be misleading and have the potential to misinform managerial decision-making. A second potential source of confusion arising from the use of utilization rates surrounds the handling of cancelled operating sessions. Approximately one-tenth of scheduled operating sessions are cancelled by NHS Trusts. 11 ^, 13 Theatre managers usually benchmark utilization after aggregation of used theatre session time over an extended period. This method of aggregating used session time can overestimate utilization rates if cancelled sessions are not included in the equation denominator. This important issue has been recognized within the toolkit developed by the Modernization Agency which presents guidance for the consistent measurement of theatre utilization within the NHS. 14 Unfortunately, there is however no evidence of homogeneity of measurement between Trusts at present and as a result comparison of performance between NHS theatre units can be unreliable. Third, theatre utilization rates fail to portray the performance of operating sessions that overrun the allocated session time. This issue is highlighted in the Audit Commission report in which numerous Trusts returned aggregate utilization rates exceeding 100%, suggesting that over-runs occurred frequently at those centres. 11 Over-runs are a source of great inefficiency as operating sessions are staffed only for the allocated scheduled session time. Any significant overrun therefore results in enhanced service costs secondary to staff overtime payments. The financial cost is relative, however: for a Trust battling long waiting lists the short-term cost implications of theatre overruns can be less important than enhancing output, although unsustainable in the long-term. Apart from the financial penalty, over-runs are a commonly cited cause of discontent amongst theatre personnel, 12 and can lead to poor staff productivity as well as incurring high indirect costs, such as agency costs, secondary to staff absenteeism and recruitment difficulties.

Both theatre session over-runs and under-runs represent sources of inefficiency; the former is a cause of high utilization rates whilst the latter is associated with low utilization rates. As such, aggregation of over- and under-utilized lists can suggest acceptable levels of theatre utilization, masking gross departmental inefficiency requiring intervention.

For the reasons cited above, caution should be exercised when using theatre utilization to benchmark performance between different theatre units. Certainly, low utilization rates represent unmistakable low theatre performance and those with very high utilization rates a problem with over-runs. What is currently difficult to identify from ‘high-level’ comparative data is the unit with a seemingly healthy aggregate utilization rate but that actually comprises a combination of extreme under-runs and over-runs.

Alternatives to operating session utilization rates

Within the NHS, assessment of operative workload and resulting theatre utilization has long relied on recording and reporting the Finished Consultant Episode (FCE). This method of ‘case counting’ does not provide a realistic assessment of operative workload as it makes no distinction between procedure complexity (i.e. an endoscopy and a liver transplant both represent a single FCE). The principle of reporting case-mix adjusted operative outputs originated with the Intermediate Equivalent system, 15 which has been used extensively in the private sector. In this system procedures are allocated a complexity score on the basis of their recommended fee value relative to that of an intermediate procedure (e.g. inguinal hernia repair). Though its origin is unclear and some discrepancies exist, the scale was developed to reflect the time and skill entailed in each operative procedure and has undergone repeated modification with developments in operative procedures. By adjusting the operative outputs for both the number and complexity of cases, it becomes more reliable to compare operative workloads and resulting theatre utilization rates between different Trusts and departments.

Since the introduction of Payment by Results 16 ^, 17 in the UK healthcare system, Healthcare Resource Grouping (HRG) now offers a potential direct measure of case-mix adjusted operative output within the NHS. The principle of HRG is to group together operations within each surgical speciality on the basis that they use comparable levels of healthcare resource and currently act as a means of determining fair and equitable financial reimbursement for care services delivered by healthcare providers. It may be that for operational and strategic decision-making, a marker of theatre productivity such as HRG payment per hour of allocated session time may offer a more useful performance measure than conventional unadjusted utilization rates. Certainly, the Diagnosis Related Group (DRG) system in the US permits financial quantification of surgical workload. The latter, along with HRG payments in the UK, facilitate managerial decision-making as well as allowing robust economic markers of performance.

Both the HRG and DRG systems allow us to assign an economic value to an operation as a surrogate marker of its complexity. Although practically useful and a vast improvement over ‘case counting’, this approach remains high-level and subject to some inaccuracy. We are seeing increasing application of electronic management systems in the day-to-day running of operating theatres. They act as a clinical governance tool, allowing for accurate prospective entry of data concerning the timing of movements of a patient through the operating department and the personnel involved at every stage of their care. In addition, they fulfil a stock-taking function; any equipment/materials used during a patient's operative episode are logged electronically. We therefore have access, on an individual basis, to data which will give us an indication of resource usage for a procedure and therefore its complexity and anaesthetic, operative and recovery times. This will allow us to determine case-mix adjusted operative outputs for individual surgeons, departments and/or Trusts, and therefore improve the reliability of benchmarking theatre utilization performance.

Optimizing theatre productivity

In a multivariate analysis of factors that drive theatre utilization at a London teaching hospital we found that the case-mix adjusted size (a classification called list scoring) of operating lists represents by far the strongest determinant of utilization in both main theatres and ambulatory theatres. 18 The effects of late starts and the impact of certain surgical and anaesthetic personnel were, by comparison, much smaller. It follows that providing adequate ‘list volume’ will have the single greatest managerial impact upon theatre performance. Having pre-determined optimal ‘list volume’, systems need to be in place to ensure that patients listed for surgery both attend for their operations and are not cancelled for medical reasons on the day of surgery. Operational advice on this issue has been published by the Modernization Agency. 19 The reasons for a high late cancellation rate of day surgery cases are often very different to those in the inpatient setting. In the day surgery environment late cancellations usually occur either because patients do not attend or are cancelled on the day by the anaesthetist. In contrast, inpatient elective admission cancellations often result from a shortage of inpatient beds (usually as a knock-on effect of emergency admissions). The priority of a surgical directorate manager is to optimize the functioning in both the day surgery and inpatient settings. Although a general panacea would be an oversimplification, a basic decision algorithm can be suggested. First, in hospitals where procedure cancellations occur commonly due to a shortage of inpatient beds, an audit of surgical bed availability may be helpful to confirm this resource deficiency. Subsequent transfer of appropriate procedures from the inpatient to the ambulatory setting should then be maximized. If bed resources remain scarce despite this measure, an expansion of inpatient surgical bed facilities may be required. Where the latter is not possible, segregation of the emergency and elective services by ‘ring-fencing’ elective beds could be considered. Certainly, small single-centre audits support the popular belief that emergency admissions compromise the availability of inpatient beds required for elective surgery. 20, 21, 22 Furthermore, evidence from Scandinavia suggests that ring-fencing may enhance hospital efficiency, although this may only be true under certain conditions of case-mix and the demand for elective surgery. 23 The concept of ring-fencing also raises potential ethical considerations with respect to the prioritization of elective surgery above the interests of the emergency patient. This problem can be partly overcome by the use of dedicated National Confidential Enquiry into Patient Outcome and Death (NCEPOD) theatres during normal working hours. Extending their use into ‘twilight shifts’ can additionally prevent the build-up of emergency cases, which can interrupt general theatre efficiency. 24 It may be that multi-site Trusts choose to provide acute and elective services at differing locations, although local circumstances will need to be considered. 25 This could afford a geographical ‘ring-fence’ that protects elective service efficiency. Competition from independent treatment centres, which are not compromised by the obligation to provide an emergency service, may mandate that NHS Trusts re-evaluate their policies on ring-fencing.

A further aspect of theatre performance that requires consideration is case-scheduling. Ideally, the number and complexity of cases listed for an operating session should match the allocated session time so that inefficiency due to either under-runs or over-runs is minimized. Obviously, efforts aimed at scheduling an optimal operating list volume are futile if the desired cases fail to attend or are then cancelled. In the NHS the surgical consultant, in collaboration with the admissions staff, is generally responsible for estimating ‘appropriate’ list volume. Certain investigators have demonstrated adequate prediction of operating time requirements using median 26 and mean log-normal 27 historical procedure times. In reality, however, the use of theatre data systems to predict expected surgical times, and thereby assist with scheduling, is not however universally accepted as an accurate method. 28 Certainly, the variance associated with a series of operations makes it difficult for a mathematical approach to be able to accurately predict list finish times as compared with informed estimates. In some instances surgeons have demonstrated superior predictive ability when compared to ‘intelligent’ scheduling software. 29 Mathematical algorithms have been used successfully for simulation modelling to help predict the effect on overall theatre efficiency as a result of hospital structural and process changes. 30 ^, 31 Although total future reliance on mathematical systems is uncertain, they could provide a useful adjunct to scheduling in the future.

Conclusions

Caution should be exercised if using utilization rates as a theatre performance indicator. Extremely high and low utilization rates do suggest inefficiency due to overrunning and underused lists respectively. In reality, however, the managerial practice of aggregating operating sessions over time to calculate overall utilization may yield apparently acceptable efficiency despite extreme individual operating session inefficiencies. With HRG now established in the NHS, the measurement of case-mix adjusted operative output and associated productivity may supersede utilization rates as a more accurate indicator of theatre performance.

Service efficiency within the NHS is becoming increasingly important as competition from the Independent sector is being actively encouraged. Safe-guarding theatre list volume underpins good theatre performance. Ensuring that patients scheduled for surgery are allocated a bed, do not fail to attend and are then not cancelled on the day of surgery is essential for optimal patient flow. These factors are easier to control in the day surgery setting away from the competing demands of the emergency service. Ambulatory operating should be maximized where possible. In the inpatient setting, ring-fencing elective beds may be necessary if service efficiency is to be prioritized.

Footnotes