A conceptual framework for selecting the most appropriate variables for measuring hospital efficiency with a focus on Iranian public hospitals

Abstract

Over the past few decades, there has been an increasing interest in the measurement of hospital efficiency in developing countries and in Iran. While the choice of measurement methods in hospital efficiency assessment has been widely argued in the literature, few authors have offered a framework to specify variables that reflect different hospital functions, the quality of the process of care and the effectiveness of hospital services. However, without the knowledge of hospital objectives and all relevant functions, efficiency studies run the risk of making biased comparisons, particularly against hospitals that provide higher quality services requiring the use of more resources. Undertaking an in-depth investigation regarding the multi-product nature of hospitals, various hospital functions and the values of various stakeholders (patient, staff and community) with a focus on the Iranian public hospitals, this study has proposed a conceptual framework to select the most appropriate variables for measuring hospital efficiency using frontier-based techniques. This paper contributes to hospital efficiency studies by proposing a conceptual framework and incorporating a broader set of variables in Iran. This can enhance the validity of hospital efficiency studies using frontier-based methods in developing countries.

Introduction

One of the major concerns of mainstream economic theory is the efficient use of scarce resources, and efficiency measurement is a useful tool for making choices between alternatives.¹ In simple words, resources go into a decision-making unit (DMU) to be processed to produce products. In economic terms, the concept of efficiency can be defined as the relationship between scarce factor inputs and products. It examines how well scarce resources are converted into products. Given the need to ensure the best use of scarce resources, increasing emphasis is being placed on efficiency measurement. Many studies have documented that public and private sector organizations do not always use resources efficiently.² From this standpoint, increasing emphasis is being placed on measures of efficiency in health-care organizations, particularly hospitals, because they are the prime resource-consuming unit in any national health system.³ Efficiency measurement, by monitoring the performance of individual hospitals and comparing them with each other, is a useful tool for improving management, rationalizing resource allocation and mobilizing additional inputs. This evidence has convinced researchers to conduct many hospital efficiency studies during the past few years.

Review of the literature has shown the diversity in techniques used and variables employed in hospital efficiency measurement in developing countries. In the literature, simple ratio analysis and frontier-based techniques are the usual techniques applied to analysing efficiency in hospitals. Simple ratios, including average length of stay and occupancy rate, can provide a generally useful but inevitably limited overview of the data structure. Having established the limits posed by simple ratio analysis in dealing with multiple inputs and multiple outputs, frontier-based techniques are being used increasingly to measure the efficiency of health-care services.⁴ The two principal types of frontier-based approaches to measuring hospital efficiency are parametric methods, such as stochastic frontier analysis (SFA), and non-parametric methods, such as data envelopment analysis (DEA) which uses mathematical programming. Both methods compare the performance of an individual hospital against an estimated efficient frontier. SFA as an advanced econometric method uses statistical methods to fit a frontier. In SFA, the form of the cost or production function is assumed to be known or is estimated statistically. SFA generates efficiency measures, computed in terms of the distance that lies between the observation and the estimated function. However, in many cases there is no known functional form. This is especially true in the case of public sector organizations, such as a health-service providers.⁵ DEA, on the other hand, as a non-parametric method, makes no assumption about the form of the underlying production function or the distributions of errors. DEA uses a linear programming model to estimate the efficient production frontier, comparing a particular DMU's performance with the efficient frontier.

Researchers working in the field of hospital efficiency measurement argue about the relative utility of the above methods. The review of existing hospital efficiency studies using frontier-based techniques in developing countries has revealed that, while the choice of measurement methods in hospital efficiency assessment has been widely argued in the literature, few authors have offered a framework to specify variables that reflect different hospital functions, the quality of the process of care and the effectiveness of hospital services. This implies that the main objective of a hospital and its full range of functions in efficiency measurement are poorly understood. This is evident from the fact that a body of empirical studies selected different sets of variables and offered only vague recommendations for selecting variables. One explanation for these inconsistencies is the lack of conceptual clarity for selecting the most appropriate variables for measuring hospital efficiency.

The specification of input and output variables is a key step in an evaluation of comparative performance because the results of any efficiency measurement depend heavily on the variables used.^6,7 It has been argued that in hospital efficiency measurement using frontier-based techniques, the hospitals' rankings are subject to a substantial variation due to changes in the specification of output variables.⁸ This phenomenon has also been noted by other studies.⁹

The breadth of literature related to hospital efficiency studies using frontier-based techniques is variable. While relatively a wide range of variables have been used for measuring hospital efficiency in developed countries, there has been little effort to propose a comprehensive framework for selecting the most appropriate variables in developing countries. In Iran, the subject has not been prominent in the literature. Currently, the efficiency of the Iranian hospital is assessed mainly based on some throughput ratios (such as average length of stay or bed occupancy rate) and/or the number of services provided. However, because of the unidimensionality of the variables used, the results run the risk of making misleading comparisons.

In selecting variables, a majority of studies rely on pragmatic criteria; a phenomenon described by Parkin et al. ⁹ as ‘common-sense rules’. Such ‘common-sense rules’ lay emphasis on covering the full range of resources used and capturing all relevant activity levels.¹⁰ However, what is meant by ‘full range of resources’ and ‘relevant activities’ awaits clarification. In the majority of hospital efficiency studies, output and input variables are selected according to a narrow view of hospital function revolving around curative care and the associated diagnostic procedures.

In the output side, most studies in developing countries have used the unweighted units of measurement for inpatient activities, including the ‘number of separations’ and ‘number of patient days’.^11,12 Measuring hospital outputs by such variables does not capture the case-mix. This may lead to higher efficiency scores for hospitals that admit and treat non-complicated cases. In developing countries, some authors have tried to address this problem by strategies such as using the number of minor and major surgical interventions.¹³ In addition to the inpatient variables mentioned above, hospital efficiency studies frequently use outpatient events, such as the number of outpatient visits and/or emergency attendances.^14,15

The input variables in the existing literature have focused on human, capital and consumable resources. The number of staff or monetary values, i.e. salaries, has been used as human resource input variables.^16,17 In terms of capital inputs, studies have used ‘number of beds’¹⁸ or ‘total amount of annual expenditure of capital’.¹² A few studies have employed variables relevant to consumables, such as the monetary value of drugs.¹⁹

The variables employed in most studies have predominantly been concerned with the quantity and/or cost of the services provided. This approach assumes a relationship between the episodes of service provision and improvement in health. From here onwards we use the term ‘service-oriented approach’ for efficiency studies that rely on output and input variables that are concerned mainly with the quantity of services provided and resources used. This approach suffers from at least two drawbacks.

Firstly, and most importantly, the use of a service-oriented approach reflects a narrow view of hospital functions. This approach emphasizes the curative care (or therapeutic) function of the hospital. Although, curative care is the core function, there is a growing emphasis on a broader set of hospital functions and objectives.³

Understanding of the hospital's role widened from patient care alone to a range of functions that include protective and preventive care, health promotion, health workforce development and interaction with and support for other components of the health-care system.

Secondly, the service-oriented approach only partially takes into account the quality of the care provided. The trade-off between resources used and quality of care, and the use of quality variables in hospital efficiency studies and their impact on efficiency scores might be regarded as a contentious point. As some studies have shown, it is possible that a hospital providing better quality care may need relatively more resources compared with a hospital providing relatively poor quality care.²⁰ This might be particularly true when the higher amount of an input in one hospital compared with other hospitals (for example, the number of doctors, which is frequently used in hospital efficiency studies as an input variable) affects the quality of care with little impact on the number of services provided by that hospital. An efficiency analysis focusing only on the quantity of services may spuriously suggest quality-oriented hospitals to be inefficient. Despite this, only a few hospital efficiency studies in developing countries have employed variables that reflect the quality of care. For example, in an attempt to include quality variables, Sahin and Ozcan²¹ used hospital mortality rate to measure hospital efficiency in Turkey. While these and some other studies have included variables that reflect quality of care, those variables are issue-specific and a systematic framework that reflects a full range of hospital functions incorporating a complete range of quality variables is not yet available.

Considering the importance in selecting the most appropriate variables in hospital efficiency studies and the absence of a well-defined framework in the literature related to the developing countries and Iran, this study is an attempt to propose a conceptual framework for choosing variables to measure hospital efficiency with an emphasis on its applicability to the Iranian public hospitals. This framework should recognize the importance of quality of health care and reflect a coherent set of key hospital functions.

Methods

Believing that a public hospital in Iran is the hub of an integrated health-care system contributing to the improvement of overall population health status, and basing its foundations on an enhanced understanding of what ‘ought to be’ the objectives and functions of a hospital, this study makes the following argument.

To propose the most appropriate variables for measuring hospital efficiency, a review was conducted of the published peer-reviewed literature on the main objectives of a hospital, the full range of hospital functions and hospital performance measurement. This review included English-language studies published from 1 January 1986 to 31 July 2006. Medline, the Cochrane database, health-promoting hospital (HPH) studies,²² quality-oriented reviews of hospital performance measurement (such as The World Health Organization [WHO] literature on hospital performance assessment and reports published by the Agency for Healthcare Research and Quality [AHRQ]) and relevant journals accessed through ISI Web of Science were searched. To retrieve these documents, the main strategies were: Internet search engines, official website search, the search of electronic databases and a check of references from selected documents. This review led to propose a framework for selecting the most appropriate variables with a focus on the Iranian hospitals. We did not, however, identify new variables in measuring hospital efficiency. Rather, we identified the most appropriate set of variables in addition to the existing appropriate variables used by other researchers – based on a full range of hospital functions and the quality of care – that can be employed in the evaluation of hospital efficiency. To achieve this, the following criteria were used:

Key hospital functions from patient, population and health system perspectives;

Values of various stakeholders including patients, staff and community;

The extent to which the desired outcomes are under hospital control;

Scientific evidence of validity and reliability;

Ease in defining and collecting at regular intervals.

Hospital functions as the basis for a health-oriented framework

To propose a coherent framework for selecting variables, an understanding of a wider set of hospital functions is required. Dyson²³ has suggested that, to properly manage an organization, the set of performance variables should reflect all relevant objectives and functions of that organization. Designing a coherent framework and selecting appropriate variables should reflect what a hospital ought to achieve and how it carries out its functions.²⁴ This immediately raises a concern regarding the difficulty in specifying any hospital's objectives, functions and valued products.^25,26

Traditionally, it has been assumed that hospitals should be acknowledged as a centre for offering a wide range of diagnostic and therapeutic services. From this point of view, hospitals are responsible for treating sick people. These services, although considered to be core functions, do not wholly reflect all hospital functions. There is a growing body of literature indicating that both the definition and functions of hospitals are changing. It has been argued that hospitals should have a multidimensional approach to the health of the patients and communities in the target/catchment areas, responding to community needs and patient expectations, and not merely focusing on therapeutic services.^2,27 ‘Population health’ in contrast to ‘individual health’ requires working beyond the provision of curative care for individuals. This view of health has been considered in the HPH concept.

The concept of HPH requires hospitals to promote health, to foster participation of target groups including patients, staff and community, to offer education and information to target groups, and to improve communication with other types of health-care providers.^28–30 To develop this idea in the context of a hospital environment, some authors indicate a number of general tendencies.³¹ Firstly, the hospital organization needs to be repositioned in order to redefine the specific range of new functions. Hospitals have to show their awareness of the patient's life before and after admission, provide an educational programme for patients and staff, run disease-prevention activities, offer a healthy workplace for staff, provide high-quality services for patients and cooperate with other health-service providers and the community. Secondly, the hospital should be committed to improving and assuring the quality of its services.

It should be pointed out that having a holistic approach to population health indicates that hospital functions should be considered as a part of the functions of a wider health-care system. From this point of view, the main objective of a hospital can be regarded as its role in improving population wellbeing and meeting societal needs. Clearly, if we define the main aim of a hospital in this manner, the variables that are selected to measure its efficiency should be defined in relation to explicit functions, reflecting the values of a variety of stakeholders including patients, staff and community.

Adopting the above perspective, a comprehensive list of hospital functions can be classified into productive and interactive functions. This paper places all functions related to the hospital's internal environment, which contribute to producing health among its productive functions. All those hospital functions reflecting its capacity for interaction with its surrounding environment are considered to be its interactive functions. The discussion is centred around the situation of public hospitals in Iran and in other developing countries comparable with Iran. These two functions (productive and interactive) are discussed below.

Productive function

The productive function refers to how the hospital directly improves patient health and ameliorates sickness through fundamental roles that include curative, preventive and protective services. This function is about the performance of various internal elements within any hospital and about the hospital's capacity to integrate these elements in order to produce health. Some measures of curative care include output volumes (such as the number of surgical procedures) and quality of services provided (such as unplanned readmission rates). Preventive care involves interventions such as health educational programmes – particularly for patients with chronic conditions, such as diabetes mellitus, asthma and hypertension.

Protective care is about providing an error-minimal service to patients (measured, for example, by the number of postoperative sepsis) and a healthy working place for the staff. Other examples of productive functions include staff training and development (for example, the provision of training workshops).

Interactive function

This function is attributed to the capacity of the hospital in providing an effective interaction with its surrounding environment, which is required to fulfil its role as a HPH. In the past, hospitals were mainly concerned with their productive functions. More recently, under the influence of movements such as the HPH, there is an emerging imperative to form an effective relationship between the hospital and other levels of the health-care system, and with society at large. This function refers to the coordination by which the hospital deals with other parts of the health system (for example, a primary health-care network). It also refers to the hospital's responsiveness to societal needs ensuring that the hospital is appropriately responding to the population health-related issues. The interactive function of the hospital can impact upon patient health outcomes. For example, an association has been found between collaboration among health providers and mortality and quality of life.³²

Consider that a full range of hospital functions potentially have a marked impact on hospital efficiency measurement. If efficiency measurement is to be used as a tool to effectively manage an organization such as a hospital, the variables used to measure efficiency should cover a full range of functions that are needed to attain the hospital's objectives. This ensures that there is a link between variables used and hospital functions, leading to the enhancement of the validity of efficiency studies.

Inspired by the productive and interactive functions defined above, and the values of various stakeholders, including patient, staff and community, a conceptual health-oriented framework for hospital efficiency measurement is proposed in the following section. However, before developing the framework, the following points should be noted. Firstly, this framework proposes those variables that are relevant to the standards, which are promoted by the European HPH Network. There is a growing body of literature to support the effectiveness of the HPH standards and functions that ought to be provided within the hospital setting.³³ Further research is now required to assess these for cost-effectiveness.

Secondly, it should be stressed that definitions of hospital missions and functions have to be addressed in the context of the type of hospital and the overall health system. For example, public hospitals may have different institutional constraints, missions (such as teaching and research) and subsequently different functions compared with private hospitals.³⁴ These hospitals are expected to provide more community-oriented services, including preventive care and health-promoting activities. In many developing countries, a substantial amount of health funds is spent on building new hospitals or developing existing hospital facilities with little attention to PHC capacities. Yet, the PHC movement encourages hospitals in developing countries to reach out to the community, offering preventive care as well as curative care. Thus, providing these community-based services by hospitals in developing countries in general, and the Iranian hospitals in particular, is appropriate.

Results and discussion

A conceptual framework to guide in selecting appropriate variables

For the sake of clarity, variables that are commonly used in the existing hospital efficiency literature are called ‘conventional (service-oriented) variables’ in this paper. These variables are often those most readily associated with episodes of service provision, for example, the total number of separations. On the other hand, variables proposed for addition to the framework, based on a comprehensive set of productive and interactive functions, are called ‘non-conventional (development-oriented) variables’. Figure 1 proposes non-conventional (development-oriented) variables for hospital structure, throughput, output and outcome in parallel with conventional (service-oriented) variables (Table 1).

Figure 1

A health-oriented framework to guide in selecting the most appropriate variables for hospital efficiency measurement

Table 1

Non-conventional variables based on the health-oriented model

Major classification	Description	Variable	Numerator	Denominator
Intermediate outcome variables	Patient safety variables	Decubitus ulcer	Discharges with related ICD code or DRG, except those with length of stay less than five days and primary diagnosis of decubitus ulcer	All medical and surgical separations
	Patient safety variables	Iatrogenic pneumothorax	Discharges with related ICD codes or DRG, except those primary diagnosis, patients with trauma, patients with any code indicating thoracic surgery or assigned to cardiac surgery	All medical and surgical separations
	Inpatient quality variables	Avoidable mortality rate	All in-hospital deaths due to maternal and perinatal events, cholelithiasis, cholecystitis, appendicitis, peptic ulcer and abdominal hernia	Number of related separations
		Postoperative mortality rate	All deaths within 30 days of selected surgeries*	Separations with the same codes or DRGs
		Non-operative mortality rates due to AMI, GH, and P	Number of deaths with principal diagnosis of AMI, GH, P identified by DRG or ICD codes, except those who are 18 years or younger	Separations with the same ICD codes or DRGs
		Unplanned readmissions for specific conditions (hypertension, diabetes and COPD)	The total number of unplanned readmissions for specific conditions identified by related ICD codes or DRG within 28 days	The total number of separations for specific conditions
Output variables	Overused utilization variables	Caesarean delivery rate	Number of caesarean deliveries identified by related ICD code or DRG, except those for which a scientific indication is identified	All deliveries
Process variables	Clinical performance variables	Administration of aspirin within 24 hours of admission for patients with AMI	Number of discharged patients with AMI, identified by related ICD code or DRG who received aspirin within 24 hours of admission, except patients with aspirin contraindications	Total number of discharged patients with AMI without aspirin contraindication
		Administration of aspirin at discharge for patients with AMI	Number of discharged patients with AMI who are prescribed aspirin at hospital discharge, except patients with aspirin contraindications	Total number of discharged patients with the same code without aspirin contraindication
		Administration of beta-blockers at discharge for patients with AMI	Number of discharged patients with AMI who are prescribed beta-blockers at hospital discharge, except patients with beta-blocker contraindications	Total number of discharged patients with AMI without beta-blockers contraindication
	Health educational programme	Health educational programme	Number of patients who can name actions in self-management for their chronic conditions (diabetes, hypertension and COPD)	All patients diagnosed with the mentioned chronic conditions
Structural variables	Promotion and preventive programmes for main target groups	Staff-oriented activities	Total number of days out of work, excluding planned holidays	Total number of days contracted
			Total number of declared work-related injuries	Total number of staff
			The existence or lack of counselling service to offer stress management plan and smoking cessation programmes for staff
		Community-oriented activities	The existence or lack of counselling services and health information centre for community access, the number of lifestyle educational programmes for public (such as monthly public sessions) and properly managing hospital waste
		All target groups	Direct costs for all activities dedicated to staff, patients and community health promotion	Total operating budget
	Cooperation with other health-care providers		Discharge letters sent to general practitioner within two weeks	All discharge letters

ICD, International Classification of Disease; DRG, diagnostic-related groups; AMI, acute myocardial infarction; GH, gastrointestinal haemorrhage; P, pneumonia; COPD, Chronic obstructive pulmonary diseases

*Aortic aneurysm repair, cataract extraction, mastectomy, knee replacement, hysterectomy, gastrectomy, hip replacement, esophagectomy and prostatectomy (Scottish Clinical Outcomes Working Group, 2002)

At the level of ‘structure’, conventional variables relate to inputs of personnel (for example, the number of staff), capital (for example, the number of beds) and consumables (for example, the cost of drugs). Although there is little doubt that these inputs have a major impact on patient outcomes, there is a growing body of literature indicating the crucial role of other factors, such as the characteristics of the organization.^35,36 Hospital organizational characteristics refer to the capacity of the hospital to achieve the best interaction between various inputs in order to obtain optimum value for patients and staff, and to achieve beneficial collaboration with the community and other health-care providers to attain better population health. Therefore, we consider it important to include variables that relate to concepts, such as preparedness, for an effective response to community needs. The impacts of organizational characteristics on health outcomes have been discussed in a number of studies. For example, Aiken and Patrician,³⁶ by using three subscales reflecting the organizational characteristics within a hospital (nurse autonomy, nurse control over the practice setting, and the relationship between nurses and physicians) have pointed out that the hospitals that value professional nursing expertise had a better patient outcome in terms of a higher patient satisfaction and a low mortality rate. This suggests that these organizational capacities have a worthwhile impact on outcomes. More importantly, the findings imply that the impact of organizational capacities is independent of the staffing pattern.

The next stage is entitled ‘throughput or process’ (the transformation of inputs into outputs). According to the service-oriented approach, the variables describing throughput relate to the extent to which hospitals are busy. One example is the length of stay, which is used in hospital efficiency measurement. However, it is known that the use of such variables may lead to perverse incentives, because hospitals may thereby have an incentive to admit patients with uncomplicated illnesses or to discharge patients too early in their convalescence. In responding to this deficiency in the existing service-oriented approach, the proposed framework suggests a broader range of process variables, such as health educational programmes and quality-oriented procedures (Table 1). Since existing service-oriented efficiency measurement studies do not emphasize such processes, these run the risk of labelling a health promotion and quality-oriented hospital as an inefficient hospital because promoting health and improving quality may require additional resources.

The next stage focuses on ‘outputs’. In the existing hospital literature, output is considered in terms of hospital activities such as the number of separations. The use of the number of services provided may encourage managers to increase efficiency scores at the expense of quality, adversely affecting the health outcomes. Here we argue that, in addition to the conventional output variables which are about the number and types of separations, there are particular types of output variables with a quality dimension that influence health outcomes. For example, a hospital performing a large number of caesarean deliveries, some of which may not be medically indicated, could still be labelled as an efficient hospital. Yet, an over-utilization of procedures, such as caesarean delivery, could lead to adverse effects or at least may not add to any population health benefits.

The last stage of the proposed framework is on health ‘outcomes’ – both individual-based and population-based. ‘Individual-based outcomes’ could be grouped into ‘biological’ and ‘patient-centred’ outcomes. A reduction in blood pressure resulting from treatment is an example of a biological outcome. Clinicians place a heavy emphasis on such biological outcomes.³⁷ A bio-medical approach to health, however, may not fully cover the patient's best interest. For example, Guyatt et al. ³⁸ have found that in patients with chronic heart and lung disease, exercise capacity in the laboratory is only weakly related to exercise capacity in daily life. Thus, some studies aim to identify and include variables that reflect the likely benefits from the patient's point of view.

Taking a broader perspective, ‘population-based outcomes’ is an expression of health services in terms of improvement in population health status and wellbeing. Very few hospital efficiency studies have used variables related to this type of health-outcome measure. This is largely because of difficulties in quantifying the extent of change in health status produced by the hospital as compared with the next best alternative form of treatment. The proposed framework has offered a set of variables as proxies for health outcomes.

Considering our proposed health-oriented framework, the next section discusses in detail the specification of variables relevant to each of the four stages – structure, process, output and outcome – of the proposed framework (Figure 1).

Non-conventional variables proposed to measure hospital efficiency

Structural variables

Efficiency measurements could benefit from the inclusion of structural variables. According to the HPH concept, hospitals should have a holistic approach to the health of all individuals, including their patients, staff and community. To achieve this, hospital services should be reoriented to a more integrated proactive approach including health promotion and prevention. This can only be attained by developing new structures that are capable of providing a supportive and healthier working environment for staff, with the potential to meet overall health development through changing people's health behaviour, leading to the improvement of population health. For this reoriented process, three main target groups can be identified, namely patients, staff and community. Evaluators who agree to use variables related to hospital organizational structure in the processes of performance measurement tend to consider a complete range of hospital functions (other than curative care) including the preventive function, health-promoting activities, human resource development, responsiveness to community needs and cooperation with other health providers.

Almost all studies focusing on the development of these variables emanate from WHO and its European Office for Integrated Heath Care Services. Out of the four reports in which standards and strategies for health promotion in hospitals have been explored by WHO, the report of the fourth WHO Workshop provides a revised format to define the health-promoting activity standards and associated variables.³⁹ Those variables include, among others, the level of patient awareness, the prevalence of work-related injuries to staff and the availability of lifestyle educational programmes for the public. In addition, the collaboration between hospitals and other health-care providers can also be evaluated. These variables are particularly appropriate for hospital efficiency studies in Iran or in other countries at a comparable stage of economic development. These hospitals suffer from the lack of health-promoting activities targeting various stakeholders, including patients, staff and community (Table 1).

Process variables

While outcome and output quality variables largely concentrate on the patient care and protection functions, the majority of process variables focus more on the disease prevention and health educational functions. Generally, process variables assess the capability of health professionals to make appropriate choices in order to achieve desired outcomes for the patients.⁴⁰ Two commonly suggested dimensions are, firstly, clinical performance and the extent of the evidence-based nature of the clinical care process and, secondly, health educational programmes adopted from the HPH concept.^41,42 We propose to include quality-oriented process variables, for example administration of aspirin within 24 hours of admission for patients with acute myocardial infarction, as proposed by the Centres for Medicaid and Medicare Services.⁴³ In addition to clinical performance, another set of non-conventional process variables focuses on health educational programmes. These are central to the philosophy of the HPH. Employing these types of variables is important for hospital efficiency measurement since they reflect hospital functions other than the curative function and relate to preventive care and respond to community needs. Health educational programmes particularly target the groups of patients suffering from chronic diseases. Diabetes mellitus, hypertension and chronic obstructive pulmonary disease are regarded as diseases whose prognosis are highly dependent upon involving patients in educational and skill development programmes.⁴⁴ From this standpoint, patient education on risk factor modification and disease treatment options could be considered for hospital efficiency measurement. Health education could be measured as a categorical variable (existence or lack of a health education programme) (Table 1).

Output variables

Utilization ratios lie at the heart of this category. Amidst various utilization variables, the caesarean delivery rate is widely used as an indicator of the quality performance of a health service. The validity of this variable has been supported by some studies.⁴⁵ Low caesarean rates correlate to fewer unnecessary surgeries, a lower cost of care and greater patient satisfaction leading to a better health outcome. Nevertheless, given the varying risk profiles in a catchment population, there is no gold standard for an optimal rate of caesarean delivery. Despite this, WHO states that no region in the world is justified in having a caesarean rate greater than 15%.⁴⁶ Taking into account WHO recommendations, a general hospital that performs more than 15 caesarean sections for every 100 births might be suspected of performing unnecessary procedures. Given that a large output in terms of a higher number of caesarean sections will not necessarily be a sign of high efficiency, we suggest that the caesarean section rate as a quality-oriented output variable should be used in hospital efficiency studies. We also consider this variable appropriate for measuring hospital efficiency in Iran because of the high rate of unnecessary caesarean delivery.⁴⁷

Outcome variables

‘Improvement in health at both patient and population levels’ is the desired outcome of a hospital. Efficiency studies need to consider variables that reflect health improvements. It might be argued that achieving better health for patients may not necessarily lead to better health at the population level (for example, very high-cost treatment for a single individual).

For this reason, it is desirable that a hospital's success is measured in terms of both the health of individuals (individual-based outcomes) and population health (population-based outcomes) in the target area.

It has been argued that variables such as quality-adjusted life-years and disability-adjusted life-expectancy measure both dimensions of health outcome (i.e. health per se and the value of health) and reflect the health status of the population.^48,49 However, for a number of reasons such variables may not be appropriate for hospital efficiency measurement. Patients' self-reported health-related quality of life strongly depends on various individual and environmental conditions, such as socio-cultural, economic and political factors, many of which are not well known and/or are beyond the control of the hospital. This implies that, without adjustment for these contextual circumstances, the use of such variables may not yield valid results. It has also been argued that there can be a long delay between the provision of care and the final population-level outcome.⁵⁰ This can lead to uncertainty in identifying which intervention or health provider is responsible for the actual outcome measured.⁵¹ Therefore, it is difficult to establish a link between these variables and the efficiency of any one hospital.

For all these reasons, hospital efficiency studies have to step back to examine variables that represent individual-based outcomes. Individual-based outcomes can be classified into two categories: (i) biomedically-centered; and (ii) patient-centred variables. A clear example of a biomedical variable is ‘effective management of placenta praevia’ as suggested by Montoya-Aguilar.⁵² However, due to professional uncertainty and disagreements on accepted standards for most medical conditions, the use of such variables in efficiency studies is arguable. Patient-centred variables in current use focus more on patient satisfaction. These variables are subjective, depending on individual and environmental factors that might be beyond hospital control, so that the existing hospital efficiency literature has tended to avoid them.

It has been shown that achieving the quality-indicator targets (for example, an effective hypertension management plan in diabetic patients) can lead to better health outcomes (for example, prevention of cardiovascular events among diabetic patients).⁵³ Because ‘quality of care’ translates into better health outcomes, we argue that quality-of-care variables can provide insight into health outcomes and could be measured routinely to allow study of changes over time. Quality variables linked to health outcomes are presented in the following section (Table 1).

Quality variables linked to outcomes

AHRQ⁵⁴ has grouped health services quality variables into categories. Some of the AHRQ ‘patient safety variables’ and ‘inpatient quality variables’ could be utilized for hospital efficiency studies since they fulfil our selection criteria. These variables have been developed for international use. Hence, they are appropriate for the purpose of this paper which particularly emphasizes their applicability to countries at a comparable stage of economic development with Iran.

Patient safety variables reflect potentially avoidable complications and iatrogenic events. By employing patient safety variables, this paper emphasizes the important role of the hospital protection function. Examples of patient safety variables that could be used for efficiency studies include incidence of decubitus ulcer and iatrogenic pneumothorax.

Inpatient quality variables reflect the quality of care provided and are mainly about mortality rates and readmission rates. The variables proposed for hospital efficiency include: (i) ‘adjusted mortality rates’ (and crude mortality rate if adjusted mortality data are not available); (ii) avoidable mortality rate;⁵⁵ (iii) postoperative mortality rate; (iv) non-operative mortality rate; and (v) unplanned readmissions for specific conditions.

Experts' validation of the variables identified

Due to lack of relevant studies in developing countries, many data sources reviewed were obtained from developed countries. This raises a concern regarding how the findings could be applied to a developing country like Iran. To compensate for this, we used qualitative methods.

A panel of 12 key informants was selected to assess the Iranian health professionals' agreement on the variables identified for measuring hospital efficiency. This panel included seven senior experts at national level (Hospital Performance Evaluation Department from the Iranian Ministry of Health and the Iranian Social Security Organization), three hospital managers and two senior hospital administrative staff.

In this study, a purposive sampling was used. The criteria for inclusion included the extent of knowledge and experience with hospital performance monitoring at the national level, and a minimum of five years' managerial (or administrative) experience in hospitals. A summary report was prepared discussing the background and the method used to propose the conceptual framework for selecting the most appropriate variables for measuring hospital efficiency. The questions asked from the experts were organized in the two sections that formed the basis for discussion. These questions were about their agreement on the framework proposed and on the appropriateness of the variables identified for measuring efficiency of public hospitals in Iran. A general consensus was found regarding the validity of the method used, the framework proposed and the variables identified for measuring hospital efficiency in Iran. This agreement, however, did not deter the experts from urging the need to identify variables that are more specific to a particular specialty. The discussion mainly focused on variables that are appropriate for measuring efficiency at a micro level within a hospital. The participants also stated that the framework proposed and the variables identified will help in developing proposals to determine deficiencies in the current Iranian hospital database and to identify new data to enhance efficiency measurement. In the current Iranian hospital database, there is no variable reflecting a broad range of hospital functions, such as preventive care, protective care, health promotion activities, staff development-related activities and interactive functions. There are also few variables reflecting the quality of care. From this standpoint, there is a pressing need for major revisions in the Iranian hospital database in order to capture a broader set of variables.

Conclusion

For hospital efficiency literature using frontier-based methods, this paper has presented a conceptual framework for selecting the most appropriate variables with a focus on its applicability to the public hospitals located in Iran. We critically appraised the service-oriented approach, as it is used in the literature, for selecting (conventional) variables for measuring hospital efficiency. We found a lack of conceptual clarity for selecting the most appropriate variables. We also found that the variables used in hospital efficiency studies mainly reflect a narrow view of hospital functions, namely curative care, with a little attention to quality variables. In this study, we argued that the critical step in efficiency studies is to provide a clear understanding of all functions of the organization (hospital in this study) and appropriate variables by which the efficiency of the organization should be assessed. To discuss a broader set of variables linked to the essential functions of hospitals, our point of departure was the concept of an HPH that gives rise to a complete range of hospital functions coupled with the hospital's commitment to improving the quality of services. Undertaking an in-depth investigation regarding the multi-product nature of hospitals, we proposed a conceptual framework with a focus on the Iranian public hospitals, discussing the necessity of the use of a broader set of variables (development-oriented) in hospital efficiency studies using frontier-based techniques.

Meeting the above objectives revealed a number of practical and theoretical implications that can be used by health professionals. We propose that for measuring hospital efficiency both conventional and development-oriented variables should be used. This can enhance the validity of hospital efficiency studies, because together they can capture a complete range of hospital functions and quality of care. This, in turn can provide a more valid tool for hospital efficiency measurement from both the policy-makers and the health administrator's point of view.

Furthermore, it is worth noting that, although the framework proposed has revealed the value of development-oriented variables in hospital efficiency measurement, it has also drawn attention to problems associated with the evaluation of hospital performance. This will help in developing proposals to determine deficiencies in the current Iranian hospital database and to identify new data to enhance efficiency measurement.

References

Robbins

. The Nature and Significance of Economic Science. London: Macmillan Press, 1984

Yaisawarng

, Puthucheary

. Performance measurement and resource allocation. Sydney: The Treasury's Office of Financial Management, 1997

McKee

, Healy

. Hospitals in a Changing Europe. Philadelphia: Open University Press, 2002

Hollingsworth

. Non parametric and parametric applications measuring efficiency in health care. Health Care Manag Sci 2003;6:203–18

Norman

, Stocker

. DEA Analysis: The Assessment of Performance. Chichester: John Wiley and Sons, 1991

Ramanathan

. An Introduction to DEA; A Tool for Performance Measurement. London: Sage Publications, 2003

Bowlin

. Measuring performance: an introduction to data envelopment analysis. J Cost Anal 1998;Fall:3–27

Magnussen

. Efficiency measurement and the operationalization of hospital production. Health Serv Res 1996;31:21–37

Parkin

, Hollingsworth

. Measuring production efficiency of acute hospitals in Scotland, 1991–1994: validity issues in data envelopment analysis. Appl Econ 1997;29:1425–43

10.

Dyson

, Allen

, Camanho

, Podinovski

, Sarrico

, Shale

. Pitfalls and protocols in DEA. Eur J Oper Res 2001;132:245–59

11.

Chang

, Chang

, Das

, Li

. Health care regulation and the operating efficiency of hospitals: evidence from Taiwan. J Account Publ Policy 2004;23:483–510

12.

Ersoy

, Kavunkubasi

, Ozcan

, Harris

. Technical efficiencies of Turkish hospitals: DEA approach. J Med Syst 1997;21:67–75

13.

Ramanathan

. Operations assessment of hospitals in the Sultanate of Oman. Int J Oper Prod Manag 2005;25:39–54

14.

Kirigia

, Emrouznejad

, Sambo

. Measurement of technical efficiency of public hospitals in Kenya: using DEA. J Med Syst 2002;26:39–45

15.

Pavananunt

. Hospital efficiency: a study of public community hospital in Thailand. In: Emrouznejad

, Podinovski

, eds. Data Envelopment Analysis and Performance Management, 2004. See www.DEAzone.com/DEA2004 (last accessed 22 November 2005)

16.

Osei

, Almeida

, George

, Kirigia

, Mensah

, Kainyu

. Technical efficiency of public district hospitals and health centers in Ghana: a pilot study. Cost Effective and Resource Allocation, 27 September 2005;3: [13 screens]. See www.resource-allocation.com/content/3/1/9 (last accessed September 2006)

17.

Owino

, Korir

. Public Health Sector Efficiency in Kenya: Estimation and Policy Implications. Kenya: Institute of Policy Analysis and Research, 1997

18.

Al-Shammari

. A multi-criteria DEA model for measuring the productive efficiency of hospitals. Int J Oper Prod Manag 1999;19:879–90

19.

Bhat

, Verma

, Reuben

. An Empirical Analysis of District Hospitals and Grant-In-Aid Hospitals in Gujarat State of India. Ahmedabad: Indian Institute of Management, 2001

20.

Morey

, Fine

, Loree

, Retzlaff-Roberts

, Tsubakitani

. The trade off between hospital cost and quality of care, an exploratory empirical analysis. Med Care 1992;30:677–98

21.

Sahin

, Ozcan

. Public sector hospital efficiency for provincial markets in Turkey. J Med Syst 2000;24:307–20

22.

Rootman

, Goodstadt

, Hyndman

, Evaluation in Health Promotion: Principles and Perspectives. Copenhagen: European Regional Office of the World Health Organization, 2001

23.

Dyson

. Performance measurement and data envelopment analysis – rankings are rank! OR Insight 2000;13:4–8

24.

Roemer

. National Health Systems of the World. New York: Oxford University Press, 1991

25.

Pauly

. Efficiency, incentives and reimbursement for health care. Inquiry 1970;7:114–31

26.

Harris

. The internal organisation of hospitals: some econometric implications. Bell J Econ 1977;8:467–82

27.

Veillard

, Champagne

, Klazinga

, Kazandjian

, Araph

, Guisset

. A performance assessment framework for hospitals: the WHO regional office for Europe PATH project. Int J Qual Health Care 2005;17:487–96

28.

World Health Organisation Regional Office for Europe. The Vienna Recommendations on Health Promoting Hospitals. Copenhagen: The Organization, 1997

29.

Rootman

. Evaluation in Health Promotion: Principles and Perspectives. Copenhagen: World Health Organization Regional Office for Europe, 2001

30.

Pelikan

, Dietscher

, Krajic

, Nowak

. Core strategies for Health Promoting Hospitals. Gamburg: World Health Organization Regional Office for Europe, 2001

31.

Barbero

. Putting HPH Policy into Action. Barcelona: World Health Organization Regional Office for Europe, 2003

32.

Baggs

, Ryan

, Phelps

, Richeson

, Johnson

. The association between interdisciplinary collaboration and patient outcomes in medical intensive care. Heart Lung 1992;21:18–24

33.

Groene

, Jorgenson

. Health promotion in hospitals: a strategy to improve quality in health care. Eur J Public Health 2005;15:6–8

34.

Pauly

. Nonprofit firms in medical markets. Am Econ Rev 1987;77:257–62

35.

Harding

. Innovations in Health Care Delivery: The Corporatization of Public Hospitals. Baltimore: Johns Hopkins University Press, 2001

36.

Aiken

, Patrician

. Measuring organizational traits of hospitals. Nurs Res 2000;49:149–53

37.

Campbell

, Roland

, Buetow

. Defining quality of care. Soc Sci Med 2000;51:1611–25

38.

Guyatt

, Thompson

, Berman

How should we measure function in patients with chronic heart and lung disease?

J Chronic Dis 1985;38:517–24

39.

World Health Organisation. Standards for Health Promotion in Hospitals: Development of Indicators for a Self-Assessment Tool. Copenhagen: The Organisation, 2004

40.

Rubin

, Pronovost

, Diette

. The advantage and disadvantages of process-based measures of health care quality. Int J Qual Health Care 2001;13:469–74

41.

Blumenthal

. Quality of care – what is it? N Engl J Med 1996 335:891–4

42.

Irvine

. Managing for Quality in General Practice. London: Kings Fund Centre, 1990

43.

Centres for Medicaid and Medicare Services, 2004. See www.cms.hhs.gov/ (last accessed 21 August 2006)

44.

Berger

, Krajic

, Paul

. Health Promoting Hospitals in Practice: Developing Projects and Networks. Gamburg: Health Promotion Publications, 1999

45.

Gould

. Quality improvement in perinatal medicine: assessing the quality of perinatal care. Neoreviews 2004;5:33–41

46.

Anderson

. Making sense of rising caesarean section rates. Br Med J 2004;329:696–7

47.

Rahnama

, Ziaie

, Faghizadeh

. Impact of early admission in labour on method of delivery. Int J Gynaecol Obstet 2006;92:217–20

48.

World Health Organisation. The World Health Report 2000, Health Systems: Improving Performance. Geneva: The Organisation, 2000

49.

Graber

, Phelps

. Economic foundations of cost-effectiveness analysis. J Health Econ 1999;16:1–31

50.

Gerard

, Mooney

. QALY league tables: handle with care. Health Econ 1993;2:59–64

51.

Ibrahim

, Majoor

, Boyce

, McNeil

. Pilot Hospital Clinical Indicators Project. Canberra: Commonwealth Department of Health and Family Services, 1998

52.

Montoya-Aguilar

. Measuring the Performance of Hospitals and Health Centres. Geneva: World Health Organization, 1994

53.

McElduff

, Lyratzopoulos

, Edwards

, Heller

, Shekelle

, Roland

. Will changes in primary care improve health outcomes? Modeling the impact of financial incentives introduced to improve quality of care in the UK. Qual Saf Health Care 2004;13: 191–7

54.

Agency for Healthcare Research and Quality. Quality Indicators-Guide to Inpatient Quality Indicators: Quality of Care in Hospitals-Volume, Mortality, And Utilization. Rockville: Agency for Healthcare Research and Quality, 2004

55.

Holland

. The European Community Atlas of Avoidable Deaths. NC: Oxford University Press, 1997