Three-Stage Data Envelopment Analysis as a Tool for Nurse Leader Performance Appraisals

Abstract

As in other sectors, increasingly higher performance and efficiency are also being required from health care employees. To achieve this goal, every health care organization should have a suitable human resource management system. The selection, education, training, effective performance appraisal, and evaluation of leaders are particularly of key importance for every organization. Data envelopment analysis (DEA) is used in this study to develop a model of practice outputs and inputs to help identify the most efficient nurse leaders. The employees’ performance appraisals are often closely related to their specific organizations. As leaders’ behaviors are also influenced by external, non-discretionary factors, the three-stage DEA was used to include inputs not controlled by individual leaders. This article proposes a performance appraisal based on competency models of leadership for a larger professional group working across several different organizations, yet in similar professional and institutional environments. The empirical data in this article are based on two surveys that were conducted in 15 Slovenian public hospitals.

Keywords

data envelopment analysis hospital performance nurse leader performance Slovenia

Introduction

An efficient and effective health care system significantly contributes to a higher quality of life and a higher level of human development. The effects of a good health care system can be observed in the productivity of the workforce, which in turn contributes to higher economic growth. Therefore, the assurance of a successful health care system should be a priority in any country. Health care expenditures in developed countries comprise 8% to 16% of GDP, whereas employees in the health and social work sector account for 6% to 20% of all employees (Organization for Economic Co-Operation and Development, 2014). The health care systems in the majority of European countries are designed so that the largest portion, in terms of both health service providers and funding, remains within the public sector (Organization for Economic Co-Operation and Development, 2014). The state should use its powers to ensure public sector organizations (including health care organizations) operate in an efficient and effective manner. One of the characteristics of the majority of public sector organizations is that their performance greatly depends on the successful management of human resources.

Employee performance appraisals (PA) can support a wide range of human resource decision-making activities including pay raise, promotion, demotion, termination, and training of employees (Osman, Berbary, Sidani, Al-Ayoubi, & Emrouznejad, 2011). Well-designed and properly used appraisal systems are essential for effective functioning of organizations (Manoharan, Muralidharan, & Deshmukh, 2009).

Research on PA has evolved from a pure focus on how to measure performance to examining the social context where performance appraisal takes place (Prasad, Suar, & Taraban, 2014; Zheng, Zhang, & Li, 2012). Evaluating and ranking the employees working in organizations are challenging tasks involving several factors. Each employee achieves certain performance levels in various factors, and the resulting information can be overwhelming (Manoharan et al., 2009).

The development of PA system is especially important for leaders. Leaders can affect organization performance in different ways. Devising more effective ways of managing the performance of managers and leaders has become a cornerstone of organizational and leadership development in recent years (Cheng, Dainty, & Moore, 2005). Effective management and leadership are critical aspects of well-performing organizations and programs.

One of the most frequently used new approaches in human resources management is the study of competencies and competency models (Cappellen & Janssens, 2008; Wu, 2013). A competence is an action that can be monitored, evaluated, and whose efficiency can be measured. On that basis, we can define a leadership competence model as a system of key competences, designed based on research of key indicators of effective behavior of leaders (Kvas, Seljak, & Stare, 2013). Competencies can only be defined on the basis of observation of the behavior of the individual (actions in real situations; Stare & Seljak, 2013). They have become an important tool for introducing changes in organizations and in changing the behavior of the individual (Plessius & Ravesteyn, 2010).

In the competency approach, we compare employees’ performance against performance standards or desired behaviors (Cheng et al., 2005). Competency based PAs have been seen as a powerful tool to underpin many human resource management activities also in healthcare organizations (Osman et al., 2011; Rosenfeld, Pyc, Rosati, & Marren, 2012). Such assessments can help to define job characteristics and desired levels of performance and hence, can provide a basis for many aspects of the human resource management in health care (Cheng et al., 2005).

Nurse leaders need to be familiar with the operation of the health care system, possess knowledge and skills from the areas of leadership, communication, and employee motivation, and possess the ability to work in a team, ability to solve complex problems, and ability to work with health and other institutions. They usually perform a dual role—a professional and a leadership role, which require additional knowledge and skills (Hogan & Nickitas, 2009; Yoder-Wise, 2011b).

A relatively long chain of leadership hierarchy has been forged in professional groups such as nursing: ranging from the nursing director, leaders of wards, clinics, and sectors at the top level to the charge nurse and team leader on the first-line management level. Not only are people in top positions (top managers and leaders) important in the organization here, but also the entire hierarchical management structure within it (Kvas & Seljak, 2013b; Twedell & Jackson Gray, 2007).

The competency model of leadership in nursing emerged in Slovenia between 2009 and 2011 (Kvas & Seljak, 2011; Kvas et al., 2013). It consists of generic leadership competencies (behaviors or actions) that show the characteristics of all leaders and role-specific competencies that are considered to be characteristic of leaders in nursing. Generic competencies comprise flexibility at work, creativity, leadership, organizational climate, organization, networking/influence, and realization skills.

Based on analysis of the theory of nursing, leadership, and competency models (Biesma et al., 2008; International Council of Nurses, 2012; Yoder-Wise, 2011b), three groups of role-specific competencies were selected:

Ethics: competencies demonstrating the ethical or non-ethical conduct of nurses in leadership positions

Interprofessional relationships: competencies demonstrating correct understanding of the position of nurses in the health care system and their relationships to doctors

A positive attitude toward knowledge and education: competencies demonstrating the attitude of nurses in leadership positions toward their own and their subordinates’ education

A PA can only be successful if it includes as many different aspects and results of employees’ work as possible. Leadership outputs in particular should cover a wide range of performance measures (Osman et al., 2011). The complex nature of the relationship between multiple inputs and multiple outputs involved must be taken into account when developing and applying competency models. The main research problem of our study was how to incorporate a large number of different aspects and results of nurse leaders’ work into a unified system that enables consistent classification. Data envelopment analysis (DEA) is a method that has often been used for performance appraisals based on a large number of inputs and outputs.

The main aim of this study is to assess leader performance in nursing care using a competency-based PA. DEA will be used to

identify the leaders with the highest and lowest performance appraisals,

identify peers for leaders with the lowest performance appraisals,

identify what areas leaders with the lowest appraisals should improve performance and quantify the performance improvement required in these areas, and

compare leadership performance among the 15 hospitals that participated in the study.

Method

Basic DEA

DEA was proposed by Charnes, Cooper, and Rhodes (1978), as a non-parametric method of efficiency analysis to compare the relative performance of units relative to their best peers (efficient frontier) rather than average performers, and to identify benchmarks for inefficient units. The virtual productivity ratio $Θ * = u_{r}^{*} Y_{o} / v_{i}^{*} X_{o}$ is maximized subject to the condition that the virtual ratio Θ of any unit j does not exceed 1, and all the weights are strictly positive. This non-linear model can then be rewritten more explicitly in a linear format as follows (Osman et al., 2011):

{Minimize Θ}_{o} = \sum_{r = 1}^{m} v_{i} x_{i j_{o}},

subject to

\sum_{r = 1}^{s} u_{r} y_{r j_{o}} = 1,

\sum_{r = 1}^{s} u_{r} y_{r j} - \sum_{i = 1}^{m} v_{i} x_{i j} \leq o; j = 1, …, n,

u_{r}, v_{i} \geq ε; i = 1, … m, & r = 1, … s,

where

$Θ_{o}$ is the efficiency score of j₀th decision-making unit (DMU) being assessed in the set of j = 1, . . ., n DMUs,

$ε$ is a non-Archimedean value to enforce strict positivity of the weights,

$y_{rj}$ is the observed amount of rth output produced by DMU j,

$x_{ij}$ is the quantity of ith input used by DMU j,

$u_{r}$ is the weight given to output r as determined by the linear programming, and

$v_{i}$ is the weight given to input i as determined by the linear programming.

Originally, DEA was designed to empirically measure productive efficiency of homogeneous DMUs. When expanding the use of DEA to other areas (public administration, measuring employee efficiency in various organizations, etc.), one should ensure that the precondition of homogeneity of the DMU is met. The efficiency of a DMU is usually influenced by a variety of factors that are difficult to define with a small number of inputs.

DEA is most commonly used for assessing the efficiency of organizations and individuals. The most commonly assessed organizations include banks (Jablonsky, 2012), police departments (Aristovnik, Seljak, & Mencinger, 2013), physician practices (Rosenman & Friesner, 2004), hospitals (Pham, 2011), and nursing homes (Garavaglia, Lettieri, Agasisti, & Lopez, 2011).

DEA-based employee PA or efficiency assessments are used for employees in industry (Tao, 2012), physicians (Chilingerian & Sherman, 2011), and nurses (Osman et al., 2011). Most of these studies assessed efficiency within a single organization or in similar cultural or working environments. To compare employees in different organizations and at different levels of leadership, the analysis must include additional variables to offset the influence of elements outside the control of an individual (or the organization that employs them).

DEA With Non-Discretionary or External Inputs

The basic DEA models assumed that all inputs were discretionary. However, in practice, the results of production processes also depend on the effect of another type of variables that are beyond the DMU control (Cordero, Pedraja, & Santín, 2009). These non-discretionary or external inputs affect the transformation of discretionary inputs into outputs (Ruggiero, 1998). Therefore, the only way the DMU can correctly be evaluated is by taking into account the influence of those external variables on efficiency scores, so that they can be interpreted as an appropriate measure of their performance.

This approach in the first stage estimates efficiency scores with basic DEA program using only the discretionary inputs. This produces a measure of the total inefficiency for the different DMUs comprising contributions both from the non-discretionary variables and non-measurable DMU inefficiency. Subsequently, a regression analysis is used in the second stage to decompose both of them. Therefore, exogenous variables (z_j) are explanatory variables, and the dependent variable is the first-stage efficiency score ( ${\overset{⌢}{Θ}}_{j}$ ; Cordero et al., 2009):

{\hat{Θ}}_{j} = f (z_{j}, β) + μ_{j} .

This regression can be estimated by ordinary least squares, although the use of a Tobit regression is more widespread (Deyneli, 2011). Tobit regression is often used for data that are censored or truncated. Censored data occur when the values of the data are clustered around some threshold, either lower (left-censored) or upper (right-censored; Walker & Maddan, 2008). From the value of estimated coefficients, it is possible to identify the influential variables and their sign (positive or negative) and also to weigh the importance of each external variable in the efficiency estimate (Cordero et al., 2009).

In the third stage, DEA will be conducted using the discretionary and non-discretionary input variables. The newly obtained values are introduced to the original DEA framework as an input, adjusting for the uncontrollable environmental factors through the use of a combined non-discretionary input. This model restricts the comparison set by removing from the potential reference set any DMU that has a more favorable environment than the unit under analysis.

There has been a growing interest in the literature to study how the effect of exogenous variables can be included in efficiency measures in such different sectors as education, banking, or sports (Cordero et al., 2009). We can find articles that have addressed this issue also in the health sector. DEAs with non-discretionary inputs are used to compare nursing homes (Fizel & Nunnikhoven, 1992), hospitals (Chilingerian & Sherman, 2011), or dialysis facilities (Kontodimopoulos, Papathanasiou, Flokou, Tountas, & Niakas, 2011).

Research Design and Participants

The study was part of a larger research project titled “Leaders in Health Care.” Following a literature review and input from an expert group, two questionnaires were designed specifically for the purposes of the research. In the first questionnaire, subordinates assessed the levels of 95 actions of their leaders. In the second questionnaire, leaders assessed what traits and characteristics they possess. Two surveys were conducted within a research project carried out from May 18, 2010, to April 30, 2011.

The survey was conducted in 13 greatest Slovenian general and special hospitals and 2 university medical centers that account for 87% of all hospital health care employees in Slovenia.

The sample from the first survey included 1,311 health care professionals who assessed the level of competencies of their immediate superiors. Of the 1,311 health care professionals, 84.3% were female. The average age of subordinates was 38.5 years; 59.2% had a secondary education, 34.4% postsecondary or professional college, 2.7% bachelor’s degree or higher.

The sample from the second survey included 236 nurse leaders, who were asked to provide information regarding their education and job experience, and assess their knowledge, job satisfaction, organizational commitment, and levels of interpersonal and interprofessional relationships. The data from second survey were used as discretionary inputs. Of the 236 nurse leaders, 7.2% were men. The average age of subordinates was 42.3 years. Those who had a postsecondary education or professional college comprised 88.6%, and those with a bachelor’s degree or higher comprised 11.4%.

The interconnection of the data from first and second surveys was ensured with a coding system. The data for non-discretionary inputs were obtained from hospital records.

Data

DEA can incorporate various types of data. Our study combined interval and ordinal data. Although interval data have been used for DEA since it was developed, the use of ordinal data has become more frequent only recently. Many authors utilized a 5-point Likert-type scale to convert qualitative data into quantitative data for use in evaluations of performance using DEA (Manoharan et al., 2009).

Our study used data from various sources and aspects; therefore, we also needed to prevent a possible data imbalance. One of the best ways of making sure there is not much imbalance in the data sets is to have them at the same or similar magnitude. A way of making sure the data are of the same or similar magnitude across and within data sets is to mean normalize the data. The process to mean normalize is taken in two simple steps. First step is to find the mean of the data set for each input and output. The second step is to divide each input or output by the mean for that specific factor (Sarkis, 2007). Mean normalization was used to normalize data for outputs and discretionary inputs.

Outputs

The focus of our model is to enhance the output, rather than decrease the inputs. The outputs for evaluation that can be controlled by the entities are important, because it is generally not very informative or motivating to be judged on the basis of factors that you cannot control (Bogetoft & Otto, 2011). On the output side, it is important that the specification encompasses the full range of leaders’ behaviors. The competency models are based on the assessment of behavior in actual situations. Contrary to the prevailing method of self-evaluation in measuring leadership behavior, we propose subordinates evaluation. The most reliable assessments of leaders are provided by people who are most familiar with their work—their immediate subordinates. Subordinates assessed 95 actions of their leaders, which were divided into four groups of competencies. This resulted in four outputs: (a) generic competencies of leaders in public sector, (b) ethical competencies, (c) interprofessional relationship, and (d) positive attitude toward knowledge and education (Kvas et al., 2013). Each of the groups of competencies was used as a single output variable, representing an aggregate performance indicator of the level of leadership in a specific area (Table 1).

Table 1.

Descriptive Statistics of Inputs and Outputs.

Variables	Number of nurse leaders	Minimum	Maximum	M	SD
Outputs
Generic competencies^a	236	50.57	135.60	96.44	19.75
Ethics^a	236	49.40	135.10	104.19	17.09
Interprofessional relationship^a	236	58.60	137.50	97.68	17.62
Positive attitude toward knowledge and education^a	236	37.40	133.70	97.86	17.62
Discretionary inputs
Years of postsecondary education	232	2.00	8.00	3.09	0.80
Years of employment	234	1.00	39.00	20.78	9.66
Self-assessment of knowledge^b	234	1.75	5.00	3.46	0.62
Job satisfaction^b	233	1.00	5.00	3.68	0.65
Communication and interpersonal relationships^b	234	2.40	5.00	4.51	0.58
Organizational commitment and commitment to the profession^b	234	2.40	5.00	3.92	0.52
Non-discretionary inputs
Type (level) of leadership position^c	236	1.00	5.00	3.88	1.20
Number of subordinates	220	1.00	82.00	13.13	12.65
Number of health care employees in hospital	236	59.00	3,653.00	1,114.46	1,406.90
Type of hospital^d	236	1.00	3.00	2.07	0.75

Values above 100 signify that the assessed leader exceeds the norm in an individual area, whereas values below 100 signify failure to meet the norm.

Ratings based on 5-point Likert-type scale.

1: team leaders; 2: senior nurses, small department heads; 3: leaders of wards, clinical departments, operational blocks, hospital units, outpatients’ clinic groups; 4: head nurses of clinics/hospital centers; 5: head nurses of hospitals, assistant directors of nursing.

1: small specialist hospitals; 2: middle-sized general hospitals; 3: large university medical centers.

Assessments of each leader and their behavior provided us with information regarding the degree to which they meet the norm defined in the competency profile. The assessed level of competencies of each nurse leader was compared with leadership competency profiles developed within the research of the leadership competency model in nursing care (Kvas & Seljak, 2011).

The relationship between the output variables was estimated using the Pearson’s coefficient. Pearson’s coefficients among the output variables scored between 0.789 and 0.876, p < .01 (Table 2), which means they were lower than the maximum threshold, which in DEA studies was 0.95 (García-Sánchez, 2009). Therefore, all four were included in the analysis.

Table 2.

Correlation Coefficients Between Variables.

	Outputs				Discretionary inputs						Non-discretionary inputs
Variables	1	2	3	4	1	2	3	4	5	6	1	2	3	4
Outputs
Generic competencies	1.00
Ethics	.86	1.00
Interprofessional relationship	.88	.85	1.00
Knowledge and education	.88	.85	.79	1.00
Discretionary inputs
Years of postsecondary education	−.10	.02	−.07	−.02	1.00
Years of employment	−.19	−.15	−.19	−.17	−.02	1.00
Self-assessment of knowledge	.12	.14	.07	.15	.14	.15	1.00
Job satisfaction	.01	.10	−.04	.02	.12	.02	.19	1.00
Communication and relationships	.11	.15	.05	.13	.05	.09	.35	.24	1.00
Organizational commitment and commitment to the profession	.06	.06	.01	.13	.21	−.03	.17	.14	.11	1.00
Non-discretionary inputs
Type (level) of leadership position	.55	.35	.51	.28	−.24	−.23	−.08	−.17	−.03	−.05	1.00
Number of subordinates	−.05	−.03	−.04	.03	.16	.09	.13	.03	−.07	.24	−.10	1.00
Number of health care employees	.07	.10	.08	.08	−.15	.06	.12	−.03	.05	.07	.02	−.06	1.00
Type of hospital	.12	.11	.12	.14	−.10	.03	.09	−.03	.03	.13	.09	.16	.74	1.00

Discretionary inputs

All discretionary inputs are (or could be) influenced by individual leaders. Input data were obtained from a survey submitted to leaders. We defined six indicators, which indicate what traits and characteristics leaders possess and which they presumably utilized for efficient leadership: (a) level of formal education, (b) skills and experience (total years of employment), (c) self-assessment of knowledge, (d) job satisfaction, (e) communication and interpersonal relationships, and (f) organizational culture and commitment to the profession (Hogan & Nickitas, 2009; Kvas & Seljak, 2013a; Westover, Westover, & Westover, 2009).

Pearson’s coefficients among the output variables scored between 0.09 and 0.35. Therefore, all six indicators were included in the analysis.

Non-discretionary inputs

The second phase of the DEA includes non-discretionary inputs that influence leadership performance, but are not directly related to the leader. Therefore, influences not directly linked to an individual but rather to the characteristics of the hospital and their job position needed to be excluded from the first phase of DEA. Non-discretionary inputs represent characteristics of the leader’s job position or the leader’s organization: (a) the type (level) of leadership position, (b) number of subordinates, (c) type of hospital, and (d) number of health care employees.

The correlation between the non-discretionary inputs was low. The only statistically significant correlation arose between the number of health care employees and the type of hospital (r = .737, p < .01). The analysis therefore includes all four variables.

Statistical Analysis

The data were analyzed using Frontier analyst 4.1.0 and SPSS 19.0. Output-oriented DEA model with constant returns to scale is used in this study. Relationships between variables were analyzed using a one-way ANOVA. A significance level of alpha = .05 was used for all statistical tests.

Results

Leaders with an efficiency score of 100 are efficient, and less than that value are categorized as inefficient (Table 3). This efficiency is a relative term, and it is related with a subgroup of individuals with whom the examined DMU is best compared.

Table 3.

Efficiency Scores for Nurse Leaders.

Efficiency score^a	Frequency (n, %)	Sex (% of men)	Age (years)	Years of postsecondary education
100 (efficient)	33 (14.0)	0 (0.0)	38.9	2.84
Less than 100 (inefficient)	203 (84.0)	17 (8.4)	42.9	3.12

Values above 100 signify that the assessed leader exceeds the norm in an individual area, whereas values below 100 signify failure to meet the norm.

A total of 203 (86%) leaders could improve their leadership performance based on their potential. The average age of inefficient leaders is 42.9 years. Seventeen of 203 (8.4%) are men. Thirty-three leaders achieved an efficiency score of 100 and represent a role model for peers displaying lower assessments.

Table 4 shows the basic demographic data and leadership performance appraisals of the two lowest scoring leaders. The performance appraisal values achieved by leaders and possibilities for improvement are shown for each of the four groups of competencies (outputs). The assessment shows the degree to which they would need to improve, based on existing inputs to achieve the performance level of their role model. Similar representations can be compiled for each of the 203 leaders performing below requirements.

Table 4.

Data for Least Efficient Leaders and Their Peers.

	Leader 1^a		Leader 2^a
Efficiency score	38.6		41.6
Demographic data
Sex	Female		Female
Age	40		29
Education	Professional college		Three-year higher education
Years of employment	20		5
Hospital no.	12		13
Outputs and improvements
Output name	Value	Potential improvements (%)	Value	Potential improvements (%)
Generic competencies	65.0	239	50.6	175
Ethics	85.1	159	49.4	182
Interprofessional relationship	68.9	218	58.6	140
Positive attitude toward knowledge and education	63.3	210	48.1	160
Peer	Peer for Leader 1		Peer for Leader 2
Efficiency score	100.0		100.0
Sex	Female		Female
Age	47		36
Education	Three-year higher education		Professional college
Years of employment	28		13
Hospital no.	12		13

Leader 1 achieved a relatively high score in ethics, but scored significantly lower in other areas. This leader has a peer within the same hospital with whom to compare himself or herself against and learn from and exchange experience with.

One of the major advantages of DEA is its ability to show the improvements necessary to achieve efficiency. In the comparison with the peer, Leader 1’s greatest potential for improvement lies in the area of generic competencies. This illustration shows that an increase in outputs does not depend merely on the absolute values of outputs of an individual leader but also on peers in their organization. The logic behind this example is that the improvement process begins with an exchange between higher and lower scoring leaders within an organization.

The second poorest score was achieved by a leader whose assessment showed the lowest results in ethics and attitude toward education. Leader 2 also has a peer within his or her hospital who could help improve Leader 2’s outputs.

We then prepared an overview of leadership performance in all 15 hospitals within the scope of the final research question of the analysis (Table 5). It is in the interest of hospitals to use their leaders as efficiently as possible, for the performance of health care organizations, particularly in the public sector, greatly depends on the efficiency of their leaders.

Table 5.

Efficiency Scores and Potential Improvements for Hospitals.

			Potential improvements (%)
Hospital^a	% of efficient leaders in hospital	Mean of efficient score	Generic competencies	Ethics	Interprofessional relationship	Positive attitude toward knowledge and education
No. 1	0.0	72.3	52.4	45.9	48.1	45.0
No. 2	0.0	70.0	81.4	57.1	78.4	62.3
No. 3	15.4	73.2	77.8	54.7	64.6	52.1
No. 4	0.0	69.9	80.9	54.4	59.3	49.4
No. 5	7.1	76.3	45.5	41.4	45.4	35.8
No. 6	25.0	81.1	45.9	32.4	39.1	27.0
No 7	0.0	71.3	64.4	50.7	71.6	45.7
No. 8	22.2	82.3	40.6	29.7	38.3	27.8
No. 9	24.0	78.8	50.2	38.5	46.1	38.5
No. 10	0.0	83.9	29.3	27.9	32.0	22.5
No. 11	8.7	76.4	48.2	39.3	47.1	37.7
No. 12	27.8	79.9	50.8	38.1	44.3	36.9
No. 13	10.3	75.7	51.6	43.6	52.0	41.4
No. 14	0.0	74.2	68.0	51.3	61.2	39.7
No. 15	0.0	75.8	58.5	41.6	43.9	35.5
All hospitals	14.0	77.7	51.5	39.7	47.7	37.1
ANOVA	F	1.2	1.5	1.2	1.2	1.1
ANOVA	Significance	.303	.098	.286	.245	.317

anonymous for data protection reasons.

This overview shows that eight hospitals employ efficient leaders who could serve as role models for their peers. Hospitals 6, 8, 9, and 12 employed more than 20 % of efficient leaders. The average efficiency score exceeded 80 in only 3 hospitals. In comparison, the two ranking orders of hospital scores show certain differences. Although the hospital with the highest score, Hospital 10, has no efficient leaders, Hospital 12, with 27.8% of all highest scoring leaders, only has an average performance score of 79.9, ranking fourth among all hospitals.

The ANOVA analysis reveals that arithmetic means of efficiency score of hospitals do not show any statistically significant differences. Therefore, we also used the post–ad hoc analysis. The lowest-ranking hospital, Hospital 4, shows statistically significant differences from Hospitals 8 (p = .021), 10 (p = .041), and 12 (p = .020). The highest ranking hospital shows statistically significant differences from Hospitals 2 (p = .049), 4 (p = .030), and 7 (p = .045). Based on these results, Hospital 4 could cooperate with Hospital 8, 10, or 12, and attempt to improve performance by exchanging good practices with them. Similarly, the top-ranking three hospitals could work together and organize leadership seminars or workshops.

Table 5 shows the necessary percentages of improvement for the four groups of competencies. The improvements are necessary in the areas of competencies related to interprofessional relationships (Hospitals 7, 10, and 13) and generic competencies (all remaining hospitals).

Discussion

The poorest score was achieved in the competence area of generic competencies. The lack of generic leadership competencies and the importance of their development in nurse leaders have also been shown in other studies. To achieve better leadership, in the future, nurse leaders will require a higher level of competencies in the areas of strategic thinking, use of financial/quantitative data, quality commitment (Bradley, Maddox, & Spears, 2008), leadership, strategic view (Furukawa & Cunha, 2011), and job planning and assignment and goal setting (Lin, Wu, & White, 2005).

Nurse leaders view nursing care as empathetic and caring work (Boeck, 2014) rather than as a rational decision-making process required from leaders and managers. They are not prepared to fully accept the role of leaders. The problem of role transition has also appeared in other research (Twedell & Jackson Gray, 2007). Nurse leaders even at the lowest leadership level should be aware of the importance of their role as leaders and become active in developing and changing health care practices and leadership (Currie, Tolson, & Booth, 2007). A survey of nurse leaders in Slovenia showed that nurse leaders were aware that they did not have enough management and leadership knowledge—generic leadership competencies (Lorber & Skela Savič, 2011). This indicates that the work of nurse leaders requires new knowledge that expands beyond the borders of specific professional fields. Most continuing education programs for nurses are organized for narrow professional fields of nurses’ work (cardiology, psychology, etc.). In addition to field-specific professional knowledge, other programs also need to be organized that would focus not only on a single topic or field but also on comprehensive education (Yoder-Wise, 2011a). One such area is leadership as it incorporates different scientific fields (human resources management, sociology, psychology, economics, etc.)

The second poorest score was achieved in the competence area of interprofessional cooperation. Leadership in nursing depends on relationships within the professional group of nurses as well as relationships with other professions in the health care sector. These relationships depend on the role of the medical profession in society and the state’s approach to health and health care. In the past, the role of leader in the health care team was assumed by the physician (Hall, 2005). However, nowadays, the successful implementation of team work can only be carried out through equal participation of all professions working in the field of health care in the form of interprofessional and trans-professional teams (Thylefors, Persson, & Hellstrom, 2005).

Nurse leaders would above all focus on interpersonal relationships between nurses and other members of the health care team (West & Lyubovnikova, 2013). Communications of nurse leaders with other leaders are frequent both inside and outside the organization. Best practices in health care teams should therefore comprise good interpersonal and interprofessional relationships (Furukawa & Cunha, 2011; Spicer et al., 2011) that incorporate respect for the work of other team members. A study of other professional groups also showed appropriate communication between team members (and with other stakeholders) to be an important component of team success, being even more important than experience, work history, and education (Stevenson & Starkweather, 2010).

The analysis based on our research provides important information for three groups of stakeholders:

Nurse leaders can use the analysis results as a guideline for which areas they need to improve their competencies. During their studies, nurses adopt professional values and accept learning and training as an integral part of their profession. Based on the PAs, individual leaders can find peers within the organization to serve as role models. The assessment also shows the degree to which they would need to improve to achieve the performance level of their role model.

Less efficient leaders and their role models (or “nurse trainers”) could form study groups in which they could exchange ideas and learn. The highest management structures of hospitals have the task of ensuring the appropriate conditions for nurses to become and remain good leaders. Hospital managers need to ensure that potential of nurse leaders is realized in the form of better leadership and a higher quality of health care.

National Health Service managers: All 15 participating hospitals are included in the public health care system and are, thereby, a part of the public sector. The state as the owner of all public organizations should understand these results as an indication that certain differences in leadership performance exist. Differences exist both among leaders and among hospitals. The state should promote better leadership in nursing care through its representatives in hospital management. The state should also, within its competencies, organize training for nurse leaders in areas showing poorer scores and promote exchanges of good practices among hospitals.

In future, it would be worthwhile conducting the surveys in all Slovenian health care organizations. The study included only nurse leaders in Slovenian public hospitals. Conditions are probably different in smaller health care organizations (health centers or nursing homes) and in private hospitals. A larger sample of leaders would also allow for the inclusion of additional inputs in the DEA.

Further research would also be required in assessing the level of competencies. Assessment of the competencies of leaders by subordinates was used in this study. The ratings would have been different if assessments of superiors or leader self-assessments had been used. Some studies suggest that these estimates could differ significantly (Kvas & Seljak, 2013b). In the future, surveys should be carried out among all three groups and the evaluations compared.

Conclusion

Tasks performed by nurse leaders are similar, as are their cultural and working environments. Nurses are a professional group, and a characteristic of professional groups is that one does not become a member solely on the basis of education and training but also by sharing the values of the professional group. The employees’ PAs are often closely related to their specific organizations. This article proposes a PA based on competency models of leadership for a larger professional group working across several different organizations, yet in similar professional and institutional environments.

Leaders’ PAs are also influenced by the characteristics of the job position and organization. By including non-discretionary inputs, the analysis also takes into account these additional factors. To assess the PA, this research used the three-stage DEA, which incorporated both discretionary inputs—inputs related to leaders’ characteristics, which they can influence—and non-discretionary inputs on which leaders have no influence, at least not in the short term. This enabled us to ensure better comparability among leaders employed in different organizations and improved the homogeneity of the DMU. The calculated values of leadership performance allow comparisons among leaders employed in different organizations.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research and/or authorship of this article.

Author Biographies

Janko Seljak is an independent researcher at the Faculty of Administration, University of Ljubljana, Slovenia. His research focuses on statistics, performance indicators, public sector, and health care.

Andreja Kvas is the chair of Nursing Division at the Faculty of Health Sciences, University of Ljubljana, Slovenia, and vice president of the Nurse and Midwifery Organization of Ljubljana. Her research interests include health care, education, public sector, and leadership.

References

Aristovnik

Seljak

Mencinger

(2013). Relative efficiency of police directorates in Slovenia: A non-parametric analysis. Expert Systems With Applications, 40, 820-827. doi:10.1016/j.eswa.2012.08.027

Biesma

G. R.

Pavlova

Vaatsra

Van Merode

G. G.

Czabanowska

Smith

Groot

(2008). Generic versus specific competencies of entry-level public health graduates: Employers’ perceptions in Poland, the UK, and the Netherlands. Advances in Health Sciences Education, 13, 325-343. doi:10.1007/s10459-006-9044-0

Boeck

P. R.

(2014). Presence—A concept analysis. SAGE Open, 4(1). doi:10.1177/2158244014527990

Bogetoft

Otto

(2011). Benchmarking with DEA, SFA, and R. New York, NY: Springer.

Bradley

L. A.

Maddox

Spears

(2008). Opportunities and strategies for nurse leader development: Assessing competencies. Nurse Leader, 6(3), 26-33. doi:10.1016/j.mnl.2008.04.001

Cappellen

Janssens

(2008). Global managers’ career competencies. Career Development International, 13, 514-537. doi:10.1108/13620430810901679

Charnes

Cooper

Rhodes

(1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 3, 429-444.

Cheng

M.-I.

Dainty

A. R. J.

Moore

D. R.

(2005). Towards a multidimensional competency-based managerial performance framework: A hybrid approach. Journal of Managerial Psychology, 20, 380-396. doi:10.1108/02683940510602941

Chilingerian

A. J.

Sherman

H. D.

(2011). Health-care applications: From hospitals to physicians, from productive efficiency to quality frontiers. In Cooper

W. W.

Seiford

L. M.

Zhu

(Eds.), Handbook on data envelopment analysis (Vol. 164, pp. 445-493). New York, US: Springer.

10.

Cordero

J. M.

Pedraja

Santín

(2009). Alternative approaches to include exogenous variables in DEA measures: A comparison using Monte Carlo. Computers & Operations Research, 36, 2699-2706. doi:10.1016/j.cor.2008.11.019

11.

Currie

Tolson

Booth

(2007). Helping or hindering: The role of nurse managers in the transfer of practice development learning. Journal of Nursing Management, 15, 585-594. doi:10.1111/j.1365-2834.2007.00804.x

12.

Deyneli

(2011). Analysis of relationship between efficiency of justice services and salaries of judges with two-stage DEA method. European Journal of Law and Economics, 34(3), 1-17. doi:10.1007/s10657-011-9258-3

13.

Fizel

J. L.

Nunnikhoven

T. S.

(1992). Technical efficiency of for-profit and non-profit nursing homes. Managerial and Decision Economics, 13, 429-439. doi:10.1002/mde.4090130507

14.

Furukawa

P. d. O.

Cunha

I. C. K.O

. (2011). Profile and competencies of nurse managers at accredited hospitals. Revista Latino-Americana de Enfermagem, 19, 106-114. doi:10.1590/S0104-11692011000100015

15.

Garavaglia

Lettieri

Agasisti

Lopez

(2011). Efficiency and quality of care in nursing homes: An Italian case study. Health Care Management Science, 14, 22-35. doi:10.1007/s10729-010-9139-2

16.

García-Sánchez

I.-M.

(2009). Measuring the efficiency of local police force. European Journal of Law and Economics, 27, 59-77. doi:10.1007/s10657-008-9079-1

17.

Hall

(2005). Interprofessional teamwork: Professional cultures as barriers. Journal of Interprofessional Care, 19(Suppl. 1), 188-196. doi:10.1080/13561820500081745

18.

Hogan

A. M.

Nickitas

M. D.

(2009). Prentice Hall Reviews and Rationales: Nursing leadership and management. Upper Saddle River, NJ: Pearson Prentice Hall.

19.

International Council of Nurses. (2012). Code of ethics for nurses. Retrieved from http://www.icn.ch/images/stories/documents/about/icncode_english.pdf

20.

Jablonsky

(2012). Multicriteria approaches for ranking of efficient units in DEA models. Central European Journal of Operations Research, 20, 435-449. doi:10.1007/s10100-011-0223-6

21.

Kontodimopoulos

Papathanasiou

Flokou

Tountas

Niakas

(2011). The impact of non-discretionary factors on DEA and SFA technical efficiency differences. Journal of Medical Systems, 35, 981-989. doi:10.1007/s10916-010-9521-0

22.

Kvas

Seljak

(2011). Foundations for implementation of leadership competency model in nursing. In Kolnik

Š. T.

Majcen

D. S.

Klemenc

(Eds.), Nurses and midwives contributing to access and equality in the health services (in Slovenian) (pp. 123-127). Ljubljana, Slovenia: Zbornica zdravstvene in babiške nege Slovenije - Zveza strokovnih društev medicinskih sester, babic in zdravstvenih tehnikov Slovenije.

23.

Kvas

Seljak

(2013a). Assessment of leadership competencies in the field of ethics for healthcare leaders. Slovenian Nursing Review, 47(1), 8-17.

24.

Kvas

Seljak

(2013b). Views of the Slovenian nursing profession regarding leadership. Journal of Health Sciences, 3(1), 30-37. Retrieved from http://jhsci.ba/docs/archives/2013-1/05-Kvas.pdf

25.

Kvas

Seljak

Stare

(2013). The use of competency models to assess leadership in nursing. Iranian Journal of Public Health, 42, 988-995. Retrieved from http://ijph.tums.ac.ir/index.php/IJPH/article/view/4137.pdf

26.

Lin

L.-M.

J.-H.

White

L. P.

(2005). Managerial activities and skills of nurse managers: An exploratory study. Hospital Topics, 83(2), 2-9. doi:10.3200/htps.83.2.2-9

27.

Lorber

SkelaSavič

(2011). Perceptions of managerial competencies, style, and characteristics among professionals in nursing. Croatian Medical Journal, 52, 198-204. doi:10.3325/cmj.2011.52.198

28.

Manoharan

T. R.

Muralidharan

Deshmukh

S. G.

(2009). Employee performance appraisal using data envelopment analysis: A case study. Research and Practice in Human Resource Management, 17, 92-111. Retrieved from http://rphrm.curtin.edu.au/2009/issue1/employee.html

29.

Organization for Economic Co-Operation and Development. (2014). Health status. Retrieved from http://stats.oecd.org/index.aspx?DataSetCode=HEALTH_STAT

30.

Osman

I. H.

Berbary

L. N.

Sidani

Al-Ayoubi

Emrouznejad

(2011). Data envelopment analysis model for the appraisal and relative performance evaluation of nurses at an intensive care unit. Journal of Medical Systems, 35, 1039-1062. doi:10.1007/s10916-010-9570-4

31.

Pham

T. L.

(2011). Efficiency and productivity of hospitals in Vietnam. Journal of Health Organization and Management, 25, 195-213. doi:10.1108/14777261111134428

32.

Plessius

Ravesteyn

(2010). The paradox of more flexibility in education: Better control of educational activities as a prerequisite for more flexibility. In Reynolds

Turcsányi-Szabo

(Eds.), Key competencies in the knowledge society (pp. 301-309). Berlin, Germany: Springer.

33.

Prasad

H. C. S.

Suar

Taraban

(2014). Antecedents and moderators of software professionals’ performance. SAGE Open, 4(1). doi:10.1177/2158244014521436

34.

Rosenfeld

Pyc

L. S.

Rosati

R. J.

Marren

J. M.

(2012). Developing a competency tool for home health care nurse managers. Home Health Care Management & Practice, 24, 5-12. doi:10.1177/1084822311416499

35.

Rosenman

Friesner

(2004). Scope and scale inefficiencies in physician practices. Health Economics, 13, 1091-1116. doi:10.1002/hec.882

36.

Ruggiero

(1998). Non-discretionary inputs in data envelopment analysis. European Journal of Operational Research, 111, 461-469. doi:10.1016/S0377-2217(97)00306-8

37.

Sarkis

(2007). Preparing your data for DEA. In Zhu

Cook

D. W.

(Eds.), Modeling data irregularities and structural complexities in data envelopment analysis (pp. 305-320). New York, NY: Springer.

38.

Spicer

J. G.

Guo

Liu

Hirsch

Zhao

Holzemer

(2011). Importance of role competencies for Chinese directors of nursing based on the forces of magnetism. Journal of Nursing Management, 19, 153-159. doi:10.1111/j.1365-2834.2010.01182.x

39.

Stare

Seljak

(2013). Vodenje ljudi v upravi (2. ponatis) (Leadership in administration (2nd ed.). Ljubljana, Slovenia: Faculty for Administration.

40.

Stevenson

D. H.

Starkweather

J. A.

(2010). PM critical competency index: IT execs prefer soft skills. International Journal of Project Management, 28, 663-671. doi:10.1016/j.ijproman.2009.11.008

41.

Tao

(2012). Multi-department employee performance evaluation based on DEA cross efficiency. Journal of Emerging Trends in Economics and Management Sciences, 3, 553-558.

42.

Thylefors

Persson

Hellstrom

(2005). Team types, perceived efficiency, and team climate in Swedish cross-professional teamwork. Journal of Interprofessional Care, 19, 102-114. doi:10.1080/13561820400024159

43.

Twedell

M. D.

JacksonGray

(2007). Role transition. In Yoder-Wise

S. P.

(Ed.), Leading and managing in nursing (pp. 515-529). St. Louis, MO: Mosby.

44.

Walker

Maddan

(2008). Statistics in criminology and criminal justice: Analysis and interpretation. Sudbury, MA: Jones & Bartlett Learning.

45.

West

M. A.

Lyubovnikova

(2013). Illusions of team working in health care. Journal of Health Organization and Management, 27, 134-142. doi:10.1108/14777261311311843

46.

Westover

J. H.

Westover

A. R.

Westover

(2009). Enhancing long-term worker productivity and performance: The connection of key work domains to job satisfaction and organizational commitment. International Journal of Productivity and Performance Management, 59, 372-387. doi:10.1108/17410401011038919

47.

J.-L.

(2013). The study of competency-based training and strategies in the public sector: Experience from Taiwan. Public Personnel Management, 42(2), 259-271. doi:10.1177/0091026013487124

48.

Yoder-Wise

P. S.

(2011a). Continuing professional development: Taking the broad view. The Journal of Continuing Education in Nursing, 42(2), 51-51. doi:10.3928/00220124-20110124-01

49.

Yoder-Wise

P. S.

(2011b). Leading and managing in nursing (5th ed.). St. Louis, MO: Mosby.

50.

Zheng

Zhang

(2012). Performance appraisal process and organizational citizenship behavior. Journal of Managerial Psychology, 27, 732-752. doi:10.1108/02683941211259548