Abstract
Surgeon experience certainly improves their technical efficiency although it also causes physiological changes with aging. The authors hypothesized that surgeons’ technical efficiency improves with increasing experience up to a point where it then decreases, which is a concave relationship. The authors collected data from all the surgical procedures performed at University Hospital from April through September in 2013–19. The dependent variable was defined as surgeons’ technical efficiency scores that were calculated using output-oriented Charnes–Cooper–Rhodes model of data envelopment analysis. Inputs were defined as (1) the number of assistants and (2) the duration of surgical operation. The output was defined as the surgical fee for each surgery. Surgeon experience was defined as the number of years since medical school graduation. Five control variables were selected: surgical volume, gender, academic rank, surgical specialty, and the year of surgery. Multiple regression analysis using pooled and random-effects Tobit models was performed for our panel data. Totally 20,375 surgical procedures performed by 264 surgeons in 42 months were analyzed. The coefficients of experience and the square of experience were not significantly different from zero. The other coefficients were also insignificant. Surgeons’ technical efficiency does not have a concave relationship with experience.
Introduction
Surgeon experience concerns not only patients but also healthcare managers. 1 Surgeon experience, which was measured either by age or years since completion of training, has been examined as a predictor of clinical outcomes with varying results. 2 Some studies have found better outcomes with an increasing number of years of experience while other studies have described worse outcomes among oldest providers.3–6 For example, obstetricians with more years of experience had fewer maternal complications. 3 On the other hand, very senior surgeons had higher morbidity and mortality in congenital heart surgery. 1 In esophagectomy for cancer, surgeons age 52–55 years had lower mortality. 4 There was no statistically significant association between surgeon age and patient outcome in bariatric surgery. 5 One of the reasons outcomes suffer is surgeons’ technical inefficiency. Healthcare managers also concern about surgeon experience from the viewpoint of surgeons’ technical efficiency because they should know how efficiently surgeons can utilize healthcare resources. Although surgeon experience is an important determinant of operative efficiency in cardiac surgery, 7 the relationship between surgeon experience and technical efficiency has not been well established because it is difficult to quantitatively define surgeons’ technical efficiency.
Despite the difficulty of definition of technical efficiency, it is a common practice in Japan for university hospitals to set a mandatory retirement age threshold at 65, probably with a primary emphasis on technical efficiency. This practice may be counterproductive because the longevity of Japanese population is increasing and the shortage of surgeons may become problematic in the near future. Moreover, medical students who want to become surgeons are currently decreasing because of long surgical training period and short active clinical career. 8
One of the quantitative definitions of technical efficiency is derived from microeconomic theory. Technical efficiency is an economic term which is in contrast to allocative efficiency. Technical efficiency reflects the ability of a decision making unit (DMU) to produce the maximum amount of output given a set of inputs while allocative efficiency reflects the ability of a DMU to use inputs in optimal proportions given their respective prices. 9 Generally speaking, senior surgeons are responsible for the technical efficiency while healthcare managers are responsible for the allocative efficiency in healthcare settings. Data envelopment analysis (DEA) is a methodology for technical efficiency that takes account of multiple inputs and multiple outputs. DEA has been widely used to measure technical efficiency of various healthcare entities.10,11 Individual surgeons’ technical efficiency can also be measured by efficiency scores calculated from DEA.11,12 As surgeons gain experience, they also suffer physiological changes with aging although their longer experience improves their technical efficiency.
We hypothesized that surgeons’ technical efficiency improves with increasing experience up to a point where it then decreases, which is modeled as a quadratic function of experience with a parabolic shape (concave relationship). Specifically, we attempt to identify the experience when technical efficiency peaks.
Methods
The Teikyo University Institutional Review Board approved our study.
Data
Teikyo University Hospital is located in the metropolitan Tokyo, Japan, serving a population of ∼1,000,000. It has 1152 beds and has a surgical volume of approximately 9000 cases annually. It has 13 surgical specialty departments. We collected data from all the surgical procedures performed in the main operating rooms of Teikyo University Hospital from April 1 through September 30 in 2013–2019. We collected data for only 6 months each year because of our time and budget constraints. One of the authors (Y.N.) looked up in person all the surgical data and extracted the necessary information. The necessary information was obtained from the Teikyo University Hospital electronic medical record system.
Exclusion criteria for surgery were as follows. First, surgical procedures performed under local anesthesia by surgeons were excluded to equalize resource utilization. They were minor surgeries performed without anesthesiologists’ involvement, and are different in resource utilization. Second, oral and dermatologic surgical procedures were excluded because most of their cases were minor surgeries performed under local anesthesia, and those under general anesthesia do not represent the activity of their surgeons. Third, the surgical procedures were excluded if the patients die within 1 month after surgery to maintain a constant quality outcome of surgery. It is true that the quality of the surgical outcome is not constant just because the patient did not die within 1 month after surgery. However, we analyzed different kinds of surgical procedures. Death was the only available outcome measure that is common to all different kinds of surgery although we understand that it is imperfect. Fourth, the surgical procedures which were not reimbursed under the surgical payment system were excluded. For example, prostatectomy using da Vinci System was not covered by the surgical payment system in Japan before 2016 and was excluded from our analysis. Fifth, the surgical procedures were excluded if their records were incomplete for any reason.
Technical efficiency (dependent variable)
The method to calculate surgeons’ technical efficiency was similar to that described in our previous studies.12,13 A decision making unit (DMU) is defined as the entity that is regarded as responsible for converting inputs into outputs in DEA. 14 We defined in this study the DMU as a surgeon with the highest academic rank that scrubbed in the surgery (senior surgeon).
Since the selection of inputs and outputs often drives the DEA model, it is important to develop a justification for selecting inputs and outputs. In this analysis we focused on the surgeons’ activity and their clinical decision. We assumed surgeons’ production function in a microeconomic model. Inputs were defined as (1) the number of medical doctors who assisted surgery (assistants) and (2) the duration of surgical operation from skin incision to skin closure (surgical duration). Assistants are considered to be a proxy variable for labor input. Surgical duration represents capital input because surgeons utilize operating rooms equipped with surgical instruments during surgery. Using these labor and capital inputs, senior surgeons produce surgical fees for the hospital. Therefore, the output was defined as the surgical fee for each surgery. Japan has maintained a universal health insurance system and most health care providers are reimbursed on a fee-for-service basis according to the fee schedule that set prices uniformly at the national level. The same fee schedule is enforced for all plans and all the surgeons studied. 15 It is classified as K000-K915 in the Japanese surgical fee schedule, and is called “K codes.” Each surgical procedure is assigned to one of the K codes which correspond with surgical fees.16–19 The fee is identical regardless of who (an experienced surgeon or a surgical trainee) performs surgery as long as they have medical licensure, how many assistants they use, or how long it takes to complete surgery. The additional reimbursements for expensive surgical devices, such as auto suture devices or imaging navigation devices, were excluded. Other fees for blood transfusion, medications, special insurance medical materials and anesthesia were also excluded.
All the inputs and output are under the control of a DMU. Senior surgeons can choose assisting doctors, usually surgical trainees, and decide how many of them to assist their surgery (assistants). They also decide how long it takes to complete surgery (duration). The senior surgeons can choose from several options of surgeries although restricted by the patients’ condition and their consents. The choice of surgery determines surgical fees.
We employed output-oriented Charnes–Cooper–Rhodes (CCR) model of DEA under the constant returns-to-scale assumptions, which was particularly relevant because of its ability to employ multiple inputs and outputs and does not require an a priori specification of a function. 20 We chose a CCR model because there are economic reasons for expecting that a surgeons’ production function exhibits constant returns to scale. Surgeons are taught that similar patients with common conditions should be taken through the same clinical process. Thus, if surgeons want to treat twice as many patients with a common diagnosis, they would expect to use twice as many resources; consequently, scaling up the quantity of patients should result in doubling of the inputs. 21 We chose output orientation because the surgeons can choose from several options of surgeries and the choice of surgery determines surgical fees.
We added all the inputs and outputs of the surgical procedures for each DMU in each year during the study period. By totaling the inputs and output in each year, we can minimize the variation of each patient’s condition and surgery, and know surgeons’ technical efficiency over the year. Otherwise, every patient is different in the difficulty of surgery, and surgeons’ technical efficiency is impossible to determine. As mentioned above, the total surgical duration represents capital utilization, and the total number of assistants represents labor input in each year. We then calculated his/her efficiency scores using DEA-Solver-Pro Software (Saitech, Inc., Tokyo, Japan). 22 The efficiency scores all lie between 0 and 1, and the most efficient surgeons are given the score of 1. All the surgeons in the sample are given an efficiency score for each in each year. 9 We collected panel data for seven study periods.
All the surgeons analyzed were employees of Teikyo University, and were salaried according to their ranks and experiences without any monetary incentives to increase surgical volume or efficiency. The hospital charges surgeons’ surgical fees to Health Insurance Claims Review and Reimbursement Services, and the reimbursement becomes the revenue of the hospital. It pays to surgeons their salary from a part of this revenue. They belong to one of the following 12 surgical specialties; orthopedics, general surgery, emergency surgery, obstetrics and gynecology, otorhinolaryngology, neurosurgery, plastic surgery, urology, ophthalmology, cardiovascular surgery, thoracic surgery, and pediatric surgery.
Independent variables
We selected six independent variables that are available to us and may predict surgeons’ efficiency scores.
Experience
Most surgeons analyzed in this study publish their years of graduation from medical schools in directories and/or websites.23–27 Surgeon experience was defined as the number of years since medical school graduation on the date of surgical procedure. Those who do not obtain medical license upon graduation from medical school are rare.
Surgical volume
Surgical volume was defined as the number of surgical cases that a surgeon performed during the six-month study period in each year. This information was extracted from the Teikyo University Hospital electronic medical record system.
Gender and academic rank
Teikyo University Hospital makes surgeons’ gender and academic ranks publicly available in its website for patients’ convenience. 27 We assigned a dummy variable of gender, with female = 1 and male = 0.
Some surgeons were promoted during the study period of 7 years. Academic ranks were defined as ones when the surgery was performed. We assigned a dummy variable of academic ranks of DMUs, with full professor = 1, associate professor = 2, and otherwise = 0. The group of “otherwise” was regarded as a reference.
Surgical specialty and year of surgery
Orthopedics has the largest number of observations in our sample. We assigned orthopedics as a reference, and defined 11 dummy variables to encode 12 surgical specialties.
The year of 2013 is the oldest in our observations. We made the year of 2013 as a reference, and defined six dummy variables to encode 7 years of surgery.
Statistical analysis
We used Stata Data Analysis and Statistical Software (Stata 14, StataCorp LP, College Station, Texas, U.S.A.) for our statistical analysis. We modeled efficiency scores as a function of experience and the square of experience. We performed multiple regression analysis using both pooled and random-effects Tobit models for our data.28,29 The efficiency score is a limited dependent variable that lies within the range of 0–1, and its distribution is best described by a censored normal distribution. 10 The appropriate regression model to use when the dependent variable has a censored distribution is the Tobit model. The right-censoring limit was set at 1. 10 We used pooled and random-effects models because it allows for independent variables that are constant over time in panel data. 30 We omitted a fixed-effects model because some of our independent variables (gender, surgical specialty) are constant over time and they would drop out of the fixed-effects analysis. 30 We previously conducted the Hausman test using the similar dependent and independent variables. 31 The Hausman test failed to reject the null hypothesis that there were no statistically significant differences in the coefficients between random-effects and fixed-effects models. The assumption of a random-effects model that the unobserved effect is uncorrelated with each independent variable was not false. Therefore, it is appropriate to use a random-effects model in this analysis. 31
We also calculated the average marginal effects of independent variables by Delta-methods. This method gives us the partial derivatives of expected value of dependent variable with respect to the vector of independent variables. 32 A p-value < 0.05 was considered statistically significant.
Results
We analyzed total 20,375 surgical cases in 42 months study period from 2013 through 2019, which added up to 1037 observations. Efficiency scores were calculated for all of the surgeons in each year.
Characteristics of dependent and independent variables.
Data are presented as mean ± SD (range) or absolute values (%). n is a number of observations.
Scatter plot of surgeons’ technical efficiency score and experience.
Results of multiple regression analysis using pooled Tobit model for panel data.
The number of observations was 879. Data are presented as mean ± standard error. * indicates that the coefficient is significantly different from zero (p < 0.05).
Average marginal effects by Delta-methods in the pooled Tobit model.
The number of observations was 879. Data are presented as mean ± standard error. * indicates that the marginal effect is significantly different from zero (p < 0.05).
Results of multiple regression analysis using random-effects Tobit model for panel data.
The number of observations was 879, and the number of surgeons analyzed was 264. Data are presented as mean ± standard error. * indicates that the coefficient is significantly different from zero (p < 0.05).
Average marginal effects by delta-methods in the random-effects Tobit model.
The number of observations was 879, and the number of surgeons analyzed was 264. Data are presented as mean ± standard error. * indicates that the marginal effect is significantly different from zero (p < 0.05).
Discussion
From our pooled and random-effects Tobit models multiple regression analysis, our hypothesis that surgeons’ technical efficiency improves with increasing surgeon experience up to a point where it then decreases was found to be wrong. Surgeons’ technical efficiency was not modeled as a quadratic function of experience with a parabolic shape, nor was a concave relationship with experience. It rather followed a small but statistically significant positive linear relationship with experience, as demonstrated in our average marginal effects analysis. Surgeons’ gender, surgical volume, or academic ranks did not have any significant predictive values for their technical efficiency. There was no peak of technical efficiency during the experience of 2–44 years. This is the first study that evaluated the relationship between surgeons’ technical efficiency and experience. Judging from our results, we can argue against mandatory retirement age threshold with a primary emphasis on technical efficiency. This finding suggests an important policy implication that the government regulatory bodies should not enforce strict age cut-offs for retirement for surgeons because technical efficiency cannot be modeled as a quadratic function of experience with a parabolic shape. The retirement age threshold must be individualized based on individual surgeons’ technical efficiency and clinical abilities.
Surgical volume did not predict surgeons’ efficiency, either. We studied heterogeneous surgery that represented 12 surgical specialties. The number of surgical cases may not reflect surgeons’ activities. For example, cardiac surgery usually lasts for several hours while ophthalmologic surgery lasts less than an hour. These difference in the surgical duration might have caused our insignificant coefficient.
Gender also did not predict surgeons’ efficiency although the ratio of female surgeon is small in our study. Academic rank did not predict surgeons’ efficiency although the promotion criteria of clinicians at Teikyo University Hospital includes clinical excellence. This is because the clinical excellence has rarely been quantitatively evaluated at promotion. Our method of evaluating efficiency may be useful to determine surgeons’ appropriate academic ranks.
There are some limitations in our study. First, this is a study conducted in a single large teaching hospital in Tokyo, Japan. Our surgeons may not represent all the surgeons. However, there is an advantage of studying surgeons’ efficiency in a single hospital. Since one of the significant resource inputs is ancillary services such as operating room nursing practices and availability of support personnel, all these factors are held constant in a single hospital. Comparing surgeons in different hospitals can be misleading if some ancillary services are more efficient than others. By comparing surgeons in the same institution, they all face the same systemic advantages and disadvantages of ancillary services. 21 Second, our sample was limited with the maximum experience of 44 years. It is possible that technical efficiency might be a quadratic function of experience with a parabolic shape if we could collect more samples from surgeons with over 44 years of experience. However, such detailed data were unavailable because no surgeons with over 44 years of experience have actively performed surgery for the last 7 years in our hospital. Third, there might be a selection bias in our sample. If surgeons have become technically inefficient, they may voluntarily stop performing surgery before their surgical outcomes deteriorate. Fourth, we made technical efficiency as a dependent variable. It is true that successful surgery requires not only technical efficiency but also clinical judgment and sustained attention. 1 The outcome of surgery is more important than technical efficiency for the patients. However, the present study is the only one that evaluated all the surgical specialties. It is impossible to evaluate surgical outcomes by a single standard. Therefore, we focused on technical efficiency which represents all different types of surgical procedures. Fifth, we assumed that the year the surgeons graduated is the year they obtained their professional qualification. We ignored the gap between surgeons among different degrees, such as master’s degree and doctor’s degree, as well as the lifetime of those who have worked for a few years and then studied for a doctor’s degree. However, most Japanese surgeons obtain medical license upon graduation from medical schools unless they fail in the national medical licensure examination. Although some surgeons pursue master’s or doctor’s degrees, they usually further their study in their surgical specialties after obtaining their medical licenses. It is reasonable to count this period as surgeons experience. Sixth, we used Tobit models in our statistical analysis although an ordinary least square model has a better effect in the two-stage DEA study of efficiency measurement and influencing factor analysis. 33 However, according to Hollingsworth and Peacock, the efficiency score produced by DEA does not represent a true continuous variable since it lies within the range of 0–1. This violates assumptions of the classic linear regression models and makes estimates derived from an ordinary least square regression inconsistent. 10 It is reasonable to use Tobit models in this analysis. Seventh, although DEA has the advantage of simultaneously evaluating multiple inputs and outputs and not assuming an a priori specification of a function, it also has some disadvantages. For example, the choice of inputs and outputs depends on data availability and clinical observation. 34 Therefore, we do not reject the idea that there might be a better model than ours. Eighth, we used simple numbers of assistants rather than man-hours of assistant input as a labor input in our microeconomic model. It is true that assistants worked for different durations according to the variable duration of surgical cases. However, the number and member of assistants changed even during one surgical case. For example, in the real clinical settings, it is usual that Assistants A and B worked in the first part of the surgery, Assistants B, C and D worked in the second part, and Assistants A and E worked in the last part during a long surgical case depending on the situation in the surgery. A senior surgeon may show up in the operating room after the surgical exposure was completed by his/her assistants, and may leave the surgical field early by letting the assistants close the skin if he/she thinks that it is safe although he/she is responsible for the outcome of surgery. Only the names of all assistants of the case (Assistants A, B, C, D, and E in the example above) were recorded and available to us; the data on the duration of their individual work was unavailable. Therefore, it was impossible to calculate man-hours of labor input. Although we know that the simple numbers of assistants was imperfect, we used them as a proxy variable for a labor input in our microeconomic model because the more assistants supported a senior surgeon, the more labor input was utilized.
In conclusion, we demonstrated by the pooled and random-effects Tobit model multiple regression analysis that surgeons’ technical efficiency does not have a concave relationship with experience. There does not exist a peak in technical efficiency. Therefore, we argue against mandatory retirement age threshold with a primary emphasis on technical efficiency.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Japan Society for the Promotion of Science (JSPS) KAKENHI (17K09247).
