Abstract
Physician skill assessment occurs at the individual and national level. National regulations are assessed by maintenance of certification and maintenance of licensure, which are on rocky ground these days. The original intent of the maintenance certification system was to assure patient safety and quality, lifelong learning, and commitment to continue in professional development. However, many feel like the process has become cumbersome, burdensome, and expensive, and may not correlate to better patient outcomes. Thus, the ability to assess surgeons at the individual and hospital level, which provides privileges, is important to discuss.
Physicians are practicing for longer durations of time, and surgeons specifically are often operating past 65 years of age. However, there are no metrics by which to assess a surgeon's skills and who should be performing these assessments. The purpose of this discussion is to engage two experts in the field—an educator and an orthopedic surgeon—to discuss this topic actively.
Surgeons should be assessed from a tri-flange approach, with the goal of optimizing outcomes for patient care. The first part is a biomedical analysis of the surgeon, the second part is an observation of the surgeon's skill, and the final part is an assessment of a surgeon's outcomes and complications. Similarly, one can test a concert violinist's strength and cognitive powers, but the ultimate test is to hear the violinist play
The biomedical analysis, which is an assessment of the physician's health, must demonstrate that a surgeon is healthy enough to perform the arduous task of reconstructing the human body, and this should be performed by a medical evaluation. An internist or a cardiologist should assess a surgeon's cardiovascular system to determine whether the surgeon is able to do difficult, intricate surgical procedures. An ophthalmologist should assess the surgeon to determine whether the surgeon has the ocular ability to perform surgery. Finally, a neurologist should assess the surgeon to ensure that the surgeon has good cognitive function and has reasonable judgment. The surgeon's memory must be good, and the surgeon should not be operating if he/she had a stroke or central nervous system damage that would preclude good neurologic function during surgery.
The second assessment is to observe the surgeon. There is no substitute for the practiced eye of a sub-specialist in the same field watching a surgeon operate, and this neutral observer should be neither a friend nor a foe. At a certain age, it is worthwhile to have a peer who performs the same surgery observe the surgeon operate. This observer can evaluate surgical speed, average blood loss, and ability to perform surgery safely.
The final assessment is a mathematical approach where one evaluates the outcomes generated by the surgeon, which include complication rates, mortality rates, and readmission rates.
When assessing surgeons, I think we should also consider a surgeon's cognitive abilities, including their critical thinking, decision making, communication skills, and teamwork skills. I would propose that in addition to observation, we need to move to a competency-based model where we look at performance competencies in each of the cognitive, affective, and technical domains in a skills laboratory environment. We should have agreed-upon performance metrics so that we are also evaluating individual surgeons against performance metrics, and we are using those metrics to give feedback and coaching and to help them achieve proficiency in complex procedures. This is driven nationally by the concerns for the general public related to quality and safety in healthcare, by the cost of healthcare, and by the numbers of medical errors that are still rising in most specialties.
I believe that the use of simulation-based education with defined metrics should be utilized where we can teach tasks, skills, procedures, and complete procedures, and have evaluation tools that we can use to measure performance so that we can give feedback. This structured feedback on performance can assure that we have standardization of performance.
I am an orthopedic surgeon, and that defines me as a simple-minded guy. This is such a complex field that you want to make the analysis about safety, which is the most important and objective metric for evaluation. Other areas, such as courage, judgment, and strength are all important, but may be more difficult to assess. Methods such as case review, observational analysis, and medical evaluations are readily available in every department in the country, and that is important—it will limit the ability to assess surgeons if you need a million-dollar tool to evaluate people. A skillful surgeon should be defined as an individual who operates quickly and has less blood loss, low readmission rates, low infections, and low mortality rates, which gives someone a very objective picture. Now, it does not tell you about judgment, which is a separate issue.
It is like looking at a nationally-ranked swimmer. You can measure their quadriceps size, muscle twitch, strength, the length of their legs, and flexibility. However, the only measurement that counts is if you throw them in a pool with three other swimmers and see who finishes first.
I would agree with you completely on that point. I think that having these standards and metrics, which many hospitals do, helps us with the problem identification and remediation for the root cause of complications. However, I think that the challenge here is whether the number of cases, mortality and morbidity rates, and infection rates translate into a competent or expert surgeon.
I do think that the other skills that you were identifying in terms of courage and confidence are hugely important, and this is one of our challenges when it comes to training surgeons. How do we pass on and transfer the skills that you have because you have been in practice for a certain number of years, and you have used evidence-based best approaches for your patient care? What is the best way to transfer those skills from you to someone who is more of an advanced beginner, a novice, or someone who is competent but not proficient?
That's an interesting issue. Should everybody be tested on a regular basis and how often? Do you assess it when you are 65, or do you pick another arbitrary age? I do not presume to know the answer. There is no profession where you are forced to retire at a specific age, other than an airline pilot. You can be a brain surgeon or a heart surgeon until you are 93 years old, and this is legal and somewhat set in custom. When it comes to assessing skills, we may consider applying it to 40-, 50-, and 60-year-old surgeons as a test of who needs remediation of their skills. We may also consider testing people every year or two after 65 years of age, as is done with a driver's license, and then you get retested because things change quickly at that juncture in your life.
Based on your points about mental acuity and hand–eye coordination, physical biomedical markers are key in determining when to start testing. It may not be fixed to a certain age, but there are assessments that could periodically occur that would be cues that a surgeon may no longer safely operate. Perhaps they are developing a neurological condition that needs further exploration, or a hand tremor that keeps them from operating. Having a system in place that acts as an early warning system, if you will, for physicians who may be having difficulties that they are not aware of may be a sounder way to think about this than having an age where we start with mandatory assessments.
Search the literature for “skill assessment for surgeons” and you will see hundreds of articles. One top-cited paper, titled, “Objective assessment of technical surgical skills,” by van Hove et al., 1 looked at studies addressing the validity and reliability of methods for objective skills assessment within surgery and gynecology in order to provide a guideline for use in clinical practice. After reviewing 104 published studies, the authors concluded that most methods of skills assessment are valid for feedback or measuring progress of training, but few can be used for examination or credentialing. So the search continues.
Surgeons should be assessed by 360-degree evaluations. Ideally, this should be conducted by people who are experts in the field, meaning their peers, by those who maybe report to them, by other colleagues who work with them on a regular basis, and their supervisor. This would provide us a snapshot of an individual's total performance. In reality, however, unless the evaluators are really trained in terms of performance evaluation, a 360-degree evaluation can become very divisive and create all kinds of additional issues for the individual and department.
I think that the assessments should be performed by two people, so I would call it a 180-degree evaluation instead of 360-degree evaluation. The first evaluation should be performed by a neutral surgeon who is skilled in the same field as the surgeon in question. This surgeon should not be an adversary or a friend. It should be the responsibility of the chairman to select this individual. Another evaluation should be performed by the operating room nurses. These individuals are extremely good observers of technical skill, as they spend their life assisting and working with surgeons, and they are very, very good analysts.
The nurses in the operating room also understand the communication necessary to conduct an operation, so they are in a position to provide feedback in terms of the team—team communication and team dynamics. This is ultimately all about patient care and achieving the best outcomes for the patient.
It is interesting that you mention communication because our operating rooms are almost silent. We need very little communication because we are so well rehearsed. It is like if you were to go to the Philadelphia Symphony Orchestra; the violinist would not have to tell the percussionist to hit the drum, and the cello player would not have to turn to the pianist and say play. The conductor is really the individual setting the rhythm and metrics, and during a half-hour symphony, a word is never spoken. A well-groomed operating room team that is performing at a high level really does not have to speak.
I will do an entire complex operation and not say a word because everybody knows the score, the rhythm, and the cadence. Observers who have watched me have said, “You must hate your team because you never talk to them.” It is not that at all. I love and respect my team, but we are so rehearsed and so orchestrated that you need only minimal verbal communication.
That is the gold standard. That is where teams want to be, and your setup and the way that your teams are assigned may allow you to accomplish that level of efficiency and teamwork. Unfortunately, that does not happen at all institutions. At other institutions where surgeons do not exhibit this level of nonverbal communication and teamwork, team dynamics and communication can be a metric of competency.
Whoever delivers the news that a surgeon should stop operating needs to make this a kind, supportive process, not an inquisition. I think that the actual news should be carefully delivered by a messenger who is a friend to the individual because, in the end, you are doing the surgeon a favor if you persuade the individual to stop operating when he/she is no longer competent. This also helps patients and society, but it ultimately helps the surgeon.
I feel very strongly that it is a surgeon's responsibility to define the standards and the methodology of assessing surgeons. The last thing I want is the federal government setting another 200 regulations about things they know very little about. We already have plenty of regulations!
I think that we as physicians, surgeons, and educators have to accept the responsibility to find simple, fair, and generally available standards of measuring performance to figure out a fair, legal way to deal with those people beyond two standard deviations of competence.
This has to be defensible in a court of law because these cases may end up in lawsuits, as there is so much money, emotion, and ego involved. I think when you tell a surgeon, as I have had to do many times, to stop operating, you are saying they are moving closer to the grave. You are taking away their life and their self-image, and you have to do this with great care, affection, and respect for the surgeon and patients.
I would agree with what you said in terms of it has to be the surgeons who are defining the standard of care. If not defined, then it will be the federal government stepping in and intervening. I would agree with you that most physicians would not like to see that happen. However, I think that the standards have to be defined, and they have to be communicated within the profession. It is then the obligation of the profession to determine how to make sure that all of its members understand the standards of practice and are able to perform to meet those standards.
At the present moment, there is very little implementation of this. I bet that no academic medical center has a set of standards for evaluating the senior surgeon, particularly for how one decides their level of competence.
I think that is absolutely correct. My concern is that without an enterprise-wide system that is defined by surgeons or by the specialty group, we may lose surgeons or they may look for other alternatives for employment, rather than continuing to care for the patient. Especially with increasing requirements for maintenance of certification and maintenance of licensure and privileging, and everything else that surgeons have to deal with in today's healthcare environment, there will be little incentive to continue practicing. We as a profession owe it to our colleagues to have systems in place that will help them obtain and maintain those practice skills.
Conclusion
There are no definitive metrics by which to assess surgeon skills, and no methodology to consider when a surgeon should cease operating. This point/counterpoint discussion may provide a launching point upon which surgical skills can be formally assessed on a regular basis, either in a laboratory setting using standardized measurements or via peer evaluation from appropriate individuals in the field. The ultimate goal is to provide the best patient care, and evaluating the surgeon should constitute a vital portion of this process.