Abstract
Virtual reality (VR) simulations have long been used for training in aviation and other professions. High-fidelity endovascular procedure simulators are now available, providing procedure simulations with real-time interactions; two-dimensional graphic displays of angiographic anatomy; mechanical interfaces with guidewires, sheaths, and catheters that provide some degree of haptic feedback; modeling of physiologic and pharmacology responses; and other features. Simulators have been incorporated into training programs for physicians learning carotid artery stenting (CAS). For the first time, US Food and Drug Administration approval of a new device (CAS systems) has included a requirement for physician training that incorporates the use of VR simulators. Early experience has shown that simulation is well accepted by trainees, performance on simulators improves with training and practice, and simulation prior to first performing endovascular procedures can improve clinical performance. Specific to CAS, the value of education programs using simulators appears to be tangible as trained but inexperienced CAS operators have clinical results comparable to those of physicians with extensive CAS experience.
Keywords
The skill set and knowledge required for modern endovascular surgery are growing and rapidly changing. Traditional training methods, such as graduate medical education (GME) and continuing medical education (CME) programs, cannot meet specialists' procedural training needs. Learning by case observation is helpful; however, hands-on experience is needed. Issues related to patient safety, credentialing, and availability of clinical experiences make “learning by doing” impractical in clinical practice, especially when considering the training for complex procedure that is associated with substantial risk, such as carotid artery stenting (CAS).
State-of-the-art training in many other high-skill and high-risk professions, such as aviation, uses virtual reality (VR) simulation as a means to reduce the time and expense of training with actual experience. (The term virtual reality generally includes computer simulations of real or imaginary scenarios in which users perform actions on the simulated system, and then they are shown the effects of their actions in real time.) For example, aviation simulators allow pilots to learn new procedures, transition to new aircraft or systems, and train in emergency procedure responses. Although simulators may reduce the number of flying hours required to meet training requirements, they do not, however, replace the need for actual time in aircraft.
Satava first proposed VR as a method for surgical procedural skills training in 1991, 1 but the lack of well-controlled clinical trials to demonstrate its efficacy, and perhaps the skepticism of the surgical community, made acceptance of VR training a gradual and slow process. 2 Despite the lack of early enthusiasm for surgical simulation, development work continued over the next decade, primarily in minimally invasive (videoendoscopic) surgery. By 2002, randomized double-blind studies had demonstrated the efficacy of simulation for improving intraoperative performance in minimally invasive surgery. 3 Similar work led to development of endovascular VR simulators that offered elements of the “look and feel” of working on actual patients, the ability to measure and record catheter and wire movements, and the ability to distinguish correct from incorrect procedural steps. 4
The early experience with CAS involved a relatively concentrated experience with a limited number of investigators, but by 2004, when the US Food and Drug Administration (FDA) considered approval of CAS systems for broader clinical use, concerns had been raised about the safety of CAS in the hands of less experienced operators. The FDA made the implementation of appropriate training programs a prerequisite to marketing of CAS systems, requiring manufacturers to educate physicians with a tiered training approach, including on-line and multimedia components. By this time, endovascular procedure simulators were commercially available and sufficiently realistic to be used as a training adjunct (Figures 1 –3). Thus, with CAS, the FDA's approval of an endovascular device included-for the first time-a requirement for training that included performance of procedural steps on a VR endovascular procedure simulator. 5,6
The SimSuite endovascular simulation system from Medical Simulation Corporation (Denver, CO) is shown in a fixed installation version.
Procedicus VIST is a vascular intervention simulation trainer produced by Mentice AB, Sweden.
ANGIO Mentor endovascular procedures simulator (Simbionix, USA, Cleveland, OH).
Simulations are categorized into two broad groups: low fidelity and high fidelity. Low-fidelity simulations use materials and equipment that are different from those used for the task being considered. An example of a low-fidelity simulation would be a verbal description of a hypothetical clinical scenario, with the learner (or person being evaluated) responding to questions about how to proceed. High-fidelity simulations use realistic materials and equipment to reproduce or represent the setting and task. The fidelity of a simulation activity may be more related to the overall conduct of the simulation scenario than to the specific capabilities of a specific device or interface capability. High-fidelity simulation can be provided without VR devices by the use of “standardized patient” actors or passive training aids (such as a mannequin) if these are used in a realistic manner. Although they have notable differences in design and features, all of the commercially available VR endovascular procedure simulators are categorized as high-fidelity simulators as they include haptic (relating to or based on the sense of touch), aural, and visual interfaces and provide a pseudorealistic representation of the clinical situation.
VR endovascular procedure simulators include portable endovascular procedure simulators and larger systems with fixed installations (Table 1). Each simulator has device-specific differences in displays, interfaces, and programming. The majority of physicians who have used endovascular procedure simulators have used portable systems through participation in device-specific training programs or simulation sessions at meetings of professional societies.
Commercially Available Virtual Reality Endovascular Simulators that Include Carotid Procedure Simulation
Shifting the Learning Curve
Performance with CAS has been documented to improve as operators and teams gain experience. Lin and colleagues reviewed their group's experience with 200 consecutive CAS procedures in 182 patients. 7 Technical success, periprocedural complications, and treatment outcomes were compared in four sequential groups of 50 consecutive interventions. The 30-day stroke and death rate in the first and second groups of 50 consecutive patients was 8% and 2%, respectively, which was reduced significantly to zero in their subsequent experience. Changes in techniques and refinement in periprocedural management were also observed to reduce complications in later patients. This procedure-associated “learning curve” was also evident by the reduced procedural fluoroscopy time and reduced use of contrast volume that occurred with an increase in physician experience.
A similar learning curve can be observed when physicians are trained in carotid angiography techniques using VR simulation. Patel and colleagues instructed 20 interventional cardiologists in carotid angiography and had the subjects perform five serial simulated carotid arteriograms on the VIST simulator (Mentice AB, Göteborg, Sweden). 8 There were measurable improvements between the first and the fifth procedure, including procedure time, contrast used, fluoroscopy time, and catheter handling errors. In addition, they were able to validate the internal consistency of the VIST simulator and its test–retest reliability with several key procedural metrics. The suggestion from this work is that some of the progress on the “learning curve” can occur in a nonpatient care setting.
Performance on an endovascular simulator has also been shown to have some ability to discriminate between novices and experienced endovascular surgeons. 9 Aggarwal and colleagues divided a group of 20 vascular surgeons into those who had performed less than 50 endovascular procedures as the primary operator (n = 8) and those who performed less than 10 procedures (n = 12). They tested endovascular skills, not procedural knowledge, using a VR simulated renal artery interventional procedure with angioplasty and stent placement. The simulator recorded procedure time, the amount of contrast used, and total fluoroscopy time. They found that surgeons with greater endovascular experience performed the simulated procedures significantly faster and used less contrast than inexperienced operators, although differences for fluoroscopy time were not significant. Notably, over the course of six simulated cases, the inexperienced group made significant performance improvements, achieving scores at the end of the training program similar to those of the experienced group. It was concluded that VR simulation might be particularly useful for the early part of the learning curve for vascular surgeons learning new endovascular procedures.
This certainly applies to resident training. Chaer and colleagues reported their experience training with the use of VR simulation training for general surgery residents learning basic endovascular procedures. 10 A group of 20 residents were provided with reading materials and didactic instruction in catheter-based interventions. Ten then received additional training with the VIST simulator. All 20 residents then participated in two consecutive mentored catheter-based interventions for lower extremity occlusive disease in an operating room or angiography suite. Resident performance on specific procedural steps was graded by attending surgeons blinded to the residents' training status. Residents who were prepared with simulation training uniformly scored higher than their peers during both the first and the second real case. Simulation training led to enhancement in almost all of the measures of performance, which included wire handling skills, catheter manipulations, positioning of angioplasty balloon catheters, introduction of stents, and overall ability to safely and efficiently complete the case.
Simulation-based training has also demonstrated efficacy in the training of vascular surgery fellows. Dawson and colleagues reported experience providing endovascular simulator training to fellows from vascular surgery programs in the western United States. 11 Fellows participated in a series of 2-day endovascular training programs that used a high-fidelity endovascular procedure simulator, didactic instruction, computer-based training, and tabletop procedure demonstrations. Performance on a standardized TransAtlantic Inter-Society Consensus (TASC) B iliac angioplasty or stenting case was used to assess endovascular skills and knowledge at the beginning of the training program, and this was repeated at completion of the training. Expressed as percent change from each individual's initial performance value (mean change from baseline), the total procedure time was shortened 54%, the volume of contrast use decreased 44%, and the fluoroscopy time was reduced by 48%. Selection of angioplasty balloon catheters and stents was improved, and the average number of catheters used and stents deployed decreased, although not reaching statistical significance. Both objective metrics and subjective faculty assessments suggested a substantial improvement in each fellow's performance. An introduction to CAS techniques was included in this program, and fellows found this content relevant and useful, although objective CAS performance metrics were not recorded in this study. Postcourse evaluations indicated that the participants strongly supported the use of endovascular simulation (including CAS) in vascular surgery fellowship programs.
Other reports have similarly suggested that simulation is especially well suited for the initial phases of procedural training. Dayal and colleagues, using the VIST to teach CAS to vascular surgeons, observed significant improvement in participants' endovascular techniques, but novice participants perceived a greater benefit from simulator training than did experienced interventionalists. 12 Similarly, Hsu and colleagues found that simulation training for CAS yielded greater improvements in simulated performance for individuals without previous carotid stent training than it did for those with significant previous experience. 13 Their program included training for medical students and procedurally inexperienced residents, as well as interventional radiology fellows, attending surgeons, and other interventionalists. As might have been expected, performance on the carotid stenting simulator correlated with previous endovascular experience. Although both novices and experienced clinicians improved their simulator performance after a proctored training session, improvement in the novice group was greater than that in the advanced group, again suggesting that novices disproportionately benefit from simulation training. Hsu and colleagues also reported that the opportunity to perform simulated carotid stenting procedures increased endovascular novices' interest in vascular surgery. 13
Team Training
A major impetus for the adoption of simulation in clinical training is the recognition that there is value in training clinical teams to work together. The importance of communication and effective management of human factors issues has long been recognized in aviation. Use of simulations for this purpose is now commonplace in aviation, and it has been suggested that patient safety can be enhanced by implementing aviation principles of crew resource management (CRM) in health care. 14 CRM training considers strategies to manage fatigue, creating and managing teams, recognizing adverse situations, cross-checking and communication, decision making, and performance feedback. Grogan and colleagues trained 489 medical personnel in clinical teams from a trauma unit, an emergency department, operative services, a cardiac catheterization laboratory, and administration in a 1-day CRM training course and then evaluated participant reactions and attitudes to CRM training. 15 An overwhelming 95% of participants agreed or strongly agreed with the notion that CRM training would reduce errors in their practices. The intermingling of the training and professional cultures of medicine and aerospace is particularly relevant in this article as two coauthors of this report from Vanderbilt University were physicians who were formerly NASA astronauts.
CRM-based team training for health settings may most effectively use simulators for procedures or processes that most resemble a cockpit environment: the operating room, emergent resuscitations, or the endovascular suite. In these settings, a defined group of physicians, nurses, and other staff come together to perform a specific and time-limited task, in contrast to more complicated areas, such as ambulatory care or a general inpatient ward. 16 Programs for team training for CAS have been developed and implemented at several medical centers with endovascular simulators. Nurses, technologists, and other staff involved in the periprocedural care of patients undergoing CAS have been trained in these programs.
Team training is more costly than individual training, although Gaba noted that the greatest costs associated with teamwork training are not the hardware but rather the “faculty time” of those engaged in developing and running the training. 17 He noted that instructors need special training for team simulations, as well as experience in postscenario debriefing. Larger-scale simulation exercises involve more participants in the training and thus increase the costs. He argued that learning and maintaining critical teamwork skills are appropriately considered requirements for the provision of high-quality care; thus, the costs should be considered within that context, with direct or indirect effects that may make the return on investment worthwhile over the long term.
Results after Implementing CAS Training Programs with VR Simulation
Recent data from clinical studies evaluating CAS in broader application suggest that the training programs that were implemented-which include instruction with VR simulation-have allowed CAS to transition safely from selected centers and a relatively small group of investigators to use by a larger number of clinicians, many of whom lack experience with the procedure. Although there are no reports in peer-reviewed journals documenting the effectiveness of CAS simulation training programs, data from larger trials have been publicly presented and are available in abstract form. These include data from a Guidant-sponsored study, Carotid ACCULINK/ACCUNET Post-Approval Trial to Uncover Unanticipated or Rare Events (CAPTURE 3000), and a Cordis Endovascular–sponsored study, Carotid Artery Stenting With Emboli Protection Surveillance-Post-Marketing Study (CASES-PMS). Abstracts from both were presented in March 2006 at the annual meeting of the American College of Cardiology.
The CAPTURE 3000 study included 3,000 CAS patients treated by 240 operators at 118 medical centers throughout the United States. This registry experience was part of the FDA-required postapproval study of the RX ACCULINK carotid stent system and RX ACCUNET embolic protection system (Guidant, St. Paul, MN). Gray and colleagues reported on an analysis of the first 2,500 patients in the registry. 18 Only 935 of the patients had symptomatic disease, 34% had diabetes, and 20% were smokers. Of the physicians, 71% had performed at least 10 carotid stent procedures as the primary operator. The 30-day complication rates included death in 1.6%, stroke in 4.2%, and myocardial infarction (MI) in 0.9% of patients. The composite rate of death, stroke, or MI was 5.7%. In asymptomatic patients (n = 2,267), the composite event rate was 4.9%, with death in 1.3%, stroke in 3.5%, and MI in 0.7% of patients. Symptomatic patients (n = 233), representing only a small minority of the patients treated, fared worse. For them, the composite event rate was 14.2%, with death occurring in 4.3%, stroke in 11.2%, and MI in 2.6% of patients. Higher event rates were reported in patients aged 80 years or older, with a composite rate of 8.2% and death in 2%, stroke in 6.6%, and MI in 0.5% patients.
Subgroup analysis of the CAPTURE data showed no difference in complication rates between physicians who started the trial with extensive, moderate, or negligible CAS primary operator experience. The physicians in CAPTURE with minimal previous CAS experience completed a 2-day training program that included technical instruction with tabletop models and practice with a variety of cases using a VR simulation system. 6 It therefore appears that physicians with the relevant clinical background, with sufficient skill and experience with endovascular techniques (including use of rapid-exchange catheters, 0.014-inch guidewires, and self-expanding stents), can be trained to perform CAS with outcomes equivalent to those of an operator experienced in CAS. In this setting, the training program does not use simulation to provide the initial endovascular skills training. Rather, the training and practice on the simulator facilitate transition to performing a new procedure that has similarities to procedures already mastered.
Cordis Endovascular initiated CASES, its comprehensive training program, in 2004. The program training included didactic review, case observations, and simulation training, with hands-on experience in regional programs. CASES-PMS was a multicenter, prospective, single-arm, open-label, periapproval study designed to assess the outcomes of stenting with the PRECISE nitinol self-expanding stent and ANGIOGUARD XP embolic protection device (Cordis, Miami Lakes, FL). The study's primary objective was to demonstrate that outcomes in a periapproval setting, including use of the detailed CASES training program for physicians not experienced in CAS, would allow outcomes similar to those obtained with CAS in SAPPHIRE, the previously performed pivotal trial. 19
CASES-PMS enrolled 1,493 high–surgical risk patients with de novo atherosclerotic or postendarterectomy restenotic obstructive lesions in native carotid arteries at more than 70 US sites. 20 Inclusion and exclusion criteria matched those in the SAPPHIRE study. The results reported from the first 1,279 enrolled patients from CASES-PMS showed that high–surgical risk patients treated with CAS had the same 30-day composite major adverse event (MAE) rates as the patients treated with CAS in SAPPHIRE. The data showed an MAE rate of 4.8% (62 of 1,279) in CASES-PMS, the same as the MAE rate (8 of 167) in CAS-treated patients in the randomized arm of the SAPPHIRE trial. Overall, the 30-day stroke rate was 3.6% (46 of 1,279), which was not different from the 3.7% (3 of 82) rate of stroke in patients treated by inexperienced physicians who had completed the entire CASES training program.
Future Directions for CAS Simulation
The commercially available endovascular simulation system represent the first generation of VR technology for CAS training. Several factors may drive continued development of this technology.
Advanced Procedural Training
Simulators have a growing library of scenarios with increasing case difficulty, but more sophisticated practitioners seek even more complexity to derive training benefit. Improved simulation technology may help with transition to the use of new CAS technologies (such as use of flow reversal embolic protection systems) or practice with special techniques (eg, use of a buddy wire to facilitate lesion crossed with an embolic protection device).
Simulators currently use predefined models of three-dimensional vascular anatomy. Future endovascular procedure simulators may incorporate anatomic information from individual patient's computed tomography or magnetic resonance imaging data, thus creating a patient-specific simulation model. This would allow for patient-specific case rehearsal, which might have benefit for team training or evaluation of alternative treatment strategies.
Skills Assessment
To date, simulation technology has been used primarily as an educational tool, but it has the potential to be used as a method for evaluation of cognitive expertise and technical proficiency. Performance in simulations is taken to represent likely future performance in a particular situation. Performance metrics can be measured, and the success of an intervention can be evaluated against established standards. Using performance in simulated procedures as a means to assess clinical competence, however, is far from being an accepted means of evaluating physician performance. There are no data that directly correlate ability to perform well in a simulation to clinical competence. Even if this relationship were to be accepted, it would then be necessary to establish thresholds for performance that represent minimum acceptable levels of cognitive and technical ability.
The initial certifying examinations in vascular surgery and other specialties that use an oral examination are examples of very low-fidelity simulations. Evaluators present case scenarios, and the candidate responds verbally with a description of actions. The adequacy of the responses is assessed in real time by the expert evaluators using established criteria. This process, however, depends on the subjective assessments of the evaluators. Although it is unlikely that the role of the expert examiner would ever be replaced by a VR-based testing system, it might be possible to have an expert examiner evaluate a candidate's performance on a simulator as a substitute for some of the oral examination questions.
Previously, it was assumed that a specific period of training or performing a certain volume of procedures meant that a physician was proficient in practice. No mechanism for measuring post-training skill has ever been generally used, but credentialing bodies now recognize the need for ongoing reevaluation of physician knowledge, skills, and performance. The American Board of Surgery has developed a program for Maintenance of Certification (MOC) to promote ongoing education, promote continuous professional development, and certify competency. As MOC evolves, it is intended to replace other multiple and often overlapping assessment standards. The MOC program consists of four parts that are intended to measure competencies on a continual basis, including evidence and documentation of professional standing, continuing education, periodic self-assessment, cognitive expertise based on performance on a secure examination, and evaluation of performance in practice. The program presumes the need for continuing education-to include advanced procedural training-as well as the need for development of better, broadly applicable assessment methods. Simulators may have a role in both areas.
Simulation as an Integral Educational Tool
The high-fidelity VR endovascular procedure simulators in use today are not “stand-alone” devices. They are tools that facilitate teaching and practice of technical skills. CAS educational programs using simulators have been successful when the simulator experiences, including a series of case scenarios with extensive interaction between the learner and the physician faculty, have been integrated into a curriculum designed to meet specific educational goals.
Realistic endovascular procedure simulation is an educational tool that has the potential to make the training of vascular specialists safer and more efficient. The experience to date suggests that trainees value and benefit from simulation training opportunities, the technology is mature enough to be widely adopted, and simulation should be included as part of a vascular specialist's GME and CME curricula.
High-fidelity simulation can provide an excellent opportunity for risk-free training in CAS and management of potential complications. Although it does not replace clinical training, it does offer a means for mentored instruction in a more realistic way than can be provided with tabletop demonstrations and is more efficient (more cases can be practiced), realistic, and less expensive than other training options. It completely avoids the risks of patient injury and medicolegal liability associated with “hands on” training in patient care settings.
Credentialing
Privileges to perform CAS are conferred by individual institutions, typically through medical staff committees that have been assigned to ensure that staff credentials and practice privileges are granted only to providers who have the appropriate training and sufficient experience to ensure safe and effective care. Often there is a requirement for proctored or supervised practice prior to assignment of full privileges, and there is usually a requirement for periodic review and renewal of privileges.
Although different institutions and different specialties have taken various approaches to credentialing issues, a multispecialty consensus recommendation on training and credentialing for CAS has been endorsed by the Society for Cardiovascular Interventions, Society for Vascular Medicine and Biology, and Society for Vascular Surgery. 21 The consensus recommendation includes specific cognitive, technical, and clinical skills requirements. Among other technical requirements (Table 2), a minimum experience of at least 30 cervicocerebral angiograms and 25 carotid stent procedures was recommended.
Technical Requirements for Performance of Carotid Stenting Recommended by the Society for Cardiovascular Angiography and Interventions, Society for Vascular Medicine and Biology, and Society for Vascular Surgery
Adapted from Rosenfield K et al. 21
Conclusions
At present, performance of CAS cases on a high-fidelity simulator has not been accepted as an alternative for actual procedural experience, but this could be considered in the future once the validity of such an approach is demonstrated. Training with simulators, however, already has direct relevance to the credentialing process as CAS training programs teach essential cognitive elements and simulations help with learning and practicing angiographic and interventional skills. Simulation may also be better for training physicians in the recognition and management of serious intraprocedural complications as these events may be rare enough to have never been encountered during the performance of actual cases during training.