Longitudinal Evaluation of Congenital Cardiovascular Surgical Performance and Skills Retention Using Silicone-Molded Heart Models

Abstract

Objective: Hands-on surgical training (HOST) for congenital heart surgery (CHS), utilizing silicone-molded models created from 3D-printing of patients’ imaging data, was shown to improve surgical skills. However, the impact of repetition and frequency of repetition in retaining skills has not been previously investigated. We aimed to longitudinally evaluate the outcome for HOST on two example procedures of different technical difficulties with repeated attempts over a 15-week period. Methods: Five CHS trainees were prospectively recruited. Repair of coarctation of the aorta (CoA) and arterial switch operation (ASO) were selected as example procedures of relatively low and high technical difficulty. Procedural time and technical performance (using procedure-specific assessment tools by the participant, a peer-reviewer, and the proctor) were measured. Results: Coarctation repair performance scores improved after the first repetition but remained unchanged at the follow-up session. Likewise, CoA procedural time showed an early reduction but then remained stable (mean [standard deviation]: 29[14] vs 25[15] vs 23[9] min at 0, 1, and 4 weeks). Conversely, ASO performance scores improved during the first repetitions, but decreased after a longer time delay (>9 weeks). Arterial switch operation procedural time showed modest improvements across simulations but significantly reduced from the first to the last attempt: 119[20] versus 106[28] min at 0 and 15 weeks, P = .049. Conclusions: Complex procedures require multiple HOST repetitions, without excessive time delay to maintain long-term skills improvement. Conversely, a single session may be planned for simple procedures to achieve satisfactory medium-term results. Importantly, a consistent reduction in procedural times was recorded, supporting increased surgical efficiency.

Keywords

congenital heart surgery cardiac anatomy/pathologic anatomy imaging modeling

Introduction

3D modeling is revolutionizing the understanding of the surgical anatomy of congenital heart defects (CHDs), impacting clinical decision-making, surgeons’ training, and parental counseling.^1-4 Hands-on surgical training (HOST) using 3D-printed or silicone-molded models created from 3D-modeling of patients’ imaging data has been proven to be a valid instrument to train residents and fellows in the specific surgical skills to achieve the correction of CHDs.^5,6 By utilizing validated self-assessment tools to investigate procedure-specific details and objectives,⁶ both junior and senior surgeons have demonstrated an enhancement in technical performance across two different attempts of the proposed surgical procedures.^7,8 Furthermore, the incorporation of HOST into congenital heart surgery (CHS) training holds tangible potential to result in improved outcomes in clinical practice.^3,4,9

To date, the assessment of CHS trainees’ surgical skills and performance is based on supervisors’ judgment and approval, while a standardized and objective-specific CHS training program implemented with HOST is still in evolution. Given the extremely wide spectrum of technical difficulty and required surgical skills for the correction of CHDs, a single training protocol is not considered sufficient. Moreover, the requirements for repeated training and the optimal time intervals between HOST simulations remain unknown.

This study aimed to longitudinally evaluate the surgical outcome, sustainability, and efficiency for HOST on two example CHS procedures of different technical difficulties. This work will help standardize CHS training programs, as well as design a procedure-specific and cost-effective HOST curriculum.

Material and Methods

Study Participants

Congenital heart surgery residents and fellows attending the cardiovascular surgery department of the Hospital for Sick Children, Toronto, were prospectively recruited. After an individual interview, informed written consent was acquired, and the participant was given study instructions and needed material for HOST simulations. The study was approved by the Institutional Ethics Review Board (protocol 1000080945).

Study Design

Repair of coarctation of the aorta (CoA) and the arterial switch operation (ASO) for d-transposition of the great arteries were selected as example procedures of relatively low and high technical difficulty, respectively (Figure 1). We evaluated these procedures as they do not involve intracardiac surgical steps and can be executed on heart models by a single operator without the need for an assistant. The timeline of HOST simulations was designed proportionately to the technical difficulty of the procedure: CoA repair was performed at baseline and after one and four weeks, while the ASO was done at baseline and after 1, 4, 9, and 15 weeks, to assess the modification/retention of surgical skills of participants.

Figure 1.

Silicone-molded models created from 3D-printed hearts used in the present study: aortic coarctation repair (A) and arterial switch operation (ASO) for d-transposition of the great arteries (B).

The CoA and ASO models consist of silicone-molded models created from 3D-modeling of the image data generated from computed tomograms of pediatric patients, as previously described.¹⁰ The HOST setup for all simulations included a silicone-molded model, a chest wall simulator allowing rotation and tilting of the model, surgical instruments, sutures, and silicone patches. Before every simulation, the participants were asked to review the procedure-specific HOST training video, which is freely available online (www.3dprintheart.ca). The HOST simulations were video recorded without identifiable information of the participant and the procedural time was measured. After the simulation, the technical performance was quantified using procedure-specific assessment tools by the participant (Supplemental Material). All videos and models were collected, randomized, and reassessed by a peer reviewer (CHS fellow) and proctor (experienced CHS staff surgeon) using the same procedure-specific assessment tools. These scoring systems have been developed specifically for assessment of the individual surgical procedures and take into account accuracy, correct procedural components, technical skill, and anatomical result. They have been shown to be reliable and reproducible.^5,6 Technical performance scores were reported as a percentage of the maximum achievable score.

Statistical Analysis

Continuous variables are summarized as median and interquartile range (IQR) or mean and standard deviation (SD), as appropriate. Paired t test was used to compare overall scores by all reviewers and procedural times across time-points. Given the relatively small number of participants, no statistical tests were run to compare score variation among each scorer. Analyses were performed using SPSS 23.0 (IBM Corporation) and GraphPad Prism 9.5.1 (GraphPad Software).

Results

Participants

Five CHS trainees were prospectively recruited. The participants have undergone a median of 4 (2-5) years of cardiovascular training and 1 (1-4) year of CHS training at the time of enrollment. None of the participants had previously performed the attempted operations using silicone-molded models.

Coarctation Repair

The technical performance scores of CoA reported by the participants remained stable during the HOST sessions: 91.9% (80.7%-97.6%) versus 90.3% (90.3%-92.8%) versus 90.3% (90.3%-96.8%) at zero, one, and four weeks, respectively (Figure 2 and Table 1). However, the peer-reviewer and proctor assessment scores actually improved after the first repetition, but remained unchanged at the follow-up HOST session: 85.2% (77.8%-90.7%) versus 88.9% (84.3%-94.4%) versus 88.9% (81.5%-88.9%) for the peer-reviewer at zero, one, and four weeks, respectively; 88.9% (84.3%-88.9%) versus 94.4% (89.8%-100%) versus 94.4% (85.2%-97.2%) for the proctor at zero, one, and four weeks, respectively (Figure 2).

Figure 2.

Modification of performance scores of repair of coarctation of the aorta (CoA). In (A), red dots represent self-scores by the participant, green dots represent scores by the senior surgeon (proctor), blue dots represent scores by the peer reviewer, and lines connect medians of every time-point. In (B), mean and standard deviation of scores from all reviewers are represented. * P < .05 at paired t test.

Table 1.

Technical Performance Scores of CoA.

	Week 0Median (IQR)	Week 1Median (IQR)	Week 4Median (IQR)
Participant	91.9% (80.7%-97.6%)	90.3% (90.3%-92.8%)	90.3% (90.3%-96.8%)
Peer Reviewer	85.2% (77.8%-90.7%)	88.9% (84.3%-94.4%)	88.9% (81.5%-88.9%)
Proctor	88.9% (84.3%-88.9%)	94.4% (89.8%-100%)	94.4% (85.2%-97.2%)

Abbreviations: IQR, interquartile range; CoA, coarctation of the aorta.

Overall scores from every reviewer showed a significant improvement between the first and second HOST simulations (mean [SD] of 87.1% [7.3%] vs 91.8% [4.7%] at 0 and 1 week, respectively, P = .039). However, no significant changes were observed between the second and third HOST simulations: 91.8% (4.7%) versus 90.2% (5.9%) at one and four weeks, respectively (P = .328).

Procedural time for CoA repair showed a significant early reduction between the first and second HOST simulations (mean [SD]: 29 [14] min vs 25 [15] min at zero and one week, respectively, P = .034), but no further significant improvements with the third HOST session (25 [15] min vs 23 [9] min at one and four weeks, respectively, P = .662, Figure 3).

Figure 3.

Modification of procedural time for repair of coarctation and the arterial switch operation (ASO) during follow-up. Lines connect mean of every time-point. * P < .05 at paired t test.

Arterial Switch Operation

The technical performance scores of the ASO reported by the participants improved during the first repetition, remained stable at the second repetition, but then showed a modest decrease during the last two HOST sessions: 88.9% (77.8%-92.5%) versus 92.8% (84.3%-94.5%) versus 92.8% (84%-93.8%) versus 90.8% (88.6%-95.8%) versus 90.8% (89.9%-95.1%) at 0, 1, 4, 9, and 15 weeks, respectively (Figure 2 and Table 2). The peer-reviewer and proctor scores showed a substantial improvement during the first repetition (85% [84.7%-95.1%] vs 91.5% [90.2%-92.5%] for peer reviewer, 87.6% [86.6%-94.1%] vs 94.1% [88.9%-96.4%] for proctor), followed by slow but steady progress over the subsequent HOST sessions. However, a decrease in scores was documented when a longer time delay between repetitions occurred: 91.5% (90.2%-92.5%) versus 92.2% (87.6%-93.8%) versus 95.4% (87.9%-96.1%) versus 89.5% (87.9%-90.2%) for peer-reviewer at 1, 4, 9, and 15 weeks, respectively; 94.1% (88.9%-96.4%) versus 94.8% (88.9%-96.4%) versus 93.5% (85.7%-96.7%) versus 85.6% (83%-87.6%) for the proctor at 1, 4, 9, and 15 weeks, respectively (Figure 4).

Figure 4.

Modification of performance scores of arterial switch operation (ASO). In (A), red dots represent self-scores by the participant, green dots represent scores by the senior surgeon (proctor), blue dots represent scores by the peer reviewer, and lines connect medians of every time-point. In (B), mean and standard deviation of scores from all reviewers are represented. * P < .05 at paired t test.

Table 2.

Technical Performance Scores of ASO.

	Week 0Median (IQR)	Week 1Median (IQR)	Week 4Median (IQR)	Week 9Median (IQR)	Week 15Median (IQR)
Participant	88.9% (77.8%-92.5%)	92.8% (84.3%-94.5%)	92.8% (84%-93.8%)	90.8% (88.6%-95.8%)	90.8% (89.9%-95.1%)
Peer Reviewer	85% (84.7%-95.1%)	91.5% (90.2%-92.5%)	92.2% (87.6%-93.8%)	95.4% (87.9%-96.1%)	89.5% (87.9%-90.2%)
Proctor	87.6% (86.6%-94.1%)	94.1% (88.9%-96.4%)	94.8% (88.9%-96.4%)	93.5% (85.7%-96.7%)	85.6% (83%-87.6%)

Abbreviations: IQR, interquartile range; ASO, arterial switch operation.

Overall scores from every reviewer showed a significant improvement between the first and second HOST simulations (mean [SD] of 88.2% [6.4%] vs 91.3% [4.9%] at zero and one week, respectively, P = .033). Subsequently, overall scores showed a modest (but not significant) increase until the fourth HOST simulation: 91.3% (5.4%) at week 4 (P = .970, week 1 vs week 4) and 92.1% (4.4%) at week 9 (P = .618, week 4 vs week 9). However, a significant decrease in overall scores was documented when a prolonged time delay occurred: 92.1% (4.4%) at week 9 versus 88.9% (3.6%) at week 15 (P = .034, Figure 4).

Procedural time for the showed modest changes across HOST sessions, with a trend toward overall reduction in time with repetition. The mean times were 119 [20] min at week 0, 126 [29] min at week 1 (P = .362, week 0 vs week 1), 120 [17] min at week 4 (P = .240, week 1 vs week 4), 105 [34] min at week 9 (P = .092, week 4 vs week 9), and 106 [28] min at week 15 (P = .799, week 9 vs week 15, Figure 3). However, a significant improvement was recorded between week 0 and week 15 (P = .049), indicating an increased surgical efficiency among the participants.

Comment

The HOST program has been part of the CHS training curriculum at our Institution for almost a decade.¹¹ After the inaugural course at the American Association for Thoracic Surgery annual meeting in 2015, the HOST program has gradually advanced, in terms of available heart models,¹⁰ validated procedure-specific assessment tools,⁵ and systematic incorporation into the CHS training program with monthly sessions.⁷ Implementing a year-long HOST program that incorporates monthly simulations of various CHS procedures is feasible with the proactive involvement of senior staff and organizing personnel, coupled with the commitment of the trainees.^2,7

With the transition from 3D printed to silicone-molded heart models, the simulation experience has been significantly enhanced¹⁰; however, the higher production costs represent a deterrent for many institutions to introduce HOST simulations in their CHS training curriculum systematically. On the other hand, the hearts from sacrificed animals are excellent for surgical simulation as they provide the tissue characteristics that are almost identical to the ones of human hearts.¹² However, they are of limited use for certain procedures that are performed for diseases occurring in otherwise normal hearts. In addition, the availability of animal hearts can be limited and ethical concerns regarding animal rights can be raised. The optimization of HOST model design, the increased experience in the models assembly process, and the introduction of new 3D printing materials and techniques are projected to reduce the cost of HOST.^10,13 At the same time, evidence-based planning of surgical procedures to be included in HOST and understanding the need for repeated simulations and the timing of repetitions can establish a more sustainable and cost-effective HOST program.

This project specifically aimed to track the role of repetition in modifying technical performance and surgical efficiency during repeated HOST simulations, using two CHS procedures (CoA and ASO) as example operations of low and high technical difficulty. Although the self-evaluation of the technical performance by the participants did not reveal any change during three HOST simulations for CoA, both the peer-reviewer and proctor scores showed an improvement between the first and second HOST simulations, without further improvement at the third simulation. This finding was confirmed also by statistical comparisons utilizing overall data from every scorer. Similarly, procedural time showed an early significant reduction between the first and second HOST simulations, but no further modifications with the following attempt. These results highlight that simple surgical procedures may benefit from single HOST simulations to achieve satisfactory results in terms of surgical skills and technical efficiency.

Conversely, the performance scores of the improved constantly during the first, second, and third HOST simulations and consistently among the different scorers (participant, peer-reviewer, and proctor). Moreover, the participants showed a retention of the acquired surgical skills when a fourth HOST simulation was performed within a month (from week 4 to week 9). However, all performance scores decreased almost to baseline values when a longer time delay occurred (from week 9 to week 15). When data from all scorers were analyzed together, we documented a significant improvement between the first and second HOST simulations, followed by a slow steady increase until the fourth HOST simulation, but a subsequent notable decline in the last attempt (performed after a six-week delay). These results suggest that complex surgical procedures require multiple HOST simulations that should be scheduled with a monthly frequency. Importantly, procedural time for ASO displayed a slow improving trend across HOST simulations, which translated into a significant reduction between week 0 and week 15 (P = .049), translating into increased surgical efficiency.

The present findings confirm with a systematic approach our previous study showing overall improved technical performance of CHS trainees after a year-long HOST program.⁷ When analyzing single procedures, technical performance for CoA showed a modest improvement between the first and second attempts, without significant changes in procedural time. On the other hand, only a trend of increased performance was documented for the ASO, suggesting that a single repetition might not be sufficient to achieve a significant improvement in skills.⁷

We previously documented that after a prolonged (2-14 months) time delay between HOST simulations (including different surgical procedures), the technical performance of trainees diminishes in almost 50% of cases.⁷ However, in the present study, we were able to better quantify the timing of repetitions between simulations to maintain surgical skills for specific surgical procedures. Although the simulation outcome data in two particular operations are not justifiably applicable to all other pathologies, they give good insights into how to build the future HOST curriculum for trainees. For instance, these findings would suggest that one HOST session alone is sufficient for simple procedures, while multiple HOST sessions with less than four-week intervals are required for more complex procedures. With these findings in mind, we will further define the minimum required number of repetitions for other procedures. Considering that the training period is limited to 1 to 2 years for CHS fellowships, training of simple procedures such as uncomplicated aortic arch reconstruction and closure of ventricular septal defects could be scheduled as a single HOST session, while complex procedures such as repair of the atrioventricular septal defect, ASO, and Norwood operation could be arranged multiple times as proctored HOSTs followed by “Do-it-yourself” HOSTs using models showing different variations of the same pathologic entity. It can also be considered to individualize the program based on the trainee's achievement.

Furthermore, the decrease in technical performance registered after a longer time delay suggests a requirement for regular HOST sessions after the completion of the training for complex procedures. It will be particularly valuable for complex lesions, such as d-transposition of the great arteries, that are regularly but infrequently encountered, and where technical proficiency is directly related to outcome. Newly appointed staff surgeons may want to rehearse again on the HOST models just prior to the planned surgery at any stage of their career, just as sports players take practice rounds before the real game.

Finally, the translatability of surgical skills acquired during HOST sessions into the intraoperative setting is still one of the unresolved questions in most studies investigating surgical simulations in CHS.² Li and colleagues specifically addressed this limitation by evaluating the impact of previous HOST simulations on the surgical outcomes for ventricular septal defect repair performed by residents and fellows.⁹ Interestingly, shorter cross-clamp time and lower incidence of residual patch leak requiring multiple pump runs were documented after HOST simulations, independently from the surgical experience of trainees. Similarly, Wang et al reported better patient outcomes (in terms of operative times, postoperative complications, and length of stay, as well as echocardiographic results) in the setting of left ventricular outflow tract obstruction, if a simulation was performed before the Morrow operation.⁴ Further studies are awaited to support the efficacy of HOST simulations on clinical outcomes of extracardiac procedures and operations of high technical difficulty as the ASO. We are currently developing a new generation of silicon models that can be pressurized to better evaluate the results of surgical reconstructions as well as the quality of anastomoses, which will help predict intraoperative outcomes in patients.

Limitations

We acknowledge several limitations of our work. Firstly, the limited number of enrolled participants precluded statistical comparisons of technical performance within scorers. Although the CoA and ASO adequately emblematize surgical procedures of low and high technical difficulties, results from this study might not adequately apply to other surgical procedures performed with silicone-molded or 3D-printed models. On the other hand, the present study was intended to optimize resources for CHS training and to design a cost-effective HOST curriculum. For this reason, we focused on two example procedures to be investigated with repeated simulations following a timeline that was proportionate to the technical difficulty of the procedure. Previous larger studies have already documented the benefit of a single HOST session in ameliorating the technical performance of participants.^1,6,7,9,14 Future studies aim to study larger cohorts of trainees who are at different stages of their residency/fellowship/early staff position and establish whether the role of repetition changes as trainees become more experienced. Moreover, the impact of HOST sessions on patient outcomes needs further investigation, especially for complex operations such as the ASO.

Conclusions

In order to design a procedure-specific and cost-effective HOST curriculum to be included in CHS training, a single HOST session may be planned for simple procedures (exemplified by CoA repair) to achieve satisfactory medium-term results in terms of surgical performance and efficiency. Conversely, complex procedures (exemplified by ASO) require multiple HOST repetitions without excessive time delay to maintain skills improvement, especially in the long term; otherwise, a significant decrease in performance score is expected. Importantly, procedural time showed a slow constant reduction during HOST repetitions, supporting improved surgical efficiency.

Supplemental Material

sj-docx-1-pch-10.1177_21501351241237785 - Supplemental material for Longitudinal Evaluation of Congenital Cardiovascular Surgical Performance and Skills Retention Using Silicone-Molded Heart Models

Supplemental material, sj-docx-1-pch-10.1177_21501351241237785 for Longitudinal Evaluation of Congenital Cardiovascular Surgical Performance and Skills Retention Using Silicone-Molded Heart Models by Matteo Ponzoni, Rawan Alamri, Brandon Peel, Christoph Haller, John Coles, Rachel D. Vanderlaan, Osami Honjo, David J. Barron and Shi-Joon Yoo in World Journal for Pediatric and Congenital Heart Surgery

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Cardiac 3D Printing Program and Hands-On Surgical Training Program in congenital heart surgery at the Hospital for Sick Children are generously supported through the work of Mr Peter and Mrs. Fabiola Butler, and Mr Joseph and Penny del Moral.

ORCID iD

Matteo Ponzoni

Supplemental Material

Supplemental material for this article is available online.

Abbreviations and Acronyms

References

Yoo

Hussein

Peel

, et al. 3D modeling and printing in congenital heart surgery: entering the stage of maturation. Front Pediatr. 2021;9:621672. doi:10.3389/fped.2021.621672

Barron

Hussein

Yoo

S-J

. Training on congenital 3D cardiac models—will models improve surgical performance? Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu. 2023;26:9-17. doi:10.1053/j.pcsu.2022.12.001

Hussein

Kasdi

Coles

Yoo

S-J

. Use of 3-dimensionally printed heart models in the planning and simulation of surgery in patients with Raghib syndrome (coronary sinus defect with left superior vena cava). JTCVS Tech. 2020;2:135-138. doi:10.1016/j.xjtc.2020.01.023

Wang

Liang

, et al. Effect of 3D-printed hearts used in left ventricular outflow tract obstruction: a multicenter study. BMC Cardiovasc Disord. 2022;22(1):200. doi:10.1186/s12872-022-02641-6

Hussein

Honjo

Barron

, et al. Assessment tool validation and technical skill improvement in the simulation of the Norwood operation using three-dimensional-printed heart models. Eur J Cardiothorac Surg. 2021;59(2):316-324. doi:10.1093/ejcts/ezaa321

Hussein

Lim

Honjo

, et al. Development and validation of a procedure-specific assessment tool for hands-on surgical training in congenital heart surgery. J Thorac Cardiovasc Surg. 2020;160(1):229-240.e1. doi:10.1016/j.jtcvs.2019.11.130

Hussein

Honjo

Barron

Haller

Coles

Yoo

S-J

. The incorporation of hands-on surgical training in a congenital heart surgery training curriculum. Ann Thorac Surg. 2021;112(5):1672-1680. doi:10.1016/j.athoracsur.2020.11.018

Hussein

Honjo

Haller

, et al. Quantitative assessment of technical performance during hands-on surgical training of the arterial switch operation using 3-dimensional printed heart models. J Thorac Cardiovasc Surg. 2020;160(4):1035-1042. doi:10.1016/j.jtcvs.2019.11.123

Hussein

Zhang

, et al. Clinical translation of surgical simulated closure of a ventricular septum defect. Interact Cardiovasc Thorac Surg. 2022;35(3):ivac122. doi:10.1093/icvts/ivac122

10.

Peel

Lee

Hussein

Yoo

S-J

. State-of-the-art silicone molded models for simulation of arterial switch operation: innovation with parting-and-assembly strategy. JTCVS Tech. 2022;12:132-142. doi:10.1016/j.xjtc.2021.12.009

11.

Yoo

Thabit

Kim

, et al. 3D printing in medicine of congenital heart diseases. 3D Print Med. 2015;2(1):3. doi:10.1186/s41205-016-0004-x

12.

Mavroudis

Jacobs

DeCampli

Tweddell

. Simulation and deliberate practice in a porcine model for congenital heart surgery training. Ann Thorac Surg. 2018;105(2):637-643. doi:10.1016/j.athoracsur.2017.10.011

13.

Peel

Voyer-Nguyen

Honjo

Yoo

Hussein

. Development of a dynamic chest wall and operating table simulator to enhance congenital heart surgery simulation. 3D Print Med. 2020;6(1):12. doi:10.1186/s41205-020-00067-4

14.

Yoo

Hussein

Barron

. Congenital heart surgery skill training using simulation models: not an option but a necessity. J Korean Med Sci. 2022;37(38):e293. doi:10.3346/jkms.2022.37.e293

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

3.95 MB