Validation of a cost-effective appendicectomy model for surgical training

Introduction

Appendicitis is a frequently occurring condition in both men and women.^1–3 Appendicectomy or appendectomy is the indicated treatment for both complicated and uncomplicated appendicitis and unsurprisingly it is a common operation worldwide.^1–6 The lifetime risk for undergoing this operation in the United States is 23.1% for women and 12% for men. ¹ The minimally invasive approach is now recommended particularly for some groups of the population. ⁷ Research has shown this technique to be superior to its open counterpart in regards to patient outcomes, such as less post-operative complications, less need for post-operative analgesia, shorter length of stay and quicker return to work.^6,8–11

While this procedure was traditionally taught in the operating theatre, working time restrictions, shift pattern working and increased needs for service provision enforced in the past decades, have reduced the hours spent training in the operating theatre.^12–16 Simulation has emerged as a safe and effective didactic medium that despite some difficulties ¹⁷ it has been successfully introduced in several training curricula in various surgical specialties.^18–24 Training models for laparoscopic appendicectomy can be largely divided into two groups, virtual reality (VR) ²⁵ or physical models. ²⁶ VR models provide high fidelity; however, cost and lack of or unrealistic haptic feedback are significant considerations. ²⁷ Although newer models like Touch Surgery® are free and can be used on a mobile phone or tablet, ²⁸ they do not reproduce essential aspects of procedures such as physically holding laparoscopic instruments or providing realistic tactile forces. Commercially available synthetic models, on the other hand, do offer tactile feedback; however, they are single-use and relatively expensive. ²⁸

The ideal training model should combine cost-effectiveness and high fidelity. Recently a private company in collaboration with an academic institution (Medical Meat Supplies and Wales Institute for Minimal Access Therapy) developed a model with these two attributes which we used extensively for training courses in our region. The aim of this study is to assess this model by determining its construct validity.

Methodology

Intervention

The appendix model used for this study consists of porcine large and small bowel. The mesentery of the small bowel and the distal end of the lumen are attached to the side wall of the large bowel, in such a way that the luminal structure (i.e. small bowel) depicts the appendix and the mesentery represents the mesoappendix. The cost of each model was 15 GBP. The models are provided already assembled on a wood board, covered by a plastic bag. They can be kept frozen for several weeks or months and they are defrosted the day before the training session.

After a demonstration of the anatomical features described above and of the procedure by one of the faculty members, all participants were asked to perform a simulated laparoscopic appendicectomy, using the Medical Meat Supplies model. The model is supplied by a company, fully assembled, and the tissues are obtained from animals which were slaughtered for the purposes of consumption (i.e. not for the purposes of creating the model). The laparoscopic camera was fixed using a retractor that was secured onto the table upon which the box trainer was placed. This eliminated the need for an assistant and therefore a variable factor that can influence the trainee’s performance. The equipment available to the participants was the same every time and consisted of a box trainer, three trocars (one 10 mm and two 5 mm), a laparoscopic stage, a laparoscopic camera, dissector, scissors, two graspers, three endoloops and a retrieval bag. The trainees’ performance was assessed by one of the faculty members (all experts – surgical consultants) under direct observation, using a validated scoring system.

The procedure included placement of trocars and identification of the appendix and the mesoappendix, followed by retraction of the tip of the appendix and exposure of the mesoappendix. One of the vessels of the mesentery “acted” as the appendicular artery which the participants had to safely ligate. The taught method was to ligate the mesoappendix horizontally until the lumen of the appendix was reached and then skeletonise the appendix vertically to the base of the caecum (Figure 1). Although the last two steps would have been done with the use of diathermy in a real-life operating room, a diathermy equipment was not available in the simulation suite, therefore laparoscopic scissors were used instead and the participants were asked to verbalise when they would use diathermy.

OPEN IN VIEWER

Figure 1.

Appendix model.

Once the appendix was skeletonised, three endoloops were placed, two at the base of the appendix and one above in a distance that would allow for the safe ligation of the appendix.

The ligation and division of the appendix was followed by the exteriorisation of the specimen using a retrieval bag.

Results

One-way ANOVA was used to compare the performance between the three groups. GOALS scores are significantly different between groups for all GOALS categories as well as the total GOALS score (Figure 2).

OPEN IN VIEWER

Figure 2.

GOALS results from Global Operative Assessment of Laparoscopic Skills.

Post-hoc comparisons with the application of Bonferroni adjustment (statistically significant p-value <0.017) demonstrate a significant difference in performance between all groups for all GOALS categories as well as the total score. Eta-squared calculated as a measure of the effect size showed moderate (>0.5 and <0.8) and significant (>0.8) impact of experience level on the performance of the simulated procedure (Table 1).

OPEN IN VIEWER

Table 1.

P-value and effect size (eta-squared) for post-hoc comparisons (p < 0.017 statistically significant after Bonferroni correction applied).

	P-value	η2
Depth perception
Novices versus Moderate  experience (N-M)	P < 0.0001	0.61
Experts versus Moderate  experience (E-M)	0.0012	0.6
Novice versus Experts (N-E)	P < 0.0001	0.76
Bimanual dexterity
N-M	P < 0.0001	0.68
E-M	P < 0.0001	0.76
N-E	P < 0.0001	0.86
Efficiency
N-M	P < 0.0001	0.61
E-M	0.0009	0.61
N-E	P < 0.0001	0.73
Tissue handling
N-M	P < 0.0001	0.6
E-M	0.0012	0.6
N-E	P < 0.0001	0.8
Autonomy
N-M	P < 0.0001	0.81
E-M	P < 0.0001	0.86
N-E	P < 0.0001	0.86
Total scores
N-M	P < 0.0001	0.89
E-M	P < 0.0001	0.88
N-E	P < 0.0001	0.94

Discussion

Appendicectomy is a commonly performed procedure globally.^1–6 A cost-effective though realistic simulation model should be available to teach trainees how to perform the procedure. The model we present here has demonstrated construct validity (ability to distinguish between novices, intermediates, and experts) and is also cost-effective as it is currently sold for approximately £15. Construct validity is confirmed for all GOALS categories as well as the total score. The effect size of experience on performance is moderate (η²>0.5) to large (η²>0.8) for all categories and total scores. As expected, the greatest effect in each category is documented for the novices – experts’ comparison.

Besides cost-effectiveness and validity, this model has several more advantages. For instance, the tip of the appendix is fairly mobile and it can be placed in the retrocaecal, paracaecal, subcaecal or rectocolic position, imitating anatomical variations one can encounter during surgery.^30–33 In addition to a model for laparoscopic appendicectomy, it can also be used for teaching the open approach. Furthermore, being an animal tissue model it can replicate haptic feedback more realistically than virtual reality simulators. ³⁴ Conversely, a clear advantage for the virtual reality simulators is the immediate provision of performance metrics and whilst this has not been available for box-trainers, new technology is emerging to change this. ³⁵ Virtual reality simulator metrics such as instrument tip path length or number of movements are meant to reflect the economy of movement principle and even though they are generally considered an objective measure of performance, ³⁶ there are some reservations in recent publications regarding their effectiveness compared to scoring systems like GOALS. ³⁷

During a simulated session, not all aspects of surgery can be reproduced. For example, patient positioning which is vital for the successful completion of a laparoscopic appendicectomy cannot be reproduced. Several steps of the procedure are also not performed during the simulation session (e.g. retraction of the small bowel and identification of the appendix). However, these shortcomings are not exclusive to the current model, and both synthetic^26,28 and virtual reality models ²⁵ have the same limitations.

The current study has some limitations. Other types of validity (e.g. face, concurrent, content and predictive validity) were not assessed; however, there is emerging evidence to suggest that these type of evidence is outdated and should be facing out from surgical simulation assessment. ³⁸ The Messick four-faceted Framework was proposed by several authors for evaluation of simulation models. ³⁹ Although the framework has not been followed to its entirety, Messick framework tried to standardise construct validity – the concept used in this study – defining it as an evaluative judgement to the degree empirical and theoretical evidence support the adequacy of score tests achieved. ⁴⁰ The test scores in our study support the ‘empirical’ difference in experience between groups, i.e. experts (highly experienced) performed significantly better than novices (less experience). Previous studies using construct validity were criticised for using groups with a significant difference in experience and ability, i.e. complete novices and experts. In such case, the results of the construct validity assessment would not necessarily reflect the validity of the simulator but rather the difference in skills and expertise between the two groups. ¹⁷ For that reason, the authors of this study decided to use a third group of intermediate experiences to ensure that there is no great discrepancy in skills between groups.

Although this study used a validated scoring tool to assess trainee performance, the ideal metric is transferability of the gained skills in a clinical environment. Unfortunately, assessing this using the majority of the participants of this study would be difficult. This is because most of the participants were PGY2, who rotate through different surgical specialties, and therefore only a small percentage would be practicing in general surgery and using the skills taught during the course in a real clinical environment immediately after the completion of the course. ⁴¹ Nevertheless, having the assertion that this model is valid, in the future we can proceed to work with regional educational authorities and introducing it into a curriculum addressed to trainees undertaking general surgery clinical placements. Such a step will assist greatly in evaluating the transferability of skills from the simulated to the clinical environment. Furthermore, repetition and structured, individualised feedback often followed during teaching curriculums will assist in the long-term retention of skills acquired. ¹⁷

Conclusion

This study provides evidence of validity for a cost-effective model, simulating appendicectomy. The model can be used for laparoscopic and open surgery as well as to reproduce anatomical variations. The introduction of this model in a postgraduate training curriculum should be explored.