Abstract
Objectives:
As the demand increases for demonstration of competence in surgical skill, the need for validated assessment tools also increases. The purpose of this study was to validate the use of a sawbones model for the assessment of performance of anterior cruciate reconstruction (ACLR) by residents. We hypothesized that the combination of a checklist and a previously validated global rating scale be a valid and reliable means of assessing ACLR when performed by residents in a dry model.
Methods:
All residents, sports medicine staff and fellows were invited to perform an ACLR on an ACL Sawbones model. Demographics regarding previous exposure to knee arthroscopy and ACLR were collected. All participants were asked to perform a hamstring ACLR using an anteromedial portal with Endobutton fixation on the femur - a detailed surgical manuscript and technique video was sent to all residents prior to the study. Residents were evaluated by faculty using a task-specific checklist created using a modified Delphi procedure, and the Arthroscopic Surgical Skill Evaluation Tool (ASSET) global rating scale. Each procedure was recorded, with videotaping of the hand movements and arthroscopic video recordings of the intra-articular procedure. These videos were scored by a fellow blinded to the year of training of each resident.
Results:
A total of 29 residents, six staff and five faculty performed an ACLR on the sawbones model (40 total). The overall reliability (Cronbach's Alpha) of the test using the total ASSET score was very high (>0.9). The reliability for the femoral checklist was 0.75, for the tibial checklist was 0.78, and 0.68 for the graft passage and fixation. One-way analysis of variance for the total ASSET score and the total checklist score demonstrated a difference between residents based upon year of training (p<0.001). Post hoc analysis demonstrated a significant difference in global ratings and checklist scores between junior residents (PGY1-3) and senior residents (PGY4&5), seniors and fellows, and fellows and staff (p<0.05). A good correlation was seen between the total ASSET score and prior exposure to knee arthroscopy (0.73) and ACLR (0.65). The inter-rater reliability (ICC) between faculty rating and blinded assessor for the total ASSET score was very high (>0.8).
Conclusion:
The use of a sawbones models to assess resident performance of ACLR using the ASSET global rating scale is valid and reliable. These models may be used to ensure a minimal level of competence prior to resident performance of ACLR in the operating room.