Can We Continue to Ignore Gender Differences in Performance on Simulation Trainers?

Introduction

Gender differences between surgical residents were reported as early as 1985, ¹ when it was noted that female residents had better scholastic abilities yet scored less well on tasks measuring confidence and task organization partly because of caution in avoiding errors. In fact, studies have previously demonstrated a similar trend in nonsurgical psychomotor skills, wherein women have slower reaction times but lower error rates, and this difference persists over increasing age, exhaustion, and sleep-deprivation and is not overcome by practice.^2–5

Acquisition of skills occurs over stages such as cognition, integration, and automation, ⁶ and acquisition of psychomotor skills on the laparoscopic trainer has been widely recognized as being a proxy measure for better performance in the operating room.^7–15 Previous studies have suggested a bias toward better performance of male students and residents on laparoscopic trainers,^16–18 yet this difference remains largely unrecognized and consequentially unaddressed.

In an environment where laparoscopic trainers are being increasingly used for training and evaluation of residents, it is important that programs are sensitized to issues with regard to gender differences to ensure fair and personalized training opportunities.

Materials and Methods

The study was carried out in the Department of General Surgery at the Creighton University Medical Center in Omaha, Nebraska, between February and March 2008. The study protocol was approved by the Institutional Review Board, and conforms to all applicable governmental agency guidelines. Each person who agreed to participate provided their informed consent. Our primary hypothesis was that students who chose surgical subspecialties performed better than those who chose medical subspecialties, and a difference in performance by gender was a secondary outcome measure. Fourth-year medical students enrolled in the School of Medicine were invited to participate in the study after the National Resident Matching Program match results. There was no financial incentive for participation, which was purely voluntary. Systematic unbiased announcements were made to recruit medical students and accrual was stopped after 32 consecutive students owing to limitations of time at our skills lab.

We included all volunteers who completed a preliminary survey, training, and testing. All volunteers were naïve to the Minimally Invasive Surgical Trainer and other laparoscopic trainers in the skills lab, which had recently been established. Exclusion criteria included volunteers who were either unable to complete the task, who had caffeine intake within the previous 6 hours, or had slept <6 hours the previous night. Information regarding exposure to surgical procedures during surgical rotations was gathered with a questionnaire that scored students on whether they had performed laparoscopic tasks, handled the camera, or only observed surgeries. Detailed information regarding video game use such as device of video game, use at different age ranges, and current and past video game use were collected, in addition to experience at other psychomotor skills, such as use of musical instruments, chopstick use, and typing. Surgical specialties were defined as general surgery, orthopedics, urology, ophthalmology, obstetrics, and gynecology, whereas the medical specialties were considered as medicine, family practice, radiology, and emergency medicine.

Performance Measurement

Participants were then given written instructions on the Minimally Invasive Surgical Trainer task “Transfer Place” describing the steps to completion and errors recorded by the program. This task essentially assesses the spatio-cognitive abilities of a participant: it requires the participant to introduce the instruments, grab a spherical object with a grasper, transfer it to the other hand, and place it inside a cube with minimum errors; feedback of which is provided visually. Additionally, participants viewed a step-by-step tutorial provided in the Minimally Invasive Surgical Trainer software and watched a live demonstration of the task. Participants were then instructed to perform three “Transfer Place” trials on the “Easy” level to gain familiarity with the task. Next, the difficulty was set at the “Medium” level and the participants were instructed to perform three additional trials. Data analysis was restricted to the three repetitions performed at the Medium or testing level.

Data collection was automated through the Minimally Invasive Surgical Trainer software. A total score was calculated using time, economy of motion, and number of errors. Each participant had six scores—three from training and three from testing. We used all three scores in the testing mode for each participant assuming an exchangeable correlation matrix.

The study data were manually reviewed before computer entry to look for omissions, illegible entries, and gross errors. The data were validated via range and logic checks to allow for quick identification of data collection problems and to minimize the amount of missing data. Statistical analysis was performed using Intercooled Stata 9.0 (Houston, TX). An alpha value of .05 was considered significant for all tests. t-test was used for variables with normal distributions (assessed graphically and with q-q plots), whereas the Wilcoxon rank sum and the chi-squared test was used for nonparametric tests. Generalized estimating equations (GEE) were used to perform multivariate modeling with scores as the dependent variable, and each participant was treated as a cluster. The use of GEE allows for population average models since the scores for each participant are correlated (clustered). Additionally, they do not assume normal distribution of the dependent variable, and estimate a nonvarying coefficient in the presence of clustering. The GEE method models within-cluster similarity of residuals to estimate parameters and standard errors. ¹⁹ Variable selection was determined after assessing the effects of individual variables after adjusting for clustering in single variable models. All variables with a P value <.20 or with biological significance (experience in the OR and current video game use) were included in the final model. Model checking using graphical representation of the predicted scores versus the actual scores to assess fit and verification of postestimation statistics and correlation matrix were used.

Results

A total of 32 students participated in our study, of whom 16 were females (n=16) and the mean age of participants was 27.4 years (SD±3.1 years). We did not exclude any participants who met the inclusion criteria for this study. Seventeen students matched in surgery or surgical subspecialties, of whom 6 were females. Of those who chose surgical specialties, 8 (8 males, 0 female) chose general surgery, whereas 7 (1 male, 6 females) chose obstetrics and gynecology, 1 chose urology (1 male), and 1 chose orthopedics (1 male). Most of the students had chosen their specialty in their third year (n=20), and the predominant reason for choosing a surgical specialty was “enjoy working with hands” (n=10), whereas the second main reason for choosing a surgical specialty was a “positive experience during the clerkship” (n=5).

Of the 16 female participants, 10 chose medical specialties, whereas 6 chose surgical specialties (P=.08). Current video game use was less frequent in females but was not statistically significant (44% males vs. 25% females, P=.26). Video game use at younger ages was more common among males with a trend toward significance (Table 1). More females than males were skilled at sewing (18% vs. 6.2%, P=.04), playing wind instruments (provide absolute numbers and remove percentages 25% versus 12.5%, P=.01), and playing keyboard instruments (provide absolute numbers and remove percentages 25% versus 18.7%, P=.03). More males than females thought that they were dexterous (provide absolute numbers and remove percentages 25% versus 0%, P=.01). No gender differences were noted in the proportion of students that believed that they were skilled in painting, sculpting, typing, chopstick use, and home repairs (P>.20).

Female participants performed worse on the trainer (Coefficient of difference +29.8 [95% CI 6.1, 53.4], P=.01 [positive coefficient indicates worse performance]), compared to male participants. The differences were notable in all components measured by the Minimally Invasive Surgical Trainer, including economy of motion (P=.01), time taken (P<.01), and errors made in 3 repetitions (median 25 errors in males versus 29 errors in females, P<.01).

Effect of repetition on performance

Repetition significantly improved performance during both training and testing, as evidenced by the lower coefficient of difference for the third repetition compared to the first during training [−19.5 (95% CI −30, −9), P<.001] and testing [−14 (95% CI −28, 0.3), P=.05]. This difference remained unchanged after adjusting for gender (Fig. 1).

OPEN IN VIEWER

FIG. 1.

Improvement in performance on MIST-VR trainer with repetition. (Lower score indicates better performance.)

Discussion

Our study highlights the significant difference in the innate performance of “trainer-naïve” male and female medical students on the Minimally Invasive Surgical Trainer. Psychomotor skill acquisition and behavior of females differ from males and have been shown in previous cognitive nonsurgical experiments.^2–5 However, this difference is largely ignored during the widespread implementation of training modules of laparoscopic trainers, even though a substantial part of the surgical workforce comprises females.

Women have been suggested to be risk averse and tend to have longer reaction times but higher precision in previous experiments.^2–5 Even though we found that female students did worse than male medical students in time, economy of motion, and error rate, the difference was less in the error rate. We attempted to adjust for confounding factors such as video game use and other psychomotor skills to understand the true inference of such a finding, yet recognize the limitation of the small sample size of our study. Nevertheless, to strengthen our conclusions, we reviewed all English-language articles published from 1990 on PubMed for gender differences in laparoscopic trainers and found other studies that have found similar results more recently (Table 3).

We also found that current or previous video game use was not associated with an improved performance on the Minimally Invasive Surgical Trainer. This is contradictory to earlier studies that have suggested a strong association but is in tune with more recent studies that have also failed to find similar associations.^18,20–22 Several reasons could explain our lack of finding such an association in addition to a small sample size. Video game use alone may not be sufficient in predicting laparoscopic skills but rather proficiency in video games may be more important in the current generation of medical students who are already experienced in hand–eye coordination skills afforded to them by devices such as computers, mouse devices, touch screens, and cellular phones. Similarly, the dexterity benefit of other psychomotor skills such as chopstick use and playing musical instruments may be offset by concurrent use of electronic devices. In either case, we believe that repetitive cognitive psychomotor exercises can do nothing but improve surgical performance and do not wish to detract from proponents of video game use.

We also found that students appear to select specialties based on their better technical abilities, at least as assessed by the laparoscopic trainer. In fact, other studies have shown that students who consider themselves to be dexterous perform well on standardized virtual reality trainer models.^23,24 We also found that repetition improved performance for both genders—again, emphasizing the well-known adage of “practice makes perfect.”

Our study does suffer from limitations of a small sample size and the use of volunteers who may introduce bias. Nevertheless, we do not believe the bias to be non-directional, and have attempted to minimize bias as far as possible. The use of statistical methods such as GEE can help eliminate some of the biases from small sample size; nevertheless, the potential for type II errors remains significant.

In conclusion, female medical students differ in their innate abilities on the laparoscopic trainer from male students, and these differences need to be studied on a larger scale and used to calibrate the training modules on laparoscopic trainers. We do not believe or endorse the use of a laparoscopic trainer to define how good a surgeon is, since the dimensions of a good surgeon far surpass psychomotor skills alone; yet, we feel that by recognizing differences in the genders in their skills on the trainers, we may cultivate the strengths of each gender to fortify our future surgeons.