Influence of Dynamic Shadowing on 2D and 3D Laparoscopic Visualization Under Visible Light and Infrared Light

Abstract

Background:

Depth perception is a significant weakness in conventional two-dimensional (2D) endoscopy. We hypothesize that dynamic shadowing improves endoscopic depth perception during laparoscopy. Two experiments were performed to investigate this hypothesis. The first experiment compared the effect of dynamic shadowing with 2D and three-dimensional (3D) displays. The second experiment compared the effect of dynamic shadowing under visible light and infrared light.

Materials and Methods:

In the experiment, a box trainer was designed and built to house three laparoscopic tasks. Six settings were investigated: 2D display without dynamic shadows under visible light, 2D display with dynamic shadows under visible light, 3D display without dynamic shadows under visible light, 3D display with dynamic shadows under visible light, 2D display without dynamic shadows under infrared light, and 2D display with dynamic shadows under infrared light. Two types of illumination were used: static overhead illumination and dynamic grasper illumination. The execution time and number of errors committed by the volunteers with no formal laparoscopic experience were measured in the experiments.

Results:

The experimental results showed that dynamic shadowing with 2D display under both visible light and infrared light reduced mean execution time. Dynamic shadowing with 3D display increased execution time for one of the tasks and increased the number of errors for all three tasks.

Conclusions:

Dynamic shadowing has the potential to be an inexpensive method to improve laparoscopic depth perception. However, subjects needed to pay special attention to the shadows, which suggests that better implementation is necessary to improve the dominance of shadows as an effective depth cue.

Introduction

Minimally invasive surgery, also called laparoscopic surgery, has evolved into a common preference as the main treatment approach for many types of surgeries such as laparoscopic cholecystectomy. There are many advantages in laparoscopic surgery for the patient, such as reduced recovery time, lesser analgesic use, and less postoperative pain.¹ However, laparoscopic surgery has its disadvantages as well. Compared with open surgery, surgeons have to work with harder visual and physical conditions.^2,3 Because of the skewed hand–eye–instrument axis, the dexterity of the surgeon is reduced. The stationary positioning of the laparoscopic trocar ports imposes a counterintuitive fulcrum effect on the mapping between the surgeon and the instrument's movement. The long and thin laparoscopic instruments amplify hand tremor and, along with frictional effects of the trocar, reduce haptic feedback to the surgeon. As most conventional endoscopic systems can only display a two-dimensional (2D) representation of the three-dimensional (3D) operating workspace, it leads to a reduced perception of depth. This is due to the removal of 3D depth cues such as binocular stereopsis that is derived from binocular disparity that occurs because of the slight difference of vision experienced in both eyes. For a 2D display, both eyes view the same image; hence there can be no binocular disparity.

Although there are many aspects to laparoscopic surgery, this article focuses on laparoscopic visualization, in particular, on depth perception. In our previous work that explored laparoscopic training methodologies with a customized laparoscopic box trainer, it took about twice as long for subjects to perform abstract laparoscopic tasks with laparoscopic visualization compared with open visualization (C.S.L., unpublished data on simulation gaming for laparoscopy, 2009). Similarly, even the hybrid hand-assisted laparoscopic surgery method, with better depth perception from the inclusion of an intracorporeal hand, was reported to be significantly faster than conventional laparoscopy.⁴ Improving depth perception is necessary as surgeons, especially inexperienced ones, tend to compensate for the reduction in depth perception by “crashing” onto the tissue surface when trying to probe for the relative depth of the surfaces, which results in past-pointing tissue damage.⁵

Depth perception is based on binocular and monocular depth cues such as stereopsis, motion parallax, accommodation, convergence, occlusion, and shadows. Literature shows that the most dominant cue, especially for close proximity sight, is binocular stereopsis,⁶ which led to the development of 3D stereoscopic endoscopes that try to recover that cue. However, 3D laparoscopic visualization products are still in their infancy and prohibitively expensive. Furthermore, earlier studies have shown that initial 3D products had mixed results and tend to induce mental fatigue as well as look unnatural.^7,8 However, recent studies have shown that using state-of-the-art 3D high-definition systems increased task efficiency while performing abstract tasks in a physical phantom.^9,10

One way to improve depth perception within conventional 2D displays is to reinforce monocular cues. In a endoscopic environment, some of the more significant monocular depth cues are motion parallax, occlusions, and shadows.⁶ Shadows are typically absent in the endoscopic view because of the coaxial arrangement of the illumination and camera. Recent research has gone into the restoration of absent cues such as shadows, through optimal secondary illumination positioning,¹¹ shadow contrast ratio,¹¹ enhanced shadowing from image processing,¹² and color contrasting shadows.¹³ However, to our knowledge, no work has been done on enhancing motion parallax for laparoscopic surgery. This is not surprising as the nature of motion parallax requires a translating viewpoint to be effective. In the small laparoscopic environment, fixed trocar positioning makes it hard to achieve a manual, steady, and translating endoscopic viewpoint. In addition, the endoscopic field of vision is very narrow; hence excessive and unstable movement of the viewpoint will induce mental fatigue to the observer.¹⁴

This leads to the first aim of our study, which is to integrate motion parallax and shadows by introducing dynamic shadowing. Conventionally, in order to achieve the motion parallax depth cue, a translating viewpoint is needed. However, in our experiment, it is the light source and shadows that are moving instead of a translating viewpoint. This is achieved by mounting a secondary light-emitting diode (LED) light source onto the laparoscopic grasper that shines onto the tip of the instrument itself, producing a dynamic silhouette of the instrument against the surface at which the tip is pointing. The background illumination from the moving LED casts dynamic shadows against objects in the workspace as well. Indirectly, motion parallax can be inferred as shadows of objects nearer to the moving light source have a higher shadow contrast and move faster than the shadows of objects farther away. In addition, moving cast shadows have also been found to be such an effective depth cue that they can override¹⁵ other depth cues such as object size changes and movement in light sources.¹⁶

The second part of this study involves the evaluation of infrared light (IRL) as the illumination for the application of dynamic shadowing and laparoscopic visualization. Aside from thermal imaging,¹⁷ IRL has long been attempted for use in medical purposes such as classification of tissue type based on IRL, which is capable of penetrating into tissue and is absorbed and re-emitted differently by different tissue types.^18,19 The second useful property of IRL is the strong optical absorption by hemoglobin. Coupled with its penetrative nature, IRL is capable of enhancing and revealing near surface vasculature,²⁰ which is very useful to help surgeons avoid unnecessary hemorrhages. However, one major drawback of using IRL is the loss of color, which is an important source of visual information for the surgeon to differentiate between tissue types. In this study, we replaced visible light with IRL and examined whether dynamic shadowing translates well into a monochrome environment where shadows are less distinguishable from the environment.

The scope of this article is to investigate two hypotheses we put forward. In the first experiment, we investigate the first hypothesis, which is that the implementation of a dynamic light source to produce dynamic shadowing will improve depth perception and enhance the performance of three abstract laparoscopic tasks under a conventional 2D view and an autostereoscopic 3D view. In the second experiment, we investigate the second hypothesis, which is that IRL can be used as an alternative invisible light source for the implementation of dynamic shadowing as well as general illumination in laparoscopic surgery.

Materials and Methods

First experiment

Participants

Ethics approval for both experiments was granted by the local institutional review board. Twelve volunteers from non-medical backgrounds were recruited for the first experiment. Their age varied from 22 to 26 years. None of them had any prior laparoscopic training. All of the subjects had good (or corrected) eyesight and no color impairment. Subject demographics are listed in Table 1.

Table 1.

Subject Demographics for the First Experiment

Demographic	Value
Mean age (years)	23.8
Gender	9 males:3 females
Master hand	10 right:2 left
Experience with and frequency of Hand–eye
coordination games	67% have experience/about twice monthly
Racquet games	100% have experience/seldom

Apparatus

A 300-×300-×270-mm laparoscopic box trainer made of acrylonitrile butadiene styrene was designed and fabricated (Fig. 1). The box trainer had a 3-mm-thick silicon rubber that was stretched over the curved top cover of the box and fastened with Velcro^® (Velcro USA, Manchester, NH). It was designed with detachable side panels for easy access into the workspace.

FIG. 1.

Overview of the laparoscopic box trainer.

Instead of a 2D endoscope, an 8.1-megapixel USB Webcam was used as the primary overhead video source (model Webcam 7100; Sensonic, Singapore). The Webcam was secured in place through the use of an adjustable mechanical arm. The 2D output was displayed onto a 17" LCD monitor (model 1708FP; Dell, Round Rock, TX), and the 3D output was displayed onto a 20" lenticular autostereoscopic 3D monitor (model AD2; Newsight, Jena, Germany).

The laparoscopic instruments used are a 5-mm-diameter 310-mm-long bullet nose grasper (BBraun, Tuttlingen, Germany) and a straight tip needle holder (model E705R; Ethicon, Somerville, NJ). Primary static overhead illumination is from six white LED bulbs built into the Webcam or by a separate overhead LED probe (Fig. 2) that houses five 3-mm clear 3.6-V visible LEDs (Nichia, Tokyo, Japan) and is secured through the silicone sheet with an external magnet. Secondary dynamic grasper illumination (Fig. 3) is provided by two 3-mm-diameter white LED bulbs powered by an external DC source (model E3616A; Hewlett Packard, Palo Alto, CA). All probes were designed to be implantable and conform to 15-mm laparoscopic trocar size restrictions. In addition to the size advantage, compared with arc-lamp light sources, LED light was reported to produce sharper shadows, uniform lighting, and less flickering.²¹

FIG. 2.

Overhead light-emitting diode probe.

FIG. 3.

Grasper light-emitting diode probe.

Procedure

We have come up with three abstract laparoscopic tasks in order to test the depth perception of our subjects. The three tasks comprised threading, pushing, and pick and place. The first was a threading task (Fig. 4). It involved the subject withdrawing and inserting a thin stick held onto the end of the needle holder into a perforated vertical surface. The perforated surface is split into four separate rows of holes, and the subject was tasked to insert the stick into a randomly generated sequence of targets. Because of the nearly top-down orientation of the view, the rows of holes are perceived as targets at different depths. The subjects were assessed based on their execution time only as the majority of the subjects did not make any gross mistakes and were able to self-correct, at the expense of slightly additional time incurred. Each run consists of eight holes, and each run was carried out twice: once with the perforated surface located frontally to the camera's orientation and once with the perforated surface located obliquely to the camera's orientation.

FIG. 4.

Task 1: threading a wooden stick through the perforated Lego brick.

The second and third tasks involved the use of a hollow box that was interlaced with wires inserted at various depths across the box (Fig. 5). By linking a light and buzzer circuit with the network of wires and the electrode of the laparoscopic instrument, we created a maze to test the subject's instrument navigation performance in an intricate 3D space. We have previously used this 3D wire maze as part of training and assessing accuracy in laparoscopic motion with a robotic platform.²²

FIG. 5.

Task 2: pushing a Lego brick horizontally across the box.

The second task was to drag a piece of Lego^® (Lego Group, Billund, Denmark) brick horizontally across the length of the box. A subject has to withdraw and insert the laparoscopic grasper at appropriate positions and orientations several times across the maze in order to avoid touching the wires and be within reach of the brick as the brick was moved. The subjects were assessed on total time taken and the number of times the instrument tip touched the wires and triggered the circuit.

The third task was to pick six sponge cubes that were randomly placed within the maze and deposit them in a designated corner of the screen (Fig. 6). Similarly, the subjects were assessed on time taken and the number of times the circuit was triggered.

FIG. 6.

Picking and placing the randomly positioned sponge cubes.

The sequence of the tasks and viewing conditions assigned to the participants was randomized to minimize any learning effect on the results. The result collected were evaluated using analysis of variance to determine if there is a significant difference in the mean time taken and error rates for the different viewing conditions. Significance was set at P<.05.

Each set of three tasks was carried out under the following four viewing conditions: (a) 2D view with static illumination only, (b) 2D view with dynamic grasper illumination, (c) 3D view with static illumination only, and (d) 3D view with dynamic grasper illumination.

The premise of this research was to see whether we could improve depth perception through augmented shadowing, thereby leading to improved laparoscopic performance. Therefore, we adapted and designed the three tasks for the maximization of depth perception as the major factor of difficulty. Specifically, our tasks were designed for repeated range of motion (especially the withdrawal and insertion motions) of the graspers normal to the viewing plane such that the ability to perceive depth continually affects a user's overall performance. In comparison, the tasks in the Fundamentals of Laparoscopic Surgery were designed for testing a broad range of generalized technical laparoscopic skills such as ambidexterity, hand–eye coordination, and tissue handling as well as depth perception.

Second experiment

Participants

Twenty-five volunteers from non-medical backgrounds were recruited for this part of the study. Their age varied between 22 and 27 years. None of them had any prior laparoscopic training. All of the subjects had good (or corrected) eyesight and no colour impairment. Subject demographics are listed in Table 2.

Table 2.

Subject Demographics for the Second Experiment

Demographic	Value
Mean age (years)	23.9
Gender	15 males:10 females
Master hand	23 right:1 left:1 ambidextrous
Experience with and frequency of Hand–eye
coordination games	60% have experience/about twice monthly
Racquet games	88% have experience/seldom

Apparatus

The setup used for the second experiment was similar to that of the first experiment except for changes to the camera and lighting. A second Webcam of the same model was modified to capture video in infrared (Fig. 7). The IRL filter on the second webcam was removed so as to make it sensitive to IRL. In addition, a visible light filter made from standard photography film negative was added to the lens of the camera. The filter was prepared by overexposing a strip under visible light for a few minutes and then developing it. The strip was subsequently sized and fitted onto the lens of the camera. A 860-nm peak wavelength infrared LED (model SFH 4350; OSRAM Opto Semiconductors, Regensburg, Germany) was also incorporated into the overhead lighting probe and the grasper lighting probe.

FIG. 7.

Performing Task 3 under infrared illumination.

Procedure

The objective of the second experiment is to investigate the second hypothesis, which is that IRL can be used as an alternative light source for the implementation of dynamic shadowing as well as for general illumination in laparoscopic surgery. The task evaluation methods for the second experiment were the same as those used in the first experiment. Volunteers were asked to perform the same set of laparoscopic tasks under four viewing conditions.

The four conditions for Experiment 2 are as follows: (a) static illumination with visible lighting, (b) dynamic illumination with visible lighting, (c) static illumination with infrared lighting, and (d) dynamic illumination with infrared lighting.

Results

First experiment's results

Task 1: threading task

The viewing condition with the lowest average threading time is the 2D display condition with dynamic illumination (56.37 seconds). Overall, results also showed that the 2D display conditions (60.53 seconds) yielded significantly better results than the 3D display conditions (79.12 seconds) (P=.008), and dynamic shadows (68.41 seconds) was slightly better than static illumination (71.24 seconds), although the difference was not significant (P=.672).

Task 2: pushing task

The viewing condition with the lowest average pushing time is the 3D display with static illumination (43.91 seconds). Overall, results showed that the time taken to complete the task under 3D viewing conditions (56.85 seconds) was generally shorter than that under 2D viewing conditions (60.46 seconds). The difference was not statistically significant (P=.736). The overall mean for static illumination was 53.25 seconds, and the overall mean for dynamic illumination was 64.06 seconds, which is not significantly different (P=.315). The least amount of errors was made under 2D display conditions with dynamic shadowing (2.42). However, overall trends indicate static illumination (2.67) to be slightly better than dynamic illumination (2.92).

Task 3: pick and place task

The viewing condition with the lowest average pick and place task was 2D display conditions with dynamic illumination (103.53 seconds). Overall trends correlate with this result, with 2D display conditions (114.82 seconds) faring better than 3D display conditions (119.65 seconds) and dynamic illumination (111.48 seconds) doing better than static illumination (123.00 seconds). The least amount of errors was recorded under the 2D display condition with dynamic illumination (3.00). However, overall trends show that the 3D display condition (3.46) had a better error rate than 2D display conditions (3.88).

Second experiment's results

Task 1: threading task

The viewing condition with the lowest mean threading time is visible light with dynamic illumination (51.98 seconds). Overall, dynamic illumination (52.02 seconds) was better than static illumination (57.17 seconds), but the difference was not statistically significant (P=.24). IRL (52.74 seconds) was found to be slightly better than conventional visible light (56.46 seconds; P=.39).

Task 2: pushing task

The viewing condition with the lowest average time taken for this task was visible light with static illumination (40.87 seconds). Overall, the performance under visible light (41.81s) was significantly better than under IRL (61.33 seconds; P=.0051), and dynamic illumination (48.21 seconds) was found to be slightly better than static illumination (54.94 seconds; P=.32).

The least number of mean errors made was under visible light and static illumination (1.04). Overall, the number of errors made under visible light (1.18) was found to be significantly lesser than under IRL (2.30; P=.019), and the number of errors made under static illumination (1.56) was less than under dynamic illumination (1.92; P=.44).

Task 3: pick and place task

Visible light with dynamic illumination was found to have the lowest mean time taken for this task (74.99 seconds). Overall, visible light (77.99 seconds) was found to be better than IRL (91.55 seconds), and dynamic illumination (81.02 seconds) was found to be better than static illumination (88.52 seconds).

Similar to the pushing task, the mean number of errors made was lowest under visible light and static illumination (2.44). Overall, there were fewer errors made under visible light (2.68) compared with IRL (3.30), and there were fewer errors made under static illumination (2.76) compared with dynamic illumination (3.22).

Discussion

First experiment

In general, from the tabulation of mean results (Table 3), for all three tasks, using the 2D display, performance improved with the use of dynamic shadows. The experiment has shown that dynamic illumination with 2D display conditions reduces execution time and the number of errors made. The user relied on the brightness of the reflected light and the size of the shadow to gauge the distance between the instrument and the target. However, this was not true when looking at 3D display results. Mean performance for Tasks 1 and 3 was almost the same, but Task 2 took 60% longer when using dynamic shadows. Feedback from the volunteers was that the 3D images had distorted depth judgment and induced dizziness. Some of them also stated that they became visually and mentally fatigued faster when using the 3D monitor. We hypothesize that this was due to the 3D-generating Pulfrich effect that the 3D monitor used to simulate depth. This effect seemed to generate a decaying ghosting effect that tended to duplicate the number of shadows and hence served to disorient the subject more than aiding him. Even though our experiments were only 1–2 minutes long per task, it was sufficient for the subjects to feel visual fatigue with the 3D display, and that might have led to the degradation in performance. It may also be that 3D visualization requires a longer accustomization period compared with a standard 2D view.

Table 3.

Averaged Results for Experiment 1

	Static	Dynamic	Overall (2D/3D)
Task 1: threading time (seconds)
2D	64.70 (17.76)	56.37 (11.54)	60.53 (15.25)^a
3D	77.79 (30.05)	80.45 (27.93)	79.12 (28.40)
Overall (static/dynamic)	71.24 (25.05)	68.41 (24.25)^a	—
Task 2: pushing time (seconds)
2D	62.58 (28.69)	58.34 (34.54)	60.46 (31.13)
3D	43.91 (19.19)	69.79 (55.37)	56.85 (42.63)^a
Overall (static/dynamic)	53.25 (25.71)^a	64.06 (45.51)	—
Task 2: pushing errors (touches)
2D	2.83 (3.24)	2.42 (1.93)	2.63 (2.62)^a
3D	2.50 (2.28)	3.42 (3.26)	2.96 (2.79)
Overall (static/dynamic)	2.67 (2.75)^a	2.92 (2.67)	—
Task 3: pick and place time (seconds)
2D	126.11 (55.22)	103.53 (50.25)	114.82 (52.91)^a
3D	119.88 (66.87)	119.43 (76.23)	119.65 (70.13)
Overall (static/dynamic)	123.00 (60.06)	111.48 (63.66)^a	—
Task 3: pick and place errors (touches)
2D	4.75 (4.47)	3.00 (1.95)	3.88 (3.49)
3D	3.17 (1.85)	3.75 (2.77)	3.46 (2.32)^a
Overall (static/dynamic)	3.96 (3.44)	3.38 (2.37)^a	—

Corresponding standard deviation is listed in parentheses.

Best performance.

2D, two-dimensional; 3D, three-dimensional.

The performance degradation in the 3D group may be due to various factors such as the type of 3D display used or the novice subjects. 3D display technologies are still in maturation, and studies have shown mixed results and inadequacies of 3D displays, especially with the earlier products on the market.^7,8

The autostereoscopic 3D display that we used has an array of lenticular lens to selectively magnify and shrink columns of pixels to generate the differential parallax view seen by the left and right eyes, which might not be ideal for displaying an endoscopic view with a narrow depth of field, and that weakness was compounded with our implementation of dynamic shadowing. In the future, we will investigate using more advanced 3D displays such as those that utilize active shuttering glasses, which can purportedly reduce the visual fatigue and decay ghosting problems that we experience with the current equipment.

Looking at the mean number of errors, compared with the static counterpart, using dynamic shadows reduced the number of mistakes with the 2D display, but the trend reversed with the 3D display. This showed that the Pulfrich effect not only increased the time taken but also increased the number of mistakes made.

When we look at the results of each task under the four viewing conditions, 2D display with dynamic illumination and 3D display with static illumination tended to perform better in terms of time taken and number of errors made compared with the remaining two viewing conditions. This trend signifies two findings. First, implementing a dynamic illumination or 3D display improves on the conventional 2D display with static illumination. Second, although individually the dynamic illumination and 3D display generated an improvement in performance over the 2D display with static illumination, their joint effect produced an opposite effect, resulting in the worst performing viewing condition in most of the results.

Although we preemptively attempted to minimize the learning effect by randomizing the task order for each subject, there were still signs of familiarization as each task were carried out four times per subject. In addition, subject feedback was generally that the dynamic shadowing did help them in perceiving depth but that they needed to be reminded to actively use the shadows to gauge depth. This may be accounted for by the evidence that individuals have different strategies for depth cue combination.²³ Subjects might need to specifically train to consciously take notice and make use of the shadows.

Second experiment

In general, when we compare static illumination to dynamic illumination (Table 4), we see that dynamic illumination has reduced the overall mean time taken for all three tasks. This trend is also evident for all three tasks under IRL, which shows that the benefit of dynamic shadows, as shown in Experiment 1, can be applied with IRL as well.

Table 4.

Average Results for Experiment 2

	Static	Dynamic	Overall (visible/IR)
Task 1: threading time (seconds)
Visible	60.93 (30.06)	51.98 (19.64)	56.46 (25.78)
IR	53.42 (17.40)	52.06 (14.18)	52.74 (15.89)^a
Overall (static/dynamic)	57.17 (24.84)	52.02 (17.13)^a	—
Task 2: pushing time (seconds)
Visible	40.87 (18.79)	42.76 (24.24)	41.81 (21.71)^a
IR	69.00 (46.99)	53.66 (36.05)	61.33 (42.58)
Overall (static/dynamic)	54.94 (38.45)	48.21 (31.20)^a	—
Task 2: pushing errors (touches)
Visible	1.04 (1.40)	1.32 (1.43)	1.18 (1.42)^a
IR	2.08 (2.43)	2.52 (3.32)	2.3 (2.92)
Overall (static/dynamic)	1.56 (2.05)^a	1.92 (2.63)
Task 3: pick and place time (seconds)
Visible	80.99 (34.31)	74.99 (30.76)	77.99 (32.72)^a
IR	96.05 (40.85)	87.05 (40.20)	91.55 (40.77)
Overall (static/dynamic)	88.52 (38.47)	81.02 (36.29)^a	—
Task 3: pick and place errors (touches)
Visible	2.44 (2.95)	2.92 (3.11)	2.68 (3.04)^a
IR	3.08 (2.76)	3.52 (2.32)	3.3 (2.56)
Overall (static/dynamic)	2.76 (2.87)^a	3.22 (2.75)	—

Corresponding standard deviation is listed in parentheses.

Best performance.

IR, infrared.

Both the first and third tasks showed that performance under IRL yielded no significant penalty compared with visible light. However, the performance under IRL was significantly worse in the second task. Both mean time and number of errors made were significantly higher. We hypothesize that it was due to the nature of the task coupled with the optical property of the block material under IRL. The red Lego brick used for the second task actually appeared white under IRL (Fig. 8). The background also appeared white so subjects might have had difficulty trying to differentiate the brick from the background. In the first and third tasks, because there were no problems in recognizing the perforated surface or sponge cubes under IRL, it explains why using IRL did not result in significantly longer execution times compared with using visible light.

FIG. 8.

The difference in color of the same Lego brick under (left) infrared light and (right) visible light.

Overall, when comparing visible light with IRL, the mean performance was faster with visible light in two tasks and less number of errors was made under visible light compared with IRL in both Task 2 and Task 3. This showed that the loss of color under IRL did reduce the laparoscopic performance of the subjects.

Conclusions

The two hypotheses we put forth are found to be valid. In Experiment 1, it has been shown that the implementation of a dynamic light source in 2D visualization provides better depth information in the form of moving shadows. In Experiment 2, the results indicated that dynamic shadowing continued to show that it improved the subjects' performance, and its improvement can be shown even under IRL. IRL was found to be generally inferior to visible light as general endoscopic illumination, especially if the task involved color recognition.

The experimental results in Experiment 1 showed that dynamic shadowing with 2D display reduced execution time and number of errors. Dynamic shadowing with 3D display increased execution time for one of the tasks and increased number of errors for all three tasks. This demonstrates that dynamic shadowing could be an inexpensive method to improve conventional 2D laparoscopic depth perception.

However, familiarization is necessary to increase the dominance of shadows as a depth cue. In the experiment, subjects had to be reminded to pay attention to the shadows. A possible improvement would be to enhance the delivery of shadows to the subject's view through image processing.

The general feedback from the volunteers was mixed. Some were in favor of IRL, citing that the lack of color created less confusion and gave a sharper tone contrast. However, there were also those who claimed that the visible lighting allowed them to perform better as people were generally more accustomed to viewing from a full color display.

For future work, we intend to improve on mounting of the grasper illumination as the current LED setup was deemed to be too bulky and frequently became entangled with the live wires. We also aim to introduce another variant of dynamic illumination using a static and flexible array of LEDs that can be magnetically mounted onto the inner wall of the operating space. Alternative 3D displays will also be used to compare their compatibility with dynamic shadowing.

Currently, the main performance indicator of the tasks was derived from time. We intend to improve on the assessment criteria by including image-based motion tracking as well as force analysis. We propose that further experiments could track the path taken by the instrument and use the path length and path deviation to further assess individual performance. Second, the assigned game could simulate an actual laparoscopic task like suturing and cutting. Third, besides using near-infrared, medium- and far-infrared can also be considered for illumination. Last but not least, the torso case should not only mimic the human body externally but also internally. Inserting synthetic organs and vessels that have a wet surface finish can make the workspace look and feel like the actual abdomen. In this way, the user can experience the real visual and tactile restrictions encountered by laparoscopic surgeons.

Currently our comparison is limited to novice subjects, and the endoscopic viewpoint is fixed. This allows for the subjects' performance to be independent of their camera manipulation proficiency. However, in the actual operating theater, experienced surgeons, with a high proficiency in camera manipulation, use a moving viewpoint and select appropriate camera orientations to accommodate the deficit in endoscopic depth perception. For future work, we intend to recruit experienced surgeons to compare the efficacy of augmenting depth perception with dynamic shadowing when the subjects are already using alternate means of augmenting depth perception.

Footnotes

Acknowledgments

The authors thank the subjects for donating their valuable time to participate in the experiments. The authors would also like to thank the Mechanical Engineering Department of the National University of Singapore for their generous funding and support.

Disclosure Statement

No competing financial interests exist.

References

Shah

, Shah

. Minimally invasive surgery. Indian J Pediatr, 2008; 75:925–929.

Heemskerk

, Zandbergen

, Maessen

, Greve

, Bouvy

. Advantages of advanced laparoscopic systems. Surg Endosc, 2006; 20:730–733.

Emam

, Hanna

, Cuschieri

. Ergonomic principles of task alignment, visual display, and direction of execution of laparoscopic bowel suturing. Surg Endosc, 2002; 16:267–271.

Manasnayakorn

, Cuschieri

, Hanna

. Hand-assisted laparoscopic surgery is associated with enhanced depth perception in novices. Surg Endosc, 2010; 24:2694–2699.

Taffinder

, Smith

, Huber

, Russell

, Darzi

. The effect of a second-generation 3D endoscope on the laparoscopic precision of novices and experienced surgeons. Surg Endosc, 1999; 13:1087–1092.

Yamauchi

. Clinical demands and evaluations of 3D and augmented visualizationMICCAI 2002 TutorialSeptember 25, 2002. http://yamau.ch/data/2002MICCAItutorial-e-pdf

Kong

S-H

, Oh

B-M

, Yoon

et al. Comparison of two- and three-dimensional camera systems in laparoscopic performance: A novel 3D system with one camera. Surg Endosc, 2010; 24:1132–1143.

Hofmeister

, Frank

, Cuschieri

, Wade

. Perceptual aspects of two-dimensional and stereoscopic display techniques in endoscopic surgery: Review and current problems. Surg Innov, 2001; 8:12–24.

Storz

, Buess

, Kunert

, Kirschniak

. 3D HD versus 2D HD: Surgical task efficiency in standardised phantom tasks. Surg Endosc, 2012; 26:1454–1460.

10.

Smith

, Day

, Rockall

, Ballard

, Bailey

, Jourdan

. Advanced stereoscopic projection technology significantly improves novice performance of minimally invasive surgical skills. Surg Endosc, 2012; 26:1522–1527.

11.

Mishra

, Hanna

, Brown

, Cuschieri

. Optimum shadow-casting illumination for endoscopic task performance. Arch Surg, 2004; 139:889–892.

12.

Nicolaou

, James

, Lo

, Darzi

, Yang

G-Z

. Invisible shadow for navigation and planning in minimal invasive surgery. Duncan

, Gerig

. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2005. Berlin: Springer, 2005; 3750:25–32.

13.

Shimotsu

, Cao

CGL

. The effect of color-contrasting shadows on a dynamic 3-D laparoscopic surgical task. IEEE Trans Systems Man Cybernet A Systems Humans, 2007; 37:1047–1053.

14.

van Beurden

MHPH

, Kuijsters

, Ijsselsteijn

. Performance of a path tracing task using stereoscopic and motion based depth cues. Paper presented at the 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX), Trondheim, Norway, June 21–23, 2010.

15.

Johnston

, Cumming

, Parker

. Integration of depth modules: Stereopsis and texture. Vis Res, 1993; 33:813–826.

16.

Kersten

, Mamassian

, Knill

. Moving cast shadows induce apparent motion in depth. Perception, 1997; 26:171–192.

17.

Gannot

. Thermal imaging bundle—A potential tool to enhance minimally invasive medical procedures. IEEE Circuits Devices Mag, 2006; 21,6:28–33.

18.

Marcucci

, Coletta

, Aceti

, Endres

, Moritz

. The use of infrared imaging in laparoscopic surgery. Paper presented at the IEEE 17th Annual Conference on Engineering in Medicine and Biology Society, Montreal, Quebec, Canada, 1995.

19.

Marcucci

, Freeman

, Quinn

et al. Infrared imaging in minimally invasive surgery. Paper presented at the IEEE 20th Annual International Conference on Engineering in Medicine and Biology Society, Hong Kong, 1998.

20.

Matsushita

, Miyati

, Nakayama

et al. Qualitative near-infrared vascular imaging system with tuned aperture computed tomography. J Biomed Opt, 2011; 16:076004–076005.

21.

Lee

, Elson

, Neil

et al. Solid-state semiconductors are better alternatives to arc-lamps for efficient and uniform illumination in minimal access surgery. Surg Endosc, 2009; 23:518–526.

22.

Lee

, Yang

et al. Designing an active motor skill learning platform with a robot-assisted laparoscopic trainer. 33rd Annu Int Conf IEEE EMBS, 2011; 4534–4537.

23.

Westerman

, Cribbin

. Individual differences in the use of depth cues: Implications for computer- and video-based tasks. Acta Psychol, 1998; 99:293–310.