Determining the Threshold of Time-Delay for Teleoperation Accuracy and Efficiency in Relation to Telesurgery

Abstract

Background: Advances in robotics have made teleoperated surgical procedures a feasible means of treating patients in remote locations. In this study a suite of experiments was performed to investigate the influence of time-delay on teleoperation accuracy and efficiency during a path-following task. Materials and Methods: Subjects used a Phantom Omni 6-degrees of freedom (dof) input device (Sensable, Triangle Park, NC) to move the end-effector of a Mitsubishi (Tokyo, Japan) PA-10 7-dof robotic manipulator along a prescribed path. End-effector motion was recorded using a video motion capture system. Time-delays ranging from 0 to 2.5 s were artificially imposed. Performance was quantified by time to complete the task, path length, and square root-mean-square (RMS) error. Randomization of time-delay order and allowance for practice runs reduced the learning effect. An imposed time limit and pacing were used to negate the move-and-pause strategy that emerged in early trials. Results: Time to complete the task and RMS error generally increased with increasing time-delay. Path length also generally increased, but not as consistently. With imposed pacing, RMS error continued to increase beyond 1.5 s, and some subjects were not able to complete the task in the allotted 90 s. Conclusions: The results suggest a threshold of time-delay in the range of 1.5–2.0 s. Beyond 1.5 s, subjects adopted a move-and-pause strategy that increased completion time to preserve path-tracking accuracy. If paced, tracking accuracy tended to degrade substantially beyond 1.5 s. A strong learning effect was evident, and experienced teleoperators performed substantially better than novices.

Introduction

There is a growing shortage of physicians and surgeons worldwide, and there is a concomitant aging of the population, which is living longer and requiring additional resources, such as surgical care. As we move forward, we will need to develop better technologies and training programs to support this growing need.

Surgical care has evolved from traditional open surgery where the surgeon, surgical team, and patient are in sterile isolation with limited access. The introduction of robotics and telecommunications into the surgical arena has made significant changes in both clinical care and pedagogy. Advances in the fields of robotics and telecommunications have enabled surgeons to robotically operate on patients from a distance—known as telesurgery.

In theory, telesurgery could be performed in remote or inaccessible locations such as rural areas, battlefields, polar regions, or even during human spaceflight, in situations where a patient would not otherwise have access to timely and capable surgical care. The presence of robotic da Vinci ^® Surgical Systems from Intuitive Surgical (Sunnyvale, CA) in thousands of medical centers worldwide has led to widespread acceptance of robotic surgery.

As we explore the utility of telesurgery in patient care, we must increase our understanding of the barriers and challenges we face in projecting surgical expertise remotely. Over the past several decades, research in telesurgery has provided a foundation from which we can build a greater understanding and appreciation for this innovative approach.

In the first telesurgery demonstration (2001), a surgical team in New York City performed a laparoscopic cholecystectomy, using the ZEUS robotic system, on a woman in a hospital in Strasbourg, France, under a 155 ms time-delay.¹ A “trauma pod” robotic system under development by SRI International (Menlo Park, CA) could potentially be deployed into combat areas or remote regions by helicopter or airdrop.² Robotic surgery during spaceflight is of particular interest to NASA because they have identified a 90% probability of serious illness/injury during a future long-range space mission,³ and such an incident might require surgical intervention before a return to Earth is possible. At the same time, NASA cannot guarantee that a surgeon would be present on all long-range flights (beyond low-Earth orbit). To address this need, the SRI International M7 robot has been used to perform telerobotic surgery on human phantoms in a weightless environment (NASA C-9 aircraft),⁴ in the Aquarius subsea habitat (NEEMO 9 and 12),⁵ and in the high desert using unmanned airborne vehicles for communication support.⁶

Although telesurgery is conceivable in these scenarios, they all include inherent time-delay due to telecommunication latency, which can reduce accuracy and efficiency. The RAVEN surgical robot, developed by the University of Washington (in Seattle) Bio-Robotics Laboratory, was used to perform the Fundamentals of Laparoscopic Surgery (FLS) training tasks with simulated time-delays of 0, 250, and 500 ms in extreme environments (NEEMO 12 and the high desert).^5,6 These experiments showed a higher error rate with increased delay.⁷

Sheridan and Ferrell⁸ performed an experiment where the user operated a system that had two translations degrees of freedom (dof) and was required to grasp a small block with the slave system under simulated time-delays of 0.0, 1.0, 2.1, and 3.2 s. The subjects had a clear view of the slave system but not of the master system. These investigators predicted the time to complete the task using the following equation: \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}t ( I ) = t_0 ( I ) + ( t_{ \rm r} + t_{ \rm d} )*N ( I ) + t_{ \rm d} \tag{1}\end{align*} \end{document}

where I is the index of task difficulty, t ₀ is the time it took the participant without a time-delay, t _r is the participant's reaction time at the start (set at 0.2 s), N(I) is the number of corrective movements, and t _d is the time-delay. As the time-delay increased, the time to complete the task increased because a move-and-wait strategy was used.

Marescaux et al.¹ showed that telerobotic surgery is feasible by conducting an experiment in which the surgeons performed a laparoscopic cholecystectomy on a pig using the ZEUS robotic system. The surgeon's console was in Strasbourg, France, and the pig was in Paris. In this experiment there was an artificially introduced time lag that increased the time-delay from 20 ms up to 551.1 ms. Surgeons believe that 330 ms was the limit of the acceptable time-delay in terms of safety. However, no statistical data were provided showing the performance degradation. They also successfully carried out remote laparoscopic surgery on six pigs using the ZEUS robotic system with the operators in New York and the pigs in Strasbourg.⁹ The total time-delay in the system was 155 ms. Three surgeons in New York evaluated the quality of the image, the impact of time-delay on their performance, the coordination and safety of the use of electrocautery, and the overall safety of the procedure. Scores were evaluated on a 0–10 scale, where 0 is the worst possible scenario and 10 is the best. The scores were 9.1 for the image quality, 8.5 for the impact of the time-delay, 9.2 for the electrocautery, and 8.7 for the overall safety.⁹ As in other studies, their experiment demonstrated that telesurgery can be performed safely if the time-delay is small, but they did not explore the upper bound of time-delay.

Anvari and co-workers^10

–13 conducted similar research in Canada and experienced similar results. They established a telerobotic surgical program between St. Joseph's Hospital and North Bay General Hospital. The distance between the two hospitals was more than 400 km. Before each surgeon performed telesurgery and telementoring, the surgeon's ability to adapt to varying latencies while suturing, cutting, and other two-handed maneuvering tasks was evaluated. The surgeons were able to complete all tasks with a latency of 500 ms. However, the higher latency was associated with a longer time to complete the task and higher error rates. After it was shown that surgeons could perform telesurgery between the two hospitals with reasonable results, they began to recruit patients who required advanced laparoscopic surgery. The patients were given the option of open surgery, teleoperated surgery, and traveling to St. Joseph's Hospital for surgery. All of them opted for the telerobotic surgery. There were two surgeons: one on-site at North Bay General and one at St. Joseph's Hospital. There was also a surgical team present at North Bay General to set up and position the robot for surgery. Additionally, there were information technology specialists present at both sites to assist during the surgical cases should a problem arise. The ZEUS-TS was used in all cases. The total time-delay between the two hospitals was between 135 and 140 ms. All 22 telerobotic surgeries were performed without any major intraoperative complications. The postoperative recovery was uneventful for all 22 patients. Anvari and co-workers^10

–13 have successfully demonstrated the safety and feasibility of robot-assisted telepresence surgery.

Rayman et al.¹⁴ conducted a study in which 8 test subjects performed four maneuvers. First, pick up a cone with the left hand and place it in a circle, then put it back to the original position with the right hand, and pass it back and forth six times. The next task was to pick up a rod with the left hand. pass the rod through three hoops without touching the hoops, and pass it back and forth through the hoops four times. The next task was to pick up an orange ring with a black line with the left hand such that the left gripper is on the line and then maneuver the ring such that the right gripper is on the line without dropping the ring; repeat this back and forth four times. Lastly, the subjects were to pick up a 6-0 needle with the right hand, pass the needle though a right dot and exit on a left dot, and then retrieve the needle with the left hand and reload it back to the right hand, repeating the process six times. Rayman et al.¹⁴ used the ZEUS robotic platform with NetDisturb software placed between the master and slave consoles. This allowed them to impose an artificial time-delay between the master and slave consoles. They did sequential trials of 0–600 ms and found that there were no significant differences between the average task times compared with the zero latency. They also found that the error rate remained low and decreased up until 500 ms. Average error at 600 ms was no worse than the error without a time-delay. They also repeated the experiment with randomized time-delays. They found that the randomized time-delay trials took longer to complete the tasks than the sequential trials. Despite the errors increasing after a 400-ms time-delay, all the errors in the randomized trial were less than the average number of errors for the no imposed time-delay case in the sequential trials.¹⁴

Lum et al.¹⁵ conducted a study by performing a block transfer with the RAVEN surgical robot. They used 3 subjects and three time-delays: 0, 250, and 500 ms. They found that as the time-delay increased, the time it took to complete the task also increased. There was a significant difference between the different time-delays for total time to finish in all the participants except for one participant who did not have a significant difference between the 0 ms and 250 ms time-delays.¹⁵

These experiments demonstrated that telerobotic surgery is feasible but becomes problematic with increasing time-delay. Some studies suggest a threshold of time-delay but do not provide strong objective evidence in support of the threshold value chosen. There is typically some description of the effect of latency on task performance, such as the time taken to tie a surgical knot, the time taken to perform the FLS procedure, or the number of errors made, but there is little quantitative description of the effect of latency on manipulator path tracking accuracy. Furthermore, there is wide disagreement on the threshold of time-delay. Values range from 250 ms to 1 s and appear to be specific to the task performed and the robotic equipment used. Our study was specifically designed to assess the time-delay threshold using a generic task and a relatively simple manipulator system so that the results can be more widely applicable.

Materials and Methods

Experiments were performed using one slave system location (a laboratory at the University of Cincinnati [UC], Cincinnati, OH) and two master system locations (a second laboratory at UC or a laboratory at SRI International in Menlo Park). The slave location contained the test article, robot, slave computer (to drive the robot), vision system, and motion capture system. The vision system consisted of a pair of Point Grey (Richmond, BC, Canada) Flea^®2 cameras aligned to provide a stereoscopic view of the test article. A PA-10 7-dof robotic manipulator (Mitsubishi Heavy Industries, Tokyo, Japan) was used to execute the teleoperated path-following task. This is a nonsurgical robot. It was chosen because its generic and simple design would avoid the complicating factors of using a specific surgical robot and would at the same time allow the results to be interpretable for any robotic surgery platform.

The test article (Fig. 1a and b) consisted of eight gates, arranged in a square pattern. Each Y-shaped gate consisted of a crossbar with vertical spring on its ends supporting markers and a single post supporting the crossbar. The gates were placed upright in holes in a plywood base arranged in a square array with a pitch of 25.4 mm that allowed for multiple configurations to be adopted. Four high gates were placed in the corners of a square with sides of 101.6 mm in length. Four low gates were placed at the midpoints of the sides, resulting in a three-dimensional path as shown in Figure 1b. A motion capture system (BTS Bioengineering, Milan, Italy) consisting of eight infrared cameras was used to capture the robot end-effector motion via a retroreflective marker placed on a small rod extending from the end-effector (Fig. 1a and c). The camera system was calibrated to provide positional accuracy of 0.5 mm (Fig. 1c). Retroreflective markers were initially mounted on the gate ends, and the gate positions were recorded in a static capture. The gate markers were then replaced with nonreflective red beads that were visible to the operator but would not be picked up by the infrared motion capture system. This avoided difficulties with marker occlusion/confusion as the end-effector marker passed close to the gate markers.

Fig. 1.

Experiment setup: (a) test article showing gates and robot end-effector marker, (b) ideal path through the gates, (c) slave location setup, including robot, test article, motion capture system, and stereo-vision cameras, and (d) master location including the Phantom Omni 6-degrees of freedom controller and the stereo-vision headset.

The master laboratory housed the control hardware consisting of a Phantom Omni 6-dof input device (Sensable, Triangle Park, NC) and the master computer (Fig. 1d). Test article visualization was provided by an eMagin (Bellevue, WA) three-dimensional headset at UC or a computer screen at SRI. Control inputs and video images were transmitted via the Internet using transmission control protocol–Internet protocol communication.

Subjects completed a questionnaire to assess their previous experience with radio controllers, video game controllers, and robot control operations. Based on the responses, the participants were classified into Experienced and Novice groups. If the participant had much experience with three-dimensional video games or if the participant had much prior experience with teleoperation of radio-controlled vehicles or teleoperation of a robotic system, he or she was placed in the Experienced group. If the participant did not have those experiences, he or she was placed in the Novice group.

The experiments were completed in four phases. The first-phase experiment was performed between the master location at SRI International and the slave location at UC. This preliminary experiment was limited to a two-dimensional path-following task. The test article was viewed from directly above, and the robot vertical axis was locked. A single video image was viewed on the master computer. Artificial time-delays of 0, 0.5, 1.0, and 1.5 s were imposed on the inherent communication time-delay (which averaged approximately 0.25 s). This first phase was performed to orient the test subjects to the input device, to build familiarity with different time-delays, and to quantify inherent public network communication time-delays. Five male subjects, ranging in age from 23 years to 46 years, were instructed to move the robot end-effector marker through the midpoints of each gate as expediently and accurately as they could. Each subject completed 16 runs, that is, four trials at each of the four time-delay conditions. The runs were completed in order of increasing time-delay and then in order of decreasing time-delay. This sequence was then repeated after a brief rest.

Experiment Phases 2–4 were completed using master and slave locations in two buildings on the main UC campus. The inherent network time-delay for control commands was less than 1 ms, but that for video transmission was of the order of 250–500 ms. Additional time-delays of 0, 0.5, 1.0, 1.5, 2.0, and 2.5 s were artificially imposed. Twelve male subjects, ranging in age from 20 years to 46 years, were instructed to maneuver the end-effector marker through the midpoints of the gates, this time along the three-dimensional path (straight line segments between the centers of successive gates; Fig. 1b) using the stereo-vision system. In Phase 2, subjects completed the runs in order of increasing time-delay. In Phase 3, the order of time-delays was randomized to mitigate the learning effect. During Phase 4, subjects were given a 90-s time limit within which to complete the task under all time-delay conditions. Subjects were provided with both an audible cue (master computer beep) and visual cue (a computer-generated graphic in the top left corner of the field of view) that indicated when each gate should be traversed and thus assisted with pacing.

In all phases, the test subjects were given the opportunity to perform practice runs of the task prior to data capture to negate the short-term learning effect. The number of practice runs was the same for all subjects in each case.

Subject performance was evaluated based on three parameters: total time to complete the task, total path length, and root-mean-square (RMS) error. The total time was recorded by the motion capture system as the time between passing through the initial gate, completing one lap around the test article, and then passing through the initial gate again. The total path length (L) was calculated as the sum of all straight line segments' lengths (L_i ) between recorded end-effector marker positions (Eqs. 2 and 3): \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}L_i = \sqrt{ \mid x_{i + 1} - x_i \mid^2 + \mid y_{i + 1} - y_i \mid^2 + \mid z_{i + 1} - z_i \mid^2} \tag{2}\end{align*} \end{document} \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}L = \mathop\sum_{i = 1}^{N - 1} L_i \tag{3}\end{align*} \end{document}

where x_i ,y_i ,z_i are the spatial position coordinates of the end-effector, as recorded by the motion capture system. The RMS error was used to quantify deviations from the ideal path and was calculated using Eq. 4: \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}RMS = \sqrt { \frac { \sum_ { i = 1 } ^N [ \left( x_i - P_ { x_ { i } } \right) ^2 + \left( y_i - P_ { y_ { i } } \right) ^2 + \left( z_i - P_ { z_ { i } } \right) ^2 ] } { 2 } } \tag { 4 } \end{align*} \end{document}

where \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document}$$P_{x_{i}} , P_{y_{i}} , P_{z_{i}}$$ \end{document} are the coordinates of the closest point on the ideal path and N is the total number of motion capture data points.

The three parameters were averaged across subjects. Statistical comparisons were performed using an analysis of variance when comparing multiple groups and using a Student's t test when comparing two groups. A significance level of α=0.05 was used for all comparisons.

Relevance to Robotic Surgery

At the time that these experiments were conducted, actual surgical robots (such as the da Vinci Surgical System) were not available to the investigators. Instead, a general purpose robot (Mitsubishi PA-10) and relatively simple input device (Phantom Omni) were used together with black and white video imagery. This system was not capable of motion scaling (up to 5:1 slave-to-master ratio), visual magnification (as much as 15 times), or haptic feedback, as commonly used in robotic surgery. The viewing system (headset) was also of much lower resolution than surgical viewing systems. However, the experiment was designed to compare relative changes in path-following accuracy and efficiency, rather than absolute measures of precision. For this reason, the results have been normalized based on the results for the zero time-delay case. For each of the time-delays in the trials, the average error was divided by the error of the nominal (zero time-delay) case. This approach allowed the investigators assess whether there is a time-delay threshold at which performance becomes more substantially degraded than in previous time-delays, or the error trend rises more steeply, or there is a change in operator strategy to cope with a marked increase in task difficulty.

Results

Phase 1

The results for Phase 1 are summarized in Figure 2 and Table 1. The inequalities provided in Table 1 indicate statistically significant differences between parameters according to the time-delay groups. The error bars in in Figure 2, as in all subsequent figures, indicate the 95% confidence interval. The most noticeable result in Figure 2 is that the average RMS error increased as the imposed time-delay increased. Furthermore, there is a greater increase in RMS error between 1 s and 1.5 s of time-delay than between shorter time-delays. This is also reflected in the fact that there is a statistically significant difference in RMS error between the 1.5 s time-delay and all of the lesser time-delays. The trend is less clear with average path length because the path length was shortest at the 1 s time-delay, but there was nevertheless a general tendency for path length to increase, and the longest path length occurred for the 1.5 s time-delay. The average time to complete the task increased consistently as the time-delay increased, and there was a significant difference between the time to complete the task at 1.5 s versus the 0 s time-delay.

Fig. 2.

Summary of root-mean-square (RMS) error, path length, and time-to-complete results for Phase 1 experiment. The robot end-effector was limited to movement only in the horizontal plane, and subjects viewed a monoscopic image of the test article from directly above.

Table 1.

Experiment Phases and Statistical Results

	TIME-DELAY GROUP (TIME-DELAY IN S)
PHASE, PARAMETER	1 (0)	2 (0.5)	3 (1.0)	4 (1.5)	5 (2.0)	6 (2.5)
1
RMS	1<4	2<4	3<4	4>1, 2, 3	NA	NA
Path length					NA	NA
Time	1<4			4>1	NA	NA
2
RMS
Path length
Time	1<4			4>1
3
RMS
Path length
Time	1<6	2<6				6>1, 2
4^a
RMS	1<4, 5, 6	2<5, 6	3<5, 6	4>1	5>1, 2, 3	6>1, 2, 3
Path length
Time

Results are shown for only the Experienced subject group in Phase 4. For all subjects combined, there were no significant differences in the parameters.

NA, not applicable; RMS, root-mean-square.

Phase 2

Between 0 to 1.5 s, the time to complete the task steadily increased (Fig. 3), and, as in Phase 1, the time at 1.5 s was greater than at 0 s time-delay. However, at 2 and 2.5 s of time-delay the time to complete was less than at 1.5 s. There were no significant differences in RMS error and path length based on time-delay (Table 1 and Fig. 4), although the path length generally increased as the time-delay increased.

Fig. 3.

Time to complete the task result for Phase 2 (three-dimensional path, time-delay order not randomized).

Fig. 4.

Comparison of average root-mean-square (RMS) error for Phases 2 and 3. Phase 3 errors were consistently higher than Phase 2 errors and with greater variance because of randomization of time-delay order.

Phase 3

With the order of time-delays randomized, there is no clear trend in RMS error through the different time-delay cases (Fig. 4). However, it is noticeable that the RMS error was consistently higher in Phase 3 than in Phase 2. The path length generally increased with time-delay, but there was a noticeable decrease in path length at 2 s time-delay (Fig. 5). Furthermore, this trend was consistent for Phases 2 and 3. Also, path length was consistently higher in Phase 3 than in Phase 2. The trends are again consistent between Phases 2 and 3 when comparing time to complete the task (Fig. 6). The time to complete the task generally increased with increasing time-delay, and the time to complete the task at 2.5 s time-delay was significantly greater than all shorter time-delays.

Fig. 5.

Comparison of average path length for Phases 2 and 3. Path length was consistently longer in Phase 3 because of the randomization of time-delay. Variance, however, was almost identical between the two experimental phases.

Fig. 6.

The average time to complete the task generally increased as the time-delay increased. The results for Phase 3 were nearly identical to those for Phase 2, although the time to complete the task was consistently slightly higher for Phase 3.

Phase 4

With the order of time-delays randomized and the timing cues implemented, the time to complete the task was consistently close to 90 s throughout this phase, as intended. With the timing cues (Fig. 7, “Timed” values), the RMS error consistently increased with increasing time delay (r ² value of 0.845 based on a linear regression). The RMS error values from Phase 3 (Fig. 7, “Non-timed” values) were consistently lower and also showed a greater amount of variance (r ² value of 0.539). When comparing the timed trial with the non-timed trial, all of the errors were divided by the nominal case of the non-timed case, in order to normalize the results.

Fig. 7.

Comparison of root-mean-square (RMS) error for timed and non-timed groups. The RMS error was higher under conditions of subject pacing (timed), but the trend was more consistent as reflected by the tighter curve fit.

To further explore the underlying trends, the RMS error data were separated between subjects who had more experience with teleoperation under time-delay or operation of remote control vehicles (Fig. 8, “Experienced” group) versus those who had comparatively little experience in these areas (Fig. 8, “Novice” group). When comparing novices with the experts, all of the errors were divided by the nominal case of the experts, in order to normalize the results. There is clearly a substantial difference between these groups. The Experienced subjects exhibited normalized RMS errors that ranged from around 1 to 1.15 with a generally consistent increase in RMS error with increasing time-delay and a smaller amount of variance (as shown by the tighter confidence interval bars). The Novice subjects exhibited a less consistent increase in RMS errors that ranged from 1.1 to 1.35 and a greater amount of variance. For the Experienced group, the RMS errors for the 1.0 s time-delay and shorter were significantly less than for the 1.5 s time-delay and longer (Table 1). The change in path length was less clear, although there was generally an increase in path length as time-delay increased, and this was more noticeable with the Experienced group.

Fig. 8.

Comparison of root-mean-square (RMS) error for Experienced versus Novice teleoperators. The experienced group performed substantially better in terms of trajectory tracking accuracy, as reflected in the substantially lower RMS values.

Discussion

All phases of the experiment demonstrated that as the time-delay increased, the task difficulty increased. This was reflected as an increase in RMS error (reduced manipulation accuracy), an increase in the time to complete the task (reduced task efficiency), or both. Path length also generally increased as time delay increased but was found to be a less sensitive indicator of manipulation accuracy than RMS error. In Phase 1, the fact that RMS error increased more substantially between the 1.0 s and 1.5 s time-delays than between the other time-delays provided some early indication that task difficulty increases greatly as the time-delay approaches 1.5 s.

The order of time-delay was not randomized in Phases 1 and 2, and it appears that this allowed a short-term learning effect to occur. In particular, there was a less noticeable increase in all performance parameters (increases reflect poorer performance) in Phase 2 compared with Phases 3 and 4 where the order of time-delays was randomized. It was reasoned that preceding trials offered the subject an opportunity to gain proficiency at the task, thus allowing better performance on succeeding trials. To further diminish the learning effect, in Phases 3 and 4 the subjects completed multiple practice runs at different time-delay levels before the data capture runs.

In Phases 2 and 3 another interesting phenomenon occurred. When the time-delay exceeded 1.5 s, the RMS error decreased or did not increase as expected. However, the time to complete the task continued to increase with increasing time-delay. It was observed that, starting at a time-delay of 1.5 s, subjects generally started to use a discontinuous movement pattern, interspersing frequent pauses and controller position resets (to avoid singular configurations) between movements. These pauses allowed the observed video images to “catch up” with the controller inputs and robot responses, thus negating the time-delay based impact on movement accuracy but at the expense of an increase in the time to complete the task. To avoid this compensatory strategy and explore the impacts of time-delay on manipulation accuracy, a time limit of 90 s was introduced in Phase 4, along with audible and visual cues to keep the subjects on pace. With this measure in place, the RMS error tended to increase more monotonically; however, the pattern was still not as clear as expected. To further dissect the underlying causes, the subject data were divided between subjects with moderate to extensive teleoperation experience and those with minimal or no experience. A clear pattern then emerged. Subjects with teleoperation experience performed significantly better on the task in terms of accuracy, as reflected in their lower RMS error values, and were also more successful in completing the task within the allotted 90-s interval. The Novice users sometimes resorted to the move-and-pause strategy observed in the untimed tests, and thus preserved positional accuracy but exceeded the time limit.

As mentioned previously, the robot used in this experiment is not a surgical robot, and this was done to allow wider interpretation of the results. At the same time, we recognize the value in performing further work to confirm if the results are consistent when a true surgical robot is used. Furthermore, additional consequences of time-delay may be identified. For instance, a time delay as short as 500 ms may be problematic if accidental bleeding occurs. The operating region will fill with blood, and further tasks would be delayed until the bleeding is stopped and the operating region is cleaned. Consequently, even a relatively brief time-delay could lead to a cascade of problems that would lengthen the procedure or increase the risk of complications or even a failed procedure. Such concerns clearly need to be addressed in detail.

Conclusions and Further Work

It was found that at around 1.5 s of imposed time-delay a simple task of maneuvering a robotic manipulator end-effector along a three-dimensional path becomes substantially more difficult than at lesser time-delays. It was also shown that there is a noticeable learning effect between the different time-delay cases and that experienced teleoperators performed far better than novices. It is clear, though, that even experienced teleoperators would not perform adequately to conduct surgical procedures with acceptable safety and efficiency beyond 2 s of time delay.

For spaceflight, this would limit telerobotic surgery to that performed by an Earth-based surgeon on a patient on a spacecraft in low-Earth orbit. For exploration-class missions beyond low-Earth orbit, an alternative solution would have to be found. One method to reduce the detrimental effects of latency would be to use semiautonomous robotic surgery procedures, that is, segmented autonomous movements with human supervisory control and operator decision/intervention points between segments.

A limitation of this study is that it was performed using a general purpose (nonsurgical) robot and a relatively abstract test article. Although this approach has advantages in terms of repeatability and performance quantification, as pointed out above, an important next step would be to perform a similar time-delay experiment using a true robotic surgery platform, such as the da Vinci Surgical System or similar instrument, to validate or adjust the results in a high-fidelity surgical setting. This study, and additional follow-on studies, will also explore practical surgery questions such as how to prepare the operating field, placement of the patient on the operating table, skin incisions, open or minimally invasive procedures, and suturing/skin closure.

The use of remote surgical robotics shows promise. Bandwidth and delay in communications are only a few of the challenges that we face in incorporating this innovation into care. Research has been conducted in many other facets of surgical care, including tele-anesthesia,¹⁶ image quality from distant sites, simulated spaceflight, and how telemedicine actually works in the operating room.^17,18

Time-delay is an important factor and may prove to be a limiting factor using traditional communication networks. However, sensors and artificial intelligence may provide significant adjuncts in ameliorating this challenge moving forward.

Footnotes

Acknowledgments

The team of researchers worked closely with surgical teams in the conduct of their research. This work was supported in part by a grant from the Ohio Space Grant Consortium.

Disclosure Statement

No competing financial interests exist.

References

Marescaux

, Leroy

, Gagner

, Rubino

, Mutter

, Vix

, Butner

, Smith

. Transatlantic robot-assisted telesurgery. Nature, 2001; 413:379–380.

Friedman

DCW

, Dosher

, Kowalewski

, Rosen

, Hannaford

. Automated tool handling for the trauma pod surgical robot. In: 2007 IEEE International Conference on Robotics and Automation. Piscataway, NJ: IEEE, 2007; 1936–1941.

Allen

, Burnett

, Charles

, Cucinotta

, Fullerton

, Goodman

, Griffith

, Kosmo

, Perchonok

, Railsback

, Rajulu

, Stilwell

, Thomas

, Tri

. Guidelines and capabilities for designing human missions. Publication Number NASA TM-2003-210785. Washington, DC: NASA, 2003.

Lum

MJH

, Rosen

, King

, Friedman

DCW

, Donlin

, Sankaranarayanan

, Harnett

, Huffnam

, Doarn

, Broderick

, Hannaford

. Telesurgery via unmanned aerial vehicle (UAV) with a field deployable surgical robot. Stud Health Technol Inform, 2007; 125:313–315.

Doarn

, Anvari

, Low

, Broderick

. Evaluation of teleoperated surgical robots in an enclosed undersea environment. Telemed J E Health, 2009; 15:325–335.

Harnett

, Doarn

, Rosen

, Hannaford

, Broderick

. Evaluation of unmanned airborne vehicles and mobile robotic telesurgery in an extreme environment. Telemed J E Health, 2008; 14:534–544.

Lum

MJH

, Rosen

, Lendvay

, Sinanan

, Hannaford

. Effect of time delay on telesurgical performance. In: Proceedings, IEEE International Conference on Robotics and Automation. Piscataway, NJ: IEEE, 2009; 4246–4252.

Sheridan

, Ferrell

. Remote manipulative control with transmission delay. In: IEEE Transactions on Human Factors in Electronics. Piscataway, NJ: IEEE, 1963; 25–29.

Marescaux

, Leroy

, Gagner

, Rubino

, Mutter

, Vix

, Butner

, Smith

. Transatlantic robot-assisted telesurgery. Nature, 2001; 413:379–380. Erratum in Nature 2001;414:710.

10.

Anvari

, McKinley

, Stein

. Establishment of the world's first telerobotic remote surgical service: For provision of advanced laparoscopic surgery in a rural community. Ann Surg, 2005; 241:460–464.

11.

Anvari

. Robot-assisted remote telepresence surgery. Semin Laparosc Surg, 2004; 11:123–128.

12.

Anvari

. Remote telepresence surgery: The Canadian experience. Surg Endosc, 2007; 21:537–541.

13.

Anvari

, Broderick

, Stein

, Chapman

, Ghodoussi

, Birch

, McKinley

, Trudeau

, Dutta

, Goldsmith

. The impact of latency on surgical precision and task completion during robotic-assisted remote telepresence surgery. Comput Aided Surg, 2005; 10:93–99.

14.

Rayman

, Croome

, Galbraith

, McClure

, Morady

, Peterson

, Smith

, Subotic

, Van Wynsberghe

, Primak

. Long-distance robotic telesurgery: A feasibility study for care in remote environments. Int J Med Robot, 2006; 2:216–224.

15.

Lum

, Rosen

, Lendvay

, Wright

, Sinanan

, Hannaford

. TeleRobotic Fundamentals of Laparoscopic Surgery (FLS): Effects of time delay—Pilot study. In: 30th Annual International IEEE EMBS Conference. Piscataway, NJ: IEEE, 2008; 5597–5600.

16.

Cone

, Gehr

, Hummel

, Rafiq

, Doarn

, Merrell

. Case report of remote anesthetic monitoring using telemedicine. Anesth Analg, 2004; 98:386–388.

17.

Doarn

, Hufford

, Low

, Rosen

, Hannaford

. Telesurgery and robotics: A roundtable discussion. Telemed J E Health, 2007; 13:369–380.

18.

Doarn

. Telemedicine in tomorrow's operating room: A natural fit. Semin Laparosc Surg, 2003; 10:121–126.