Accuracy of Touch-Based Registration During Robotic Image-Guided Partial Nephrectomy Before and After Tumor Resection in Validated Phantoms

Abstract

Aim:

Image-guided surgery (IGS) allows for accurate, real-time localization of subsurface critical structures during surgery. No prior IGS systems have described a feasible method of intraoperative reregistration after manipulation of the kidney during robotic partial nephrectomy (PN). We present a method for seamless reregistration during IGS and evaluate accuracy before and after tumor resection in two validated kidney phantoms.

Materials and Methods:

We performed robotic PN on two validated kidney phantoms—one with an endophytic tumor and one with an exophytic tumor—with our IGS system utilizing the da Vinci Xi robot. Intraoperatively, the kidney phantoms' surfaces were digitized with the da Vinci robotic manipulator via a touch-based method and registered to a three-dimensional segmented model created from cross-sectional CT imaging of the phantoms. Fiducial points were marked with a surgical marking pen and identified after the initial registration using the robotic manipulator. Segmented images were displayed via picture-in-picture in the surgeon console as tumor resection was performed. After resection, reregistration was performed by reidentifying the fiducial points. The accuracy of the initial registration and reregistration was compared.

Results:

The root mean square (RMS) averages of target registration error (TRE) were 2.53 and 4.88 mm for the endophytic and exophytic phantoms, respectively. IGS enabled resection along preplanned contours. Specifically, the RMS averages of the normal TRE over the entire resection surface were 0.75 and 2.15 mm for the endophytic and exophytic phantoms, respectively. Both tumors were resected with grossly negative margins. Point-based reregistration enabled instantaneous reregistration with minimal impact on RMS TRE compared with the initial registration (from 1.34 to 1.70 mm preresection and from 1.60 to 2.10 mm postresection).

Conclusions:

We present a novel and accurate registration and reregistration framework for use during IGS for PN with the da Vinci Xi surgical system. The technology is easily integrated into the surgical workflow and does not require additional hardware.

Introduction

Image-guided surgery (IGS) during robotic partial nephrectomy (PN) facilitates accurate, real-time identification of surgical anatomy and subsurface features of the kidney by registering preoperative cross-sectional imaging to intraoperative patient anatomy (i.e., image space to physical space). There have been many efforts to develop IGS systems with goals of improving operative and warm ischemia times, procedural learning and safety, as well as preserving renal parenchyma during tumor resection.^1
–3 To do this, accurate registration is essential. Prior registration methods lack precision or require invasive fiducial placement and additional hardware, which interfere with surgical workflow. We have previously described a robotic, touch-based registration method using the da Vinci robotic system.⁴ By intraoperatively tracing the tip of a robotic instrument over surface anatomy, a three-dimensional (3D) surface point set can be generated and registered to a preoperatively segmented patient image.^5
–7 Both the anatomy from segmented patient images and robotic instruments can be displayed in TilePro™ and synchronously move and track accurately with manipulation of the robotic instruments and endoscope.

Although we have shown the accuracy of this method in identifying critical structures during PN, registration accuracy can be affected over time by changes in patient anatomy during dissection, kidney mobilization, and tumor resection. Therefore, it is necessary to periodically reregister during IGS for PN to ensure precise, real-time identification of patient anatomy. However, there have been no studies evaluating reregistration accuracy for IGS after manipulation of the kidney for robotic PN on validated phantoms. We propose a novel method of achieving this by identifying ink fiducials in the operative field, using robotic touch-based registration. These fiducials are then periodically reidentified by touching them with the robotic instruments during PN, enabling a fast and simple method of reregistration.

This is the first system that utilizes a point-based reregistration method to provide updated patient anatomy for IGS during PN with the da Vinci Xi surgical system. In this article, we evaluate the accuracy of this method after kidney tumor resection on validated phantoms.

Materials and Methods

System overview

We have developed a method of IGS relying solely on data obtained from the clinical da Vinci robotic system to enable a touch-based registration method.⁴ Using the da Vinci application programming interface to obtain real-time kinematic data, our system computes the position and orientation of each link of the da Vinci's manipulators.⁸ This enables us to generate a set of surface data points by lightly tracing the anterior surface of the patient's kidney using a da Vinci manipulator (shown as red dots in Fig. 1). We have previously found that ∼28% of the anterior face of the kidney surface needs to be traced for accurate registration of the entire kidney.⁹ These surface data enable registration of 3D images of patient anatomy (segmented from preoperative axial imaging) to the physical anatomy using a standard surface-based registration algorithm.¹⁰ Graphical models of the manipulators are automatically displayed synchronously with motion of the robot manipulators using a custom software module built with 3D Slicer, an open-source medical image computing and visualization platform.¹¹ Graphical models are displayed directly in the da Vinci surgeon console through the console's TilePro interface (Fig. 1b, c).

FIG. 1.

(a) Conceptual illustration of touch-based registration and point-based reregistration. Initial image registration is carried out with a touch-based approach by lightly tracing the kidney surface with the robotic instrument (red dots). Reregistration updates are carried out with a faster point-based approach by touching discrete surgical ink markings placed on the kidney surface (yellow and purple dots). (b) Image guidance display during touch-based registration. (c) Image guidance display during point-based reregistration.

Our system also enables periodic reregistration after kidney manipulation during PN. Motion of the kidney surface—relative to its location at the time of initial registration—leads to misalignment between the physical organ and the organ shown on the image guidance display. Point-based reregistration helps correct such misalignment by enabling image registration to be periodically updated by identifying surface fiducial points throughout the case (shown as purple dots in Fig. 1). Immediately after initial registration, these fiducial points are identified on the physical kidney surface by lightly touching them with a robotic manipulator (shown as yellow dots in Fig. 1c). Locations of these points are therefore known relative to the initial image registration and can be reidentified throughout the case to maintain alignment between the physical system and the image guidance visualization. Point-based registration is not used for the initial registration itself as the ink marks are only introduced intraoperatively and are not present at the time of CT scanning. The initial surface-based registration enables us to accurately identify where these arbitrarily placed ink marks are in image space with respect to anatomy, facilitating quicker subsequent reregistrations.

Experimental

We investigated the accuracy of point-based reregistration during simulated robotic PN procedures on two validated, patient-specific, hydrogel kidney phantoms: (1) a left kidney with an ∼4 cm exophytic tumor, and (2) a right kidney with an ∼2 cm predominantly endophytic tumor. Phantoms were fabricated at the Simulation Innovation laboratory of University of Rochester from 3D-printed casts of segmented cross-sectional patient images. The casts were injected with polyvinyl alcohol hydrogel and assembled to create accurate hilar, renal parenchymal, and tumor anatomy. Prior experiments have validated these phantoms as adequate surgical rehearsal platforms with identical mechanical properties to living tissue.^12,13

For each experiment, the phantom was adhered to a rigid platform containing six spherical, “ground truth” fiducial markers (Fig. 2). Before experiments, the phantom platform was CT scanned with a section thickness of 0.3 mm using an xCAT ENT Scanner (Xoran Technologies LLC, Ann Arbor, MI). The kidney, tumor, hilar structures, and ground truth fiducials were manually segmented from the CT images. The segmentations produced 3D models of the kidney and tumor that were used for image guidance during experimentation. Segmentation of the fiducial markers enabled computation of the ground truth position of the kidney and tumor. Before surgery, surgeons drew a planned resection surface in each of the 3D kidney models. During surgery, the image guidance display included the planned resection contour as well as a 3 mm tumor margin (Fig. 3).

FIG. 2.

Experimental setup for evaluating registration and reregistration accuracy during simulated partial nephrectomy (R: right; S: superior).

FIG. 3.

Image-guided tumor resection enabled by our system.

Experiments were performed with a clinical da Vinci Xi system. The robot was draped and docked over the surgical bed in a position approximating a standard transperitoneal surgical approach. To determine the ground truth position of the kidney phantom relative to the robot, the surgeon first localized each of the ground truth fiducial markers on the phantom platform using a tracked large needle driver (LND) instrument. Information from the ground truth fiducials enabled post hoc analysis of the accuracy of our touch-based registration and point-based reregistration; this information was not used intraoperatively by the image guidance system. At the start of the registration process, the surgeon lightly traced a patch of the anterior surface of the kidney using the LND instrument to acquire data for touch-based registration. After computing the initial touch-based registration, 15 ink dot fiducials were manually marked on the anterior surface with a surgical marker; each ink dot was then immediately localized using the LND instrument. After identifying the ink fiducials, the phantom platform was moved to simulate manipulation of the kidney. Point-based reregistration was then performed by once again localizing the ink fiducials. Ground truth fiducials were also localized again for post hoc analysis of reregistration accuracy. Following reregistration, the surgeon resected the tumor using the image guidance display, as shown in Figure 3. After completing tumor resection, the ink fiducials and ground truth fiducials were localized a final time using the LND instrument.

Results

Accuracy of initial touch-based registration was evaluated by comparing vertex positions of the registered 3D kidney model with the corresponding vertex positions in the ground truth 3D model. The target registration error (TRE) at each vertex was computed as the distance between that vertex and the corresponding vertex of the ground truth model (see Fitzpatrick and Labadie¹⁴ for a thorough discussion of and mathematical definitions of TRE). A visualization of TRE of the initial registration across the entire kidney surface was generated based on the surface tracing for each phantom (Fig. 4a, c). The root mean square (RMS) average TRE over the entire kidney surface was 2.53 mm in the endophytic case and 4.88 mm in the exophytic case. Overall, registration accuracy decreased as distance from the surface tracing increased. The average time to achieve the initial registration and subsequent point-based reregistration was 280 and 59 seconds, respectively.

FIG. 4.

An illustration of the residual TRE across the kidney surface, after initial touch-based registration. (a) Kidney surface for an endophytic case. (b) Planned resection surface for the endophytic case. (c) Kidney surface for an exophytic case. (d) Planned resection surface for the exophytic case. TRE, target registration error.

When evaluating registration accuracy in the context of the resection surfaces, only the component of TRE normal to the surface needs to be considered. The component of registration error parallel to the surface does not affect the accuracy of cutting along the said surface. The RMS average of the normal TRE over the entire resection surface was 0.75 mm in the endophytic case and 2.15 mm in the exophytic case (Fig. 4b, d).

The point-based reregistered image was compared with its initial registration, rather than the ground truth reference, to evaluate the isolated TRE of reregistration. The accuracy of point-based registration also depended on the number and distribution of fiducial markers used to calculate the registration. This dependence was investigated in post hoc analysis by computing numerous “leave-N-out” point-based registrations using subsets of the fiducial markers localized during experimentation. Mean TRE for all possible combinations using the specified number of fiducials was calculated. The impact on RMS TRE of point-based reregistration was minimal compared with our initial registration and almost instantaneous (increasing from 1.34 to 1.70 mm preresection and from 1.60 to 2.10 mm postresection, depending on the number of points used in reregistration). The RMS TRE improved with an increase in number of fiducials identified and reached nadir at ∼8 fiducials. The change was similar for both the endophytic and exophytic tumors in the phantoms (Fig. 5).

FIG. 5.

RMS TRE vs number of ink fiducials used in point-based reregistration. Note that reported error values represent an error unrelated to the initial registration error. RMS = root mean square.

Discussion

We have developed a novel IGS system for PN utilizing the da Vinci Xi robotic surgical system and evaluated point-based reregistration on validated kidney phantoms after tumor resection. By registering segmented preoperative axial imaging, such as CT scan images, to intraoperative patient space, IGS enables identification of hilar anatomy as well as subsurface kidney features related to the tumor. The technology has the potential to improve operative time and make margin selection easier for surgeons, optimizing the preservation of renal parenchyma. The ability to reregister during IGS is essential as dissection and manipulation inevitably lead to a change in the orientation and configuration of a target organ. Point-based reregistration by identifying fiducials marked with a sterile surgical marker provides a quick and simple way to reregister, to maintain accuracy during surgery. Furthermore, our system provides both digitized anatomical surfaces and robotic manipulators in the TilePro display, complementing surgical workflow.

Our method of touch-based registration for IGS complements surgical workflow and only requires a software update to a standard, widely available da Vinci robot, for effective implementation. The idea of touch-based registration with the da Vinci system was first suggested by our research group in Ong and colleagues.⁵ We previously demonstrated that the da Vinci manipulators can be localized with sufficient accuracy for image guidance.⁴ By tracking the positions of the robot as the surgeon moves a manipulator across the surface of the patient anatomy, the IGS system creates a set of 3D surface point data. This data set can be aligned with the surface of a preoperative segmented imaging of the kidney, allowing for accurate registration. Our prior analysis has shown this technique to be capable of achieving an average TRE of 3.69 mm in a set of 700 registration experiments when using the da Vinci Si robotic system.⁷ In the current study, we have translated our techniques to the da Vinci Xi robotic system and have generated similar TRE results.

Multiple other methods of registration for IGS have been previously evaluated. Teber and colleagues first evaluated a method for IGS for laparoscopic PN by inserting fiducial markers on barbed needles directly into the kidney.² Intraoperatively, the fiducials were imaged with a mobile C-arm and segmented, allowing point-to-point registration between the imaged fiducials and those seen endoscopically. Although accurate with an error margin of 5 mm, the technique requires invasive placement of fiducials as well as intraoperative OR-based CT imaging and segmentation, which interrupt surgical workflow and add to operative time.

A less invasive registration technique, such as manual registration, involves the surgeon visually aligning 3D images to the surgical field in the endoscopic view. Although manual registration might benefit the surgeon by having the preoperative imaging information displayed during surgery, the technique is often inaccurate as it depends on human hand/eye coordination and spatial reasoning to perform the registration. Multiple trials have shown wide variation in precision for manual registration.^15
–17

Other groups have used stereoendoscopes for instrument tracking and organ registration by triangulating surface points using a 3D stereoscopic endoscope and matching those to patient anatomy identified on preoperative imaging.¹⁸ For example, Su and colleagues developed a multistep CT-to-endoscope registration method by aligning a segmented kidney surface with stereoscopic video.¹ Similarly, Pratt and colleagues utilized a combination of manual and endoscopic registration to align translation of the endoscope during PN.¹⁹ Although endoscopic registration might eventually allow for a seamless way of integrating preoperative patient anatomy with the endoscopic display, the approach requires persistent direct line of sight between the camera and patient anatomy, which is often changed intraoperatively or obscured by smoke and blood. Furthermore, the method reduces the accuracy of tracking the robotic manipulators, as their positions in space are known only relative to the endoscopic image.

Although some of these methods have evaluated a means of updating IGS during surgery, no one has implemented a point-based reregistration method that can be rapidly performed during robotic PN without interruption of the procedure.²⁰ It is necessary to maintain an up-to-date registration due to frequent changes in kidney orientation and configuration during mobilization, manipulation, and incision. In a study evaluating tissue deformation, Altamar and colleagues found a 4.4 mm average change in position of surface fiducials after making a transverse incision in porcine kidneys.²¹ Furthermore, changes in renal perfusion during hilar clamping also can lead to anatomic and tissue deformation. Previous perfusion modeling demonstrated an average nonrigid kidney shift of 3 mm from lack of vascular perfusion.²² Thus, to minimize warm ischemia time and maintain accuracy during IGS, it is essential to have a flexible and fast method for reregistration. Our point-based technique enables seamless reregistration that can quickly maintain accuracy during manipulation and incision of the kidney in a validated phantom.

Despite implementing a novel, point-based reregistration method for robotic PN, there are some limitations to our study. The technology has only been evaluated by a small number of surgeons and requires further evaluation for widespread real-world application. Furthermore, although the phantoms used in our experiment have previously been validated for PN, they still may not accurately replicate living, human kidney tissue, including the changes associated with surgical ischemia and parenchymal incision.¹² In addition, intraoperative ultrasound, which is commonly used to enhance resection and tumor identification intraoperatively, was not evaluated in our experiment. It is possible that the incorporation of ultrasound may further improve accuracy during IGS and demonstrating this will necessitate additional studies. We plan to continue research into tissue deformation modeling as well as a clinical evaluation of our point-based reregistration method in vivo.

Conclusions

We have developed the first system that uses a point-based reregistration method to update patient anatomy during IGS for PN using the da Vinci Xi surgical system and shown good accuracy. The technology is easily integrated into the surgical workflow and does not require additional hardware other than a software upgrade. In vivo studies are being planned to further evaluate the technology and its real-world application.

Footnotes

Acknowledgments

Any opinion, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the National Institutes of Health.

Author Disclosure Statement

N.K.—Consultant for Virtuoso Surgical.

N.N.—Consultant for Virtuoso Surgical.

R.J.—Founder and board member of Virtuoso Surgical and receives grant funding from NIH.

S.D.H.—Founder and board member of Virtuoso Surgical and receives grant funding from CSATS, BTG, and NIH.

Funding Information

This article is based upon work supported by the National Institutes of Health under R01-EB023717.

Abbreviations Used

References

, Vagvolgyi

, Agarwal

, et al. Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3D-CT to stereoscopic video registration. Urology, 2009; 73:896–900.

Teber

, Guven

, Simpfendorfer

, et al. Augmented reality: A new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur Urol, 2009; 56:332–338.

Porpiglia

, Checcucci

, Amparore

, et al. Three-dimensional augmented reality robot-assisted partial nephrectomy in case of complex tumours (PADUA >/ = 10): A new intraoperative tool overcoming the ultrasound guidance. Eur Urol, 2020; 78:229–238.

Ferguson

, Pitt

, Remirez

, et al. Toward practical and accurate touch-based image guidance for robotic partial nephrectomy. IEEE Trans Med Robot Bionics, 2020; 2:196–205.

Ong

, Glisson

, Burgner-Kahrs

, et al. A novel method for texture-mapping conoscopic surfaces for minimally invasive image-guided kidney surgery. Int J Comput Assist Radiol Surg, 2016; 11:1515–1526.

Simpson

, Burgner

, Glisson

, et al. Comparison study of intraoperative surface acquisition methods for surgical navigation. IEEE Trans Biomed Eng, 2013; 60:1090–1099.

Ferguson

, Cai

, Reed

, et al. Toward image-guided partial nephrectomy with the da Vinci robot: Exploring surface acquisition methods for intraoperative re-registration. Proc SPIE Med Imaging, 2018; 10576:1057609.

DiMaio

, Hasser

. The da

Vinci research interface

. In: MICCAI Workshop on Systems and

Arch

. for Computer Assisted

Interventions

, Midas

Journal

. 2008. Available at: http://hdl.handle.net/10380/1464.

Benincasa

, Clements

, Herrell

, et al. Feasibility study for image-guided kidney surgery: Assessment of required intraoperative surface for accurate physical to image space registrations. Med Phys, 2008; 35:4251–4261.

10.

Yang

, Li

, Campbell

, et al. Go-ICP: A globally optimal solution to 3D ICP point-set registration. IEEE Trans Pattern Anal Mach Intell, 2015; 38:2241–2254.

11.

Kikinis

, Pieper

, Vosburgh

. 3D Slicer: A platform for subject-specific image analysis, visualization, and clinical support. In: Jolesz

, ed. Intraoperative Imaging and Image-Guided Therapy. New York, NY: Springer, 2014, pp. 277–289.

12.

Melnyk

, Ezzat

, Saba

, et al. Mechanical and functional validation of a perfused, robot-assisted partial nephrectomy simulation platform using a combination of 3D printing and hydrogel casting. World J Urol, 2019; 38:1631–1641.

13.

Stone

, Melnyk

, Wu

, et al. Patient specific rehearsal using 3D printing for complex partial nephrectomy cases (V4-01). J Urol, 2017; 197:e375.

14.

Fitzpatrick

, Labadie

. Image-Guided Surgery: Fundamentals and Clinical Applications in Otolaryngology. San Diego, CA: Plural Publishing, 2016, pp. 117–138.

15.

Volonté

, Buchs

, Pugin

, et al. Augmented reality to the rescue of the minimally invasive surgeon. The usefulness of the interposition of stereoscopic images in the Da Vinci™ robotic console. Int J Med Robot Comput Assist Surg, 2013; 9:e34–e38.

16.

Ukimura

, Gill

. Imaging-assisted endoscopic surgery: Cleveland Clinic experience. J Endourol, 2008; 22:803–810.

17.

Nakamura

, Naya

, Zenbutsu

, et al. Surgical navigation using three-dimensional computed tomography images fused intraoperatively with live video. J Endourol, 2010; 24:521–524.

18.

Puerto-Souza

, Mariottini

. A fast and accurate feature-matching algorithm for minimally-invasive endoscopic images. IEEE Trans Med Imaging, 2013; 32:1201–1214.

19.

Pratt

, Mayer

, Vale

, et al. An effective visualisation and registration system for image-guided robotic partial nephrectomy. J Robot Surg, 2012; 6:23–31.

20.

Glisson

, Ong

, Simpson

, et al. The use of virtual fiducials in image-guided kidney surgery. In: Wong

, Holmes D

III

, eds. Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling. Buena Vista (Orlando), Florida: International Society for Optics and Photonics, 2011, p. 796402.

21.

Altamar

, Ong

, Glisson

, et al. Kidney deformation and intraprocedural registration: A study of elements of image-guided kidney surgery. J Endourol, 2011; 25:511–517.

22.

Ong

, Herrell

, Miga

, et al. A kidney deformation model for use in non-rigid registration during image-guided surgery, In: Proceedings of SPIE—Medical Imaging 2008: Visualization, Image-Guided Procedures, and Modeling. p. 6918. https://doi.org/10.1117/12.771669