Abstract
The multidisciplinary nature of artificial intelligence (AI) has allowed for rapid growth of its application in medical imaging. Artificial intelligence algorithms can augment various imaging modalities, such as X-rays, CT, and MRI, to improve image quality and generate high-resolution three-dimensional images. AI reconstruction of three-dimensional models of patient anatomy from CT or MRI scans can better enable urologists to visualize structures and accurately plan surgical approaches. AI can also be optimized to create virtual reality simulations of surgical procedures based on patient-specific data, giving urologists more hands-on experience and preparation. Recent development of artificial intelligence modalities, such as TeraRecon and Ceevra, offer rapid and efficient medical imaging analyses aimed at enhancing the provision of urologic care, notably for intraoperative guidance during robot-assisted radical prostatectomy (RARP) and partial nephrectomy.
Introduction
With the advent of electronic health records, widespread digitization of the health care field has produced large amounts of data that physicians encounter while establishing diagnoses, tailoring patient care, and predicting clinical outcomes. This has enabled the development of artificial intelligence (AI) applications based on large health care data to improve the management of patients by health care professionals. Specifically, there has been a rapid growth in the application of AI in medical imaging. These advancements are fueled by the multidisciplinary nature of AI in medical imaging involving experts from various fields, including computer science, machine learning, data science, and radiology. Interpretative uses of AI include breast cancer imaging, thoracic radiology for lung cancer, neuroradiology for the detection of stroke and hemorrhages, and musculoskeletal radiology for the detection of fractures and osteoarthritis to name a few. 1 Noninterpretative applications of AI include creating study protocols for radiologists. For example, radiologists create study protocols based on specific clinical parameters, such as relevant laboratory values, prior images, and radiology reports of patients, which requires chart review and is a time-consuming task. Research has shown that machine-learning algorithms can determine clinical protocols for studies in both brain and body magnetic resonance imaging (MRI). 1 AI can also improve image quality, reduce radiation dosage in computed tomography (CT), and decrease scan time within MRI machines. 1 AI’s significance lies in its ability to augment human capabilities, enabling more efficient and accurate decision-making processes. AI systems can alleviate the workload of physicians, preventing medical errors, saving time, and increasing the accuracy and reliability of clinical decision making and treatment plans. 2 AI algorithms can enhance diagnostic and planning accuracy by providing quantitative measurements, flagging subtle abnormalities, and improving image quality through reduced noise, fewer artifacts, improved resolution, and enhanced contrast. 3 By improving accuracy and reducing the risk of misdiagnoses and medical errors, AI algorithms can also contribute to health care cost savings. 1 AI continues to evolve and accomplish this by integrating multimodal, complex data from different domains and leveraging vast amounts of data and advanced image recognition techniques. In this study, we will summarize landmark developments in the use of AI in medical imaging for urologic surgery.
Methods
A literature search using PubMed and Web of Science was done to perform a comprehensive nonsystematic review of articles using search terms “artificial intelligence,” “AI medical imaging,” “urologic surgery,” “Ceevra,” “TeraRecon,” “Fujifilm’s Synapse,” “3-D modeling,” “deep learning,” and “machine learning” to identify studies involving AI medical imaging for urologic surgery. Articles selected were required to be original articles written in English. Systematic reviews, original articles, and case reports/series were included (Tables 1, 2, and 3). Commentaries and news articles were excluded. The studies were independently reviewed. References of articles were reviewed for potential missed studies. Information regarding individual AI medical imaging platforms was obtained using each platform’s original website.
Summary of Landmark Papers Evaluating AI Applications of 3-D Augmented Reality and Virtual Reality Modeling in Urology
RAPN, robot-assisted partial nephrectomy; 3-D, hree-dimensional; VR, virtual reality; CT, computed tomography; MRI, magnetic resonance imaging; OR, odds ratio; AAA, abdominal aortic aneurysm; D2P, DICOM to Print; CBCT, cone-beam computed tomography; AR, augmented reality; NS, nerve-sparing; RARP, robot-assisted radical prostatectomy; mpMRI, multiparametric MRI.
Summary of Landmark Papers Evaluating AI Applications of Computer-Aided Design Imaging Analysis in Urology
CAD, computer-aided diagnostic; MRI, magnetic resonance imaging; IRA, inter-rater agreement; PI-RADS, prostate imaging reporting and data system; mpMRI, multiparametric MRI; 3-D, three-dimensional; CN, cytoreductive nephrectomy; RCC, renal cell carcinoma; CT, computed tomography; APF, adherent perinephric fat; MAP, mayo adhesive probability; ROC, receiver operator characteristic; CNN, convolutional neural network.
Summary of Landmark Papers Evaluating AI Applications of 3-D Printed Models and Reconstruction from Standard 2-D Imaging Modalities (CT/MRI) in Urology
3D, three-dimensional; CT, computed tomography; HA 3-D, hyperaccuracy three-dimensional; RAPN, robot-assisted partial nephrectomy; 2-D, two-dimensional; fMRI, magnetic resonance imaging. GRE, gradient-recalled echo; MRU, magnetic resonance urography; CTU, computerized tomography urogram; PDUS, penile Doppler ultrasound; MRA, magnetic resonance angiography; AVM, arteriovenous malformation; mpMRI, multiparametric magnetic resonance imaging; 3-D-RVI, 3-D reconstructed virtual image; RENAL-NS, RENAL nephrometry score.
Overview of AI Medical Imaging Platforms
AI-driven techniques have begun to make an impact in clinical settings by expanding the power of medical imaging. 32 Deep learning is a subset of machine learning that focuses on training artificial neural networks with multiple layers to learn and make predictions. Deep learning algorithms excel at handling unstructured and complex data, leveraging their layered architectures to capture intricate patterns and relationships. 32 Another technique involves neural networks, which are computational models inspired by the structure and function of the human brain. They consist of interconnected nodes organized into layers and implement mathematical operations to learn complex relationships in the data. 32 For example, a convolutional neural network can analyze an input image by passing it through layers of the network. Health care can leverage this machine learning to achieve data-driven, machine/human optimized workflow collaboration. 32 AI algorithms can also augment various imaging modalities, such as X-rays, CT, and MRI, to generate high-resolution three-dimensional (3-D) images. By mimicking human intelligence, AI equates to the ability to discover, infer, and reason. 32 Machine learning is a technique to achieve artificial intelligence by providing a machine with the capability to learn from data and experience through algorithms. As more data are gathered, it can inform the creation and fine-tuning of a predictive model to answer questions in medicine. 32 For instance, a machine is given thousands of images and trained using a label or diagnosis attached to each of the images in the dataset, such as cancerous or noncancerous. This training allows the machine to recognize patterns in identifying cancer in new images. 33
These various platforms can leverage advanced imaging technologies and AI algorithms to analyze and interpret medical images. AI in medical imaging also has a significant role in the operating room, such as in the preoperative assessment. AI can provide surgeons with valuable insights into potential risks associated with a specific procedure by integrating various parameters, such as anatomical features, comorbidities, and procedural factors. 33 By identifying factors that contribute to potential complications or suboptimal outcomes, surgeons can determine best practices, improve surgical techniques, enhance preoperative assessments, and reduce postoperative complications. 33 Preoperatively, AI reconstruction of 3-D models of patient anatomy from CT or MRI scans can better enable surgeons to visualize structures and accurately plan surgical approaches. 33 AI can go further to create virtual simulations of surgical procedures based on patient-specific data, giving surgeons more hands-on experience and preparation.
To give surgeons more intraoperative control, image segmentation algorithms can automatically outline and differentiate tumor boundaries, helping surgeons visualize the precise location, size, and extent of the tumor. 34 This tool aids in patient counseling, planning optimal surgical resection, and minimizing damage to surrounding health tissues, nerves, and vessels. AI can also be used to overlay preoperative imaging data onto the live surgical view to provide surgeons with navigational guidance during procedures. 34 This aids in precisely localizing target structures with real-time feedback and helping surgeons avoid accidental damage by highlighting critical areas. Finally, AI is revolutionizing image fusion in medicine by automating and optimizing the process of combining multiple images from different modalities or time points. For instance, CT and MRI images can be combined using an AI-assisted CT/MRI image fusion technique. This would build a personalized 3-D image with more detailed anatomical information that was previously neglected, informing our understanding of complex medical conditions and procedures. This ability to fuse images can also be applied to medical images acquired at different time points to monitor disease progression, longitudinal changes, and treatment response. For instance, this tool could offer better visualization and precise measurement of tumor size and shape before and after interventions. 34
Virtual reality (VR) and augmented reality (AR) technology in conjunction with 3-D imaging has been increasingly implemented within the health care field, and more specifically in the realm of surgical management for urologic cancers. 35 Among these emerging technologies is Ceevra®, an advanced visualization technology that develops 3-D digital models from standard CT and MRI scans to facilitate surgical planning and intraoperative guidance. The use of 3-D VR models has been linked to improved operative times, shorter hospital stay, reduced clamp time, and minimized blood loss in patients undergoing robot-assisted laparoscopic partial nephrectomy when compared with standard operative approaches that do not utilize VR technologies. 35 VR has also been studied in patient education of patients receiving vascular and oncologic surgery. Preoperative implementation of AI through sharing of 3-D models and VR with patients led to a notable decrease in preoperative patient anxiety scores. Additionally, the use of VR facilitated patients’ feelings of increased enjoyment and engagement in their health care. 4
Review of Relevant Tools and Technologies
Increasingly, commercially available platforms providing AI algorithms for 3-D image generation, interpretation, and integration are being integrated into routine clinical practice and clinical trials. 5 These instruments offer rapid and efficient analyses aimed at enhancing the provision of urologic care. 5 FDA 510(k) approved Bot Image Incorporated’s prostate AI software, in partnership with Terarecon, known as ProstatID, which functions as a computer-aided diagnosis (CADx) software interface for postprocessing biparametric MRI (bpMRI) and multiparametric MRI (mpMRI). 36 This product’s ability to blend AI with MRI to automatically register images, segment and analyze the prostate, and detect and assign cancer probability scores to potentially cancerous lesions is a tremendous aid to physicians in patient diagnosis (Fig. 1). 37 ProstatID accomplishes prostate detection and segmentation by utilizing a 3-D convolutional neural network trained to segment the prostate. 38 It then employs a pixel-wise random forest model, which has been trained on patient data with proven cancerous/benign diagnoses to classify the cancer on a continuous scale, and subsequently with a colorized overlay of the postprocessed image set. 38 Support for the addition of a random forest method-based, computer-aided diagnostic (CAD)-generated MRI series comes from a multireader, multicase study of interreader agreement of nine radiologists’ interpretations of 150 prostate MRI examinations with and without CAD. 39 Those results indicated improved interreader agreement and diagnostic performance for detection of clinically significant prostate cancer with the addition of a CAD-generated MRI image. 39 It has also been shown that the addition of a CAD system can improve experienced reader detection of patients with ≥10 mm prostate cancer lesions, and can significantly improve the performance of less-experienced readers in distinguishing benign from malignant lesions to the level of experienced readers. 12,40 This generates a single, unified, and simplified clinical workflow that processes advanced 3-D urologic medical imaging to provide urologists with a user-friendly visualization platform.

Clinical workflow and AI application of ProstatID platform by Bot Image Incorporated and TeraRecon. AI: artificial intelligence, MR: magnetic resonance.
Another such AI tool for 3-D medical imaging is Ceevra, which has shown promising benefits in the use of 3-D VR models before and during surgery. 13,14,35 Ceevra is a software that generates detailed, interactive 3-D digital models from standard CT and MRI scans (Fig. 2). 6 Patient CT/MRI can be uploaded to Ceevra’s HIPAA-compliant cloud, where it generates a 3-D colorized model accessible from phones, tablets, VR headsets, and other displays. 14 These models provide many benefits to surgeons such as an improved understanding of patient anatomy notably to optimize surgical planning for robot-assisted radical prostatectomy and partial nephrectomy. 7,35,41 Patients benefit from improved surgical outcomes, a better understanding of diagnosis and proposed surgical approach, less anxiety, and greater confidence and trust in physicians. 7,35,41 Hospitals also benefit from improved patient experiences, reduced risks, and reduced costs from shorter operating room time and patient length of stay. 35

Clinical workflow of AI-enhanced medical imaging with TeraRecon, Ceevra, or FujiFilm Synapse and its application in AR/VR, 3-D reconstruction, preoperative and intraoperative guidance. PACS, picture archiving and communication system; HIPAA, Health Insurance Portability and Accountability Act; CAD, computer-aided diagnosis; AI, artificial intelligence; CT, computed tomography; MRI, magnetic resonance image; 3-D, three dimensional; AR, augmented reality; VR, virtual reality.
A third AI surgical application is Fujifilm’s AI platform called Synapse 3D, which is an advanced visualization technology used to create segmented 3-D imaging of renal anatomical structures for kidney analysis. 21 This software has specific use for surgical planning for partial nephrectomy. 21 The planning software enables extraction of the kidney, renal cortex, vessels, renal sinus fat, and surrounding organs from contrast-enhanced CT images for analysis. 21 These extracted regions can be separated, and partial nephrectomy simulations can be performed. 21,22 The platform can simulate selective arterial clamping on the arteries in the area to be resected, improving surgical planning without increasing intraoperative and postoperative complications. 21,22 The application also automatically calculates the functional volume of renal parenchyma in the area of selective arterial clamping or remaining after partial resection to compare and observe changes over time and plan for function after surgery. 21 A study by Komai et al. reporting on the initial surgical experience with this tool found that the 3-D images and surgical simulation aided not only surgeons in their performance of partial nephrectomies, but also patients in their understanding of their disease status, the operation, and the surgical risks. 7,42,43 This 3-D model system can decrease postoperative complications and reduce turnaround times in the clinical workflow (Fig. 2). 21
Finally, another 3-D reconstruction software provided by Intuitive Surgical, named IRIS, similarly provides segmented reconstructions for renal masses based on preoperative CT. In a study by Iarajuli et al, 18 propensity-matched high-complexity partial nephrectomy pairs demonstrated improved precision of postpartial nephrectomy surgical volume but no difference in postoperative change in GFR by using the IRIS software. 8
Applications of AI Medical Imaging Platforms in Urology
Urologic oncology
Within urologic oncology, 3-D reconstruction using AI allows for extensive and accurate appraisal of complex surgical anatomy in patients with renal masses and may confer a beneficial impact on surgical planning and patient outcomes. Indeed, 3-D kidney models have been shown to improve localization of tumors in comparison to MR and CT imaging and changed the course of surgical approach to kidney masses in 60% of cases. 15,23 In another study by McDonald et al., 3-D VR models generated from imaging of patients undergoing robotic partial nephrectomy influenced presurgical planning in 6 of 15 cases. Operative planning is crucial for robotic partial nephrectomy, and 3-D reconstructions before and during the surgery offer a superior viewpoint for surgeons in comparison to routine planning. 14 Silagy et al. studied the relationship between fraction of overall disease resected and postoperative survival in cytoreductive nephrectomy (CN) for clear cell renal cell carcinoma (RCC), for which primary and metastatic disease was measured using TeraRecon semiautomated segmentation analysis of preoperative CT scans. Segmentation analysis provided information for volume calculations, as well as tumor compactness measurements as an assessment of tumor sphericity. It was found that fraction of disease resected did not impact overall survival. 24 To further assess the relationship between visceral adherent perinephric fat (APF) and perioperative outcomes during robotic partial nephrectomy, Jericevic et al. used TeraRecon for independent analysis of preoperative imaging to calculate the Mayo Adhesive Probability (MAP) score for patients with renal cell carcinoma. 25 Using AI allowed for the objective quantification of visceral and adherent perinephric fat through MAP scoring, and it was found that APF patients had significantly higher blood loss (p = 0.02) and longer operative times (227 minutes vs. 193 minutes, p < 0.01), without any differences in complication and malignancy rates. 16 Moreover, 3-D reconstructions have been shown to assist in surgical planning for prostate cancer by allowing for comprehensive assessment of proximity of the index cancer to the prostatic capsule and neurovascular bundles. 44 In the evaluation of small urothelial tumors, 3-D gradient-recalled echo (GRE) magnetic resonance urography (MRU) helped detect 74% of small urothelial carcinomas, with nephrographic and excretory phase images found to be essential for helping detect small urothelial carcinomas. 17
Benign urologic disease
Yamashita et al. measured mean stone density for predicting the outcome of shock wave lithotripsy by creating 3-D reconstructions of stones generated by TeraRecon using pretreatment noncontrast CT imaging. It was found that mean stone density and the area under the curve of the mean stone density measured with 3-D reconstruction was significantly higher in comparison to manual methods. The area under curve of 3-D images for shock wave lithotripsy was 0.6330, which was significantly higher compared with either elliptical region of interest (ROI) in abdominal windows (0.5836, p = 0.006), average of three points in abdominal windows (0.5756, p = 0.004), elliptical ROI in bone windows (0.5797, p = 0.008), or average of three points in bone windows (0.5794, p = 0.006). 26 Additionally, greater stone volume (p = 0.002) and greater mean stone density (p = 0.023) detected by 3-D imaging were found to be significant independent predictors of the treatment outcome on multivariate analysis. 26 Sudah et al. investigated the diagnostic performance of 3-D imaging in the visualization of upper urinary tract pathology using 3.0T-MRU, during which 3-D volume rendering reconstruction of the urinary tract was viewed against a faded background from CT urography imaging. It was found that comprehensive 3.0T-MRU was comparable to CTU in accuracy. Given that 3.0T-MRU would not expose patients to radiation, it may potentially serve as the primary diagnostic technique in certain patients. 27 In the evaluation of patients with clinically suspected vascular anomalies contributing to erectile dysfunction, 3-D image datasets were reconstructed from time-resolved MR angiography (MRA) using TeraRecon. This allowed for the creation of subvolume maximum intensity projections that were enhanced to evaluate penile arterial supply and venous drainage, and aided in identification of arteriovenous malformations and fistulas that were then correlated with intraoperative findings. 28
Advancements in AI surrounding stone disease improve diagnostic accuracy in both accurate stone detection as well as determining stone composition. Given that CT is the primary imaging method for the diagnosis of kidney stones, AI models trained on CTs have been developed with convolutional neural networks to facilitate diagnosis and management in patients with stone disease. Parakh et al. used 535 CT scans to develop two convolutional neural networks (CNNs) with distinct purposes: one for identifying the urinary tract and another for detecting the presence of renal stones within it. 18 By integrating the outputs of these two CNNs, the researchers attained a stone detection sensitivity of 94% and specificity of 96%, with an area under the curve (AUC) of 0.954. 18 Langkvist et al. trained an AI algorithm to identify ureteral stones on 465 CT images labeled by a radiologist and evaluated it on an additional 88 scans. 19 The CNN demonstrated 100% sensitivity in detecting ureteral calculi. However, this algorithm resulted in an average of 2.68 false positives per patient on the test set of 88 scans, which was clinically significant and necessitated radiologist review for each scan. 19 Given diverse stone compositions react uniquely to specific modalities used for fragmentation or disintegration, accurate determination of stone composition can facilitate precise treatment selection. For example, Kazemi et al. used machine learning with 936 stone patients to create a predictive model, which was found to have a 97% accuracy in anticipating stone composition. 9 Black et al. used images of 63 human kidney stones with known compositions to train a more sophisticated deep learning algorithm designed for stone composition identification. 20 Through this approach, they achieved a sensitivity of 85% in predicting composition. Notably, the algorithm demonstrated the highest sensitivity of 94% for uric acid stones, whereas sensitivity was lowest at 71% for brushite stones. 20 Initial application of the model to new images obtained during clinical endoscopic stone treatment revealed a recall rate of 67% for calcium oxalate monohydrate stones and 100% for uric acid stones. 20
Intraoperative guidance
AI has contributed to growing use of 3-D models for surgical training and intraoperative guidance, including partial nephrectomy, prostatectomy, and prostate biopsies. 29,35 Rather than only depending upon presurgical interpretation of standard medical imaging, AI allows for the use of live anatomically accurate models during kidney and prostate cancer procedures. These tools can aid surgeons in decreasing global ischemia time and guiding biopsies to improve the rate of true positive diagnoses. 10,11 VR has been used in conjunction with 3-D modeling to assist surgeons during robotic prostate and kidney surgery, during which the surgical DaVinci video stream and a 3-D view of the 3-D virtual model is overlapped in real-time. 30 Schiavina et al. assessed operative outcomes of 15 patients undergoing robot-assisted partial nephrectomy (RAPN) during which exact identification of the tumor’s localization and the dissection of the renal hilum was guided by the AR-3-D video stream. Preoperative surgical planning for arterial clamping based on 2D images was no clamping in 3 (20%), clamping of the main artery in 10 (66.7%), selective clamping in 1 (6.7%), and superselective clamping in 1 (6.7%) cases. Following intervention with the 3-D model, the plan of arterial clamping was changed to no clamping in 1 (6.7%), clamping of the main artery in 2 (13.3%), selective clamping in 8 (53.3%), and superselective clamping in 4 (26.7%) cases (p = .03). 11 Clear clinical benefits for this technology also justify the time and expenses associated with 3-D reconstruction and AR/VR content in clinical practice. The 3-D technologies have been associated with shorter warm ischemia times, shorter duration of intraoperative ultrasound times, decreased blood loss, and decreased in positive surgical margins in the management of renal cancer. 41,45
Clinical Patient Outcomes with 3-D AI Models for Urologic Surgery
Increased utilization of 3-D VR models for surgery and promising anecdotal evidence have generated significant interest in clinical trials to investigate surgical outcomes involving patient-specific 3-D digital models. 33,41,46 In a single-blind randomized clinical trial involving 92 patients undergoing RAPN from 2017 to 2018, Shirk et al. investigated surgical outcomes when preoperative planning imaging was supplemented with a 3-D VR model. 35 Patients were assigned to either the intervention group, in which the surgeon reviewed CT and/or MRI scans along with a 3-D VR model, or the control group, in which the operation was planned with surgeon review of solely the CT and/or MRI scans. 35 The 3-D VR models, provided by Ceevra, Inc., were viewed through a mobile application installed on surgeons’ smartphones and through a Google Cardboard-compatible VR headset. 35 The study demonstrated that patients in the control group were significantly more likely to have a length of stay longer than 2 days (OR, 2.86; 95% CI: 1.59–5.14) and estimated blood loss greater than 200 mL (OR, 1.98; 95% CI: 1.04–3.78). 35 These reported positive outcomes for the intervention group suggest reduced potential for various in-hospital complications and reduced rates of 30-day perioperative complications, respectively. 31,35,47 It is suggested that an enhanced understanding of patient anatomy and a reduced cognitive load demanded of surgeons may have contributed to improvements seen in outcomes and overall successful surgeries. 35 Moving forward, using these images to perform AR procedures through overlayed 3-D virtual models to the real anatomy and subsequently accommodate new findings could further enhance RAPN outcomes. 14
In a multicenter, randomized, single-blind clinical trial from —2019 to 2020, Shirk et al. investigated surgical outcomes for robot-assisted radical prostatectomy with the supplementation of 3-D, VR models as a roadmap for surgeons. 13 A total of 92 patients were randomized into a control group undergoing conventional preoperative planning with prostate biopsy results and MRI only or to an intervention group where biopsy and MRI results were accompanied by a 3-D VR Ceevra model composed of individually segmented and labeled anatomical structures and masses. Baseline characteristics were well matched between groups, apart from a higher number of patients of unknown race in the intervention group. To investigate oncologic control, sexual function, and urinary function, the study observed the outcomes of margin status, urinary pad usage, nerve sparing, bladder neck sparing, postoperative prostate-specific antigen (PSA), and Sexual Health Inventory for Men (SHIM) score. Trends toward lower positive margin rates (33% vs 25%, p > 0.05) and better urinary continence over the course of 6 months after surgery (1.4 vs 0.9 pads/day, p > 0.05) in the intervention group were found. Detectable postoperative PSA was also significantly lower in the intervention group (p < 0.05). While no significant difference in nerve sparing, bladder neck sparing, and SHIM score was identified, an additional subset of 32% of the intervention cases in which the surgeon reported changing the preoperative plan after reviewing the model before surgery was also investigated. 13 Within this subset and when compared with the control group, there was a significantly lower rate of postoperative detectable PSA (31% vs 0%, p < 0.05) and a strong trend toward increased bilateral nerve sparing (78% vs 92%, p > 0.05). 13 This suggests that 3-D reconstructive AI imaging allowed the surgeon to more appropriately select for non-nerve sparing, which resulted in significant improvement in postradical prostatectomy PSA. Although there were no significant differences in functional outcomes like urinary continence and sexual function, significantly lower PSA detection after surgery is an indicator of good cancer control. Significantly better oncologic outcomes while still preserving functional outcomes could be attributed to more aggressive and confident surgical approaches after viewing the 3-D VR models. 13
In another case series, Schiavina et al. evaluated the intraoperative application of AR technology to guide surgical dissection and the level of arterial clamping during RAPN. 11 They enrolled 15 patients for RAPN from 2018 to 2019 and the presurgical plan based on conventional 2D imaging was compared with the presurgical plan based on the 3-D model. The 3-D model reconstructions were obtained using 3-D Systems Inc.’s DICOM-to-Print (D2P) software. The study noted a significant difference between preoperative planning based on 2D imaging and the reassessment after revision of the 3-D model, with a higher rate (80% vs. 13.4%, p = 0.03) of selective and superselective arterial clamping. 11 Moreover, the surgical complexity of renal masses according to nephrometry scores was evaluated with and without the 3-D model, in which there was a significant difference in scores, owing to better comprehension of tumor anatomy after revision of the 3-D virtual reconstruction. 11,42,48 The study also highlighted the ability of AR technology to allow fast and real-time overlapping of the 3-D models inside the robotic console to guide surgeons during arterial clamping and dissection without needing to stop the intervention. 11,48 While 3-D overlay may not be feasible currently for most users, AI-generated 3-D models can be viewed concurrent (and close to) the operative view by feeding the image into tile-pro or viewing on a cell phone or tablet during the case. Overall, this series showcased the ability of 3-D-AR models to help induce ischemia targeted to the renal area near the tumor, eliminate ischemia of the remnant kidney while limiting hemorrhage, and improve the understanding of the size, location, and depth of a renal mass and its vascular anatomy before PN. 11 Based on these efforts, AI-powered 3-D surgical planning softwares have encouraging potential to overcome the limitations of conventional planning in other urology case types.
Challenges and Future Directions
The integration of AI into the realm of surgical procedures is not void of limitations. AI also has an excess of social and ethical impacts to consider regarding education and training. 49 AI medical imaging in surgery also introduces dilemmas in the areas of patient consent, transparency, and accountability. 49 Patients must be assured of the role of AI in their procedures and how situations in which AI systems make decisions that differ from a surgeon’s judgment will be handled. 49 Moreover, a strong data governance protocol, which involves determining how data are stored, secured, maintained, accessed, and safeguarded, is crucial to protect patients’ privacy. 49 AI-assisted surgery often demands substantial financial investments in terms of technology acquisition, training, maintenance, and ongoing support. A comparison of potential costs and benefits must be done to evaluate how AI resources can be used judiciously.
While many of the studies listed in our review demonstrate a benefit in some domains, the studies may not have been sufficiently designed to detect errors attributable to AI models. Such errors make it difficult to identify the origin of error when inaccurate decisions are made by surgeons, thereby shifting responsibility for adverse outcomes.
Moving forward, deep learning methods are currently dominating new research endeavors, attributing to increased success in optimizing the accuracy of image processing and decision making across a wide array of applications. Increased use of 3-D reconstruction and VR can enhance spatial awareness, provide real-time critical information and haptic feedback, and guide surgical actions. AI-powered robotic systems can also enable surgeons to remotely assist with complex procedures, allowing for wider access to prompt, high-quality care. Given the value-adding potential of these applications, AI is on its way to revolutionizing the interpretation of medical imaging data and to becoming an indispensable tool in medical imaging and individualized medicine.
Conclusion
5AI medical imaging platforms, such as TeraRecon, IRIS, and Ceevra, leverage advanced imaging technologies, thereby improving diagnostic accuracy, surgical planning, and patient outcomes in urology. 11 These tools aid in patient counseling, planning optimal surgical resection, and minimizing damage to surrounding structures. AI can also be used to overlay preoperative imaging data onto the live surgical view (or at least provide it a picture alongside live view) to provide surgeons with navigational guidance during urologic oncologic procedures. 11 The use of 3-D VR models has been linked to improved operative times, shorter hospital stay, reduced clamp time, and minimized blood loss in patients undergoing robot-assisted laparoscopic partial nephrectomy and prostatectomy when compared with standard operative approaches that do not utilize VR technologies. 35 In this study, we have shown AI’s emerging role in remodeling urologic patient care and emphasized the potential AI has to cause a paradigm shift in medical imaging.
Footnotes
Acknowledgment
The authors would like to acknowledge the Robert Wood Johnson University Hospital Innovation Center for their commitment to applying artificial intelligence into the realm of robotic urologic surgery.
Authors’ Contributions
S.E.E. and N.Y. conceived the project idea. L.K. performed the literature review. L.K., V.B., and S.K. performed article writing. L.K., V.B., S.K., N.Y., B.L., J.P., and S.E.E. performed article editing.
Author Disclosure Statement
No competing financial interests exist.
Funding Information
No funding was received for this article.
