Usefulness of Three-Dimensional Modeling in Surgical Planning,Resident Training,and Patient Education

Introduction

Imaging has a critical impact on diagnostic and surgical decision making, and three-dimensional (3D) digital models of patient pathology can now be made commercially. However, we are limited by the use of flat screens for the observation of 3D imaging data. Emerging techniques such as augmented reality and 3D printing overcome this limitation by producing high-resolution stereoscopic observation and graspable 3D objects.^1,2

We developed a 3D digital model of a cancer at the head of the pancreas by integrating actual CT data with a 3D modeling process performed by the Immersive Touch team. This reconstructive process resulted in a 3D-augmented reality system using high-resolution stereoscopic display of the patient's CT scan for detailed observation and manipulation of a virtual 3D patient-specific model. In this case, the virtual pancreatic model was also produced using a high-quality 3D printer.

Patient and Methods

We discuss a 56-year-old female with pancreatic head adenocarcinoma who presented with biliary obstruction and jaundice. The patient underwent endoscopic retrograde cholangiopancreatography with ductal stenting and six chemotherapy cycles with Folfirinox.

In collaboration with the Immersive Touch team, we used multiple series from the CT scan and studied the relevant anatomy to understand the physical location of the tumor. The relevant series from the DICOM (Digital Imaging and Communications in Medicine) data set were used to build the 3D digital models. Series were chosen based on the relevance to the area in question and the resolution. Multiple series were taken into ITK-SNAP (Insight Segmentation and Registration Toolkit) and the relevant anatomy was segmented. This includes the stomach, pancreas, tumor, arteries, veins, liver, kidneys, and spleen. Each series model was then brought into 3Ds Max, where another model was created to overlap the segmented model. This was done to lower the number of polygons for each model and create a complete picture if there were any holes in the segmentation that is common with small structures like arteries. The model was then ready to be saved as VRML (Virtual Reality Modeling Language) files and put on the Immersive Touch platform. Once the models are loaded into the simulator, the surgeon can use a haptic stylus to move the models around and interact with them. Rotation occurs by pressing a button on the haptic device. The stereoscopic display adds a level of depth perception.

Data files from the Immersive Touch system were then converted for printing a 100% size-rendering model. The polygon efficient models were saved as one single STL file and sent for printing. The 3D printer we utilized was a Stratasys Object 500 Connex 3 and the materials used were photopolymers in Vero Clear, Vero Magenta, Vero Yellow, plus a water-soluble support. The whole process took 64 hours to be finalized.

Results

The initial CT scan showed a borderline resectable tumor and neoadjuvant chemotherapy was started. After six cycles of Folfirinox, the patient underwent a repeat CT scan with contrast that documented a clear involvement of the gastroduodenal artery (Fig. 1) but doubtful relationships with hepatic artery and celiac axis (Fig. 2).

OPEN IN VIEWER

FIG. 1.

CT scan, involvement of the gastroduodenal artery.

OPEN IN VIEWER

FIG. 2.

CT scan, relationship between tumor and common hepatic artery.

Reconstructing and compositing the different series together, creating a manipulable 3D model, enhanced the imaging, which allowed clearer observations of the relationship between the mass and the blood vessels, and provided confirmation that the mass was invading the hepatic artery—evidence that the tumor was unresectable (Fig. 3).

OPEN IN VIEWER

FIG. 3.

Digital reconstruction of the tumor (beige colored).

The 3D printed model (Figs. 4 and 5) was helpful to discuss with the patient about her clinical condition and to explain why a surgical procedure was not indicated. The model, at the same time, was used for didactic purposes to residents and clinical staff.

OPEN IN VIEWER

FIG. 4.

3D printed model.

OPEN IN VIEWER

FIG. 5.

Detail of the tumor infiltrating the hepatic artery.

Discussion

This study showed that (1) augmented reality enhances traditional imaging by merging and modeling different data series together for a 3D view of an organ with diverse angles and transparency, allowing the observation of previously unapparent anatomical details; (2) with this new technology, surgeons and residents can preobserve their planned surgical intervention, explore the patient-specific anatomy, and sharpen their procedure choices; (3) high-quality 3D-printed models are increasingly useful not only in the clinical realm but also for personalized patient education.

Today, medical imaging techniques produce a huge amount of information, but sometimes data sets are difficult to be interpreted. One reason is that viewing axial sections with a large number of slices may be problematic. Second, as the complexity of image data grows, restricting the view modality to a traditional two-dimensional observation may be suboptimal. ³ Segmenting and compiling 3D composite view of the relevant anatomy can be critical in understanding patient pathology. Immersive Touch is an augmented reality system that integrates haptic device and a high-resolution stereoscopic display. The simulation platform uses multiple sensory modalities (visual, aural, tactile, and kinesthetic) creating a realistic 3D environment. ⁴ This process allows the observation of anatomical details barely visible in the CT scan.

Correct interpretation of patient's anatomy is crucial in a preoperative process to ensure optimal surgical treatment. The pancreas is a retroperitoneal organ surrounded by several critical structures. The anatomical location of the pancreas and relationship with adjacent structures contribute to the complexity of treatment. A complete understanding of patients' anatomy is essential before proceeding with surgery. However, surgical planning for pancreatic cancer is still challenging. 3D-rendered images provide additional information; however, these reconstructed models are still limited to viewing on a computer monitor. Consequently, 3D-rendered images may not provide a significant advantage over traditional observation methods.^5,6 With the Immersive Touch system, surgeons and residents can preobserve and “premanipulate” the patient's specific anatomy, and sharpen their procedure choices based on a stereoscopic observation of the 3D patient-specific model.

High-quality 3D printed models are increasingly useful, not only in the clinical realm but also for personalized patient care and education.^7–11 By generating a physical 3D model, the patient had the opportunity to visually explore her condition, resulting in a facilitated mutual understanding between patient and physician. Moreover, using this kind of models not only for patient counseling but also for students, residents and fellows may help to improve surgical training. This has already been shown to have benefits in the fields of craniofacial and maxillofacial surgeries, orthopedic surgery, vascular surgery, cardiac and thoracic surgeries, among others.^1,12

Unfortunately, virtual image creation and model printing are time consuming, and the cost of production can be high depending on how large, how precise, and what material the model is. Recent research has shown that the costs are decreasing and production is faster as automatic segmenting processes improve. ¹¹

Conclusions

This study suggests that the process of reconstructing and completing 3D images is promising for patient care, both directly through preoperative preparation and indirectly through the realms of surgical training and patient education.