General Surgery 4.0 – A Systematic Review of Extended Reality Interventions in General Surgery

Abstract

Background

Surgery is a central component of healthcare but involves significant risks, with complications occurring in 16.4% patients, accounting for 7.7% of worldwide fatalities. “Surgery 4.0” or digitisation of surgery, has introduced extended reality (XR) technology, offering opportunities to enhance peri-operative care. This study explored the current uses of XR to improve outcomes for general surgery patients.

Method

A systematic search of MEDLINE, EMBASE, and Cochrane databases was performed in August 2024 to include studies using XR for pre-operative planning, navigation or patient experience for adult patients undergoing general surgery. Data on pre-operative planning, post-operative complications, patient experience, image segmentation and study reporting were presented using a narrative approach.

Results

The search returned 966 articles. 26 studies were included featuring 1142 patients. The most investigated procedure was liver resection (n = 11, 42%), with XR interventions showing significant reductions in length of stay, blood loss, operative time and complication rates. Improved outcomes were only seen for patients undergoing liver resection. For patient experience (n = 5, 19%), XR systems were shown to significantly improve anxiety, pain and mood scores. Most studies (n = 11, 73%%) utilised manual methods for image segmentation, costing up to €650 and taking 3-6 hours per model. Reporting of the XR technology, assessment and future development was variable.

Conclusion

The benefits of XR technology to improve patient outcomes in liver surgery are emerging but are yet to materialise in other general surgical procedures. Future research should focus on automatic image segmentation to improve workflow efficiency and innovation frameworks to generate robust evidence.

Keywords

general surgery surgical oncology virtual reality augmented reality holography clinical decision-making systematic review

Introduction

Around 310 million major surgical procedures are performed each year worldwide.¹ Despite advances in surgery and perioperative care, surgery still involves significant risks, accounting for 7.7% of all worldwide fatalities.² One or more surgical complications occur in 16.4% patients, with 9% being potentially life-threatening and requiring intervention. The risk of surgical complications increases with the complexity of the procedure. For complex surgery, such as pancreatic or liver resections, the risk of surgical complications has been reported as high as 48% and 60%, respectively.^3,4 One way to reduce the rate of complications is to improve surgical planning. Implementation of new technologies, which enhance the review of individual patient anatomy, might improve surgical planning and improve patient outcomes. Surgical navigations systems, leveraging data from radiological imaging, could improve efficiency and safety in the operating theatre, reducing operative duration, complication rates and length of stay for patients.⁵

“Surgery 4.0” is a concept that encompasses the digitisation of surgery, including the internet of things, artificial intelligence, surgical data science, and extended reality (XR).⁶ XR is an umbrella term for all devices that alter the human-computer interaction.⁷ This includes virtual reality (VR), where the user sees digitally rendered images without the physical world; augmented reality (AR), involving the physical world augmented by digital information, and mixed reality (MR), which uses a mixture of methods to blend the physical and digital world. Individual patient anatomy can now be displayed in XR using medical imaging segmentation, a process involving partitioning of images into distinct regions, enabling three dimensional reconstructions.⁸ XR technology has been mainly used in general surgery for surgical training and education.⁹ Clinical interventions using XR for patient care are emerging, but their efficacy is largely unknown. This systematic review aims to determine the current uses of XR to improve outcomes for patients undergoing general surgery.

Methods

Study Design and Definitions

The study protocol was developed in accordance with the PRISMA and AMSTAR 2 guidelines and was prospectively registered with PROSPERO (registration number CRD42024569448).^10,11 XR interventions were defined as XR technology (VR, AR and MR) used as an intervention to influence patient outcomes in all pre-operative, intra-operative and post-operative aspects of care. The review focused on applications in general surgery, defined as gastrointestinal, endocrine, breast, trauma or organ transplantation procedures, in adult patients (aged 18 and over).

Selection Criteria

To enrich the data captured in this systematic review, all study types, excluding conference abstracts and case reports, were included. The studies included had to use XR as a patient intervention for perioperative care in adult patients undergoing a general surgical procedure. There were no time or language restrictions. Specific exclusion criteria included.

Conference abstracts or case reports;

Studies involving other modes of surgical visualisation technology rather than XR, such as robotic platforms or indocyanine green (ICG) fluorescence used without XR technology;

Studies focusing on procedures not considered to be general surgery.

Systematic Literature Search

Embase (Ovid), MEDLINE (Ovid) and Cochrane Library databases were systematically searched in August 2024. All identified studies were reviewed against the inclusion and exclusion criteria to assess eligibility. Referenced studies within identified literature were accessed and considered for inclusion. Screening was performed by two independent investigators (MK and TW) and studies identified were analysed for relevance to the systematic review prior to full inspection. Any discrepancies between the independent investigators were addressed by a third senior investigator (DJ) until consensus was achieved. The search strategies used are displayed in full in Appendix S1.

Primary and Secondary Outcomes

The primary outcomes included post-operative complications (blood loss, resection margin status, operative time, complications, length of stay, mortality) to assess the impact of XR technology for pre-operative planning and surgical navigation. The secondary outcomes were collected to explore other XR interventions as part of the peri-operative pathway and the current reporting of XR technologies in the literature. Secondary outcomes were grouped into.

Surgical planning and navigation uses (pre-operative review of anatomy, anatomical structure identification);

Patient experience (pain levels, anxiety);

Modality of primary source imaging;

Mode of segmentation;

Reporting of performance, safety and preparatory steps for definitive clinical trials.

Data Extraction

Two independent investigators (MK & TW) extracted data in duplicate using a standardised data collection form. The senior author (DJ) was consulted on any discrepancies. Data were extracted on patient characteristics (age, sex, type of surgery), intervention (type of XR, peri-operative stage), study characteristics (number of patients, study type, country of origin) and primary and secondary outcomes.

Data Synthesis

Included studies were tabulated and grouped according to type of general surgical procedure. Data relating to the outcomes of interest were recorded, but due to the heterogeneity amongst study types, meta-analysis of effect estimates were not performed. When applicable, treatment effects were displayed in odds ratios (ORs) or risk ratios (RRs) with 95% confidence intervals (CIs) for dichotomous data. Continuous data treatment effects were expressed as mean differences with 95% CIs. Alongside the traditional approach to reporting data, a structured narrative synthesis was conducted in line with the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews from the Economic and Social Research Council.¹²

Risk of Bias

Risk of bias assessment were conducted independently by MK and TW. Any disagreements were consulted with the senior author DJ. For studies which involved a comparison group, the Cochrane’s tool for assessing risk of bias was used.¹³ Where applicable, the RoB-2 score was used for randomised controlled trials (RCT) and the ROBINS-I for non-randomised studies (NRS). The risk of bias was assessed as “low risk”, “high risk” or “unclear risk” for each study. The results were depicted graphically using a visualisation tool by McGuinness et al¹⁴

Results

Included and Excluded Studies

The search strategy returned 966 articles after removing duplicate studies. Following abstract screening, 907 articles were excluded. Full-text reviews excluded 36 studies and three studies meeting inclusion criteria were identified through reference searches. The 26 included studies subsequently underwent data extraction. The selection process together with reasons for exclusion are outlined in the PRISMA flowchart in Figure 1.

Figure 1.

PRISMA flow diagram for study selection. Adapted from¹⁰

Study Characteristics

The 26 included studies featured a total of 1142 patients within case series (1 study, 9 patients), NRS (19 studies, 503 patients) and RCTs (6 studies, 630 patients). RCTs were all single-centre and involved 23% of the included studies. The most investigated general surgical procedure using XR technology was liver resection in eleven studies (42%). Colorectal resection was the next most frequent in five studies (19%), followed by pancreatic resection (n = 3, 11%), cholecystectomy (n = 2, 8%), splenectomy (n = 1, 4%), pelvic exenteration (n = 1, 4%) and thyroidectomy (n = 1, 4%). Two studies involved a mixture of abdominal procedures (8%). The most frequent XR technology used was AR in 18 studies (69%), followed by VR (n = 6, 23%) and MR (n = 2, 8%). The research was primarily focused on intra-operative use (n = 21, 81%), and included interventions dedicated to patient education and recovery (n = 5, 19%). The full study characteristics are displayed in Table 1 and the main findings are depicted graphically in Figure 2.

Table 1.

Study and Patient Characteristics

Ref.	Study type	Origin	n	Patient and study characteristics					Risk of bias
Ref.	Study type	Origin	n	Age	Sex (M/F%)	Surgery	XR technology	Interventional use	Risk of bias
¹⁵	NRS	China	12	54	83/77	Liver resection	AR	Intra-operative	Moderate risk
¹⁷	NRS	China	16	58	19/81	Liver resection	AR	Intra-operative	Moderate risk
²⁶	NRS	China	85	53	82/18	Liver resection	AR	Intra-operative	Moderate risk
²⁵	NRS	China	98	55	38/62	Liver resection	AR	Intra-operative	Moderate risk
³⁹	NRS	China	76	52.5	84/16	Liver resection	AR	Intra-operative	Moderate risk
²⁴	NRS	China	10	64.5	80/20	Liver resection	AR	Intra-operative	Moderate risk
¹⁸	RCT	Germany	16	70	75/25	Liver resection	VR	Intra-operative	Low risk
⁴⁰	NRS	China	11	63.4	55/45	Liver resection	AR	Intra-operative	Moderate risk
⁴¹	NRS	China	48	NA	NA	Liver resection	AR	Intra-operative	Serious risk
¹⁶	NRS	France	3	60	33/67	Liver resection	AR	Intra-operative	Serious risk
³⁴	NRS	Switzerland	2	75	50/50	Liver resection	AR	Intra-operative	Serious risk
²⁹	RCT	Germany	62	NA	NA	Colorectal resection	VR	Patient use	Low risk
⁴²	NRS	Japan	13	NA	NA	Colorectal resection	MR	Intra-operative	Serious risk
³⁰	RCT	Spain	126	64	57/43	Colorectal resection	AR	Intra-operative	Low risk
²¹	NRS	Italy	3	NA	NA	Colorectal resection	VR	Patient use	Serious risk
²²	NRS	Ireland	26	68	55/45	Colorectal resection	MR	Pre-operative	Moderate risk
¹⁹	NRS	Germany	5	66	20/80	Pancreatic resection	AR	Intra-operative	Serious risk
⁴³	NRS	Japan	19	NA	NA	Pancreatic resection	AR	Intra-operative	Serious risk
²⁰	NRS	Japan	7	75	43/67	Pancreatic resection	AR	Intra-operative	Serious risk
³¹	RCT	Iran	150	41	15/85	Cholecystectomy	VR	Patient use	Low risk
²⁷	NRS	Japan	27	51	56/44	Cholecystectomy	AR	Intra-operative	Serious risk
²⁸	NRS	China	34	49	25/65	Splenectomy	AR	Intra-operative	Low risk
²³	Case series	South Korea	9	39	22/78	Endocrine surgery	AR	Intra-operative	Not applicable
⁴⁴	NRS	Russia	8	NA	NA	Pelvic exenteration	AR	Intra-operative	Low risk
³²	RCT	Turkey	225	40	59/41	Mixed abdominal	VR	Patient use	Some risks
³³	RCT	Romania	51	NA	NA	Mixed abdominal	VR	Patient use	Some concerns

AR: Augmented reality; MR: Mixed reality; NRS; Non-randomised study; Randomised Controlled Trial; VR: Virtual reality.

Figure 2.

Graphical illustration of immersive reality technology applications and characteristics in general surgery

Quality Assessment

For studies involving a comparison between two groups (case series excluded), quality assessments were performed for 25 studies (19 NRS and 6 RCTs). Using ROBINS-I, the risk of bias in NRS was assessed as “low” in two studies (n = 2/19, 11%), “moderate” in eight studies (n = 8/19, 42%) and “high” in nine studies (n = 9/19, 47%). The reasons most likely to lead to an overall “high” rating included the presence of confounders or poorly defined interventions. For RCTs, the risk of bias according to ROB-2 was assessed as “low” in four studies (n = 4/6, 67%) and “some concerns” in two studies (n = 2/6, 33%). The results are shown graphically in Appendix S2 and S3.

Operative Planning and Navigation

When examining pre-operative uses of XR, there were no studies solely dedicated to anatomical assessment or surgical planning. All studies involving the use of XR technology for pre-operative visualisation also used their technology intra-operatively. The studies reporting on operative planning and navigation are outlined in Table 2. There were no studies that investigated the use of XR to enhance MDT meetings. In liver surgery, pre-operative AR has been used to calculate predicted resected liver volume (PRLV) from preset resection plans. Zeng et al¹⁵ performed this for laparoscopic anatomical liver resection in 12 patients, and found good agreement when compared with actual resected liver volume (ARLV) (r = 0.98, P < 0.05, relative error = 8.62 +/− 6.66%). For colorectal liver metastasis (CRLMs), Ntourakis et al¹⁶ used an AR system to locate tumours which were not identifiable post-chemotherapy using intra-operative ultrasound. Using AR models created from imaging before chemotherapy, all CRLMs were identified and resected successfully. AR navigation systems were also found to be useful in another liver study, involving displays of a dissection plan for left hepatectomy.¹⁷ Huber et al disputed these navigation findings in their RCT of 16 patients. They compared two groups (VR and control) of patients undergoing liver tumour resection. Their primary endpoint was the quotient of planned to intra-operative resection volume, which did not show any significant difference between the groups (0.94 vs 1.11, P = 0.305).¹⁸

Table 2.

XR Interventions for Pre-operative Planning and Surgical Navigation

Ref.	Surgery	XR technology	Pre-operative planning	Surgical navigation
¹⁵	Liver resection	AR	PRLV calculated using AR technology, good correlation with ARLV
¹⁶	Liver resection	AR		AR used to identify CRLMs not visible using intra-operative ultrasound
¹⁷	Left hepatectomy	AR		Display of dissection plane
¹⁸	Liver resection	VR		No difference in the quotient of planned to intra-operative resection volume between VR and control groups
¹⁹	Pancreatic resection	AR		Identification of vascular structures
²⁰	Pancreaticoduodenectomy	AR		Identification of vascular structures
²¹	Colorectal resection	VR	Identification of arterial anatomical abnormalities
²²	CME and CVL	AR	Identification of arteriovenous anatomical abnormalities
²³	Thyroidectomy	AR		Identification of nerve structures

ARLV: Actual resected liver volume; CME: Complete mesocolic excision; CRLM: Colorectal liver metastasis; CVL: Central venous ligation; FLR: Future liver remnant; PRLV: Predicted resected liver volume.

In pancreatic surgery, XR technology has been used to identify key anatomical structures. Javaheri et al¹⁹ used an AR system for pancreatic resection in five patients. Using AR intra-operative overlay, surgeons were able to identify all planned anatomical structures including the gastroduodenal artery, superior mesenteric artery, hepatic artery, splenic vein, inferior and superior mesenteric vein and portal vein. These structures were confirmed with intra-operative ultrasound. Onda et al²⁰ also found their AR system useful in seven patients undergoing pancreaticoduodenectomy. They were able to identify jejunal arteries and the inferior pancreaticoduodenal artery, which facilitated early intraoperative vascular ligation.

For colorectal procedures, Guerriero et al²¹ used VR for pre-operative planning in three patients undergoing colorectal resections. The use of VR was crucial for one patient where an anatomical variant was detected with the left colic artery having an abnormally short origin from the inferior mesenteric artery, enabling the surgeon to anticipate the anomaly and execute safe surgery. Kearns et al²² also tested their AR system for laparoscopic complete mesocolic excision and central venous ligation. Their 3-D models displayed in AR were able to detect 96% of arteriovenous variations in a cohort of 26 patients.

XR technologies have also been used for surgical planning and navigation in other general surgical procedures. An AR system was tested in the setting of robotic thyroidectomy to aid location of the recurrent laryngeal nerve.²³ In a pilot study involving six patients, all recurrent laryngeal nerves were successfully identified using an AR system with a 1.9 mm mean difference between the AR image and actual nerve position. To conclude, XR technologies have been used for pre-operative planning (such as calculating FLR) and intra-operative navigation, frequently to identify specific anatomical structures (vessels or nerves) or aid tumour identification. The current literature is limited to individual specific uses within small, early studies.

Intra-and Post-operative Outcomes

For studies which compared XR intervention and control groups, intra- and post-operative outcomes were analysed and detailed in Table 3. Liver surgery involved the most comparative studies, featuring four papers (3 NRS and 1 RCT).^18,24-26 Three studies found significantly reduced estimated blood loss (EBL), with mean differences of 100 mL (P = 0.002), 100 mL (P = 0.005), and 156.7 mL (P < 0.001), when AR was used for navigation in laparoscopic anatomical hepatectomy.^24-26 Zhang et al²⁶ also showed reduced length of stay (LOS) by 2 days (P = 0.003). Lou et al²⁴ found additional significant differences for AR navigation, including reduced operative time (mean difference 63.9 minutes, P < 0.001) and complication rate (31% difference, P = 0.021). In contrast, Huber et al¹⁸ did not demonstrate any differences between their AR and control groups undergoing liver resection. Similar results were seen in the study by Onda which used an AR navigation system for pancreaticoduodenectomy with no differences seen when compared to their control group.²⁰ Laparoscopic cholecystectomy was investigated in one AR study.²⁷ The authors did not find any significant differences, but qualitative results suggested the technology was more beneficial for trainee grade surgeons compared to those who had completed training. Tao et al²⁸ also investigated an AR system, but in the setting of laparoscopic splenectomy. Their comparison revealed significantly reduced LOS (3.8 days vs 4.5 days, P = 0.040) and EBL (306.6 mL vs 462.6 mL, P = 0.047) for the AR navigation group. In summary, improved outcomes as a result of XR interventions have been displayed in liver resections. It is important to note the small sample sizes (n < 100) and moderate risk of bias for these studies, limiting these findings to a potential signal of clinical benefit rather than strong evidence of efficacy.

Table 3.

Intra- and Post-operative Outcomes From Comparison Studies

Ref.	n	Intervention/Comparator	Surgery/Aetiology	EBL (ml)	Operative time (min)	Morbidity	Mortality	LOS (days)	R0 rate (%)	Significant findings
²⁶	85	AR navigation system/standard lap resection	Lap anatomical hepatectomy/primary liver cancer	AR: 200	AR: 300	AR: 18	NA	AR: 8	NA	Decreased LOS and EBL in AR group
				Control: 300	Control: 300	Control: 19		Control: 10
				P = 0.002		P = 0.614		P = 0.003
²⁵	98	AR navigation with ICG system/standard lap resection	Lap anatomical hepatectomy/primary liver cancer	AR: 100	AR: 290	AR: 14	NA	AR: 7	AR: 45	Decreased EBL in AR
				Control: 200	Control: 240	Control: 26		Control: 7	Control: 51
				P = 0.005	P = 0.086	P = 0.072		P = 0.788	P = 0.498
²⁴	45	MR navigation combined with IOUS/standard lap resection	Lap anatomical hepatectomy/primary liver cancer	MR: 102.7	MR: 135.4	MR: 1	NA	MR: 8.4	MR: 100	Decreased EBL, complication rates and operative time in MR
				Control: 259.4	Control: 199.3	Control: 7		Control: 8.9	Control: 90
				P < 0.001	P < 0.001	P = 0.021		P > 0.05	P > 0.05
¹⁸	16	VR navigation/standard resection	Lap & open liver resection/CRLM & HCC	NA	VR: 190	VR: 1	NA	VR: 9	VR: 100	No differences
					Control: 185	Control: 1		Control: 7	Control: 100
					P > 0.05	P > 0.05		P > 0.05	P > 0.05
²⁰	7	AR navigation/standard PD	PD/Pancreatic, bile duct & ampulla of Vater carcinoma	AR: 901	AR: 415	NA	NA	NA	NA	No differences
				Control: 828	Control: 425
				P > 0.05	P > 0.05
²⁷	27	AR navigation/standard lap cholecystectomy	Lap cholecystectomy/cholelithiasis & cholecystitis	AR: 0	AR: 74	AR: 0	AR: 0	NA	NA	No differences, efficacy may depend on experience
²⁷	27	AR navigation/standard lap cholecystectomy	Lap cholecystectomy/cholelithiasis & cholecystitis	Control: 0	Control: 58	Control: 0	Control: 0	NA	NA	No differences, efficacy may depend on experience
²⁸	34	AR navigation/standard lap splenectomy	Lap splenectomy/massive splenomegaly	AR: 306.6	AR: 178.2	AR: 3	AR: 0	AR: 3.8	NA	Decreased LOS, EBL and complication rate with AR
				Control: 462.6	Control: 166.8	Control: 5	Control: 0	Control: 4.5
				P = 0.047	P > 0.05	P = 0.688		P = 0.04

CD: Clavien-Dindo classification grade⁴⁵; CRLM: Colorectal liver metastasis; HCC: Hepatocellular carcinoma; ICG: Indocyanine green; IOUS: Intra-operative ultrasound; Lap: Laparoscopic; LOS: Length of stay; R0: Resection for cure; PD: Pancreaticoduodenectomy.

Patient Experience

A total of five studies investigated the use of XR for patient education and rehabilitation (Table 4). In colorectal surgery, Schrempf et al²⁹ used VR for post-operative bedside fitness exercises for patients who had undergone a colorectal resection. The results showed significantly improved mood scores (Distress thermometer difference = 0.76, P < 0.001) and a numerical reduction in LOS, which failed to reach statistical significance. Turrado et al³⁰ also focused on colorectal surgery, using VR pre-operatively for patient education and found significantly reduced pre-operative anxiety (Hospital Anxiety and Depression Scale difference = 2, P < 0.001) in their VR group. One study involved two intervention groups for laparoscopic cholecystectomy, using VR for either distraction or education.³¹ Both VR groups showed significantly improved pre-operative anxiety (State-Trait Anxiety Inventory Scale difference = 6, P < 0.001) as well as post-operative pain (Visual Analogue Scale difference = 2.13, P < 0.001) compared to controls. Similar results were found in two studies including mixed abdominal procedures. Okutan et al compared VR exposure post-operatively for distraction against standard of care, and showed improved post-operative pain (Visual Analogue Scale difference = 1.03, P < 0.05) and comfort in the VR group.³² Droc et al³³ performed a similar study on patients undergoing major abdominal surgery and also found improved post-operative pain levels (Visual Analogue Scale difference = 1.2, P < 0.05) in the VR group. The use of XR has been shown to improve patient education and post-operative rehabilitation in multiple ways. The evidence presented is derived from five RCTs, with quality assessments determining three at low risk of bias and two with some concerns, strengthening the evidence of XR benefit for this part of peri-operative care.

Table 4.

XR Interventions for Patient Experience and Recovery

Ref.	n	Intervention/Comparator	Surgery/Aetiology	XR technology	Pain	Other patient outcomes	Significant findings
²⁹	62	Daily VR bedside fitness exercises/no VR exposure	Colorectal resection/CRC	VR		Mood	VR improved mood, possible significant reduction in LOS
						VR vs control
						Distress thermometer difference = +0.76 (P < 0.001)
						LOS difference = 2 (P < 0.076)
³⁰	126	16-minute VR exposure detailing all peri-operative steps/no VR exposure	Colorectal resection/CRC	VR		Pre-operative anxiety VR vs control	VR reduced pre-operative anxiety
						HAD-D difference = −1 (P < 0.001)
						HAD-A difference = −2 (P < 0.001)
						STAI_A/S difference = −6 (P < 0.001)
						STAI_A/T difference = −6 (P < 0.001)
³¹	150	10-minute VR exposure for education & 10-minute VR exposure for distraction pre- & post-surgery/no VR exposure	Lap cholecystectomy/cholecystitis	VR		Anxiety	VR improved pre-operative anxiety and post-operative pain
					Education vs control	Education group vs control
					VAS difference = 2.13 (P < 0.001)	STAI difference = 22.8 (P < 0.001)
					Distraction vs control	Distraction group vs control
					VAS difference = 2.52 (P < 0.001)	STAI difference = 20.24 (P < 0.001)
³²	225	VR exposure post-operatively for distraction/no VR exposure	Mixed lap abdominal surgery/NA	VR		Comfort	VR improved post-operative pain and comfort
					VR group vs control	VR group vs control
					NRS difference = 1.03 (P < 0.05)	GCQ difference = 0.14 (P < 0.05)
³³	51	VR exposure post-operatively for distraction/no VR exposure	Major abdominal surgery/NA	VR		Cognition	VR improved post-operative pain
					VR vs control	VR vs control
					VAS difference = 1.2 (P < 0.05)	MMSE difference = 3.2 (P > 0.05)

CRC: Colorectal cancer; GCQ; General comfort questionnaire; HAD-D/A: Hospital Anxiety and Depression Scale – Depression/Anxiety; Lap: Laparoscopic; LOS: Length of stay; MMSE: Mini-mental state examination; NRS: Numerical rating scale; STAI_A/S: State-Trait Anxiety Inventory Scale – State; STAI_A/T: State-Trait Anxiety Inventory Scale – Trait; VAS: Visual analogue scale.

XR Technology Reporting

Reporting of XR technology methodology, primary source imaging modality, assessment and steps for definitive research was variable (Table 5). For studies that required segmentation of imaging to produce XR displays, six (n = 6/21, 29%) did not report on their methods. Eleven (n = 11/21, 52%) studies reported manual methods of segmentation, taking up three to six hours per patient in one paper.²⁰ Kearns et al was the only group to discuss cost implications, stating that each virtual model cost 650 euros to produce. One (5%) study used semi-automatic systems and three (14%) used automatic systems. Out of these, two studies utilised a commercially available system^18,34 whilst the other utilised locally developed software. With regards to the origin of imaging for XR models, a majority (n = 14/21, 67%) used Computed Tomography (CT). Four studies (n = 4/21, 19%) used a combination of CT and Magnetic Resonance Imaging (MRI), whilst three (n = 3/21, 14%) did not report their source of imaging data. A minority of studies (n = 6/21, 29%) provided detail on their imaging protocols and quality assurance. 21 (81%) studies reported at least one measure of performance, whilst only four (15%) reported specifically on safety and four (15%) discussed steps for a definitive trial of their technology. In summary, the majority of the literature uses manual methods for XR image production, with limited reporting of primary image source and quality, safety outcomes and performance outcomes.

Table 5.

XR Technology Reporting

Ref.	n	Surgery	Imaging modality	Record of imaging	XR technology	Segmentation methodology	Record of performance	Record of safety	Record of steps for definitive RCT
¹⁵	12	Liver resection	CT, MRI	Not reported	AR	Manual	Correctly identified resection boundaries	Not reported	Not reported
¹⁷	16	Liver resection	Not reported	Not reported	AR	Manual	Correctly identified resection boundaries	Not reported	Not reported
²⁶	85	Liver resection	CT	Protocol, quality control	AR	Not reported	Reported 10 minutes for setup	Not reported	Not reported
²⁵	98	Liver resection	CT	Not reported	AR	Manual	Able to use anatomical markers for registration	Not reported	Not reported
³⁹	76	Liver resection	CT	Protocol, quality control	AR	Manual	Able to use anatomical markers for registration	Not reported	Not reported
²⁴	10	Liver resection	CT	Not reported	AR	Manual	Able to use anatomical markers for registration	Not reported	Not reported
²⁴	10	Liver resection	CT	Not reported	AR	Manual	Registration error was 6.3 mm	Not reported	Not reported
¹⁸	16	Liver resection	CT, MRI	Not reported	VR	Automatic	Able to use anatomical markers for registration	Safety was demonstrated	Defined as feasibility study
⁴⁰	11	Liver resection	CT	Not reported	AR	Not reported	Not reported	Safety was demonstrated	Not reported
⁴¹	48	Liver resection	Not reported	Not reported	AR	Automatic	Not reported	Not reported	Not reported
¹⁶	3	Liver resection	CT, MRI	Protocol, quality control	AR	Manual	Registration within 5-6 min	Not reported	Not reported
¹⁶	3	Liver resection	CT, MRI	Protocol, quality control	AR	Manual	All metastasis found with AR guidance	Not reported	Not reported
³⁴	2	Liver resection	Not reported	Not reported	AR	Automatic	5 mins to register	Not reported	Not reported
²⁹	62	Colorectal resection	NA	NA	VR	NA	Not reported	Not reported	Not reported
⁴²	13	Colorectal resection	CT	Not reported	MR	Not reported	TLX score	Not reported	Not reported
³⁰	126	Colorectal resection	NA	NA	AR	NA	Not reported	Not reported	Not reported
²¹	3	Colorectal resection	CT	Not reported	VR	Manual	Abnormal anatomy identified	Not reported	Defined as feasibility study
²²	26	Colorectal resection	CT	Protocol, quality control	MR	Reported software used but unclear segmentation method	96% correct anatomy identification	Safety improved with MR	Defined as feasibility study
¹⁹	5	Pancreatic resections	CT	Not reported	AR	Manual	Correct anatomy identification	Not reported	Not reported
⁴³	19	Pancreatic resections	CT	Not reported	AR	Not reported	Registration error was 5 mm	Not reported	Not reported
²⁰	7	Pancreatic resections	CT	Protocol	AR	Manual, 3-6 hours per patient	Unsuccessful identification of structure 1, 6.2 mm registration accuracy	Not reported	Not reported
³¹	150	Cholecystectomy	NA	NA	VR	NA	Not reported	Not reported	Not reported
²⁷	27	Cholecystectomy	CT	Not reported	AR	Manual	Image quality rates as good or very good	No safety concerns from operating time difference	Not reported
²⁸	34	Splenectomy	CT	Protocol	AR	Not reported	Not reported	Not reported	Discussed need for large RCT
²³	9	Endocrine surgery	CT	Not reported	AR	Semi-automatic	1.9 mm recognition error	Not reported	Not reported
⁴⁴	8	Pelvic exenteration	CT, MRI	Not reported	AR	Manual	Concordance with scan, no surprises	Not reported	Not reported
³²	225	Mixed abdominal	NA	NA	VR	NA	Not reported	Not reported	Not reported
³³	51	Mixed abdominal	NA	NA	VR	NA	Not reported	Not reported	Not reported

AR: Augmented reality; CT: Computed Tomography; MR: Mixed reality; MRI: Magnetic Resonance Imaging; NRS; Non-randomised study; Randomised Controlled Trial; TLX: NASA Task Load Index score; VR: Virtual reality.

Discussion

XR systems have been found to be useful in surgical planning and navigation, particularly in identifying vascular structures and tumours in hepato-pancreato-biliary (HPB) and colorectal resections. The benefits for patient outcomes have been reported in liver surgery for primary and metastatic tumours, with reduced LOS, EBL, operative time and complication rates. These results are from small, unpowered studies; however, they provide a signal of potential benefit. Improved outcomes are yet to be defined for other general surgical procedures. For patient education and recovery, XR systems have been shown to produce significant improvements in pre-operative and post-operative anxiety, pain and mood. There are currently no studies which focus on the use of XR for patient consent or for MDT purposes within general surgery. Current XR technologies rely on primarily manual methods for imaging segmentation, which were inefficient, taking 3-6 hours per model in one study, and costly, at 650 euros per model extraction and processing. The quality of current literature is variable, with over 75% of studies at moderate or high risk of bias, and infrequent reporting of imaging source and quality, intervention safety and next steps for definitive trials and technology development. To our knowledge, this is the first systematic review to report on XR interventions in general surgery.

VR technology was first described in a science-fiction story in 1935 and has evolved considerably.⁷ Current MR systems enable display of digital and real-world content through a merged reality. This facilitates their use in healthcare environments, such as the operating theatre, where space and sterility are frequent barriers for technology implementation. Despite the technological evolution, clinical uses with an evidenced benefit for patient outcomes are limited. The most recent scoping review of XR research found that much of the XR literature (40%) is aimed at surgical education rather than clinical interventions.³⁵ When examining studies aimed at clinical uses, literature reviews in other surgical specialities also agree that current evidence for the benefit on patient outcomes is limited.³⁶ The versatility of XR technology has the potential to enhance all aspects of peri-operative care, but the opportunities are yet to be developed and investigated in high-quality studies.

The strengths of this systematic review include defined inclusion criteria for XR use in general surgery. The results provide a granular description of XR utilisation as well as the benefits for patient outcomes, with significant results for patients undergoing liver surgery. The review also provides a detailed analysis of study reporting, which is variable within this emerging technological field. The quality assessments performed enable a comprehensive review of the current evidence, which is shown to be of limited quality at present. In addition, the review provides an assessment of XR technology, such as imaging source, segmentation methods, along with current limitations and the research required for wider adoption. The limitations of this study are also acknowledged. The heterogeneity of studies prohibited quantitative synthesis by meta-analysis, however, the narrative synthesis provides a meaningful analysis of research across general surgery and the XR field. The low quality of evidence found precludes recommendations for clinical practice at the current stage, but it does highlight the need for robust, high-quality research to assess the evolving XR technology.

The areas for further research in surgical XR have been highlighted. The main factor limiting widespread adoption is manual segmentation. Having experienced clinicians performing image analysis for three to six hours per patient is not feasible in a large healthcare system, such as the National Health Service (NHS). External services providing such analysis are costly, and therefore unlikely to be cost-effective. Artificial intelligence (AI) offers a possible solution for automated segmentation, enabling surgeons to view patient anatomy, operative plans, and intra-operative navigation on demand, and unlocking the potential of XR for widespread use.37 Another challenge to address will include fusion of multiple imaging modalities, as in this review 19% of studies were noted to include more than one type of imaging in their XR models. Automatic segmentation will have to account for variables, such as patient movement and timing of scans, in the image co-registration, but if solved XR models will harness the benefits of different imaging modalities. For example, CT and MRI scans can miss lesions close to the liver surface, whereas ultrasound is able to locate these, but lacks the definition to find deep small lesions in the liver parenchyma, showing that a fusion of the modalities could enhance MDT review, surgical planning and operative navigation. Another barrier to adoption involves the diversity of current XR systems. Nearly all studies involved locally developed early-stage XR systems aimed for one specific surgical application. A generic XR platform that can display multiple organ systems would be beneficial, but would require further innovation. The platforms must be versatile to cater for all surgical approaches. The included studies frequently involve an additional screen or headset to display 3D reconstructions next to laparoscopic stack systems.²² To reduce the task load associated with new surgical tools, XR technologies must provide a seamless experience integrated within existing surgical equipment, such as laparoscopic stack systems or robotic platforms. Headsets will need to remain an option for open surgery, providing a similar experience to integrated laparoscopic or robotic solutions. In this systematic review, only four studies reported further plans for clinical trials. Ideally, XR systems should be subjected to the same prospective evaluation frameworks as other surgical technologies with minimum standards for reporting, which should include primary imaging source and quality, methods for segmentation, along with safety, efficacy, and utility.³⁸ XR technology has the potential to improve all aspects of peri-operative care and further studies are required to investigate its utility across the whole surgical care pathway, including multi-disciplinary decision-making and post-operative follow-up. There is scope for more research in other surgical specialties, rather than the current focus on HPB surgery. XR could also explore challenges within surgical subspecialties, such as anastomotic leak or advanced/recurrent cancer.

Conclusion

The use of XR technology has been reported for pre-operative planning, patient education, intra-operative navigation and patient rehabilitation in the field of general surgery. A signal towards improved outcomes has been shown for patients undergoing liver resections, including reduced LOS, EBL, operative time and complication rates. Similar patient benefits are yet to be documented for other general surgical procedures. The findings are limited by the quality of evidence and varied reporting standards of primary imaging source, methods of segmentation and safety. Areas for further research include AI-powered automatic image segmentation, fusion of multiple imaging modalities, interdisciplinary XR platforms, and the use of XR systems across surgical care pathways.

Supplemental Material

Supplemental material - General Surgery 4.0 – A Systematic Review of Extended Reality Interventions in General Surgery

Supplemental material for General Surgery 4.0 – A Systematic Review of Extended Reality Interventions in General Surgery by Mikolaj R Kowal, Tom Williams, Alexios Dosis, Samir Pathak, Shahid Farid, Deborah D Stocken, Peter Lodge, Sharib Ali, Damian Tolan and David G Jayne in Surgical Innovation

Footnotes

ORCID iD

Mikolaj R Kowal

Author Contributions

Conceptualization, MRK, SF, SP, DDS, PL, DT, SA, DGJ; resources, MRK, TW, AD; writing—original draft preparation, MRK; writing—review and editing, MRK, TW, AD, SF, SP, DDS, PL, DT, SA, DGJ; visualization, MRK; supervision, DT, SA, DGJ. All authors have read and agreed to the published version of the manuscript.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: MRK receives funding from the European Union, under Grant Agreement No 101137233. Co-funded by the European Union. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HADEA). Neither the European Union nor the granting authority can be held responsible for them. AD receives funding from Leeds Hospital Charities (A2002555) and Bowel Research UK Ref: BRUK_SG_24012. DGJ and DDS receive funding support from the National Institute of Health and Care Research (NIHR213331; NIHR205280; NIHR302439). The project is supported in part by the National Institute for Health and Care Research (NIHR) Leeds Biomedical Research Centre (BRC) (NIHR213331). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Author PL has previously acted as a paid consultant for HoloCare Limited but declares no non-financial competing interests. All other authors declare no financial or non-financial competing interests.

Supplemental Material

Supplemental material is available online.

References

Dobson

. Trauma of major surgery: a global problem that is not going away. Int J Surg. 2020;81:47-54.

Nepogodiev

Martin

Biccard

Makupe

Bhangu

Nepogodiev

. Global burden of postoperative death. Lancet. 2019;393(10170):401.

Topal

Aerts

Laenen

, et al. Survival after minimally invasive vs open surgery for pancreatic adenocarcinoma. JAMA Netw Open. 2022;5(12):e2248147.

Egeland

Rostved

Schultz

, et al. Morbidity and mortality after liver surgery for colorectal liver metastases: a cohort study in a high-volume fast-track programme. BMC Surg. 2021;21(1):312.

Future of Surgery Report: Royal College of Surgeons of England; 2019 Available from: https://futureofsurgery.rcseng.ac.uk/report/Future/20of/20Surgery/20Report.pdf

Feussner

Park

. Surgery 4.0: the natural culmination of the industrial revolution? Innov Surg Sci. 2017;2(3):105-108.

Andrews

Southworth

Silva

JNA

Silva

. Extended reality in medical practice. Curr Treat Options Cardiovasc Med. 2019;21(4):18.

Quan

Huang

Chen

Liu

. Advances in medical image segmentation: a comprehensive review of traditional, deep learning and hybrid approaches. Bioengineering. 2024;11(10):1034.

Chiu

Billy Cheung

. Extended reality in surgical education: a systematic review. Surgery. 2023;174(5):1175-1183.

10.

Page

McKenzie

Bossuyt

, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Br Med J. 2021;372:n71.

11.

Shea

Reeves

Wells

, et al. Amstar 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Br Med J. 2017;358:j4008.

12.

Campbell

Katikireddi

Sowden

McKenzie

Thomson

. Improving conduct and reporting of narrative synthesis of quantitative data (ICONS-Quant): protocol for a mixed methods study to develop a reporting guideline. BMJ Open. 2018;8(2):e020064.

13.

Higgins

JPT

Altman

Gøtzsche

, et al. Cochrane Bias Methods Group, Cochrane Statistical Methods Group. The cochrane Collaboration’s tool for assessing risk of bias in randomised trials. Br Med J. 2011;343:d5928.

14.

McGuinness

Higgins

JPT

. Risk-of-bias VISualization (robvis): an R package and shiny web app for visualizing risk-of-bias assessments. Res Synth Methods 2020.

15.

Zeng

Deng

Dong

Fang

Xiang

. A pilot study of virtual liver segment projection technology in subsegment-oriented laparoscopic anatomical liver resection when indocyanine green staining fails (with video). Surg Endosc. 2024;38(7):4057-4066.

16.

Ntourakis

Memeo

Soler

Marescaux

Mutter

Pessaux

. Augmented reality guidance for the resection of missing colorectal liver metastases: an initial experience. World J Surg. 2016;40(2):419-426.

17.

Deng

Zeng

, et al. Laparoscopic left hemihepatectomy using augmented reality navigation plus ICG fluorescence imaging for hepatolithiasis: a retrospective single-arm cohort study (with video). Surg Endosc. 2024;38(7):4048-4056.

18.

Huber

Tripke

Baumgart

, et al. Computer-assisted intraoperative 3D-navigation for liver surgery: a prospective randomized-controlled pilot study. Ann Transl Med. 2023;11(10):346.

19.

Javaheri

Ghamarnejad

Bade

Lukowicz

Karolus

Stavrou

. Beyond the visible: preliminary evaluation of the first wearable augmented reality assistance system for pancreatic surgery. Int J Comput Assist Radiol Surg 2024;20:117-129.

20.

Onda

Okamoto

Kanehira

, et al. Identification of inferior pancreaticoduodenal artery during pancreaticoduodenectomy using augmented reality-based navigation system. J Hepatobiliary Pancreat Sci. 2014;21(4):281-287.

21.

Guerriero

Quero

Diana

, et al. Virtual reality exploration and planning for precision colorectal surgery. Dis Colon Rectum. 2018;61(6):719-723.

22.

Kearns

Moynihan

Dalli

, et al. Clinical validation of 3D virtual modelling for laparoscopic complete mesocolic excision with central vascular ligation for proximal Colon cancer. Eur J Surg Oncol. 2024;50(11):108597.

23.

Lee

Kim

, et al. Vision-based tracking system for augmented reality to localize recurrent laryngeal nerve during robotic thyroid surgery. Sci Rep. 2020;10(1):8437.

24.

Lou

Zhang

L, HL

. Application effect of contrast-enhanced ultrasound combined with mixed reality technology in laparoscopic anatomical hepatectomy. Med J Chin Peoples Lib Army. 2023;48(10):1208.

25.

Wang

Zhang

, et al. Efficacy of augmented reality combined with Indocyanine green fluorescence imaging guided laparoscopic segmentectomy for hepatocellular carcinoma. J Am Coll Surg. 2024;238(3):321-330.

26.

Zhang

Zhu

Yang

, et al. Augmented reality navigation for stereoscopic laparoscopic anatomical hepatectomy of primary liver cancer: preliminary experience. Front Oncol. 2021;11:663236.

27.

Kitagawa

Sugimoto

Haruta

Umezawa

Kurokawa

. Intraoperative holography navigation using a mixed-reality wearable computer during laparoscopic cholecystectomy. Surgery. 2022;171(4):1006-1013.

28.

Tao

Lin

Luo

, et al. Application of real-time augmented reality laparoscopic navigation in splenectomy for massive splenomegaly. World J Surg. 2021;45(7):2108-2115.

29.

Schrempf

Zanker

Arndt

, et al. Immersive virtual reality fitness games to improve recovery after colorectal surgery: a randomized single blind controlled pilot trial. Game Health J. 2023;12(6):450-458.

30.

Turrado

Guzman

Jimenez-Lillo

, et al. Exposure to virtual reality as a tool to reduce peri-operative anxiety in patients undergoing colorectal cancer surgery: a single-center prospective randomized clinical trial. Surg Endosc. 2021;35(7):4042-4047.

31.

Abbasnia

Aghebati

Miri

Etezadpour

. Effects of patient education and distraction approaches using virtual reality on pre-operative anxiety and post-operative pain in patients undergoing laparoscopic cholecystectomy. Pain Manag Nurs. 2023;24(3):280-288.

32.

Okutan

Saritas

. The effect of virtual reality practice and music on patients' pain, comfort, and vital signs after laparoscopic abdominal surgery. Surg Laparosc Endosc Percutaneous Tech. 2024;34(3):259-267.

33.

Droc

Isac

Nita

, et al. Postoperative cognitive impairment and pain perception after abdominal surgery-could immersive virtual reality bring more? A clinical approach. Medicina. 2023;59(11):2034.

34.

Buchs

Volonte

Pugin

, et al. Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery. J Surg Res. 2013;184(2):825-831.

35.

Zhang

Khanduja

. The impact of extended reality on surgery: a scoping review. Int Orthop. 2023;47(3):611-621.

36.

Lan

Mao

Qiu

Kay

de Sa

. Immersive virtual reality for patient-specific preoperative planning: a systematic review. Surg Innov. 2023;30(1):109-122.

37.

Kowal

Ibrahim

Mihaljevic

Kron

Lodge

. Technological Advances in Pre-Operative Planning. J Clin Med 2025;14(15).

38.

McCulloch

Altman

Campbell

Flum

Glasziou

Marshall

Nicholl

Balliol Collaboration Aronson

Barkun

Blazeby

Boutron

Clavien

Cook

Ergina

Feldman

Maddern

Nicholl

Reeves

Seiler

Strasberg

Meakins

Ashby

Black

Bunker

Burton

Campbell

Chalkidou

Chalmers

de Leval

Deeks

Ergina

Grant

Gray

Greenhalgh

Jenicek

Kehoe

Lilford

Littlejohns

Loke

Madhock

McPherson

Meakins

Rothwell

Summerskill

Taggart

Tekkis

Thompson

Treasure

Trohler

Vandenbroucke

. No surgical innovation without evaluation: the IDEAL recommendations. Lancet. 2009;374(9695):1105-1112.

39.

Zhu

Zeng

, et al. Perioperative and disease-free survival outcomes after hepatectomy for centrally located hepatocellular carcinoma guided by augmented reality and indocyanine green fluorescence imaging: a single-center experience. J Am Coll Surg. 2023;236(2):328-337.

40.

Wang

Tao

Wang

, et al. Laparoscopic right hemi-hepatectomy plus total caudate lobectomy for perihilar cholangiocarcinoma via anterior approach with augmented reality navigation: a feasibility study. Surg Endosc. 2023;37(10):8156-8164.

41.

Zhu

Zeng

Xiang

, et al. [application of augmented reality and mixed reality navigation technology in laparoscopic limited right hepatectomy]. Zhonghua Wai Ke Za Zhi. 2022;60(3):249-256.

42.

Ryu

Kitagawa

Goto

, et al. Intraoperative holographic guidance using virtual reality and mixed reality technology during laparoscopic colorectal cancer surgery. Anticancer Res. 2022;42(10):4849-4856.

43.

Okamoto

Onda

Yasuda

Yanaga

Suzuki

Hattori

. Navigation surgery using an augmented reality for pancreatectomy. Dig Surg. 2015;32(2):117-123.

44.

Ivanov

Krivtsov

Smirnov

, et al. Experience in the application of augmented reality technology in the surgical treatment of patients suffering primary and recurrent pelvic tumors. J Personalized Med. 2023;14(1):19.

45.

Dindo

Demartines

Clavien

. Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg. 2004;240(2):205-213.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.52 MB