Artificial Neural Network System to Predict the Postoperative Outcome of Percutaneous Nephrolithotomy

Abstract

Purpose:

To construct, train, and apply an artificial neural network (ANN) system for prediction of different outcome variables of percutaneous nephrolithotomy (PCNL). We calculated predictive accuracy, sensitivity, and precision for each outcome variable.

Methods:

During the study period, all adult patients who underwent PCNL at our institute were enrolled in the study. Preoperative and postoperative variables were recorded, and stone-free status was assessed perioperatively with computed tomography scans. MATLAB software was used to design and train the network in a feed forward back-propagation error adjustment scheme. Preoperative and postoperative data from 200 patients (training set) were used to analyze the effect and relative relevance of preoperative values on postoperative parameters. The validated adequately trained ANN was used to predict postoperative outcomes in the subsequent 254 adult patients (test set) whose preoperative values were serially fed into the system. To evaluate system accuracy in predicting each postoperative variable, predicted values were compared with actual outcomes.

Results:

Two hundred fifty-four patients (155 [61%] males) were considered the test set. Mean stone burden was 6702.86 ± 381.6 mm³. Overall stone-free rate was 76.4%. Fifty-four out of 254 patients (21.3%) required ancillary procedures (shockwave lithotripsy 5.9%, transureteral lithotripsy 10.6%, and repeat PCNL 4.7%). The accuracy and sensitivity of the system in predicting different postoperative variables ranged from 81.0% to 98.2%.

Conclusion:

As a complex nonlinear mathematical model, our ANN system is an interconnected data mining tool, which prospectively analyzes and “learns” the relationships between variables. The accuracy and sensitivity of the system for predicting the stone-free rate, the need for blood transfusion, and post-PCNL ancillary procedures ranged from 81.0% to 98.2%.The stone burden and the stone morphometry were among the most significant preoperative characteristics that affected all postoperative outcome variables and they received the highest relative weight by the ANN system.

Introduction

Thanks to enormous advances in the field of minimally invasive urologic surgery during the last three decades, percutaneous nephrolithotomy (PCNL) has been accepted as the preferred minimally invasive modality for managing renal calculi. Despite recent advances, however, clinical decision-making and patient counseling remain challenging in certain situations. Several research groups have studied the role of preoperative parameters (e.g., body mass index, stone burden, and previous surgery) on the outcome of PCNL in terms of stone-free rate (SFR), postoperative complications, and the need for ancillary procedures.^1

–4 As expected, the outcome of PCNL is affected not only by preoperative variables but also by the relative weights of each variable, which are crucial in determining the surgical outcome.

Artificial neural networks (ANNs), as the most widely applicable form of artificial intelligence (AI) in medicine, simulate interconnecting neural synopses and networks which can be trained by processing input and output data sets.⁵ The most commonly used model is the feed forward back-propagation system (Fig. 1), consisting of input, hidden, and output layers, which are interconnected to provide various combinations and permutations. The hidden layer represents input data processing by the human cerebral cortex.

FIG. 1.

An ANN model consists of input, hidden, and output layers. The connections determine the weight of each input variable on the outcome variables. ANN = artificial neural network.

Every ANN system has a training phase during which synaptic weights and thresholds are adjusted and optimized. In supervised training, input data are presented to the system and a random set of weights are initially assigned for the synaptic connections between the layers. The output calculated by the system is then compared with actual clinical outcomes, and the errors detected are then used to adjust and optimize the synaptic and connection weights. Self-optimization of errors is done intuitively by the system, and the weight and relevance of each variable on the actual outcome are determined. The system thereby learns to identify variables with strong, intermediate, and minimal influence on the actual outcome, and as more data are fed to the system, the process of self-optimization becomes more mature and the predicted outcomes more closely match the actual outcomes, which in turn results in higher predictive accuracy rates for a given outcome.^5
–7

We constructed and trained an ANN system to predict PCNL outcomes and then used the trained model to predict the outcome of PCNL in a second series of patients and calculated predictive accuracy. To our knowledge this is the first report of the use of an AI technology to evaluate PCNL outcomes.

Patients and Methods

Ethics

Our institutional review board approved the study, and all patients were informed about the purpose of the study and provided their informed consent.

Patients

Between March 2014 and March 2016 all adult patients who underwent PCNL at our institute were enrolled in the study. Preoperative imaging studies consisted of complete abdominopelvic ultrasound and intravenous urography or CT scan. After preoperative values were recorded, the patients underwent standard fluoroscopy-guided PCNL in the prone position, with a pneumatic lithotriptor for stone fragmentation.⁸ All cases were done by fully qualified and competent surgeons (A.A., D.I.). Stone-free status was assessed perioperatively in CT scans, and the need for any ancillary procedures and occurrence of postoperative complications was documented during follow-up visits. Table 1 summarizes the preoperative, intraoperative, and postoperative data sets used in this study. All participants had a normal preoperative coagulation profile and sterile urine (negative urine culture) before the operation. A single intravenous dose of ceftriaxone was administered before the surgery.

Table 1.

Variables Considered for Analysis

Preoperative variables	Intraoperative variables	Postoperative variables
Age	Access point	Postoperative stone volume (mm³)^a
Gender	Radiation exposure during access (s)	Postoperative hemoglobin
Stone volume (mm³)^a	Total radiation (s)	Postoperative creatinine
Stone size (mm)^b	Operative time (minutes)	Need for blood transfusion
Stone side (right to left)		Postoperative urine leakage (Double-J insertion)
Previous surgery in target kidney		Postoperative ancillary procedures (SWL, PCNL, TUL)
Hx of Diabetes
Hx of Hypertension
Preoperative hemoglobin (g/dL)
Preoperative creatinine (mg/dL)
Body mass index
Renal anomaly
Skeletal anomaly
Stone location (upper, mid, lower calix, renal pelvis, staghorn, multiple)
Stone lucency (opaque, semiopaque, lucent)
Degree of hydronephrosis (mild, moderate, severe)

Stone volume = a × b × c × π ÷ 6.

Largest diameter.

PCNL = percutaneous nephrolithotomy; SWL = shockwave lithotripsy; TUL = transureteral lithotripsy.

Design, training, and validation of the ANN system

We used MATLAB software (Mathworks, Natick, MA) to design and train the network with a feed forward back-propagation error adjustment scheme.^9,10 Preoperative and postoperative data from the first 200 patients (training set) were used to analyze the effect and relative relevance of preoperative values on postoperative parameters. To optimize the system, variables with no significant influence on the outcome were excluded. The Wilk generalized likelihood ratio test was used to analyze the relative relevance and weight of each input variable.^9,10 The k-fold cross-validation statistical method was used to construct, train, and test the ANN model. Figure 2A demonstrated this method where k = 10 in our study.

FIG. 2.

(A) The k-fold cross-validation statistical method was used to make, train, and test the ANN model. The data of 200 patients were randomly divided into k = 10 equal subsamples. Of these 10 subsamples, 1 of them was retained randomly as validation data to test the model, while the other 9 (k-1) samples were used as training data. The software randomly repeated the process of cross-validation for 30 times, therefore all subsamples were used thrice as the validation data and the system was trained and validated with the data of all 200 patients. (B) Determination of the value of the ANN system to predict the outcome of PCNL done on next 254 patients. PCNL = percutaneous nephrolithotomy.

The constructed ANN had 25 neurons in the input layer, and each neuron was assigned to one preoperative variable. The output layer consisted of six nodes, each corresponding to one postoperative outcome (Fig. 3).

FIG. 3.

The ANN model designed to predict PCNL outcomes. SWL = shockwave lithotripsy; TUL = transureteral lithotripsy.

Clinical application of the trained ANN (test set)

The validated adequately trained ANN was used to predict the postoperative outcome in the subsequent 254 adult patients (test set) whose preoperative values were serially fed into the system. To evaluate predictive accuracy of the system for each postoperative variable, the predicted values were compared with the actual outcomes (observed values), and true positive, false positive, accuracy, and precision rates of the system were calculated for each postoperative outcome variable (Fig. 2B).

Because machine learning systems are based on AI, the more input is fed in, the better the training and more accurate the predictions.

Results

During the study period we used data from 454 patients. The first 200 patients were enrolled as the training set, and their data were used to construct and validate the ANN. The subsequent 254 patients (155 males, 61%) were considered the test set. Mean age in this group was 46.64 ± 12.16 years, and mean stone burden was 6702.86 ± 381.6 mm³.

Table 2 summarizes the preoperative and intraoperative data for the test group, and Table 3 shows the actual postoperative outcomes in these patients. Overall SFR was 76.4%. Prolonged urine leakage was seen in 26.4% and was managed with a Double-J catheter. Fifty-four out of 254 patients (21.3%) required ancillary procedures such as shockwave lithotripsy (SWL, 5.9%), transureteral lithotripsy (10.6%), or repeat PCNL (4.7%). Significant blood loss requiring blood transfusion occurred in 58 (22.8%) patients.

Table 2.

Demographic Preoperative and Intraoperative Characteristics of Patients (Test Set)

Preoperative and intraoperative variables (n = 254)
Age (mean ± SD) (years)	46.64 ± 12.16
Gender (male/female)	155/99
Stone volume (mm³)^a	6702.86 ± 381.6
Stone size (mm)^b	21.587 ± 9.09
Stone side (right to left)	123–131
Previous surgery in target kidney, n (%)	88 (34.6)
Hx of diabetes, n (%)	48 (18.9)
Hx of hypertension, n (%)	72 (28.3)
Preoperative hemoglobin (g/dL)	13.40 ± 1.77
Preoperative creatinine (mg/dL)	1.15 ± 0.48
Body mass index (kg/m²)	26.5 ± 3.52
Renal anomaly, n (%)	14 (5.5)
Skeletal anomaly, n (%)	2 (0.78)
Stone location, n (%)
Upper calix	13 (5.1)
Mid calix	25 (9.8)
Lower calix	63 (24.8)
Renal pelvis	72 (28.3)
Staghorn	52 (20.4)
Multiple	29 (11.4)
Stone lucency, n (%)
Opaque	171 (67.3)
Semiopaque	64 (25.2)
Lucent	19 (7.5)
Degree of hydronephrosis, n (%)
Mild	112 (44.1)
Moderate	90 (35.4)
Severe	52 (20.5)
Radiation exposure during access^c (mean ± SD) (s)	103.17 ± 49.74
Operative time (mean ± SD) (minutes)	40.5 ± 10.5

Stone volume = a × b × c × π ÷ 6.

Largest diameter.

Defined as duration of X-ray exposure from insertion of the access needle to the start of nephroscopy.

Table 3.

Postoperative Outcomes in Our Patients (Test Set)

Postoperative outcome variables (n = 254)
Stone free, n (%)	194 (76.4)
Postoperative stone volume (mm³)^a	193.4 ± 87.7
Postoperative hemoglobin (g/dL)	11.3 ± 1.17
Postoperative creatinine (mg/dL)	1.16 ± 0.23
Postoperative urine leakage (Double-J insertion), n (%)	67 (26.4)
Need for blood transfusion, n (%)	58 (22.8)
Postoperative ancillary procedures, n (%)
SWL	15 (5.9)
TUL	27 (10.6)
PCNL	12 (4.7)

Stone volume = a × b × c × π ÷ 6.

To determine the performance of the ANN system in predicting postoperative events and outcomes, we compared the actual results with the results predicted by the software. As shown in Table 4, accuracy and sensitivity of the system for predicting different postoperative variables ranged from 81.0% to 98.2%. The area under the receiver operating characteristic curve for the ANN model for predicting the SFR was 0.861 (Fig. 4).

FIG. 4.

Receiver operating characteristic curve for stone-free status (area under the curve = 0.861).

Table 4.

Performance of an Artificial Neural Network System in Predicting Stone-Free Status, Need for Different Ancillary Procedures, and Blood Transfusion in the Test Group

	True positive rate (sensitivity)	False positive rate	Accuracy (%)	Precision (positive predictive value)
Stone-free status	0.83	0.19	82.8	0.83
Need for repeat PCNL	0.97	0.39	97.7	0.99
Need for SWL	0.98	0.12	98.2	0.88
Need for TUL	0.92	0.32	92.5	0.92
Need for stent insertion^a	0.81	0.32	81.1	0.80
Need for blood transfusion	0.85	0.25	85.8	0.85

To manage urine leakage.

Table 5 summarizes the three most significant preoperative variables that affected different postoperative outcomes. The system assigned a relative weight for each variable. For example, the most significant variables that controlled the SFR were the stone burden and the stone morphometry (i.e., staghorn stones). This was true regarding other postoperative values such as the need for blood transfusion and ancillary procedures (Table 5).

Table 5.

Three Highly Weighted Preoperative Factors Determined by Artificial Neural Network Analysis to Affect Postoperative Outcome

Preoperative data	Relative weight
Stone-free status
Stone burden	12.439912649514316
Staghorn stone	12.138547341684561
Multiple stones	−9.6453385347247793
Need for blood transfusion
Preoperative hemoglobin	−11.259442484302262
Staghorn stone	9.617860264428179
Stone burden	8.798457665983767
Need for SWL
Stone burden	2.5171304273848163
Staghorn stone	2.3370552443585066
Renal anomalies	1.24671896769625
Need for PCNL
Stone size	2.8167100613631777
Staghorn stone	2.6146986859257194
Multiple stones	2.548713695350154
Need for stenting^a
Staghorn stone	15.67164546125897
Stone burden	7.303966793608051
Body mass index	4.789742574691487
Need for TUL
Stone burden	6.873974574379403
Renal anomalies	5.374205493427698
Hydronephrosis grade	−3.244856196300365

To manage urine leakage.

Discussion

In the era of minimally invasive urologic surgery, PCNL has been used as the standard surgical modality to manage large renal calculi. Because it involves major surgery, PCNL can be associated with significant complications, so the SFR after PCNL and the need for subsequent procedures after PCNL are important concerns. The Clinical Research Office of the Endourological Society (CROES), through a global survey, reported an overall SFR of 75.5% and complication rate of 20.5% for PCNL.¹¹ In recent years the need to develop predictive models has been noted by many investigators¹² to optimize patient selection and patient counseling and to further develop ways to determine the most likely surgical outcomes before surgery is performed.

Traditionally, stone burden and morphometry have been considered the main prognostic variables of SFR after PCNL. On the basis of this notion, in 2011 Thomas and colleagues developed the Guy's stone score system for grading PCNL complexity based on radiologic findings and stone burden.¹³ Although this model is straightforward to apply, it was not based on a data-driven analysis, and the grading system was designed based mainly on previous studies and expert opinion.

Based on preoperative CT images, stone burden, and a morphometric scoring system, Jeong and colleagues designed another renal stone complexity scoring model to predict SFR after PCNL with an accuracy of 0.86.¹⁴ They included only single tract PCNL procedures in their study, and access was obtained by radiologists before the operation.

Using preoperative clinical and radiologic data from a global cohort of patients recruited by the CROES PCNL study group from 96 centers worldwide, Smith and colleagues developed a nomogram to predict SFR after PCNL with an accuracy rate of 0.76.¹⁵ The model was designed and validated based on a comprehensive prospective data set, and the weight of each input was calculated and considered.

In the present study we developed an ANN model to predict the outcome of PCNL. In contrast to previous studies, which were based on statistical models (i.e., regression analysis), once they are developed, neural network systems require no previous knowledge of the data, and interestingly, their continuous learning capability by exposure to the input data make them valuable tools for recognizing patterns and relationships between variables.^5,9,16

The application of ANNs has advanced particularly in the field of urolithiasis in parallel with their promising predictive accuracy in the field of uro-oncology as an alternative decision-making modality to nomograms.¹⁷ Rajan and Tolley reviewed the accuracy of ANNs for predicting stone composition, genetic factors in recurrent stone formers, spontaneous passage of ureteral stones, and stone clearance after SWL. They reported a predictive accuracy of ANNs for these applications in the field of urolithiasis between 76% and 100%, which seems promising in comparison to the accuracy of statistical models (75%–93.2%).⁵

Gomha and colleagues compared the accuracy of ANN vs a logistic regression model to predict SFR after SWL for ureteral stones and found significantly better accuracy with ANN (93.2%) compared to logistic regression (77.7%).¹⁸

In the present study we designed and applied a similar model to predict the outcome of PCNL. Unlike previous applications of ANN in urolithiasis, our end points addressed not only SFR but also postoperative complications and the need for ancillary procedures. Our model had different predictive accuracies for each postoperative parameter (82.8% for SFR and 97.7% for repeat PCNL) (Table 4). This model can also easily calculate and report the relative weight of each preoperative variable on any postoperative outcome (Table 5). For most postoperative outcome variables, the largest weights among all preoperative variables that might predict the outcome were for stone burden and morphology. These findings are compatible with the nomogram model recently developed by Smith and colleagues.¹⁵

As complex nonlinear mathematical models, ANN systems represent an interconnected data mining modality, which prospectively analyze and “learn” the relationships between the variables. Our system was able to “intuitively” optimize the interconnected weights between parameters as more input data were fed into the system. This resulted in a close approximation of the predicted values to the actual values. Although no studies to date have compared the prognostic accuracy of ANN vs regression analysis methods in the field of PCNL, earlier research on the outcomes of SWL reported the same (if not better) predictive accuracy of ANN systems in comparison to statistical models.^5,7,18,19

A potential shortcoming of ANNs is the absence of underlying casual relationships between data processing algorithms and clinical realities—a situation which may sometimes result in inconsistencies in the apparent relevance of associations between variables. In the current study, as in the experience of other pioneers in ANN,^18,19 the Wilk generalized likelihood ratio was used as a reliable mathematical modality to identify and avoid potential confusion in the relationships between inputs and clinically relevant outputs.

Although ANN, unlike statistical models, can be easily deployed using user-friendly software to simplify individual-based prediction and decision-making, we are aware that the results of this study and the application of ANN in the field of PCNL should be considered a preliminary step. All procedures in this study were done by fully competent surgeons; therefore, the effect of surgeon's experience and case volume on the outcome was not included. The same is true regarding the details of the PCNL operation (such as patient's position, rigid vs flexible nephroscopy, and type of lithotriptor). Notwithstanding, one of the advantages of ANN is that the software can be easily updated with more preoperative and intraoperative variables in future versions and the institutes can easily add other features to their software to customize it. Hence, the effect and the weight of any concerned variable can be easily calculated.

Comparative prospective trials to confirm the predictive value of ANN vs statistical data mining models and to validate the system for making the decision of whether or not to proceed with PCNL will be needed.

Conclusion

ANN system, as an interconnected data mining tool, can prospectively analyze the relationships between variables. The accuracy and sensitivity of the system for predicting the stone-free rate, the need for blood transfusion, and post-PCNL ancillary procedures ranged from 81.0% to 98.2%.The stone burden and the stone morphometry were among the most significant preoperative characteristics that affected all postoperative outcome variables and they received the highest relative weight by the ANN system.

Footnotes

Acknowledgments

The present article is based on the thesis by Dr. Alireza Aminsharifi for the MS degree in Master of Public Health (MPH) awarded by Shiraz University of Medical Sciences (No. 95-01-01-11894). The authors thank K. Shashok (AuthorAID in the Eastern Mediterranean) for improving the use of English in the article.

Author Disclosure Statement

No competing financial interests exist.

Abbreviations Used

References

Streeper

, Radtke

, Penniston

, McDermott

, Nakada

. Percutaneous nephrolithotomy in patients with BMI >50: Single surgeon outcomes and feasibility. Urology, 2016; 87:33–39.

Sfoungaristos

, Gofrit

, Pode

, Landau

, Duvdevani

. Percutaneous nephrolithotomy for staghorn stones: Which nomogram can better predict postoperative outcomes?. World J Urol, 2016; 34:1163–1168.

Sfoungaristos

, Gofrit

, Yutkin

, Landau

, Pode

, Duvdevani

. External validation of CROES nephrolithometry as a preoperative predictive system for percutaneous nephrolithotomy outcomes. J Urol, 2016; 195:372–376.

, Deng

, Chong

, Zhang

, Li

, et al. Predictors of clinical outcome after minimally invasive percutaneous nephrolithotomy for renal calculus. Urolithiasis, 2015; 43:355–361.

Rajan

, Tolley

. Artificial neural networks in urolithiasis. Curr Opin Urol, 2005; 15:133–137.

Hamid

, Dwivedi

, Singh

, Gopi Kishore

, Mahmood

, Singh

, et al. Artificial neural networks in predicting optimum renal stone fragmentation by extracorporeal shock wave lithotripsy: A preliminary study. BJU Int, 2003; 91:821–824.

Chiang

, Chiang

, Chen

, Tsai

. Prediction of stone disease by discriminant analysis and artificial neural networks in genetic polymorphisms: A new method. BJU Int, 2003; 91:661–666.

Aminsharifi

, Hosseini

, Khakbaz

. Laparoscopic pyelolithotomy versus percutaneous nephrolithotomy for a solitary renal pelvis stone larger than 3 cm: A prospective cohort study. Urolithiasis, 2013; 41:493–497.

Lawrence

. Introduction to Neural Networks. Design, Theory, and Applications, 6th ed. Nevada City, CA: California Scientific Software, 1994.

10.

Golden

. Mathematical Methods for Neural Network Analysis and Design. Cambridge, MA: MIT Press, 1996.

11.

de la Rosette

, Assimos

, Desai

, Gutierrez

, Lingeman

, Scarpa

, Tefekli

; CROES PCNL Study Group. The clinical research office of the endourological society percutaneous nephrolithotomy global study: Indications, complications, and outcomes in 5803 patients. J Endourol, 2011; 25:11–17.

12.

Matlaga

, Hyams

. Stones: Can the Guy's stone score predict PNL outcomes?. Nat Rev Urol, 2011; 8:363–364.

13.

Thomas

, Smith

, Hegarty

, Glass

. The Guy's stone score—Grading the complexity of percutaneous nephrolithotomy procedures. Urology, 2011; 78:277–281.

14.

Jeong

, Jung

, Cha

, Lee

, Jeong

, et al. Seoul National University Renal Stone Complexity score for predicting stone-free rate after percutaneous nephrolithotomy. PLoS One, 2013; 8:e65888.

15.

Smith

, Averch

, Shahrour

, Opondo

, Daels

, Labate

, et al.; CROES PCNL Study Group. A nephrolithometric nomogram to predict treatment success of percutaneous nephrolithotomy. J Urol, 2013; 190:149–156.

16.

Niederberger

. This month in investigative urology. Commentary on the use of neural networks in clinical urology. J Urol, 1995; 153:1362.

17.

Anagnostou

, Remzi

, Lykourinas

, Djavan

. Artificial neural networks for decision-making in urologic oncology. Eur Urol, 2003; 43:596–603.

18.

Gomha

, Sheir

, Showky

, Abdel-Khalek

, Mokhtar

, Madbouly

. Can we improve the prediction of stone-free status after extracorporeal shock wave lithotripsy for ureteral stones? A neural network or a statistical model?. J Urol, 2004; 172:175–179.

19.

Poulakis

, Dahm

, Witzsch

, de Vries

, Remplik

, Becht

. Prediction of lower pole stone clearance after shock wave lithotripsy using an artificial neural network. J Urol, 2003; 169:1250–1256.