Abstract
Background
The multiparametric magnetic resonance imaging (mpMRI)-based Prostate Imaging for Recurrence Reporting (PI-RR) system has been proposed to evaluate local recurrence in patients with prostate cancer (PCa) who have been treated with radiation therapy (RT) or radical prostatectomy (RP).
Purpose
To evaluate the diagnostic performance and interreader agreement of the PI-RR system in the diagnosis of locally recurrent PCa remains.
Material and Methods
A total of 110 patients who have biochemically recurrent PCa after RT (n = 35) or RP (n = 75) were included in this retrospective study. All patients underwent mpMRI, PSMA-PET/CT, and biopsy. Four radiologists with varying levels of expertise independently assessed the local recurrence of PCa using PI-RR. The reference standard was the biopsy pathology. The receiver operating characteristic (ROC) curve was used to evaluate the performance of PI-RR and PSMA-PET/CT, and areas under the ROC curve (AUC) were calculated. Interreader agreement across four readers was evaluated using the intraclass correlation coefficient (ICC).
Results
Among 110 patients with biochemically recurrent PCa, 28 had local recurrence and 82 had no local recurrence. Using a cutoff of 4, the AUCs of PI-RR in the diagnosis of local recurrence were in the range of 0.61–0.84 in patients treated with RT and 0.71–0.89 in patients treated with RP. The ICC was 0.86 (95% confidence interval = 0.81–0.91).
Conclusion
PI-RR using a cutoff of 4 has a favorable diagnostic performance and interreader agreement, which might be alternatively used for detecting local recurrence in patients with biochemically recurrent PCa treated with RT or RP.
Introduction
Prostate cancer (PCa) is the third most prevalent malignancy among men and the second leading cause of tumor-related death globally (1). The majority of patients with PCa can be cured by radical prostatectomy (RP) or whole-prostate radiation therapy (RT). However, approximately 30%–50% of patients may experience biochemical recurrence (BCR) after either intervention (2). In the case of BCR, the decisions of further treatment choice rely heavily on confirming the presence of local recurrence, which might benefit from local salvage therapy. Thus, accurate detection of the local recurrence in PCa patients with BCR is of paramount importance for tailoring therapeutic approaches to impede or delay the disease progression effectively (3).
Histologic biopsy was usually recognized as the reference standard of local recurrence in patients with PCa after curative treatment, while the invasive nature and side effects restricted its clinical application. Prostate-specific membrane antigen (PSMA)-positron emission tomography/computed tomography (PET/CT) was usually adopted as a diagnostic modality for the identification of PCa recurrence, with a high diagnostic sensitivity of up to 80% (4). However, PSMA-PET/CT is not readily available, and the exposure to ionizing radiation further restricts its widespread application. Instead, multiparametric magnetic resonance imaging (mpMRI) has become one of the main diagnostic workups of PCa, which was not only widely used in the assessment of primary PCa, but also in the detection of local recurrence (5,6).
Upon mpMRI, the Prostate Imaging for Recurrence Reporting (PI-RR) system has been proposed to standardize the assessment of local recurrence of PCa in 2021 (7). PI-RR is a 5-point scale for the prediction of the likelihood of local recurrence in patients with PCa after being treated with RT or RP. It has been reported that PI-RR provided a reproducible and accurate evaluation of local recurrence, with an accuracy in the range of 77%–88% across different readers (8). However, the application of the PI-RR scoring system in clinical practice remains limited thus far. The main reason may be that the diagnostic performance and repeatability of PI-RR in detecting local recurrence of PCa remains to be adequately confirmed (9).
In this study, mpMRI scans of patients with BCR who were treated with RP or RT were retrospectively analyzed. The aim of the present study was to determine the diagnostic performance and interreader agreement of PI-RR in the diagnosis of local recurrence in patients with BCR PCa, with histopathological results used as the reference standard.
Material and Methods
Patients
This retrospective study was approved by the institutional review boards and ethical committees. Written informed consent was waived due to the retrospective nature of the study. Between January 2019 and December 2021, PCa patients who have been treated by RT or RP were identified from the electronic medical record system of our hospital. The inclusion criteria were: (i) patients diagnosed with BCR with evidence of prostate-specific antigen (PSA) persistence, according to the European Association of Urology Guidelines on Prostate Cancer (10); (ii) patients who underwent mpMRI for local recurrence assessment; (iii) patients who underwent PSMA-PET/CT within 20 days after pelvic mpMRI to exclude distant metastases; and (iv) patients who underwent target biopsy to confirm the diagnosis of local recurrence and non-local recurrence. BCR in RP-treated patients was defined as an undetectable PSA level after RP followed by two or more subsequent PSA levels >0.2 ng/mL (11); in the case of RT, BCR was defined as an increase >2.0 ng/mL over the nadir in PSA levels after RT (12). The exclusion criteria were as follows: (i) patients who received other treatment for PCa, such as androgen deprivation therapy during mpMRI; and (b) poor image quality due to artifacts of mpMRI. The flowchart of patient enrollment is shown in Fig. 1. Finally, 110 patients were included for analysis and the demographic and clinical data of these patients are shown in Table 1.

Study flowchart of patient enrollment and PI-RR assessment. PI-RR, Prostate Imaging for Recurrence Reporting.
Demographic and histopathologic characteristics of enrolled patients.
Values are given as n (%) or median (IQR).
CT, computed tomography; PET, positron emission tomography; PSMA, prostate-specific membrane antigen; RP, radical prostatectomy; RT, radiation therapy.
mpMRI protocols
mpMRI was performed using 3.0-T MR scanners with a 32-channel phased-array body coil (Achieva, Philips Healthcare; Discovery MR750, GE Healthcare; Intera, Philips Healthcare). The protocol for all patients consisted of axial and sagittal T2-weighted (T2W) imaging, axial echo-planar diffusion-weighted imaging (DWI) with b-values of 0, 800, or 1400 s/mm2, axial T1-weighted (T1W) imaging, and axial dynamic contrast-enhanced (DCE) imaging. ADC maps were automatically calculated on a pixel-by-pixel basis using b-values of 0 and 800 or 0 and 1400 s/mm2. The detailed acquisition parameters of each sequence are provided in Table 2.
Parameters of the main mpMRI sequences.
DCE, dynamic contrast-enhanced; DWI, diffusion-weighted imaging; FOV, field of view; T1WI, T1-weighted imaging; T2WI, T2-weighted imaging; TE, echo time; TR, repetition time.
Image analysis
The mpMRI images were independently assessed by four readers, with 10, 8, 6, and 3 years of experience in prostatic imaging, respectively, who were not blinded to the clinical data but blinded to the histological results. Each reader independently assigned the PI-RR assessment category to each MRI sequence as depicted in Fig. 2, the overall PI-RR assessment score for local recurrence after RT and RP was summarized in the Supplementary Material (Figure S1) according to the previously proposed PI-RR guidelines (7). For patients with a single lesion, the PI-RR score of this lesion was uses as the final score of the patient for analysis. If patients have multiple lesions, the lesion with the highest PI-RR score was used as for analysis. The PSMA-PET/CT was reviewed by an experienced nuclear medicine physician 15 years of reading experience.

ROC curves of PI-RR in assessment of local recurrence in patients with biochemically recurrent PCa who have been treated with RT. PCa, prostate cancer; PI-RR, Prostate Imaging for Recurrence Reporting; ROC, receiver operating characteristic; RT, radiation therapy.
Statistical analysis
The continuous variables (patient age and PSA level at the time of BCR) were presented as median (interquartile range [IQR]) and the categorical variable (International Society of Urological Pathology Gleason [ISUPG] grade category of prostate biopsy) was expressed as frequencies (percentagesReceiver operating characteristic (ROC) curve analysis was used to assess the diagnostic performance of PI-RR and PSMA-PET/CT at the patient level, with sensitivity, specificity, and the area under the ROC curve (AUC) calculated. In addition, the AUC values were computed by dichotomizing the PI-RR scores into <3 versus ≥3 and <4 versus ≥4. The results of the target biopsy were used as the diagnostic reference standard. The interreader agreement among the four readers was assessed using the intraclass correlation coefficient (ICC). MedCalc software (version 15.2.2) was used for all statistical analyses. P < 0.05 (two-sided) was considered indicative of a statistically significant difference.
Results
Clinicopathologic characteristics of patients
Among 110 patients with BCR (35 patients treated with RT and 75 patients treated with RP), 28 of them were diagnosed with local recurrence by target biopsy and 82 of them had no local recurrence. The median serum PSA level was 0.67 ng/mL (IQR = 0.19–16.67 ng/mL) for patients treated with RT and 0.24 ng/mL (IQR = 0.08–0.72 ng/mL) for patients treated with RP as shown in Table 1.
Diagnostic performance of PI-RR
Among 110 patients with BCR, the AUCs of PI-RR with a cutoff of 3 and 4 in the diagnosis of local recurrence were in the range of 0.67–0.82 and 0.69–0.87, respectively. The AUC of PSMA-PET/CT in the diagnosis of local recurrence detection was 0.85 (95% confidence interval [CI] = 0.77–0.91). The detailed AUC values of PI-RR in the diagnosis of local recurrence in patients who have been treated with RT or RP are shown in Tables 3 4. Using a cutoff of 3 and 4, the AUCs of PI-RR in the diagnosis of local recurrence were in the range of 0.56–0.83 and 0.61–0.84 across the four readers in the RT group, and 0.73–0.80 and 0.71–0.89 in the RP group, respectively. The ROC curves of PI-RR in the assessment of local recurrence in patients who have been treated with RT and RP are shown in Figs. 2 and 3, respectively.

ROC curves of PI-RR in assessment of local recurrence in patients with biochemically recurrent PCa who have been treated with RP. PCa, prostate cancer; PI-RR, Prostate Imaging for Recurrence Reporting; ROC, receiver operating characteristic; RP, radical prostatectomy; RT, radiation therapy.
Diagnostic performance for each reader for recurrence detection in patients who underwent radiation therapy.
Values in parentheses are 95% CI.
AUC, area under the receiver operating characteristic curve; CT, computed tomography; NPV, negative predictive value; PET, positron emission tomography; PPV, positive predictive value; PSMA, prostate-specific membrane antigen.
Diagnostic performance for each reader for recurrence detection in patients who underwent radical prostatectomy.
AUC, area under the receiver operating characteristic curve; CT, computed tomography; NPV, negative predictive value; PET, positron emission tomography; PPV, positive predictive value; PSMA, prostate-specific membrane antigen.
The per-reader AUC values for each MRI sequence and the overall PI-RR scores are shown in Fig. 4. The AUC values of DWI, DCE, and overall PI-RR (DWI + DCE) assessment for four readers in the RT groups were 0.55–0.88, 0.54–0.86, and 0.51–0.87, respectively. The AUC values of DWI, DCE, and PI-RR assessment for all readers in the RP groups were 0.75–0.89, 0.74–0.90, and 0.73–0.90, respectively. No significant differences were observed in either group (all P > 0.05). The detailed sensitivity, specificity, and AUC values of DWI, DCE, and overall PI-RR (DWI + DCE) assessment for four readers are shown in the Supplementary Material (Tables S1 and S2).

ROC curves of PI-RR assessed by four readers on each mpMRI sequence. (a–d) The ROC curves of PI-RR were assessed in patients treated with RT on DWI and DCE across four readers: (a) reader 1; (b) reader 2; (c) reader 3 (d) reader 4. (e–h) The ROC curves of PI-RR assessed in patients treated with RP on DWI and DCE across four readers: (e) reader 1; (f) reader 2; (g) reader 3; (h) reader 4. DCE, dynamic contrast enhancement; DWI, diffusion-weighted imaging; PCa, prostate cancer; PI-RR, Prostate Imaging for Recurrence Reporting; ROC, receiver operating characteristic; RP, radical prostatectomy; RT, radiation therapy.
Interreader agreement of PI-RR
The overall ICC was 0.77 (95% CI = 0.69–0.83) across the four readers when using a cutoff of 3 and 0.86 (95% CI = 0.81–0.91) when using a cutoff of 4. There was disagreement among the four readers for 11 patients treated with RT and 15 patients treated with RP. Disagreement between more than two readers was found in 4/11 patients treated with RT and in 7/15 patients treated with RP. The detailed ICC values for each pair of readers in patients who have been treated with RT and RP are shown in Table 5. An example case of disagreement among the four readers is shown in Fig. 5. The PI-RR scores were 3, 3, 4, and 4 among the four readers.

PSMA-PET/CT and mpMRI images of a 68-year-old man experiencing BCR (PSA level = 0.94 ng/mL) with a history of PCa and RP. (a) PSMA-PET/CT shows the local recurrence; (b) DWI shows the hyperintense signal of local recurrence (arrow); (c) early-phase DCE image shows focal enhancement of local recurrence(arrow). The PI-RR scores were 3, 3, 4, and 4 among the four readers. BCR, biochemical recurrence; CT, computed tomography; DCE, dynamic contrast enhancement; DWI, diffusion-weighted imaging; mpMRI, multiparametric magnetic resonance imaging; PCa, prostate cancer; PET, positron emission tomography; PI-RR, Prostate Imaging for Recurrence Reporting; PSA, prostate-specific antigen; PSMA, prostate-specific membrane antigen; ROC, receiver operating characteristic; RP, radical prostatectomy.
Inter-reader agreement for PI-RR scoring by each of the six reader pairs.
PI-RR, Prostate Imaging for Recurrence Reporting; RP, radical prostatectomy; RT, radiation therapy.
Discussion
In this study, the diagnostic performance and interreader agreement of the PI-RR system in detecting local recurrence in PCa patients with BCR who have been treated with RP or RT were evaluated by four readers with varying experience. Our results showed that with a PI-RR cutoff of 4, the AUC values of PI-RR of the four readers were in the range of 0.61–0.84 in patients treated with RT and 0.71–0.89 in patients treated with RP. The interreader agreement among the four readers was satisfactory (ICC = 0.86).
Due to the biological characteristics of malignancy and possible residual foci, the risk of local recurrence after curative treatments including RT and RP remains in patients with PCa (13), which represents a stage of disease progression that usually occurs after BCR. Its diagnosis commonly requires confirmation via invasive biopsy. Instead, several non-invasive means, such as transrectal ultrasound, mpMRI, and PET/CT, have been used to detect local recurrence of PCa. Among them, transrectal ultrasound has a lower detection rate of 40%–50% for identifying local recurrence in patients treated with RP (14). In contrast, mpMRI exhibits a sensitivity of 94% in detecting recurrence in patients with PSA >0.5 ng/mL after RP, albeit with a specificity of 45%. The sensitivity of mpMRI was even higher than that of 11C-choline PET/CT in detecting recurrence in patients with PSA levels below 1 ng/mL after RP and RT (15–17). Consequently, PI-RR based on mpMRI has been proposed as a valuable tool for predicting the risk of local recurrence in patients previously treated with RT and RP.
However, the diagnostic performance and interreader variability of the PI-RR system have only been investigated in one multireader study (8), in which the AUC values of PI-RR were reported to be in the range of 0.77–0.92 across different readers in patients treated with RT and 0.80–0.88 in patients treated with RP (ICC = 0.87). In that study, all readers were experienced radiologists with over 10 years of experience, yielding an unexpectedly high level of interreader agreement. In addition, only one of the following was used as the reference standard of local recurrence: histopathology, imaging follow-up, or clinical response to treatment.
In our study, the readers involved in PI-RR assessment had varying levels of reading experience (range = 3–10 years). Our results showed that using the cutoff of 4, the AUC values of PI-RR in reader 1 (with 3 years of experience) were 0.61 and 0.74 in patients after RT and RP, while PI-RR in reader 4 (with 10 years of experience) reached 0.84 and 0.80, respectively. These results suggest that the diagnostic performance of PR-RR is overall moderate to high among readers with different experiences. Notably, in our study, all patients underwent histologic biopsy to confirm the diagnosis of local recurrence. In addition, all patients underwent PSMA-PET/CT to exclude distant metastases, which is a prerequisite for PI-RR assessment in detecting local recurrence (7). As a comparative imaging technique for evaluating diagnostic performance against PI-RR, the AUC of the PSMA/PET-CT in the detection of local recurrence was 0.85, which is similar to that reported previously (AUC = 0.88) (18). Our results showed that when using 4 as the threshold, the AUC of PI-RR was higher than that of PSMA-PET/CT (0.89 vs. 0.84) in the experienced reader (reader 4). Although the statistical difference was not significant (P = 0.589), it indicated that the PR-RR might be an alternative to PSMA-PET/CT in the diagnosis of local recurrence in patients with BCR.
Moreover, our results showed that the PI-RR assessment system was reproducible among the four readers, with excellent interreader agreement (ICC = 0.77 and 0.86 when using a cutoff of 3 and 4, respectively), which was similar to the result reported by Pecoraro et al. (overall ICC = 0.87) (8). In addition, the ICC of PI-RR in our study was slightly higher than that of DCE (0.63–0.70) and T2W (0.18–0.39) imaging reported by Rouvière et al. (19) in assessing local recurrence in patients treated with RT. Comparatively, the PI-RR assessment system has good repeatability among different readers.
The present study has some limitations. First, it is a retrospective, single-center study with a relatively small sample size. Although use biopsy pathology was used as the reference standard, the invasive nature of the procedure resulted in limited pathological data. Second, the analysis was stratified by treatment modality (RP/RT) rather than PSA levels, as all participants experienced BCR, leading to a relatively narrow PSA range. Finally, the T2W imaging from the mpMRI protocol was excluded from analysis. In patients who have undergone RT, the irradiated prostate tissue typically exhibits reduced zonal differentiation and poor contrast between tumor and background parenchyma on T2W imaging, which complicates interpretation. To minimize diagnostic ambiguity, T2W imaging was not analyzed in this study.
In conclusion, our study demonstrated that PI-RR provides moderate to high diagnostic accuracy across radiologists with varying levels of experience, along with good interreader agreement for assessing local recurrence in patients with biochemically recurrent PCa after RT or RP. Using a PI-RR cutoff of 4 yielded favorable diagnostic performance and interreader agreement, suggesting it may serve as a useful tool for detecting local recurrence in this patient population.
Supplemental Material
sj-docx-1-acr-10.1177_02841851251334364 - Supplemental material for The diagnostic performance and reader agreement of the Prostate Imaging for Recurrence Reporting system in the evaluation of local recurrence in patients with biochemically recurrent prostate cancer
Supplemental material, sj-docx-1-acr-10.1177_02841851251334364 for The diagnostic performance and reader agreement of the Prostate Imaging for Recurrence Reporting system in the evaluation of local recurrence in patients with biochemically recurrent prostate cancer by Xiang Liu, Shuyi Yang, Wenhui Deng, Dongye Li and Jun Shen in Acta Radiologica
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This study was supported by Sun Yat-Sen Pilot Scientific Research Fund (No. YXQH202513).
Supplementary material
Supplementary material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
