Objective performance assessment of five computed tomography iterative reconstruction algorithms

1 Introduction

Computed tomography (CT) is an essential modality in medical imaging and its use has increased significantly in recent years [1–4]. Concern over stochastic radiation risk arising from the increased use of CT has led to several initiatives and developments targeting dose reduction [5–8]. One recent development has been the commercialization of iterative reconstruction algorithms. Iterative algorithms are being explored for dose reduction in CT while maintaining image quality. They complement already existing dose reduction techniques such as tube-current modulation and automatic kV selection [2, 9].

Generally speaking, iterative algorithms use models of data acquisition and noise statistics to improve the tradeoff between image quality and dose. Filtered back projection (FBP), on the other hand, assumes idealized data acquisition and does not account for data statistics, leading to amplification of image noise [10–13]. Iterative algorithms require significantly more computational resources compared to FBP [4]. With the availability of fast parallel processing such as that performed by graphic processor units, CT iterative algorithms have become a clinical reality.

General Electric (GE, Milwaukee, WI, USA) introduced the first FDA-approved clinical iterative method in 2008, called adaptive statistical iterative reconstruction (ASIR^™). ASIR^™ utilized a physical model of the scanner’s geometry. The final image could be a blend of iterative and FBP reconstructions, as determined by the user [14]. The amount of blend with FBP, determined the ‘strength’ of ASIR^™ used in the reconstruction, usually referred to as percentages. GE later introduced a full model-based iterative method, VEO^™, which employed a comprehensive physical model of the scanner’s geometry, system optics and data statistics. It used 3D voxel volumes of the scanned object, and complex mathematical computations to account for scatter, beam hardening and metal attenuation artifacts [10, 14–16]. VEO^™ required significantly longer processing times compared to FBP and ASIR^™.

Siemens Healthcare (Erlangen, Germany) introduced its first generation iterative algorithm, iterative reconstruction in image space, (IRIS^™) in 2009. IRIS^™ operated in the image domain. A second generation algorithm was released in 2011, named SAFIRE^™ (sinogram affirmed iterative reconstruction). SAFIRE^™ operated in the data and image domains, using a dynamic raw-data-based noise model for image noise reduction [17–19]. The level of noise removal and image noise texture was determined by the ‘strength’ of SAFIRE^™, with up to five user-selected strength levels [19].

Toshiba Medical Systems (Tochigi, Japan) introduced adaptive iterative dose reduction 3D (AIDR3D^™) in 2012. AIDR3D^™ was also designed to work both in the data and image domain. Noise reduction in the data domain involved the combination of scanner and statistical models to characterize and reduce noise. Iterations in the image domain resulted in further image quality enhancements leading to the final image [20–23]. The level of noise reduction was adjustable by four preset strengths namely: ‘weak’, ‘mild’, ‘standard’, and ‘strong’.

Philips Healthcare (Eindhoven, Netherlands) employs an algorithm commercially referred to as iterative dose reconstruction 4th generation (iDose^4™). iDose^4™ performs iterative processing both in the data and image domains. It uses a maximum-likelihood de-noising algorithm to remove noise in the data domain. A combination of physical and dynamic multi-frequency models are then used to reduce image noise while preserving high frequencies [10, 25]. The strength of the iterative process is adjustable by selecting one of seven levels.

Several studies have investigated the performance of iterative algorithms using objective quantitative parameters or subjective qualitative assessments [10, 26–33]. A useful review of different studies and a description of several algorithms can be found in Willemink et al. [4].

Hara et al. [26] reported the possibility of dose reduction in the range of 32%–65% with 40% ASIR^™ strength, following a preliminary study in adult abdomen/pelvic examinations. Chang et al. [15] reported up to 46% dose reduction using model-based VEO^™ with respect to a regular dose FBP, with superior depiction of lesion conspicuity in the liver. More recently, Mieville et al. [10] reported dose reduction possibilities of 67%–86% using quantitative physical metrics and subjective assessments of lesions in phantom images reconstructed with ASIR^™ and VEO^™.

For SAFIRE^™, Takx et al. [34] reported improved diagnostic accuracy in coronary CT angiography of adults using 50% of radiation dose. Ebersberger et al. [29] reported improved noise and signal-to-noise ratio (SNR), with equivalent image quality assessments using SAFIRE^™ at half-dose with respect to FBP at full dose. Baumueller et al. [17] reported improved image quality and diagnostic accuracy in non-enhanced low dose lung CT. Using quantitative physical metrics, Ghetti et al. [30] reported up to 60% noise reduction with SAFIRE^™, without significant resolution improvements regardless of the choice of reconstruction kernel.

Yamada et al. [21] reported a 64% dose reduction using AIDR3D^™, maintaining diagnostic acceptability in a low-dose chest CT examination. Improved SNR and decreased noise was obtained relative to FBP and an earlier version of the algorithm. Chen et al. [22] suggested that a 50% dose reduction produces comparable image quality with the standard FBP, using AIDR3D^™ in a retrospective clinical coronary CTA study. Gervaise et al. [32] reported a mean noise reduction of 40% using phantoms, with improved SNR when compared to FBP. However, no spatial resolution improvement was observed.

For iDose^4™, Li et al. [24] reported improved or equivalent image quality metrics in contrast-enhanced chest CT of 80 patients after 71% dose reduction at low-tube voltage (80kVp) with respect to standard FBP (120 kVp). Mieville et al. [10] reported improved contrast-to-noise ratio (CNR) and lower noise in phantom scans, but no spatial resolution change was observed.

The purpose of this study was to measure objective image quality metrics on four clinical CT scanners, and to investigate the performance of their respective iterative algorithms. We investigated the following algorithms: ASIR^™ and VEO^™ (GE Healthcare, WI, USA); SAFIRE^™ (Siemens Healthcare, Erlangen, Germany); AIDR3D^™ (Toshiba Medical Systems, Tochigi, Japan) and iDose^4™ (Philips Healthcare, Eindhoven, Netherlands). We focus on routine abdomen/thorax protocols, which are the most clinically common protocols in our jurisdiction [35].

2 Methods and materials

4 Discussion

Image-domain standard deviation measurements and NPS calculations demonstrated that image noise decreases with the use of iterative algorithms or with increasing iterative reconstruction strength, at equivalent dose values. The shape of the NPS curve produced from FBP images is comparable to data available in literature, as the NPS increases at lower frequencies, and decreases at higher frequencies consistent with ramp and apodization filters [37, 39]. Moreover, the noise spectrum shifted towards lower frequencies with increasing iterative strength, suggesting that the image noise texture appeared coarser. With the two systems (Siemens and Philips) that employ “effective mAs” to compensate for the effect of helical pitch, the magnitude of the NPS did not vary or varied slightly with pitch. With the other two systems (GE and Toshiba), the magnitude of the NPS is higher for higher pitch, as expected. The effect of pitch, however, diminishes with increasing strength of iterative processing.

We also presented coefficients for fitting the standard deviation as a function of dose for the various algorithms, using the formula SD = a · CTDI^-x where x = ½ if the Poisson model applies. GE’s VEO^™ and all of Toshiba’s reconstructions did not conform to the Poisson model. Model-based algorithms employ regularization terms to enforce smoothness in the reconstructed images. Regularization terms control the tradeoff between noise and resolution. The extent of regularization depends on the local differences in pixel values [40]. Our results are similar to other reports which demonstrated this difference in noise-dose power law in model-based algorithms with an exponent less than ½ [41]. The smaller exponent suggests that image noise in model-based algorithms has a weaker dependence on dose. In our results, for example, a 50% reduction in dose will increase VEO^™ image noise by 26%, compared to the Poisson relationship which predicts a noise increase of 40%. This helps explain the superior noise reduction observed for model based algorithms at very low doses.

The CNR improves with increasing strength of iterative reconstruction on all scanners. The model-based iterative reconstruction algorithm from GE, VEO^™, has a different performance pattern. It offers significant improvements in CNR and noise performance at low doses. The improvements however diminish as dose increases. For example, given the scan parameters employed in our study, the CNR obtained with VEO^™ becomes equal to the CNR obtained with ASIR100 at 1 mGy for PF = 0.516 and 2 mGy for PF = 0.984. A similar pattern can be observed from another study [10], although the dose value at which equivalent performance occurs was higher (∼3.5 mGy). The pitch factor in that study was 1.375. This suggests that VEO^™ is more beneficial at low doses and high pitch factors. The NPS results obtained for VEO^™ in our study further demonstrate this noise behavior.

Generally speaking, spatial resolution did not depend on choice of algorithm (provided the same kernels were used) or the strength of iterative reconstruction, over the range of doses of this study. In the case of GE reconstructions, VEO^™ increases spatial resolution by 30% over FBP and ASIR^™. This is the result of employing an image acquisition model which incorporates and compensates for sources of image blur. In this manner, VEO^™ has an additional benefit over the noise-reduction capabilities of the other algorithms. Promotional claims of spatial resolution improvements due to the use of iterative algorithms should be interpreted with care. They often refer to scenarios where sharper or high resolution reconstruction kernels are used without a noise penalty due to the superior noise suppression of iterative methods.

Table 3 compares our results with those published in other reports [30]. Differences in the scanning conditions between our study and other reports are also listed in Table 3. Given these differences, the comparison of results should proceed with caution and is qualitative at best. Mieville et al. [10] reported up to a 70% relative noise reduction for pediatric scans using VEO^™ with respect to FBP, at a lower dose of 0.5 mGy. The larger noise reduction is consistent with the observation of more aggressive noise reduction with VEO^™ at lower dose. Although the results Mieville et al. were presented in the context of pediatric scanning, they actually used a medium body x-ray filter. They also reported MTF improvements of 6% and 37% using ASIR100 and VEO^™ respectively, compared to FBP. It should be noted however that their measurements were obtained with thinner reconstruction slice thickness (1.25 mm).

For SAFIRE^™, our results are comparable to those of Ghetti et al. [30], who reported up to 35% and 59% noise reduction with SAFIRE-L3 and SAFIRE-L5 respectively. They also reported no change in the MTF between FBP and SAFIRE^™. For Toshiba’s AIDR3D^™, Gervaise et al. [32], reported less noise reduction for the earlier version of the Toshiba iterative algorithm (AIDR^™) using phantom scans. Joemai et al. [42] reported up to 65% noise reduction using AIDR3D over a similar dose range as used in our study. Both studies [32, 42] reported no spatial resolution improvements between FBP and AIDR^™ and AIDR3D^™.

For iDose^4™, our results are in agreement with the CT vendor’s claim of a noise reduction of 11% to 45% for iDose⁴-L1 to L6 [25]. Our MTF measurements were consistent with those reported by Mieville et al. [10], showing no difference between the MTF obtained with FBP and iDose^4™. The 50% MTF we obtained (0.29 mm^-1) is slightly lower than that of Mieville et al. [10] (0.33 mm^-1).

This study is limited in that it restricts the analysis to abdomen/thorax protocols at 120 kVp using the standard reconstruction kernel. Previous surveys of clinical protocols showed that these were the most common clinical parameters, in our jurisdiction and others [35, 43]. Also, only two pitch factors for each scanner were considered. It is conceivable that the performance of the different algorithms will vary with pitch and reconstruction kernel. We already alluded to the fact that the performance benefits of VEO^™ are enhanced when a high pitch is used, resulting in the reduction of the dose per slice. Different kernels will affect image noise and the visibility of fine details differently [37, 39] and may alter the relative performance obtained from using iterative algorithms. In addition, some studies have demonstrated that spatial resolution in the context of iterative algorithms can be dependent on subject contrast. In model based iterative algorithms, image noise depends on the regularization parameters which in turn are affected by local pixel gradients in the image. While our results demonstrate the relative performance of iterative algorithms, care must be exercised in generalizing these results to other clinical protocols or scanning conditions.

Another limitation of this study is in that it uses image quality metrics based on linear system theory and assumptions of linearity and stationarity. Iterative algorithms are nonlinear [44] and the geometric models are shift variant [40]. There have been suggestion to use local point-spread functions [44] or task-based MTF, evaluated under different noise and contrast conditions [45]. However, linear system-based methods are well established and several studies have used them [10, 46] and recent work has shown that the NPS of a model-based image reconstruction maintains radial symmetry [44], which justifies our use of ROI’s along a fixed radius from a small number of images. There has also been recent work on using phantoms with textured backgrounds for CT performance assessment, which was not considered in this work [47].

This study focused on objective image quality metrics obtained from images of a quality control phantom. This is useful in quantifying the performance of different algorithms, comparing them and verifying vendor claims. To facilitate comparisons between algorithms and vendor platforms, in Fig. 7 we show the noise reduction achieved by each algorithm over FBP at two fixed doses, for the lower pitch factors used in our study. We also show the dose at which each algorithm achieves image standard deviations of 5 and 10 in Fig. 8. The results, should, however be interpreted with care as they apply to phantom scans using specific scan parameters. Clinical adoption in particular, requires assessment using patients or realistic anthropomorphic phantoms. Further, clinical assessment looks at a broader range of issues related to image quality than QC phantom assessment. Recent work in our group, for example, showed that a model based algorithm yielded superior results to statistical or FBP reconstruction, when examined using a porcine model and a clinically relevant rubric [48]. The results of this work are a useful starting point for more clinically relevant assessments. They provide results from several vendors and can be useful in guiding the implementation of protocols across different platforms.

5 Conclusion

The results presented in this study characterized the performance of modern CT scanners equipped with commercially available iterative algorithms using objective image quality metrics. Several of the algorithms investigated (ASIR^™, SAFIRE^™ and iDose^4™) are effective noise reduction algorithms, not affecting spatial resolution or the Poisson noise-dose relationship significantly. Both the Toshiba AIDR3D^™ and GE VEO^™ algorithms alter the Poisson relationship, with the change more pronounced in the fully model-based VEO^™.

VEO^™ offers improvement in spatial resolution over the standard FBP reconstruction. It appears to be most effective when used in low dose scenarios. As dose increases, ASIR^™ results in CNR values equivalent or superior to those of VEO^™. VEO^™ requires long processing times and does not produce reconstructed images in real time.

Several studies [4, 49–51] have explored the clinical applications of iterative algorithms. While our study focused on an objective assessment using phantom images, it is useful in that it reports on the performance of algorithms from 4 different vendors. Further, the data for our study were obtained from the different vendors under similar acquisition conditions, which facilitate inter-system comparisons.