Validation of radiographic quality of simulation software

Abstract

BACKGROUND:

Virtual radiographic simulation has been found educationally effective for students to practice their clinical examinations remotely or online. A free available virtual simulator-ImaSim has received particular attention for radiographic science education because of its portability, free of charge and no constrain of location and physical facility. However, it lacks evidence to validate this virtual simulation software to faithfully reproduce radiographs comparable to that taken from a real X-ray machine to date.

OBJECTIVE:

To evaluate image quality of the virtual radiographs produced by the ImaSim. Thus, the deployment of this radiographic simulation software for teaching and experimental studying of radiography can be justified.

METHODS:

A real medical X-ray examination machine is employed to scan three standard QC phantoms to produce radiographs for comparing to the corresponding virtual radiographs generated by ImaSim software. The high and low range of radiographic contrast and comprehensive contrast-detail performance are considered to characterize the radiographic quality of the virtual simulation software.

RESULTS:

ImaSim software can generate radiographs with a contrast ranging from 30% to 0.8% and a spatial resolution as low as 0.6mm under the selected exposure setting condition. The characteristics of contrast and spatial resolution of virtual simulation generally agree with that of real medical X-ray examination machine.

CONCLUSION:

ImaSim software can be used to simulate a radiographic imaging process to generate radiographs with contrast and detail detectability comparable to those produced by a real X-ray imaging machine. Therefore, it can be adopted as a flexible educational tool for proof of concept and experimental design in radiography.

Keywords

Virtual simulation density contrast image quality

1 Introduction

Virtual simulation makes use of computer software program to model and visualize the physics of X-ray radiation and a virtual projection across the examined objects onto a detector. It has been found useful and educationally effective for students to practice their radiographic examinations in a simulated environment prior to contacting real patients [1]. By using virtual simulation the students can gain more hands-on experience without worrying about receiving any unnecessary radiation [2]. Moreover virtual simulation enables learning and teaching remotely or online, which has received particular attention during the spread of the COVID-19 pandemic and makes it become a global trend in the high education of medical imaging [3, 4].

Several simulation packages have being developed and become commercially available for students to improve their understanding and practice their skills during studying for radiography program. For example, the ProjectionVR distributed in various institutes or universities across several countries uses three-dimensional digital radiographic reconstruction to simulate the projection of X-ray radiography and allow learners to repeatedly practice common projections and positioning operations for the X-ray radiographic examination [5]. The SIMTICS is another similar product designed for students to practice various radiographic positioning and all other exposure settings on a cloud platform; however the effectiveness of the user interface and the limited number of cases for study are the major concerns for its wide deployment [6]. The CETSOL VR Clinic software developed for the Medical Imaging program is featured with 3D hand gesture interaction and positional stereoscopic tracking facilities. Students preferred this type of augmented patient interaction and communication during learning the general procedural workflow in hand X-ray examination. The simulation package needs to be upgraded to comprise more models for examining other anatomies [7]. Different from these commercial products, ImaSim, radiographic simulation software developed by Landry for his master degree, is free available and able to run on ordinary personal computers without any expensive requirement on the support from peripheral high-performance computing hardware. Meanwhile it allows users to customize the exposure parameters and create the phantoms or objects according to the purpose of their study, which makes it possible investigate the principles of radiography as well as different physical factors influencing radiographic image quality. Therefore it is very suitable for individuals to study independently as well as practice their skills at their own pace for their radiographic science education [8].

Although the above virtual simulation packages have been found helpful in reinforcing the learning of radiographic conception and the practicing of virtual examination, there is a lack of work to verify them in producing virtual radiographs with satisfactory radiographic quality for education or research purposes [9]. Early Tore verified the accuracy of perspective projection of dental radiographic simulation system developed by them. By comparing the theoretical distances of two artificially modeled dots and that measured from its virtual projection radiograph, the geometric accuracy of the simulation system was found within 0.5mm [10]. Cosson et al. verified the geometric accuracy of 3D digital radiographic reconstruction of the ProjectionVR through comparing to actual radiographic results of an anthropomorphic skull phantom and observed the mean error within±1.5 mm [11].

The above work focuses on the validation of geometric accuracy of virtual radiographs, probably because the distance is relatively easy to measure and quantify. Actually, the overall radiographic quality of an X-ray imaging system is determined by geometric as well as density characteristic. The density representing the amount of blackening of the radiographic images is fundamental to form the contrast and visibility of anatomic details under the investigation [12]. Because all this information is very important for final radiographic interpretation and disease diagnosis, it’s necessary to quantify the characteristic of density of virtual radiographic simulation systems to justify their deployment for educational or research purposes.

This study validates the radiographic quality mainly on the density characteristic of the ImaSim virtual simulation software. Three commonly used QC phantoms are taken as standards to build virtual models and radiographs via the ImaSim. The virtual radiographs are compared to those digital radiographs produced by a real medical X-ray examination machine to verify the faithfulness of the virtual simulation software for reconstructing radiographic images.

2 Materials and methods

2.1 Virtual simulation software-ImaSim

ImaSim is highly interactive software developed to simulate an X-ray imaging process through a series of modules responsible to model the X-ray radiation resource, object being examined, projection geometry, flat panel detector and image post-processing tools respectively. As shown in Fig. 1, users can flexibly parameterize each module and finally generate the photographic images. At present, the software contains three modes including planar projection imaging, computed tomography and cone beam computed tomography. However, this study only focuses on the planar projection imaging, which is the most commonly used mode in X-ray examinations [13].

Fig. 1

Virtual radiograph generated via setting each module of an imaging process.

2.2 Design experiments

As it is not realistic to keep all imaging parameters exactly same with every imaging process, we do not use the quantity of density directly, but compare the contrast of virtual radiographs to that of real digital radiographs. Three QC phantoms including an aluminum step wedge, DIGI-13 and CDRAD 2.0 low-contrast detail phantom are used as standards to characterize the contrast from high to low level. The step wedge manufactured with high-purity aluminum has a size of 50 mm×132 mm×35 mm and each of 11 steps has a height difference of 3 mm. The aluminum step wedge is mainly used to evaluate the high contrast characteristic of the imaging systems. The DIGI-13 is designed for testing several radiographic qualities including geometrical and contrast resolution. According to the DIN 6868-13 standard, six aluminum disks with different thickness sitting on the copper substrate has a contrast of 0.8%, 1.2%, 2%, 2.8%, 4% and 5.6% respectively under 70 kV exposure conditions [14]. The CDRAD 2.0 phantom is a 265 mm×265 mm×10 mm plexiglass plate distributing 225 cylindrical holes of various depths and diameters manufactured with tolerances of 0.02 mm. It is taken as a comprehensive test target to evaluate both detail and low contrast characteristics of diagnostic imaging systems. Figure 2 shows above mentioned physical phantoms and corresponding virtual phantoms built via the ImaSim.

Fig. 2

Aluminum step wedge, DIGI-13 and CDRAD 2.0 phantoms (left) and 3D models built in the virtual simulation software ImaSim (right).

In order to validate radiographic quality of the simulation software, a reliable medical X-ray examination machine, Siemens Multix Select DR system, was selected to scan three phantoms to produce real digital radiographs. In order to make equivalent comparison, the SID is set to 150 cm throughout for both the simulation software and Multix Select DR system and the radiation field covers the whole phantom. The ImaSim was parameterized with total 3.5 mm Al filtration, 12° anode angle and using the amorphous silicon detector. Meanwhile 70 kV, 90 kV and 125 kV were used as three representative tube voltages to investigate the influence of the X-ray energy on radiographic contrast. The quantity of X-ray was set to ensure the thinnest part of phantoms visible.

2.3 Assessment criteria

Rather than to validate the characteristic of density directly, radiographic contrast, i.e. density difference of the adjacently interested areas, is mainly used to assess radiographic quality of the simulation software. As shown in Fig. 3, the average density of two aluminum steps are $I_{0}^{1}$ and $I_{0}^{2}$ , and their contrast can be expressed as ( $I_{0}^{1} - I_{0}^{2}$ )/ $I_{0}^{1}$ . Assuming that the incident X-rays can be simplified as equivalent single-energy rays I_i and follow the Beer-Lambert exponential attenuation law when they pass through the phantoms [15], the radiographic contrast can be further simplified from (I_ie^-μd₁ - I_ie^-μd₂)/(I_ie^-μd₁) to 1 - e^{-μ(d₂-d₁)}, where μ is the linear attenuation coefficient of the X-ray with a specific photon energy penetrating through the phantoms [16]. It can be found that radiographic contrast is inherently determined by the thickness and the material of the object being examined but insensitive to the quantity of X-ray sources and therefore suitable to be used for verifying the imaging performance of radiographic systems.

Fig. 3

X-rays penetrated through two adjacent aluminum steps (a) and resultant density map (b).

More specifically each step of the aluminum step wedge has same thickness difference of 3 mm from its neighbor step and its radiographic contrast should keep constant under certain X-ray energy. Whereas six aluminum circular disks sitting on the copper substrate of the DIGI-13 phantom have different thickness and their radiographic contrasts are obviously determined by the relative thickness of the aluminum disks and copper. The CDRAD 2.0 phantom containing holes with different diameter and depth allows verification of the detectability of the density contrast as well as geometric details of the imaging systems via the contrast detail curves of the radiographic images or the inverse value IQFinv of the image quality factor [17].

3 Results

3.1 Experimental results of aluminum step wedge

Figure 4 lists two columns of radiographic images of the aluminum step wedge generated by the virtual simulation software ImaSim and the actual X-ray imaging machine. The results demonstrate that the images of aluminum step wedges generated from the virtual simulation are clearly comparable to that from the actual X-ray imaging system. Meanwhile, the resultant images from both systems are able to reflect the influence of the energy of X-ray radiation on radiographic contrast, i.e. the contrast tends to be lower at higher tube voltages, and the number of observable aluminum steps increases as the tube voltage increases. However the noticeable quantum noise associated with the real radiographs disappeared in the simulated radiographs in the left column of the Fig. 4.

Fig. 4

Radiographs of aluminum step wedge produced by the simulation software (left) and real X-ray system (right) under three different X-ray energies.

In order to further quantify and analyze the contrast of the images produced by the two systems, the image processing method is used to obtain the average density of each aluminum step. Figure 5 shows the curves of normalized density and calculated contrast of aluminum steps under different exposure tube voltages. The curves of density of both virtual simulation and real radiographic systems in Fig. 5(a) and Fig. 5(b) look similar and exhibit the form of exponential attenuation. The curves of contrast shown in Fig. 5(c) and (d) are nearly flat with a slight downward as the thickness of aluminum steps increases. While the general contrast decreases as the tube voltages applied increases, i.e. the contrast is between 25% –30% under 70 kV and reduces to 18% –20% at 125 kV. There is a sharp drop at the thickest part of aluminum steps (29 mm∼32 mm) in the real radiographic imaging system.

Fig. 5

Image density (top) and contrast (bottom) of each aluminum step generated from the virtual simulation software (left) and real X-ray machine (right) under three types of high voltages.

3.2 Experimental results on DIGI-13 phantom

Figure 6 (a) and (b) are the radiographs of the low-contrast aluminum disks of the DIGI-13 phantom generated by ImaSim virtual simulation software and actual X-ray imaging machine at 70 kV. The density map produced by the actual X-ray radiographic system looks darker than that from ImaSim simulation, and the visibility of the thinnest aluminum disk obtained by both methods is poor. Figure 6(c) shows the calculated contrast of aluminum disks based on the density of the aluminum disks and its surrounding regions in two radiographs, and the standard contrast provided in the DIN 6868-13 standard as a reference for comparison.

Fig. 6

Radiographs and radiographic contrast of six aluminum disks of DIGI-13 phantom obtained from ImaSim, the X-ray machine and the DIN 6868-1 3 standard.

3.3 Experimental results on CDRAD 2.0 phantom

Figure 7 shows the radiographs of low-contrast detail phantom CDRAD 2.0 obtained by the simulation software and the actual X-ray machine system at 70kV, and their image contrast detail curves and the inverse image quality factors automatically generated by the analysis software accompanied with the phantom. The calibration artifacts of flat-panel detector appear indistinctly in the background of real radiograph, but they are generally not seen on clinical images and didn’t affect the following assessment of both the image contrast detail curves and the inverse image quality factors either.

Fig. 7

Radiographs (top) and contrast-detail curves (bottom) from simulation software (left) and actual DR machine (right).

4 Discussions

Generally X-ray beam interacting with objects is attenuated by the photoelectric effect and Compton scattering at the energy of diagnostic range. Photoelectric effect is the dominant mechanism at lower energies, and the scattering effect gradually plays a major role as the energy of X-ray beam increases [18]. This causes the beam of X-ray attenuated more and produce higher radiographic contrast at the lower energy as demonstrated in Fig. 5. Meanwhile the part of thicker steps presents lower contrast compared to that of thinner part as the X-ray beam will become harder with higher average energy when it penetrates deeper into the aluminum object.

The ImaSim employs the ray tracing technology to simulate the interaction between photons and matters in order to achieve efficient calculation. It takes more consideration on the attenuation caused by photoelectric effects but ignores part of the scattering effect. So the curves of radiographic contrast are presented with good separation in Fig. 5(c), whereas the curves of contrast produced by the actual X-ray system become merged especially under the tube voltages of 90 kV and 125 kV. The sharp drop of the contrast at the thickest part of the aluminum step in Fig. 5(d) may be caused by the perspective projection of the beam which needs to travel longer path before reaching the detector. The ImaSim simulation hasn’t reproduced this phenomenon possibly because it hasn’t considered these factors during reconstructing radiographs.

Comparing to the aluminum step wedge, the DIGI-13 is used to test the performance of imaging systems in producing the medium and low contrast. It can be observed from Fig. 6(c) that the low contrast produced by the simulation software is much closer to the standard values suggested by the DIN 6868-13, while the radiographic contrast of the image produced by the actual X-ray machine is a little lower than the standard values. Meanwhile the contrast produced by the actual X-ray machine deviates more from the standard values as the thickness of the aluminum disks increases.

As for the low-contrast detail CDRAD 2.0 phantom, the virtual radiograph appears sharper with more discernible geometric details as shown in Fig. 7(a). But the results of automatic analysis show that the radiographs from the actual X-ray machine have better overall quality factor. By observing the contrast detail curves, it can be found that the minimum distinguishable detail at the radiographs produced by the virtual simulation and real X-ray systems are close to 0.6 mm, but the minimum depth of hole identifiable of the simulation software and the actual system is 6.3 mm and 2 mm respectively. This difference in distinguishing low-contrast details should come from the technology of ray tracing which may have difficult to trace the rays for those holes with small diameter and shallow depth.

5 Conclusions

ImaSim, free available and flexible virtual radiographic simulation software, has been validated by modeling and generating radiographs of three standard QC phantoms and comparing them to those scanned by one actual medical X-ray examination machine. The experimental results demonstrated that the ImaSim can faithfully simulate the radiographic imaging process through several logically arranged functioning modules, and its radiographic quality including contrast and spatial resolution agrees with that of actual system reasonably well. Meanwhile we found that the virtual simulation hadn’t reproduced all the possible factors within a real radiographic imaging process. Therefore, it is necessary to fully understand these features when this simulation software will be promoted for teaching or research in radiography.

In this study, experiments have been designed mainly for validating the fidelity of the radiographic quality provided by the ImaSim software. To complete the validation, the stability of the simulation software is another important factor to be examined during its deployment in the real educational circumstances. In our next work we will evaluate its stability through investigating the student experience and perception on the software when building their knowledge and confidence in clinical skills.

References

Vestbostad

, Karlgren

and Olsen

N.R.

, Research on simulation in radiography education: a scoping review protocol, Systematic Reviews 9(1) (2020), 263–269.

Kong

, Hodgson

and Druva

, The role of simulation in developing clinical knowledge and increasing clinical confidence in first-year radiography students, Focus on Health Professional Education 16(3) (2015), 29–44.

Alvin

M.D.

, George

, Deng

et al., The impact of COVID-19 on radiology trainees, Radiology 296(2) (2020), 246–248.

Gill

, Whitehead

and Wondimagegn

, Challenges to medical education at a time of physical distancing, The Lancet 396(10244) (2020), 77–79.

Shanahan

, Student perspective on using a virtual radiography simulation, Radiography 22(3) (2016), 217–222.

Wertz

, Wagner

, Ward

et al., Using simulation in radiographic science education, Louisiana State University: IGI Global (2019), 37–67.

Sapkaroski

, Baird

, McInerney

et al., The implementation of a haptic feedback virtual reality simulation clinic with dynamic patient interaction and communication for medical imaging students, Journal of Medical Radiation Sciences 65(3) (2018), 218–225.

Landry

, Deblois

and Verhaegen

, ImaSim, a software tool for basic education of medical x-ray imaging in radiotherapy and radiology, Frontiers in Physics 1(22) (2013), 1–7.

Shiner

, Is there a role for simulation based education within conventional diagnostic radiography? A literature review, Radiography 24(3) (2018), 262–271.

10.

Nilsson

, Ahlqvist

, Johansson

et al., Virtual reality for simulation of radiographic projections: validation of projection geometry, Dento Maxillo Facial Radiology 33(1) (2004), 44–50.

11.

Cosson

and Lu

, Geometric validation of a computer simulator used in radiography education, Bjr|Open 2(1) (2020), 20190022–20190027.

12.

Roding

S.T.

, Benwell

G.M.

, Bijwaard

et al., Digital imaging and radiographic practise in diagnostic radiography: An overview of current knowledge and practice in Europe, Radiography 24(2) (2018), 137–141.

13.

Landry

, ImaSim, a simulation software package for the teaching of medical X-Ray imaging. medical physics unit, Paris: McGill University, 2009.

14.

Sshreiner-Karoussou

, Review of image quality standards to control digital X-ray systems, Radiation Protection Dosimetry 117(1-3) (2006), 23–25.

15.

Hoheisel

, Bernhardt

, Lawaczeck

et al., Comparison of polychromatic and monochromatic x-rays for imaging, Proc SPIE 6142 (2006), 71–78.

16.

Dance

D.R.

, Christofides

, Maidment

A.A.

et al., Diagnostic radiology physics: a handbook for teachers and students, Vienna, Austria: International Atomic Energy Agency, 2014.

17.

Choopani

M.R.

and Chaparian

, Introduction of a new parameter for evaluation of digital radiography system performance, Journal of Medical Signals and Sensors 10(3) (2020), 196–200.

18.

Fredenberg

, Spectral and dual-energy X-ray imaging for medical applications, Nuclear Instruments and Methods in Physics Research Section a: Accelerators, Spectrometers, Detectors and Associated Equipment 878(11) (2018), 74–87.

Validation of radiographic quality of simulation software - ImaSim