Receiver operating characteristic analysis for evaluating a proper experimental condition of eddy current tests under a low signal-to-noise ratio

Abstract

This study proposed a method to evaluate proper experimental conditions of eddy current tests under a low signal-to-noise ratio. This method was applied to quantify the detection reliability of the eddy current test to detect corrosion pits on the surface of austenitic stainless-steel weld cladding from the perspectives of probability of detection and probability of false alarm. The fluctuations of flaw signal and noise were described using truncated bivariate normal distributions of a two-dimensional vector composed of the real and imaginary components of a signal. The probabilities of detection and false alarms were formalized as cumulative distributions of the estimated distributions of the maximum flaw signal and the noise exceeding a decision threshold. The receiver operating characteristic curves were created by coupling the probabilities of detection and false alarms at various thresholds. The area under the curve was used to quantify the detection reliability, simultaneously considering the probabilities of detection and false alarms. The results obtained in this study support the applicability of this method in quantitatively evaluating the detection reliability of a non-destructive inspection under a low signal-to-noise ratio.

Keywords

Weld inspection pitting corrosion detection reliability evaluation non-destructive testing probability of detection probability of false alarm

1. Introduction

Non-destructive testing plays an important role in ensuring the integrities of structures. Knowing the presence of a harmful flaw by non-destructive testing is vital for discussing proper maintenance activities. However, non-destructive testing cannot always detect flaws, especially in a noisy condition. This indicates the importance of quantitatively assessing the reliability of a non-destructive testing method to detect a flaw.

Eddy current testing (ECT) is an electromagnetic non-destructive testing method suitable for the rapid inspection of a surface flaw even in the inspection of welds where large noise appears [1,2]. Whereas recent studies to evaluate the detection reliability of ECT basically focuses only on the probability of detection (POD) [3–6], it is also essential to evaluate the probability of false alarm/detection (POFA). However, how to consider the influence of noise on detection reliability quantitatively remains unclear. A potential method to consider POD and POFA simultaneously is based on the receiver operating characteristics (ROC), which has been applied to evaluate the detection reliability of ultrasonic testing [7].

Given the above background, this study developed a method quantifying the detection reliability of ECT in a noisy condition using ROC. In our previous studies, a POD model incorporating multiple signal features and flaw parameters was proposed [6]. To distinguish signals due to a flaw from noise more clearly, this study incorporated not only the amplitude but also the phase of eddy current signals. In addition, how to quantify POFA was also proposed. Surface flaws on stainless steel weld cladding were prepared and measured by ECT to simulate an inspection in a noisy condition [8]. Flaw detection reliability was evaluated based on the receiver operating characteristics curve and quantified using the area under the ROC curve.

2. Weld inspection using ECT

2.1. Weld cladding samples

To gather eddy current signals due to flaws in a noisy condition, this study used welded plate specimens fabricated in our previous study [7] shown in Fig. 1. The samples were fabricated by cladding the base material of a steel plate, ASTM A387 Gr22, with austenitic stainless steel of US-B309L through electro-slag welding. The thickness of cladding and bead width were about 5 mm and 50–60 mm, respectively. The cladding surface was ground, and the resultant surface roughness Rz (JIS2001) was about 4.7–5.6 μm. To simulate the flaws with irregular shapes in practical inspection, artificial corrosion pits were introduced to the center of each weld bead in width by the corrosion test, using a ferric chloride-based etchant (37–39 wt.% ferric chloride, H-1000A; Sunhayato, Tokyo, Japan). The total number of corrosion pits was 47. The profiles of the corrosion pits were measured using a 3D shape measuring machine (VR-5000; KEYENCE Co. LTD., Japan). The size information of all the corrosion pits is described in Fig. 2. In addition, flawless weld cladding samples were prepared to gather the noise generated by the weld structure.

Since the uneven electromagnetic properties, and surface undulation of weld structure can cause large noise in ECT signal. Detecting flaws on the weld cladding using ECT is able to simulate an inspection in a noisy condition.

Fig. 1.

Photographs of the welded sample.

Fig. 2.

Size of corrosion pits.

2.2. Welds inspection using ECT

2.2.1. Experimental set-up

The weld inspection was conducted using two probes, as shown in Fig. 3. Each probe was fixed at an X–Y stage, which controlled the movement of the sample, and was energized by the ECT instrument of aect-2000N (Aswan ECT Co. Ltd., Osaka, Japan) with a 300 kHz alternating current. Generated eddy current was parallel with the weld path. The lift-off was 1 mm. The 25 mm² × 25 mm² square areas centered on the corrosion pits and the same size flawless areas were measured. The experimental configuration is shown in Fig. 4. Because the ECT signals suffer from the “end effect” due to the sample edges and the joint of two beads, only the signals within the range of 14 mm × 14 mm square were extracted. The signal with the largest amplitude was regarded as the signal from a pit.

Fig. 3.

Illustration of ECT probes.

Fig. 4.

Experimental configuration.

2.2.2. Discussion about measured flaw signal

The relationships between signals measured using Probe 1 and flaw parameters are qualitatively summarized in Fig. 5. The signal features considered here were the real and imaginary parts of signal amplitude, amplitude, and phase angle, which are labeled as “Real”, “Imag”, “Amp”, and “Phase”, respectively in the figure. The flaw parameters used to characterize a pit are the surface area, depth, and volume of a pit. The figure reveals the difficulty in representing a flaw parameter using a single signal feature, and the relationship between phase angle and flaw parameters also indicates it is difficult to express phase angle with flaw parameters.

Fig. 5.

Relationships between signal features and flaw parameters.

2.2.3. Discussion about measured noise

The noise of the flawless area measured by Probe 1 is shown in Fig. 6. According to the noise distribution, there is a difference in the deviation between phase angles 1 and 2. Simply using a distribution of a single variable shown in Fig. 7, a signal feature of amplitude, cannot reasonably describe the realistic distribution because the phase information of noise cannot be included.

Fig. 6.

Difference in deviation between phase angles 1 and 2 of noise distribution.

Fig. 7.

Unidimensional distribution of noise.

Fig. 8.

Estimated regression equations when using Probe 1.

Fig. 9.

Estimated regression equations when using Probe 2.

3. Detection reliability evaluation

3.1. Method

To consider the phase of ECT signal, bivariate normal distribution of the real and imaginary parts of the signal was built to describe the distribution of flaw signal and noise to determine POD and POFA for the ROC analysis of ECT.

3.1.1. Modeling of the measured flaw signal and noise

This study describes the fluctuations of the maximum flaw signal and noise, $\hat{\boldsymbol{a}}$ , using a bivariate normal distribution N₂: $\begin{eqnarray}\hat{\boldsymbol{a}}\sim N_{2}(\boldsymbol{{\mu}},\boldsymbol{{\Sigma}}),\end{eqnarray}$ (1) where μ is a vector of mean value consisting of the real and the imaginary parts of signal amplitude, $\hat{a}_{1}$ and $\hat{a}_{2}$ , 𝛴 is the covariance matrix. They can be decided through maximum likelihood estimation. For the probability density function of the maximum flaw signal $f(\hat{\mathbf{a}}|a)$ , this study used the following two regression equations: $\begin{eqnarray}\displaystyle \text{} & \text{} & \displaystyle \hat{a}_{1}(a)=C_{1}\ln (a)+C_{2}\end{eqnarray}$ (2) $\begin{eqnarray}\displaystyle \text{} & \text{} & \displaystyle \hat{a}_{2}(a)=C_{3}\ln (a)+C_{4},\end{eqnarray}$ (3) where a denote the flaw size, and C₁, C₂, C₃ and C₄ are coefficients. Three kinds of flaw size a, the maximum depth, the surface area, and the volume, were used to express the regression equations. To decide the most reliable regression, we reasonably describe the relationship between the maximum flaw signal $\hat{\mathbf{a}}$ and the logarithmically transformed flaw size a. The error between the regression equations and the measured flaw signal was evaluated using the coefficient of determination R². Sometimes, the flaw signal is submerged by noise or above some saturation value so that it cannot be correctly recorded. In this case, it is not reasonable to compute the error between the observation and the regression [9]. Such data were censored by the visual check.

The probability density function of noise $g(\hat{\boldsymbol{a}})$ was obtained by fitting the measured noise. Due to the symmetry of the ECT signal measured by the abovementioned probes, both $f(\hat{\boldsymbol{a}}|a)$ and $g(\hat{\boldsymbol{a}})$ were truncated into the first quadrant: $\begin{eqnarray}\displaystyle f_{tr}\text{} & \text{} & \displaystyle (\hat{\boldsymbol{a}}|a)=\frac{f(\hat{\boldsymbol{a}}|a)}{\displaystyle \int _{0}^{\infty }\int _{0}^{\infty }f(\hat{\boldsymbol{a}}|a)∼\text{d}\hat{a}_{1}∼\text{d}\hat{a}_{2}}\end{eqnarray}$ (4) $\begin{eqnarray}\displaystyle g_{tr}\text{} & \text{} & \displaystyle (\hat{\boldsymbol{a}})=\frac{g(\hat{\boldsymbol{a}})}{\displaystyle \int _{0}^{\infty }\int _{0}^{\infty }g(\hat{\boldsymbol{a}})∼\text{d}\hat{a}_{1}∼\text{d}\hat{a}_{2}}.\end{eqnarray}$ (5)

3.1.2. ROC analysis

Once the truncated probability density functions of the maximum flaw signal $f_{tr}(\hat{\boldsymbol{a}}|a)$ and noise $g_{tr}(\hat{\boldsymbol{a}})$ are given, considering the decision threshold $\hat{a}_{th}$ a variable of signal amplitude so that it can be presented as a quarter of a circle whose center is the origin on the complex plane, both the POD and the POFA have the following forms: $\begin{eqnarray}\displaystyle \text{} & \text{} & \displaystyle POD(\hat{a}_{th},a)=\int _{\hat{a}_{th}}^{\infty }\int _{0}^{\frac{{\pi}}{4}}f_{tr}(\hat{\boldsymbol{a}}|a)r∼\text{d}{\theta}∼\text{d}r\end{eqnarray}$ (6) $\begin{eqnarray}\displaystyle \text{} & \text{} & \displaystyle PFA(\hat{a}_{th})=\int _{\hat{a}_{th}}^{\infty }\int _{0}^{\frac{{\pi}}{4}}g_{tr}(\hat{\boldsymbol{a}})r∼\text{d}{\theta}∼\text{d}r,\end{eqnarray}$ (7) where 𝛾 and θ denote the amplitude and the phase of the signal on the complex plane. For a given threshold, the ROC is given by coupling the POD and the POFA. Considering the decision threshold $\hat{a}_{th}$ in Eqs. (6) and (7), a continuous variable ranging from zero to infinity, the POD and POFA couples given by a series of flaw sizes were documented in a curve called the ROC curve [10]. The area under the ROC curve (AUC) can be used as an effective index to conclude the reliability, including a simultaneous consideration of the POD and the POFA [11], and calculated through numerical integration using MATLAB. The 95% confidence bounds were calculated to evaluate the uncertainties of the ROC and AUC curves using the percentile bootstrap method [12]. The number of bootstrap samples was 200. The 95% confidence interval is described as twice the distance between the evaluated curve and the 95% lower confidence bound (LCB).

3.2. Results

3.2.1. Results of modeling of measured flaw signal and noise

Table 1 lists the calculated R² when relating the maximum flaw signal to different flaw size a. The result indicates that relating to the pit volume provides the most reliable regression, since the logarithmically transformed pit volume provides the highest R² in either Probe 1 or 2. The regression equations $\hat{a}_{1}(a)$ $\hat{a}_{2}(a)$ expressed by the pit volume, and the estimated truncated probability density functions of the maximum flaw signal $f_{tr}(\hat{\boldsymbol{a}}|a)$ given by a specific size when using the two probes, are shown in Figs 8–9. The flaw signals submerged by noise were plotted as crosses. The 95% predicted intervals were given by adding and subtracting the mean function with 1.96 times the standard deviation. The estimated truncated probability density functions of noise $g_{tr}(\hat{\boldsymbol{a}})$ are also shown as the contours in Fig. 10 and can reasonably describe the measured noise.

Fig. 10.

Estimated probability density function of noise.

Table 1

Calculated R² when using different types of flaw parameters a

	Probe 1		Probe 2
Flaw size a	Real part	Imaginary part	Real part	Imaginary part
Depth (mm)	0.646	0.635	0.722	0.714
Surface area (mm²)	0.697	0.761	0.769	0.683
Volume (mm³)	0.716	0.762	0.799	0.725

Fig. 11.

ROC curves at the volume of 0.16 mm³.

Fig. 12.

AUC curves.

3.2.2. Results of the ROC analysis

Figure 11 presents the ROC curves corresponding to a 0.16 mm³ pit volume estimated based on the experimental data collected by Probes 1 and 2. This result reveals that Probe 2 of smaller size reduces the POFA by more than 20% while maintaining the same POD as Probe 1. It also has higher reliability than probe 1, since the AUC curve of Probe 2 is located above the curve of Probe 1 in Fig. 12. Smaller confidence intervals of the ROC and the AUC curves of Probe 2 also suggest higher reliability in the evaluation result of Probe 2.

4. Conclusion

This study proposed a method for modeling the realistic distribution of the maximum flaw signal and the noise measured by ECT through truncated bivariate normal distributions. The POD and the POFA were formalized with two signal features: amplitude and phase. The proposed method was applied to quantify the flaw detection reliability of two ECT probes having a difference in size used in weld inspection.

The results reveal that the proposed method was able to model the realistic distributions of the maximum flaw signal and the noise reasonably. The applicability of the proposed method to the quantitative reliability evaluation of ECT under a low signal-to-noise ratio was confirmed.

References

Nadzri

N.A.

Ishak

Saari

M.M.

and Halil

A.M.

, Development of eddy current testing system for welding inspection, in: 2018 9th IEEE Control and System Graduate Research Colloquium, pp. 94–98.

Camerini

Rebello

J.M.A.

Braga

Santos

Chady

Psuj

and Pereira

, In-line inspection tool with eddy current instrumentation for fatigue crack detection, Sensors18(7) (2018), 2161.

Keprate

and Ratnayake

R.M.C.

, Probability of detection as a metric for quantifying NDE capability: The state of the art, J. Pipeline Eng.14(3) (2015), 199–209.

Mohseni

Habibzadeh Boukani

Ramos França

and Viens

, A study of the automated eddy current detection of cracks in steel plates, Journal of Nondestructive Evaluation39(6) (2020), 1–12.

Tomizawa

Song

and Yusa

, Probabilistic evaluation of detection capability of eddy current test to inspect pitting on a stainless steel clad using multiple signal features, International Journal of Applied Electromagnetics and Mechanics64 (2020), 47–55.

Tomizawa

Song

Yusa

and Yuya

, Probability of detection model based on hit/miss data considering multiple signal features and multiple flaw parameters from eddy current test suffering low signal to noise ratio, Nondestructive Testing and Diagnostics1-2 (2021), 13–18.

Schoefs

Clément

and Nouy

, Assessment of ROC curves for inspection of random fields, Structural Safety31(5) (2009), 409–419.

Yusa

Hashizume

Uchimoto

Takagi

and Sato

, Evaluation of the electromagnetic characteristics of type 316L stainless steel welds from the viewpoint of eddy current inspections, Journal of Nuclear Science and Technology51(1) (2014), 127–132.

Annis

Aldrin

J.C.

and Sabbagh

H.A.

, What is missing in nondestructive testing capability evaluation?Materials Evaluation73(1) (2015), 44–54.

10.

Ahmad

and Bond

L.J.

, Reliability of flaw detection by nondestructive inspection, ASM Handbook17 (2018), 23–46.

11.

Bismut

and Straub

, A unifying review of NDE models towards optimal decision support, Structural Safety97 (2022), 102213.

12.

Knopp

J.S.

Grandhi

and Aldrin

J.C.

, Statistical analysis of eddy current data from fastener site inspections, Journal of Nondestructive Evaluation32 (2013), 44–50.