Automatic segmentation of medical images using a novel Harris Hawk optimization method and an active contour model

Abstract

Medical image segmentation is a key step to assist diagnosis of several diseases, and accuracy of a segmentation method is important for further treatments of different diseases. Different medical imaging modalities have different challenges such as intensity inhomogeneity, noise, low contrast, and ill-defined boundaries, which make automated segmentation a difficult task. To handle these issues, we propose a new fully automated method for medical image segmentation, which utilizes the advantages of thresholding and an active contour model. In this study, a Harris Hawks optimizer is applied to determine the optimal thresholding value, which is used to obtain the initial contour for segmentation. The obtained contour is further refined by using a spatially varying Gaussian kernel in the active contour model. The proposed method is then validated using a standard skin dataset (ISBI 2016), which consists of variable-sized lesions and different challenging artifacts, and a standard cardiac magnetic resonance dataset (ACDC, MICCAI 2017) with a wide spectrum of normal hearts, congenital heart diseases, and cardiac dysfunction. Experimental results show that the proposed method can effectively segment the region of interest and produce superior segmentation results for skin (overall Dice Score 0.90) and cardiac dataset (overall Dice Score 0.93), as compared to other state-of-the-art algorithms.

Keywords

Image segmentation swarm optimization algorithm active contour model skin lesion segmentation and machine learning

1 Introduction

Medical image segmentation delineates the lesions or organs from the background images like in Magnetic Resonance Imaging (MRIs), Computed Tomography (CT) scans, Ultrasounds. In different medical imaging modalities, the segmentation results are highly affected due to the presence of different artifacts such as intensity inhomogeneity, noise, low contrast, ill-defined boundaries, and many more. These challenging artifacts make the image segmentation task very difficult. Segmentation accuracy highly affects the subsequent medical treatment and diagnosis [1]. State-of-the-art techniques for medical image segmentation involve manual segmentation, which is performed by relevant experts and it is a cumbersome and extremely time taking process [2]. The manual segmentation of medical images also suffers from inter-observer or intra-observer variability [3].

Different medical imaging modalities (such as Cardiac Magnetic Resonance Imaging (CMR), Ultrasound, X-rays, CT scans, Positron Emission Tomography (PET), Skin lesion images) are suffered from different challenges. Some problems are common in all medical imaging; however, few are specific for some imaging types, for example in skin lesion segmentation, the main challenges are irregular and fuzzy lesion borders, intrinsic skin-related features such as blood vessels, skin lines, hairs, and distortion [4]. Similarly, for Left Ventricle (LV) segmentation in CMR, low contrast and inhomogeneity are present but some specific challenges of LV segmentation are variability in types of slices, different phases of cardiac cycles [5].

In recent years, several full/semi-automated segmentation methods for medical images have been published. Though, there is no single algorithm that is suitable for different types of medical images. This is due to the variations in anatomic structures, and each structure has its own specific challenges/artifacts. Some automated segmentation methods are proposed for a specific imaging modality and may be it would be more accurate for one type and less accurate for the other imaging type [1, 6].

Over the past decades, many supervised and unsupervised methods have been proposed for medical image segmentation. Supervised segmentation methods such as the multi-atlas method [7], convolutional neural network [8, 9], random forest [10], and many more are used for the segmentation, however, they generally require large amounts of annotated datasets, which is a very time-consuming procedure [8 , 12]. These methods require several stages of intensity standardization and normalization. In addition, these models are very sensitive to the variations in the scanner, acquisition protocols, and optimization of a large number of hyper-parameters. Recently, deep learning networks have achieved great attention of researchers due to significantly improved segmentation results, but they usually require huge datasets to be annotated, which is a tedious and expensive procedure in medical images. However, data augmentation and transfer learning have been used to overcome the requirement of large annotated datasets but they have their own issues and problems and they are also suffered from the overfitting problem [12]. On the other hand, unsupervised learning algorithms do not require labeled datasets for medical segmentation; these methods use image-guided features to delineate Region of Interest (ROI). These characteristics make unsupervised algorithms more robust and applicable to the medical image segmentation [13]. Some common models of unsupervised learning include K-means clustering, self-organizing maps and multi-thresholding [14, 15] to name a few.

In last few years, Thresholding for image segmentation is very common and has attracted the attention of many researchers in the last decade [6, 14]. Thresholding is good for segmentation when image has distinct homogeneous regions. To date, many researchers have proposed different methods for optimal gray-level threshold such as using Kapur and Otsu but they only work well in the case of the bi-level thresholding method. Bi-level thresholding is useful if ROI is clearly distinguished from the remaining image using a single threshold value [17], which is generally not true in the case of medical imaging modalities due to the presence of noise, low contrast, and other artifacts problems. Multilevel thresholding techniques are more accurate than the traditional bi-level thresholding for the medical image segmentation because a diverse number of intensities are used to represent distinct regions in the image. For medical image segmentation, thresholding is mostly used as the first stage, which is followed by a series of image processing steps that make it computationally expensive [14]. Many algorithms have been proposed to get the optimal thresholding value for image segmentation [14 –18]. Although an exhaustive search is required for finding the optimal thresholds, which results in the application of swarm intelligence and evolutionary algorithms [14, 16]. Many researchers have used different entropies based multilevel thresholding (like Shannon [16], fuzzy [14], Tsallis [19]) using evolutionary algorithms and nature-inspired algorithms for the image segmentation [18].

The key limitation of the thresholding method is that it does not consider the spatial characteristics of the image, which makes the segmentation results very sensitive to intensity inhomogeneity and noise. ACMs are widely used methods for medical image segmentation being robust to intensity inhomogeneity and low contrast artifacts [5 , 21]. ACMs can be categorized into edge-based [22], and region-based methods [23, 24]. Edge-based models utilize the gradient to stop the evolving segmenting curve near the ROI’s boundary. Though these techniques produce poor segmentation results for regions with weak boundaries and they are sensitive to noise in the gradient estimates [22]. Region-based models separate the image into the foreground and background, based on local/global intensity-guided features [24, 25]. These models [23 –26] are more robust to noise as compared to edge-based methods. Local intensity information is very useful in handling the intensity inhomogeneity and noise problems but they are very sensitive to the contour initialization and poor initialization can easily trap into a local optimum [23 , 26].

For medical image segmentation, many algorithms based on ACMs have been proposed which are robust, but they are semi-automatic methods and require human intervention to provide the proper initialization for active contour models [21]. Similarly, thresholding methods are also considered as a good choice for segmentation, but they failed to produce if the ROI contains inhomogeneous regions or it has ill-defined boundaries. To combine the features of the afore-mentioned segmentation methods, we propose a method for medical image segmentation; which overcomes the intrinsic problem of thresholding (inability to handle inhomogeneous regions) and active contour models (initialization problem). The proposed method uses Harris Hawk optimization, which is used to produce an initial contour using an optimal threshold value. This contour is further refined using an active contour model. Harris hawk algorithm [18] considers the Minimum Cross Entropy Thresholding (MCET) as a fitness function to calculate multilevel thresholding values. To get one preeminent threshold value out of multiple values, we define an objective function, which is used to get the initial segmenting curve. The initial contour is further refined using an ACM, which is based on a spatially varying Gaussian kernel [26] to get the final segmentation for medical imaging.

The main contributions of this paper can be summarized as follows:

A new initialization scheme using Harris hawk optimizer, is proposed, which is used to obtain the initial contour for the active contour model.

An automated framework is developed, which is able to handle intensity inhomogeneity, low contrast images, noise, weak edges, and other artifacts that are usually present in medical images.

To the best of our knowledge, this is the first method which incorporates a metaheuristic algorithm with an active contour model to solve segmentation problems effectively.

The proposed framework is compared with different state-of-the-art thresholding algorithms and segmentation methods for skin and LV (deep-learning and non-deep learning methods).

The remaining paper is presented as follows: Section II describes the background and related work of medical image segmentation; Section III presents the proposed model for medical image segmentation. In Section IV, a set of experiments are performed, which empirically evaluate standard thresholding algorithms, evolutionary method, deep learning models, and different active contour models. Section V and VI present discussions and conclusions, respectively.

2 Background

2.1 Notations

In this paper, we use following notations as described below: pixel coordinates of the input Image I are denoted by x, y ∈ Ω (imagedomain ⊂ R²). φ (x, t) is a zero-level-set function at a certain time t and it denotes a closed curve C, which can be defined as: {x∈ Ω|φ (x, t) = 0 }. ACMs define curves that move within input image to find object boundaries. ACMs are applied for separating the image into distinct regions; foreground and background. Foreground. Image foreground and background regions are described as Ω₁ (C) ={ x ∈ Ω : Φ (x) < 0 }, and Ω₂ (C) ={ x ∈ Ω : Φ (x) > 0 } respectively. α, γ₁ and γ₂ are penalized parameters which are used in active contour model (ACM), they influence the contribution of each energy term in the minimization process of ACM.

2.2 Related work

Image segmentation is required as a preliminary phase for examining the medical images. Because of the complex nature of medical images, segmentation itself is a very challenging and difficult task [27]. Different segmentation methods have been proposed for different types of medical images [20 , 28]. We can categorize them as supervised and unsupervised learning methods. Thresholding is one of the most popular method for image segmentation [16]. However, bi-level thresholding produces the desired segmentation results if the object has distinct homogeneous regions, but medical images have main challenge of intensity inhomogeneity and low contrast. Many multilevel thresholding methods have been proposed in the last few years for the medical image segmentation [29, 30] however they are still failed to produce good segmentation results due to severe inhomogeneity and ill-defined boundaries. Recent algorithms are based on the use of entropies as objective functions in evolutionary methods to select the optimal threshold values for segmentation [16, 19].

The main problem with thresholding method is that it does not consider the spatial characteristics of the input image, due to which it is not a good choice to use them solely for the segmentation of medical images because the results are very sensitive to intensity inhomogeneity and noise. For medical images, ACMs are considered as a good choice for medical image segmentation being robust to intensity inhomogeneity and low contrast artifacts [20, 21]. ACMs can be categorized into edge-based [22], and region-based methods [23 , 26]. The main problem with ACM is dependency of each model on the initial placement of the evolving contour [24, 26]. In last decade, different hybrid and two stage ACMs have been proposed to overcome the initialization problem [25], but they are computationally very expensive. In different ACMs which are based on the spatial information of the input image, local intensity information is very useful in handling the intensity inhomogeneity and noise problems but they are very sensitive to the contour initialization and poor initialization can easily trap into a local optimum [26, 27].

Different supervised and un-supervised algorithms have been proposed for different types of medical images. An unsupervised technique was proposed for skin lesion segmentation which was based on the sparse coding method [31], the method outperformed state-of-the-art methods but still couldn’t attain very promising results (DSC = 0.80) on ISIC 2016 dataset due to presence of ill-defined boundaries of skin lesions. In 2019, a hybrid approach was proposed using classical gradient-based histogram and clustering-based methods, it outperformed the previously defined methods but still, it was able to achieve DSC < 0.80 for the Dermofit skin dataset [32]. In another paper, two adaptive approaches were used to identify the global threshold for skin lesion segmentation, they produced good segmentation results, but they tested the proposed approach only on 25 skin images [33].

In different medical imaging modalities, CMR plays a vital role in the diagnosis of several congenital heart diseases [34]. Segmentation of the Left Ventricle (LV) is very important in the assessment of ejection fraction, regional function parameters like end-systolic and end-diastolic volumes, these parameters are very significant to detect the abnormalities of the heart [7]. Many supervised and unsupervised algorithms have been proposed for endocardial segmentation. In the last decade, different active contour models (edge-based, region-based, or hybrid) have been proposed for LV segmentation. Hybrid of edge-based and region-based models are good for segmenting the LV in the presence of intensity inhomogeneity and low contrast. A two-stage active contour model was proposed by Soomro et al. [25], which was computationally expensive as the contour evolution was performed in each stage of the model. Active contour models are very sensitive to the initialization, to have good initialization, many semi-automated techniques and few fully automated algorithms have been published [20]; though these models are based on some computationally expensive pattern matching [35] or some complex preprocessing pipelining [5]. Many deep learning networks have been proposed in the last few years for LV segmentation but the main problem is that they require either a very large annotated dataset or need to combine deep learning with some deformable model for LV segmentation as proposed by Avendi et al. [36]. Several evolutionary algorithms have been proposed for image segmentation [19, 29], but no nature-inspired algorithm produces better results than ACM for LV segmentation. Many recent LV segmentation methods are not fully automated and these methods require human intervention or ground truth to initialize the contour [21].

As mentioned above, different methods have been proposed for different types of medical images, but they are specific for one type of imaging modality and fail to produce on some other medical imaging type. Some segmentation methods are good for LV segmentation [35, 36], some methods are good for Brain MRIs [28], some are good for segmenting skin lesions [32, 37], some produce good segmentation for liver on CT scans [38], however, There is no method that can be adapted for different types of medical images.

3 Methodology

3.1 Data

This paper focuses on two datasets from two different medical imaging types; namely the ISIC 2016 skin database and the ACDC database. The skin data consists of 900 skin lesion images with different artifacts. In 2016, the skin dataset was released by the International Skin Imaging Collaboration (ISIC) at the International Symposium on Biomedical Imaging (ISBI) challenges [39]. A manually drawn ground truth is provided with each skin image.

The second image modality which we use in this study is CMR and the dataset is taken from Automated Cardiac Diagnosis Challenge (ACDC) MICCAI 2017 [40]. CMR is acquired in breath-hold with a retrospective or prospective gating in short-axis orientation (SAX), The data covers images from different phases (systolic and diastolic) and types of slices (Apical, Left Ventricular Outflow Tract (LVOT), Basal). The pixel spacing varies from 1.37 to 1.68 mm² per pixel. All images are acquired over a 6-year time period using MRI scanners of different magnetic strengths (3.0 T and 1.5 T).

3.2 Proposed method

We can precisely define our segmentation method in six steps as follows:

Automatically determine multilevel threshold values for the input image using the Harris hawk algorithm.

Using an objective function, automatically calculate one optimal threshold from multiple threshold values.

Convert input image to binary image using optimal threshold value.

Remove objects that are touching the image boundary using imclearborder (MATLAB function) as shown in Fig. 1.

Determine ROI among different connected components, using prior information of the input image.

Calculate the final segmenting contour using an active contour model.

Fig. 1

Overall diagram of proposed segmentation methodology.

The overall process of proposed method is shown in Fig. 1. Further details of thresholding and active contour model is given below:

3.2.1 Multilevel thresholding

For the input image, we search for the optimal threshold values for multilevel thresholding. We use Harris hawk optimization algorithm [18], which uses cross entropy as the fitness function. This function is quantitatively (as fitness score) used to evaluate different threshold values for the given input image. Cross_Entropy is the entropy between the input image and its segmented image, it calculates the information theoretic distance between the probability distributions of input image I ={ I₁, I₂, … I_n } and segmented image S ={ S₁, S₂, … S_n }, which can be defined as: $Dist (I, S) = \sum_{i = 1}^{N} I_{i} \log \frac{I_{i}}{S_{i}}$ (1)

Smaller value of the cross-entropy depicts greater homogeneity and less uncertainty.

First, the image is converted into gray scale and then the histogram of the converted image is acquired. We used Harris hawk method [18] to obtain the vector of optimal threshold values, for clarity, all the steps of Harris hawk algorithms are described in Algorithm 1.

The inputs to the proposed method are the number of the Harris hawks in the population, the number of thresholds and the maximum number of iterations to execute the algorithm.

3.2.2 Initial segmentation

We use another objective function to get one final threshold value from the vector P_best which consists of multiple thresholding values. This final threshold value is used to get the initial segmenting curve for the final segmentation. For this, we calculate the minimum score of the objective function using Eq. (2). The generalized objective function that can be used for different types of medical imaging modalities is defined below: $Objective_function {(th)}_{\min} = {Cross}_{Entropy} + {penalty}_{1} + {penalty}_{2} + {prior}_{modality dependent}$ (2)

Cross_Entropy calculates the entropy between the input image and its segmented image, it is defined in Algorithm 1 (Equation 3).

Algorithm 1: Harris Hawk for medical image segmentation

1	Input Read image in 2D I (x, y) or 3D I (x, y, z) and convert it into grayscale I_gray.
2	Calculate the histogram hist_gray of I_gray.
	Set initialization parameters N = population size (number of Harris hawk) T = maximum number of iterations tn = total number of thresholds
3	Initialize the position P of N random particles with thn dimensions (sets of numbers from 0 –255)
	Repeat (until convergence) (t ⩾ T)
4	Evaluate: The multilevel thresholding is a vector th = [th₁, th₂, th_tn], which contains tn different thresholds values, each element of P is evaluated by the following fitness function (f_obj) based on cross entropy which is given in equation 1. $f_{obi} (th) = \sum_{t = 1}^{n} t hist (t) \log (t) - \sum_{t = 1}^{tn} H_{t}$ (3) where n is total number of gray levels and H_t is defined as: $H_{1} = \sum_{t = 1}^{t h_{1} - 1} t hist (t) \log (μ (1, t h_{1}))$ $H_{j} = \sum_{t = t h_{j} - 1}^{t h_{j} - 1} t hist (t) \log (μ (t h_{j - 1}, t h_{j})) 1 < j < tn H_{tn} = \sum_{t = t h_{tn}}^{n} t hist (t) \log (μ (t h_{tn}, n + 1))$
5	Set P_best as the best location of rabbit.
6	Calculate energy parameter E for each Hawk P_t which can be acquired using the following rule: $E = 2 E_{0} (1 - \frac{t}{T})$ where E₀ is a random number between (–1,1).
7	Update the location of each hawk P using the energy value of its prey.
8	end loop
9	Output: P_best as a vector which contains tn multilevel thresholding values

We use two penalty terms penalty₁ and penalty₂ which are calculated using Eq. (4) and (5) respectively. They are same for any medical imaging modality. B(x,y) denotes value of the pixel located at row x, column y of the thresholded binary image array. An M × N has total M rows and N columns in the image. penalty₁ is used to determine if the final output of a thresholded image is all white region, using the following condition:

${\begin{matrix} if \sum B (x, y) = = M \times N, & {penalty}_{1} = \infty \\ otherwise & {penalty}_{1} = 0 \end{matrix}$ (4)

Similarly, penalty₂ is used to determine if the final output of a thresholded image is all black region, using the following condition: ${\begin{matrix} if \sum B (x, y) = = 0, & {penalty}_{2} = \infty \\ otherwise & {penalty}_{2} = 0 \end{matrix}$ (5)

Though the first three terms in Eq. (3) are the same for any medical imaging modality, however, the last term/s prior_{modalitydependent} is solely dependent on the type of the medical image. In Eq. (2), prior_{modalitydependent} is one or more priors that can be used to segment the ROI, for instance in LV Segmentation, we have prior knowledge that LV is usually most circular, next to Right Ventricle (RV) and located close to the center of the CMR as shown in the Fig. 3. We incorporate the priors using this term as a prior_{modalitydependent}, for instance, we add normalized Euclidean distance of each object from the center of the image and circularity of each object (we prefer circular objects over non circular), the prior_LV comprises of the following terms: ${prior}_{LV} = {Eulcidean}_{normalised} + {circularlity}_{term}$ (6)

Fig. 2

Binary images produced using different threshold values. Fitness scores calculated using objective functions is shown for skin dataset.

Fig. 3

Binary images produced using different threshold values. Fitness scores calculated using objective functions is shown for CMR dataset.

Similarly, for skin lesion segmentation, we use prior knowledge such as skin lesions are usually present close to the center of the image and some properties of the markers and hair artifacts removal are also used. Marker or borders are present in some skin images, so we create masks to identify them and hair artifacts are identified using the circularity of the objects. The prior_skin comprises of the following term: $prio r_{skin} = Eulcidea n_{normalised} + marke r_{removal} + hairartifac t_{removal}$ (7)

This objective function is calculated for each value in the vector P_best which was obtained from Algorithm 1. This step is explained in Algorithm 2.

Algorithm 2: Initial segmentation for medical image segmentation

1	Input: The output vector P_best (p₁, p₂, … p_tn) from Algorithm 1
2	Repeat (∀ i (1 … tn) , pi ∈ P_best)
	Set final_threshold = p₁
	Let next = next element in the vector P_best
3	if more than one objects found in one thresholded image, select one object using prior of the imaging modality (e.g. for CMR, use circularity and Euclidean distance for the ROI)
4	Calculate VALUE[next] = Cross_Entropy + prior_{modality dependent} + penalty₁ + penalty₂
	if VALUE [next]≤VALUE[final_threshold] then final_threshold = next
5	end loop
6	Output: Generate initial segmentation I_seg with the threshold value in final_threshold

3.2.3 Final Segmentation

We use region-based ACM to refine the segmentation result, which we have obtained in the previous step using the optimization algorithm. Region-based models [23 , 26] are considered to be better as compared to the edge-based models [22] due to the global nature of these methods. We use an ACM, which utilizes a region-based energy term E_Reg augmented with two regularization energy functional R_L and R_D i.e., a length penalty term and a distance regularization term, respectively. The total energy functional is described as follow: $E_{Total} (φ) = α E_{Reg} (φ) + γ_{1} R_{L} (φ) + γ_{2} R_{D} (φ)$ (8)

We use a Local Gaussian Distribution Fitting Energy (E_Reg), which utilizes a spatially-varying Gaussian kernel to weight the pixel intensities in the ROI. Using local statistics of the image, this region term (E_Reg) is capable to handle the intensity inhomogeneity artifacts and variable contrast problems. The model partitions image into two distinct regions; foreground and background [26] which are denoted by Ω₁ and Ω₂ respectively. E_Reg minimizes the following energy functional: $E_{Re g} = \sum_{i = 1}^{2} \iint_{Ω_{i} (C)} - K (x - y) log (p_{i (x)}) (I (y)) dy dx$ (9)

where K is a truncated Gaussian kernel that assigns larger weights to nearby pixels i.e. $\lim_{d \to \infty} K (d) \to 0$ (localization property) [26] and p_i (x) is defined as: $p_{i (x)} (I (y)) = \frac{1}{\sqrt{2 π} σ_{i} (x)} \exp (- \frac{{(u_{i} (x) - I (y))}^{2}}{2 σ_{i}^{2} (x)})$ (10) $i = 1 for background and 2 for foreground$

u_i(x) and $σ_{i}^{2} (x)$ are spatially varying mean and variance respectively. This method produces good segmenting regions with weak boundaries and low contrast [26]. The level-set formulation of the region-based energy functional (11), E_Reg (φ), is described using the regularized Heaviside function $(H_{ɛ} (φ)), which is \frac{1}{2} (1 + \frac{2}{π} \arctan (\frac{φ}{ɛ}))$ and ɛ controls the smoothness of the functions, E_Reg (φ) can be defined as follows: $α \sum_{i = 1}^{2} \iint_{Ω_{i} (x)} - K (x - y) log (p_{i (x)}) (I (y)) (H_{i} (φ)) dydx$ (11)

where H₁ (φ) represents regularized Heaviside function for background and H₂ (φ) represents Heaviside function for foreground using (1 - H₁ (φ)).

In addition to the region term (E_Reg), we used two regularization terms to have smooth and better contour evolution. The first regularization term R_L is used, which penalizes the length of the contour for smooth contour evolution [23 , 27] and can be described as follows: $R_{L} (φ) = γ_{1} \int | \nabla H_{ɛ} (φ (x)) | dx$ (12)

The second regularization term R_Dev (φ) is used to obtain an accurate and stable level set function at each iteration to penalize the deviation of the contour from a Sign Distance Function (SDF) [41]: $R_{D} (φ) = γ_{2} \int \frac{1}{2} {(| \nabla φ (x) | - 1)}^{2} dx$ (13)

Iteratively, the curve evolution is performed by minimizing the following functional: $φ^{*} = \arg \min E_{Total} \equiv α E_{Reg} (φ) + γ_{1} R_{L} (φ) + γ_{2} R_{D} (φ)$ (14)

Using the gradient descent algorithm, we obtain the following gradient flow equation minimizing $Φ^{k + 1} = Φ^{k} + Δ t (\frac{\partial E_{Reg}}{\partial φ} + \frac{\partial R_{L}}{\partial φ} + \frac{\partial R_{D}}{\partial φ}) | Φ^{k}$ (15)

All hyper parameters (α, γ₁ and γ₂) are set using grid search method, however we used same values for both datasets and they give good segmentation results.

4 Experiments and results

All the experiments were performed on a processor with a 2.5 GHz Intel Core i5-7200 U processor and 8 GB of RAM. We performed all the experiments in MATLAB R2017a. We used two different types of medical images, both are publicly available; one dataset is from ISIC2016 for segmentation and the other dataset is obtained from an automated cardiac diagnosis challenge (ACDC). All the results are evaluated and compared with other state-of-the-art methods.

4.1 Performance measures

The experimental results are compared with the segmented ground truths, which are determined manually by experienced cardiologists and dermatologists for ACDC and ISIC, respectively. We quantitatively analyzed the segmentation obtained from different algorithms and our proposed method. We used Dice Score (DSC), Jaccard Index (JI), Sensitivity (SE), and Specificity (SP) to compare our method with other algorithms. The formulas for these evaluating metrics are given below: $Sensitivity Rate (SE) = \frac{TP}{TP + FN}$ (16) $Specificity Rate (SE) = \frac{TN}{TN + FP}$ (17) $Prescion (PR) = \frac{TP}{TP + FP}$ (18) $Accuracy (ACC) \frac{TP + TN}{TP + TN + FP + FN}$ (19) $Dice Score (DSC) = 2 \times \frac{PR \times SE}{PR + SE} or 2 \times \frac{A \cap B}{A + B}$ (20)

where A and B are manual and automated segmentations, respectively $Jaccard Index (JI) = \frac{DSC}{2 - DSC}$ (21) $Percentage of Good Contours = \frac{Number of good slices}{Total number of slices}$ (22)

where Good Contours (GC) are calculated with Average perpendicular distance (APD)≤5 mm and APD is the distance from the ground-truth (manual) segmentation to the automated segmented contour averaged over all contour points [42].

4.2 Comparison with state-of-the-art methods using skin dataset

In recent years, many deep learning [43 –45] and non-deep learning algorithms [33, 46] have been proposed for skin lesion segmentation. We compared skin lesion segmentation with standard thresholding (Otsu), evolutionary method (Harris Hawk) [18], ACM [26] and some recent deep learning methods [4 , 47]. Table 1 shows that our proposed algorithm outperforms non-deep learning methods, and it has produced comparable results to the deep learning methods. This table also shows that ACM, which is used to refine the segmentation results, significantly improves the dice score (from 0.83 to 0.90 with p-value≤0.001) Fig. 4 presents a qualitative analysis of proposed method for skin lesion segmentation.

Table 1
Comparison of proposed method with state-of-the-art methods for skin lesion segmentation

DSC Deep Learning SE SP ACC JI

Proposed method 0.90 ± 0.03 No 0.90±0.02 0.98 ± 0.01 0.95 ± 0.02 0.83 ± 0.03

Otsu method 0.80±0.04 No 0.78±0.04 0.88±0.02 0.87±0.03 0.74±0.04

ACM [26] 0.60±0.05 No 0.58±0.04 0.92±0.02 0.82±0.03 0.53±0.05

Harris Hawk [18] 0.83±0.04 No 0.80±0.03 0.96±0.02 0.92±0.02 0.75±0.04

CUMED [47] 0.89±0.02 Yes 0.91 ± 0.01 0.95±0.01 0.94±0.01 0.82±0.02

Deep FCN [4] 0.89±0.03 Yes 0.89±0.02 0.96±0.01 0.94±0.01 0.82±0.02

Bi. et al [47] 0.844 Yes - - - 0.765±0.02

Yuan [44] 0.849±0.02 Yes - - - 0.765±0.02

segXNorm [49] 0.88±0.02 Yes - - - 0.79±0.02

	DSC	Deep Learning	SE	SP	ACC	JI
Proposed method	0.90 ± 0.03	No	0.90±0.02	0.98 ± 0.01	0.95 ± 0.02	0.83 ± 0.03
Otsu method	0.80±0.04	No	0.78±0.04	0.88±0.02	0.87±0.03	0.74±0.04
ACM [26]	0.60±0.05	No	0.58±0.04	0.92±0.02	0.82±0.03	0.53±0.05
Harris Hawk [18]	0.83±0.04	No	0.80±0.03	0.96±0.02	0.92±0.02	0.75±0.04
CUMED [47]	0.89±0.02	Yes	0.91 ± 0.01	0.95±0.01	0.94±0.01	0.82±0.02
Deep FCN [4]	0.89±0.03	Yes	0.89±0.02	0.96±0.01	0.94±0.01	0.82±0.02
Bi. et al [47]	0.844	Yes	-	-	-	0.765±0.02
Yuan [44]	0.849±0.02	Yes	-	-	-	0.765±0.02
segXNorm [49]	0.88±0.02	Yes	-	-	-	0.79±0.02

Table 2

Comparison of proposed method with state-of-the-art methods for LV segmentation

	Deep Learning	DSC	JI	GC (%)	HD	APD
Proposed method	No	0.93 ± 0.02	0.87 ± 0.02	92	2.0±0.02	1.2
ACM [18]	No	0.89±0.03	0.80±0.02	71	2.4±0.02	2.2
Harris Hawk [24]	No	0.86±0.04	0.85±0.02	85	2.3±0.01	2.0
HTSM [25]	No	0.91±0.02	0.83±0.02	65	2.3±0.02	1.9
ACM-LV [16]	No	0.87±0.03	0.77±0.02	83	2.3±0.03	2.2
Avendi [21]	Yes	0.93±0.02	0.87 ± 0.02	94	1.8 ± 0.01	1.3
Zotti [50]	Yes	0.93±0.04	0.87 ± 0.03	-	-	-

Fig. 4

Different types of skin images, with irregular boundaries, intensity inhomogeneity, variable contrast, presence of markers, hair artifacts. Segmenting curves are shown with each image where green contours are manually determined skin lesions and red contours are generated using the proposed approach.

4.3 Comparison with state-of-the-art methods using cardiac dataset

We used 40 random subjects from the ACDC dataset and found that our proposed algorithm outperformed other state-of-the-art algorithms p≤1 × 10^–8 [18 , 26]. The proposed method also produced a very high percentage (92%) of good contours (GC) (APD≤5 mm). This method yielded superior segmentation results from non-deep learning methods and comparable results to the deep learning methods. We have analyzed the results using different evaluating measures. Figure 5 presents a qualitative analysis of LV segmentations obtained from proposed method using different types of slices.

Fig. 5

Different types of Cardiac MRIs, with severe intensity inhomogeneity, presence of different organs, variation in LV sizes. Segmenting curves are shown with each image where green contours are manually determined skin lesions and red contours are generated using the proposed approach.

Figures 4 and 5 show the robustness of our proposed algorithm for different types of images (skin lesion and LV). It is very challenging to produce good segmentation results in the presence of markers, hair, intensity inhomogeneity, different organs, low contrast, and ill-defined boundaries.

4.4 Validation of the proposed method

In this study, 500 dermoscopic images were taken from another publicly available ISIC archive to validate the proposed method and priors which we used for ISIC 2016 dataset. Our proposed hybrid segmentation model achieved an average Jaccard index score of 0.81. This results to an improvement of 10.5% over the accuracy of the winner of the ISBI 2017 challenge (Jaccard Index = 0.765).

4.5 Time complexity

In medical image segmentation, accuracy and efficiency are both very important evaluating measures. Figure 7 shows our method produces superior segmentation results as compared to other state-of-art algorithms. Figure 8 shows Harris Hawk optimization method is most efficient in segmenting LV boundary and our proposed method is the second most efficient method; however. Figure 7 depicts that the Harris Hawk fails to produce good segmentation results as compared to our proposed hybrid model.

Fig. 6

Overestimated, underestimated and estimated segmentation results produced by thresholding, ACM & proposed methods, respectively.

Fig. 7

Dice Score of different methods for LV segmentation.

Fig. 8

Computational efficiency of different methods for LV segmentation.

Our method takes approximately 1.5 seconds to segment one CMR (2D) image, which is more efficient than other state-of-the-art methods.

5 Discussion

Medical image segmentation is the key to the diagnosis of several diseases and accurate segmentation plays an important role for the assessment of several skin, lungs, brain and cardiac diseases [21 , 48]. Many methods have been proposed to get ROIs; however, there is no single framework that can be adapted and applied to different modalities of medical imaging. Recently, deep learning networks have outperformed the image segmentation, but they all require a large size of annotated data for training, tuning of a large number of hyper-parameters and requirement of Graphics Processing Units (GPUs) to solve the segmentation task. Data augmentation has resolved the problem of large datasets, however, overfitting can still cause misclassification on the unseen data examples [12]. Thresholding is suitable for those types of images where we can distinctly divide the image into n groups; however, the traditional thresholding algorithm (Otsu method) produces ineffective and undesired segmentation results in many cases, especially for images with low-intensities and ill-defined boundaries. Similarly, the Histogram Thresholding algorithm (HT) is also good to produce accurate delineation if there is a sharp difference between the ROI and the surrounding region [32].

ACMs are very robust to inhomogeneity and low contrast artifacts, which make them a suitable choice for medical image segmentation [20 , 35]. However, ACMs are very sensitive to initialization [23 , 27], which makes ROI segmentations mainly dependent on the initial placement of the contour and the initial contour can easily be trapped into a local minimum [24, 26]. Several semi-automated methods and fully automated algorithms have been proposed to have an appropriate contour initialization, though they involve computationally expensive pattern matching [35] or complex preprocessing pipeline [5], which may not generalize to multi-center datasets. Moreover, in the past recent years, some two-stage ACMs [25] have been published in which the first stage is used to get the initial curve (or contour), which is utilized as the initial placement of the contour in the second stage for the final segmentation. However, these two-stage methods are computationally very expensive because there is a complete execution of ACM in each stage [25].

In order to overcome the limitations of ACMs and thresholding methods for medical image segmentation where one may cause overestimation or underestimation of the ROI [32] as shown in Fig. 6, we need to design a framework which incorporates the advantages of both techniques to achieve a better estimation. In the proposed method, we have used a metaheuristic algorithm to get the optimal threshold value to get the initial segmentation, which is further refined by a region-based ACM to produce the final segmenting curve.

The experimental results in the last section validate our method using two publicly available medical imaging datasets. Figures 4 and 5 show the detailed qualitative analysis of our approach and proved that this proposed method is able to accommodate the artifacts present in the medical images and it is capable of handling the challenges like variable contrast, indistinct boundaries, shape differences and undesirable residues (markers, borders and many more) to get the ROI [45]. The accurate delineation of any method is a relevant achievement to produce particular features for machine learning procedures. One more advantage of our proposed method is that we have a new framework for medical image segmentation which can be adapted for different types of medical images, we have validated our approach on two entirely different datasets (a skin dataset and a CMR dataset). As given in Eq. 3, we can define prior that is mainly dependent on the type of the input image, this method can be applied to any medical dataset. In the future, we plan to apply this framework to other medical imaging modalities as well.

6 Conclusion

We propose a new framework for medical image segmentation, consisting of an evolutionary method and an active contour model. The evolutionary algorithm is used to obtain the optimal threshold value to get the initial segmentation, which is further refined by the evolution of an active contour model. The region-based active contour model includes a Gaussian based spatially-varying, local statistical model to handle intensity inhomogeneity and other artifacts. A length penalty term and a distance regularization term are incorporated to ensure the smooth evolution of the segmenting curve. The model was robust to low contrast, ill-defined boundaries, intensity inhomogeneity, and other artifacts as exhibited in challenging medical images. It also showed that the proposed method was able to handle diverse types of images using one generalized algorithm and it can be adapted for other types as well.

References

Zhang

and Song

, An adaptive fuzzy level set model with local spatial information for medical image segmentation and bias correction, IEEE ACCESS 7 (2019), 27322–27338.

, et al., Learning active contour models for medical image segmentation, in IEEE conference on computer vision and pattern recognition, 2019.

Luijnenburg

S.E.

, et al., Intra-observer and interobserver variability of biventricular function, volumes and mass in patients with congenital heart disease measured by CMR imaging, The International Journal of Cardiovascular Imaging 26(1) (2009), 57–64.

Zhang

, et al., Automatic skin lesion segmentation by coupling deep fully convolutional networks and shallow network with textons, Journal of Medical Imaging 6(2) (2019), 024001.

, et al., Automatic segmentation of the left ventricle in cardiac MRI using local binary fitting model and dynamic programming techniques, PLOS One 9(12), 2014.

Despotović

, et al., MRI segmentation of the human brain: challenges, methods, and applications, Computational and Mathematical Methods in Medicine (2015), 450341.

Shahzad

, et al., Automated cardiovascular segmentation in patients with congenital heart disease from 3D CMR scans: Combining multi-atlases and level-sets, in Lecture Notes in Computer Science, 2017.

Navab

and Ahmadi

S.A.

, V-Net: Fully convolutional neural networks for volumetric medical image segmentation, in Computer Vision and Pattern Recognition, 2016.

Yang

, et al., DCU-Net: Multi-scale U-Net from brain tumor segmentation, Journal of X-ray Science and Technology 28(4) (2020), 709–726.

10.

Margeta

, et al., Layered spatio-temporal forests for left ventricle segmentation from 4D cardiac MRI data, in Lecture Notes in Computer Science, 2011.

11.

Papernot

, et al., The limitations of deep learning in adversarial settings, in 2016 IEEE European Symposium on Security and Privacy, Saarbrucken, Germany, 2016.

12.

Hesam

, et al., Deep learning techniques for medical image segmentation: achievements and challenges, Journal of digital Imaging 32 (2019), 582–596.

13.

Aganj

, et al., Unsupervised medical image segmentation based on the local center of mass, Scientific Reports 8(1) (2018), 13012.

14.

Tobias

O.J.

and Seara

, Image segmentation by histogram thresholding using fuzzy sets, IEEE Transactions on Image Processing 11(12) (2002), 1457–1465.

15.

Nameirakpam

, et al., Image segmentation using K-means clustering algorithm and subtractive clustering algorithm, Procedia Computer Science 54 (2015), 764–771.

16.

Naidu

, et al., Shannon and fuzzy entropy based evolutionary image thresholding for image segmentation, Alexandria Engineering Journal 57(3) (2018), 1643–1655.

17.

Elaziz

and Lu

, Many-objectives multilevel thresholding image segmentation using Knee Evolutionary Algorithm, Expert systems with Applications 125 (2019), 305–316.

18.

Rodríguez-Esparza

, et al., An efficient Harris Hawks-inspired image segmentation method, Expert Systems with Applications 155 (2020), 113428.

19.

Bhandari

, et al., Tsallis entropy based multilevel thresholding for colored satellite image segmentation using evolutionary algorithms, Expert systems with Applications 42(22) (2015), 8707–8730.

20.

Khamechian

M. B.

and Saadatmand

, FoCA: A new framework of coupled geometric active contours for segmentation of 3D cardiac magnetic resonance images, Magnetic Resonance Imaging (2018), pp. 51–60.

21.

Hajiaghayi

, et al., A 3D active contour method for automated segmentation of the left ventricle from magnetic resonance images, IEEE Transactions on Biomedical Engineering (2017), pp. 134–144.

22.

Caselles

, et al., Geodesic active contours, International Journal of Computer Vision, (1997), pp. 61–79.

23.

Chan

T.F.

and Vese

L.A.

, Active contours without edges, IEEE Transactions on Image Processing 10(2) (2001), 266–277.

24.

Lankton

and Tannenbaum

, Localizing region-based active contours, IEEE Transactions on Image Processing 17 (2008), 2029–2039.

25.

Soomro

, et al., Hybrid two-stage active contour method with region and edge information for intensity inhomogeneous image segmentation, PLOS One 13(1) (2018), e0191827.

26.

Wang

, et al., Active contours driven by local Gaussian distribution fitting energy, Signal Processing 89(12) (2009), 2435–2447.

27.

, et al., Implicit active contours driven by local binary fitting energy, in IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, 2007.

28.

Meng

, et al., Brain MR image segmentation based on an improved active contour mode, PLOS One, 2017.

29.

Cuevas

and Sossa

, A comparison of nature inspired algorithms for multi-threshold image segmentation, Epert systems with Applications 40(4) (2013), 1213–1219.

30.

Ramadad

and Abraham

, Detecting tumours by segmenting MRI images using transformed, Neural Computing and Applications, (2020), pp. 6139–6149.

31.

Bozorgtabar

, et al., Sparse coding based skin lesion segmentation using dynamic rule-based refinement, Machine Learning in Medical Imaging 10019 (2016), 254–261.

32.

Pereira

, et al., Image segmentation using gradient-based histogram thresholding for skin lesion delineation, Bioimaging 2 (2019), 84–91.

33.

Thanh

, et al., A skin lesion segmentation method for dermoscopic images based on adaptive thresholding with normalization of color models, in 6th International Conference on Electrical and Electronics Engineering (ICEEE), 2019.

34.

Peng

, et al., A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging, Magnetic Resonance Materials in Physics, Biology and Medicine 29(2) (2016), 155–195.

35.

Queirós

, et al., Fast automatic myocardial segmentation in 4D cine CMR datasets, Medical Image Analysis 18 (2014), 1115–1131.

36.

Avendi

M.R.

, et al., A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI, Medical Image Analysis 30 (2016), 108–119.

37.

Ozturk

and Ozkaya

, Skin lesion segmentation with improved convolutional neural network, Journal of Digital Imaging 33 (2020), 958–970.

38.

Ibrahim

, Deep learning and level set approach for liver and tumor segmentation from CT scans, Journal of Applied Clinical Medical Physics 21(10), 2020.

39.

Gutman

, et al., Skin lesion analysis toward Melanoma detection: A challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC), Computer Vision and Pattern Recognition, 2016.

40.

Bernard

, et al., Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved?pp, IEEE Transactions on Medical Imaging 37 (2018), 2514–2525.

41.

, et al., Level set evolution without re-initialization: A new variational formulation, in CVPR’05, 2005.

42.

Radau

, et al., Evaluation framework for algorithms segmenting short axis cardiac MRI, The MIDAS Journal-Cardiac MR Left Ventricle Segmentation Challenge 49, 2009.

43.

Lii

, et al., Automatic tooth roots segmentation of cone beam computed tomography image sequences using U-net and RNN, Journal of X-ray Science and Technology 28(5) (2020), 905–922.

44.

Yuan

, et al., Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance, IEEE Transactions on Medical Imaging, (2017), pp. 1876–1886.

45.

Brinker

, et al., Enhanced classifier training to improve precision of a convolutional neural network to identify images of skin lesions, PLOS One 14(6), 2019.

46.

Cui

, et al., Fully-automatic segmentation of coronary artery using growing algorithm,–, Journal of X-ray Science and Technology 28(6) (1186).

47.

, et al., Automatic skin lesion analysis using large-scale dermoscopy images and deep residual networks, arXiv preprint arXiv:1703.04197, 2017.

48.

Meng

, et al., Brain MR image segmentation based on an improved active contour mode, PLOS One 12(8) (2017), e0183943.

49.

Thanh

50.

Zotti

, et al., Convolutional neural network with shape prior applied to cardiac MRI segmentation, IEEE Journal of Biomedical and Health Informatics 23(3) (2019), 1119–1128.