Optimizing cardiovascular image segmentation through integrated hierarchical features and attention mechanisms

2.1 Evolution of machine learning in segmentation

Compared to traditional methods, deep learning iteratively improves segmentation accuracy through continuous training, especially with uniform characteristic images. Deep learning offers faster computation and adaptability for batch processing. U-Net’s [9] paired encoder-decoder structures with skip connections address limitations and enhance pixel-level and semantic information for better segmentation. Wu et al. [10] introduced multiple supervised pathways for richer scale features. Angio-Net [11] enables continuous preprocessing modification, while SD-Net [12] simplifies the network for efficiency. CE-Net [13] captures advanced semantic information with a multiscale context encoding module. Despite successes, challenges persist due to cardiovascular image heterogeneity. Researchers explore multiscale features, attention mechanisms, and novel architectures [14] to address these challenges.

2.2 Attention mechanisms in segmentation

Inspired by human visual attention, attention mechanisms crucially enhance segmentation accuracy. The self-attention mechanism proposed by Vaswani et al. [15] allows focusing on relevant image regions, while polarized self-attention [16] selectively retains essential features. These innovations aim to provide adaptive and robust segmentation models, vital for clinical practice. The effectiveness of attention mechanisms has garnered attention, with many attention mechanisms proposed and proven effective in improving network performance and reducing computational costs, among which the most widely used is CBAM (Convolutional Block Attention Module) [17] and its variants. Gou et al. [18] introduced SA-UNet based on SD-UNet, using dropblock instead of dropout and adding a spatial attention module. The improved network can better extract small vessels and reduce noise. ESA-UNet [19] introduced an enhanced spatial attention module (ESA), which can better adaptively redistribute features based on spatial context content, enabling the network model to understand more contextual information about the image. Automated segmentation methods contribute to increased efficiency, reducing the time burden on clinical practitioners for timely interventions. Overall, the transition from traditional to advanced machine learning technologies in cardiovascular image segmentation emphasizes addressing image heterogeneity and complex anatomical structures, aiming for better healthcare outcomes.

OPEN IN VIEWER

3.1 Animal experiments to measure ABR

The suggested network architecture, shown in Fig. 1, adopts a novel method to represent the links between inter-class homogeneity to concentrate on cardiovascular image segmentation. Using an encoder to extract important features from cardiovascular pictures, the method generates segmentation masks for each kind of tissue by accurately separating cardiovascular tissues from the backdrop using a decoder. The method ensures adaptability by adjusting the feature map sizes to fit the feature map dimensions of the input cardiovascular image. By improving the precision and effectiveness of cardiovascular picture segmentation, this method seeks to offer a more dependable clinical medical image processing solution.

HRNetV1 is selected as the spine for feature extraction during the encoding step of the cardiovascular image segmentation procedure. There are two main benefits that this backbone has. First off, the network structure as a whole maintains high-resolution characteristics, which helps identify small tissue blocks in the images with accuracy. Second, HRNetV1 ensures efficient use of information contained in each resolution’s feature map by combining feature maps of various resolutions, facilitating effective information interchange. Each input image yields four feature maps from the backbone.

During the decoding process, a Region Weighted Fusion (RWF) module is employed for multi-scale feature fusion. Subsequently, the fused feature maps are input into the Shape Feature Refinement (SFR) module to fine-tune learning of shape features, enabling interaction and influence among features. Finally, a classifier with 1 × 1 convolutions is used to adjust the channel numbers of the input image to match the desired classes, and the results are down sampled to the input image’s size, yielding the semantic segmentation results for cardiovascular pathological images.

Introducing and integrating a Self-Attention module into the suggested RWF module is a creative departure from traditional feature fusion techniques. This module, which is based on the polarized self-attention mechanism, correlates channel features and learns the significance of spatial information at various scales. It then adaptively and selectively keeps important features while removing unnecessary ones. The model performs much better in cardiovascular image segmentation tasks thanks to its innovative design. For a thorough schematic of the RWF structure, see Fig. 2.

OPEN IN VIEWER

Three phases make up the RWF module in cardiovascular picture segmentation: feature fusion, weight computation, and reshaping. The first stage involves adjusting the feature map sizes ( t ∈ { 1 , 2 , 3 , 4 } ) to yield ( E O ′ ) by using nearest-neighbor interpolation. Then, a 3 × 3 convolution ( U 3 ) yields enhanced feature information, which becomes U 3 ( E O ′ ). The weights of U 3 ( E O ′ ) are determined by introducing a self-attention mechanism in the second stage. Subsequently, the four weighted feature maps undergo 1 × 1 convolutions to modify their channel numbers, yielding the output weighted feature map At ( t ∈ { 1 , 2 , 3 , 4 } ). In the third and final stage, the feature map with modified weights is obtained by multiplying U 3 ( E O ′ ) and At pixel-by-pixel. The final feature map is created by concatenating these fused feature maps.

Inspired by the properties of polarizing lenses, a polarized self-attention mechanism is utilized in the second step of the RWF module. This mechanism, allows filtering of light in random directions, orthogonal to the horizontally passing light. In the attention calculation, a polarized filtering mechanism is established, maintaining high-resolution attention computation dimensions and reducing information loss. This innovative design enhances the model’s performance in cardiovascular image segmentation tasks.

Equaattion (1) is used to calculate the feature tensor of a sample for cardiovascular image segmentation. This feature tensor is represented as E t O ∈ R h × w × c , where h , w , and c stand for the feature tensor’s height, width, and channel number, respectively. The channel attention weight A C ⁢ ( X ) plus the spatial attention weight A S ⁢ ( X ) add up to the attention weight H( ⋅ ) that the attention mechanism outputs.

H ⁢ ( ⋅ ) = A C ⁢ ( X ) + A S ⁢ ( X )(1)

More specifically, Eq. (2) is used to determine the channel attention weight A C ⁢ ( X ) :

A C ⁢ ( X ) = F 𝑆𝑖 ⁢ ( Y ⁢ ( U 3 ⁢ ( B ⁢ ( U 1 ⁢ ( X ) ) × F 𝑆𝑜 ⁢ ( B 2 ⁢ ( U 2 ⁢ ( X ) ) ) ) ) )(2)

And the weight of spatial attention Eq. (2). Equation (3) is utilized to calculate A S ⁢ ( X ) :

A S ⁢ ( X ) = F S ⁢ i ⁢ ( B 3 ⁢ ( F S ⁢ o ⁢ ( B 1 ⁢ ( G ⁢ ( U 2 ⁢ ( X ) ) ) ) × B 2 ⁢ ( U 1 ⁢ ( X ) ) ) )(3)

The softmax operation, which increases the dynamic range of attention by normalizing, is represented by F s ⁢ o in the functions above. The sigmoid operation is represented by F s ⁢ i , and the probability distribution function that makes use of the high-resolution data kept in the attention branch is the softmax-sigmoid combination. G stands for global average pooling, X is the cross product, B i is tensor reshaping, U i is the i × i convolution operation, and Y is layer normalization.

The weights from each feature map are concatenated and fused in the third stage, known as “Feature Fusion,” in order to preserve the weights of particular features within a single feature map as well as the significance of weights from separate feature maps. By removing unnecessary characteristics, this procedure helps to retain important feature information for tissue segmentation in an adaptable and selective manner.

For comprehensive training, an effective loss function is crucial in image segmentation tasks. The Dice Loss is commonly used, defined as the sum of predicted and ground truth binary values divided by the total number of pixels. For multi-class segmentation, adding Cross-Entropy Loss is beneficial. To prevent overfitting, an L2 norm term for the network’s parameters can be introduced. The final weighted loss function combines these components, ensuring efficient training and generalization of the cardiovascular image segmentation model. Adjusting weights allows fine-tuning based on the significance of each loss component for specific segmentation tasks.

𝐷𝑖𝑐𝑒𝐿𝑜𝑠𝑠 = 1 - 2 × ∑ i n ( p i × g i ) ∑ i n p i 2 + ∑ i n g i 2(4) Cross - EntropyLoss = - ∑ i n ( g i × log ⁡ ( p i ) )(5) L ⁢ 2 ⁢ 𝑅𝑒𝑔𝑢𝑙𝑎𝑟𝑖𝑧𝑎𝑡𝑖𝑜𝑛 = λ × ∑ i N || W i || 2(6) 𝑇𝑜𝑡𝑎𝑙𝐿𝑜𝑠𝑠 = w 𝐷𝑖𝑐𝑒 × 𝐷𝑖𝑐𝑒𝐿𝑜𝑠𝑠 + w 𝐶𝐸 × 𝐶𝑟𝑜𝑠𝑠 - 𝐸𝑛𝑡𝑟𝑜𝑝𝑦𝐿𝑜𝑠𝑠 + w L ⁢ 2 × L ⁢ 2 ⁢ 𝑅𝑒𝑔𝑢𝑙𝑎𝑟𝑖𝑧𝑎𝑡𝑖𝑜𝑛(7)

Where p i and g i are the predicted and ground truth binary values for each pixel, respectively, and n is the total number of pixels. W i denotes the network’s parameters, N is the total number of parameters, and λ is the regularization strength. The weights w 𝐷𝑖𝑐𝑒 , w 𝐶𝐸 , and w L ⁢ 2 are modifiable to equalize the contributions of each loss component.

4.1 Datasets

Our dataset for coronary computed tomography (CCTA) includes the proximal descending aorta (DA), inferior vena cava (IVC), coronary sinus (CS), right ventricular wall (RVW) and left atrial wall (LAW). These annotations were created and validated by board-certified cardiologists and were used as “ground truth” for the deep learning model.

We used standard segmentation evaluation metrics, including Hausdorff Distance (HD), Intersection over Union (IoU), and Dice Coefficient (Dice), to quantify the performance of our method and baseline approaches. These metrics measure boundary dissimilarity, segmentation accuracy, and spatial overlap, respectively.

𝐷𝑖𝑐𝑒 = 2 ⁢ | A ⁢ ∩ B | | A | + | B | = 2 ⁢ 𝑇𝑃 2 ⁢ 𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁(8) 𝐼𝑜𝑈 = | A ⁢ ∩ B | | A ⁢ ∪ B | = 𝑇𝑃 𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁(9) 𝐻𝐷 ⁢ ( A , B ) = max ⁡ ( 𝑠𝑢𝑝𝑎 ∈ A ⁢ inf ⁢ b ∈ B ⁢ d ⁢ ( a , b ) , sup ⁢ b ∈ B ⁢ inf ⁢ a ∈ A ⁢ d ⁢ ( a , b ) )(10)

The distance between midpoints ( d ⁢ ( a , b ) ) measures the dissimilarity between sets A and B , representing two segmentation masks. TP, TN, FP, and FN denote true positives, true negatives, false positives, and false negatives, respectively.

We evaluated our suggested approach’s performance by contrasting it with a number of cutting-edge techniques for segmenting cardiovascular images, such as Attention U-Net [21], DeepLab [14], and U-Net [9]. These techniques serve as the foundation for our comparison study and are the industry standards as of right now. The suggested approach was trained using the same training parameters and in the same experimental setup as the comparison model to guarantee the experiment’s fairness.

The experimental findings, which are shown in Table 1, demonstrate how successful our suggested strategy is in comparison to baseline techniques. In every evaluation indicator, the suggested approach surpassed the current ones with consistent results. The suggested approach outperformed U-Net by 0.08 and 0.09, respectively, and showed a notable increase in segmentation accuracy, obtaining a Dice of 0.85 and an IoU of 0.78. When compared to baseline approaches, this shows improved boundary alignment and spatial overlap. Furthermore, the HD is 2.9% less than U-Net’s. The reduced HD provides more evidence of our method’s effectiveness in precisely capturing anatomical details.

A visual comparison of the segmentation findings for CS, DA, IVC, LAW, and RVW is presented in Fig. 3. Red indicates the segmentation outcomes of various techniques, while green indicates the ground truth in the segmentation findings. The suggested technique shows that it can correctly identify cardiac components even under difficult conditions with fluctuating contrast and intricate anatomical features. The unique incorporation of the polarized self-attention mechanism, which selectively preserves relevant aspects while suppressing irrelevant information, is credited with our suggested method’s higher performance. The robustness of this approach is further demonstrated by its capacity to adapt to various imaging situations.

OPEN IN VIEWER

In these sampled regions, all models used for comparison returned some errors and missed detection results. For the CS structure, both Attention U-Net and DeepLab failed to completely identify the CS structure, and U-Net struggled to differentiate CS structure from other relevant matrix parts, resulting in the segmentation of other relevant matrix parts as CS. In the DA structure, except for our proposed model, other comparative models failed to correctly segment the boundaries of the DA. For the IVC structure, DeepLab and U-Net enlarged the segmented part of the IVC structure, significantly impacting the segmentation results. In the LAW structure, although parts of the LAW structure are relatively small, noticeable defects and distorted segmentation shapes are evident in the results of Attention U-Net, DeepLab, and U-Net. Finally, in the RVW structure, our proposed method, along with Attention U-Net and DeepLab, achieved segmentation results closely resembling the Ground Truth, while U-Net led to incomplete segmentation structures.

The results show, in summary, that our suggested method outperforms existing methods in cardiovascular image segmentation. Experimental results demonstrate that the proposed improvement method effectively enhances the segmentation outcomes of cardiovascular images. Comparative analysis with other state-of-the-art segmentation methods indicates the superiority of the proposed segmentation approach, showcasing excellent segmentation results.

To validate the effectiveness of our method, ablation experiments deconstructed the effect of RWF module components on cardiovascular image segmentation. The experimental configuration includes baseline model (BM), no self-retention (NSA), no polarization self-retention (NPSA), no feature fusion (NFF), and no remodeling (NR) modules. Evaluation metrics such as Dice coefficient and IoU on the validation set show that the results of the ablation experiments indicate that each element of the RWF module contributes significantly to the segmentation performance of the model. The effective capture of spatial features depends heavily on the self-attention mechanism, especially its polarized version. The feature fusion stage also illustrates the importance of retaining important data for organizing segmentation. Although the contribution of the remodeling stage was not significant, good results were achieved.

In conclusion, the accuracy and efficiency of the cardiovascular image segmentation model was improved by the new components of the proposed RWF module. The thorough analysis makes it easier to understand the different functions of each module and also provides suggestions for future iterations of the model for greater improvement and refinement.

5.1 Conclusions and future directions

Cardiovascular disease, a significant health threat, necessitates advanced segmentation methods. Our study introduces an innovative cardiovascular image segmentation approach, optimizing segmentation through hierarchical features and attention mechanisms. Utilizing a polarized self-attention strategy, the method achieves refined segmentation, excelling in distinguishing cardiac structures. Key contributions and conclusions include:

Improved Segmentation Accuracy: Experimental results show enhanced precision in metrics like Intersection over Union (IoU) and Dice similarity coefficient.

Adaptability to Varied Imaging Conditions: The method exhibits high segmentation capabilities under contrast, resolution, and anatomical structure variations.

Enhanced Understanding of Image Features: Integration of polarized attention improves the model’s recognition and utilization of cardiovascular image features.

5.2 Future work

Future research will explore:

Vascular Segmentation: Addressing cardiovascular vessel segmentation and extending the method to other vascular segmentation tasks.

Data Challenges: Tackling segmentation challenges in cardiovascular image data with small samples or limited annotations.

Enhanced Accuracy: Improving segmentation accuracy for tasks like organ size assessment through more powerful feature extraction networks.

Clinical Integration: Exploring how segmentation outcomes can specifically contribute to clinical-assisted diagnostic processes.

Integrated Approaches: Implementing an integrated approach involving multiple segmentation models to establish a comprehensive diagnostic platform for cardiovascular diseases.