Improving U-Net architecture and graph cuts optimization to classify arterioles and venules in retina fundus images

Abstract

The central retinal artery and its branches supply blood to the inner retina. Vascular manifestations in the retina indirectly reflect the vascular changes and damage in organs such as the heart, kidneys, and brain because of the similar vascular structure of these organs. The diabetic retinopathy and risk of stroke are caused by increased venular caliber. The degrees of these diseases depend on the changes of arterioles and venules. The ratio between the calibers of arterioles and venules (AVR) is various. AVR is considered as the useful diagnostic indicator of different associated health problems. However, the task is not easy because of the lack of information of the features being used to classify the retinal vessels as arterioles and venules. This paper proposed a method to classify the retinal vessels into the arterioles and venules based on improving U-Net architecture and graph cuts. The accuracy of the proposed method is about 97.6%. The results of the proposed method are better than the other methods in RITE dataset and AVRDB dataset.

Keywords

Arterioles venules U-Net architecture graph cuts retinal blood vessels

1 Introduction

In recent years, many dangerous diseases have appeared around the world. Some different systemic and ophthalmic ills can be diagnosed by basing on the morphology of retinal blood vessels. In fact, the retinal vessels include arteries and veins. The arteries transport blood rich in oxygen to organs, so they are brighter. While the veins carry the deoxygenated blood from the organs back to the heart, thus their color is darker [1 –3].

Doctors have used the retinal images to diagnose some health problems such as narrowing arteriole [4], diabetic retinopathy [5], glaucoma [6], age-related macular degeneration [7], arteriosclerosis and hypertension [8], and so on. These diseases come from the changes of arterioles and venules. The evidence of this, the decreased arteriolar caliber caused the coronary arterial disease [9]. The arteriole narrowing at the early step is related to hypertensive retinopathy [10]. The narrowed arteriole is a sign of the pancreas [11]. The diabetic retinopathy and risk of stroke are caused by increased venular caliber [5]. The degrees of diseases depend on the changes of arterioles and venules. The ratio between the calibers of arterioles and venules (AVR) is various [12]. AVR is considered a useful diagnostic indicator of different associated health problems. The evidence from here is the contribution based on the abnormal AVR to diagnose high blood pressure or high cholesterol levels [13]. Thus, to find out the abnormal AVR is necessary to serve for diagnosing doctors. However, the task is not easy because of the lack of information of the features being used for the classification of the retinal vessels as arterioles and venules.

Acute hypertension often causes reversible vasospasm in the retina, and malignant hypertension can cause papilledema. Prolonged or severe hypertension leads to vascular changes, resulting in endothelial damage and necrosis. Other changes, such as thickening of the vessel wall and arteriovenous crossing, often require years of hypertension to progress. Symptoms usually do not develop until late in the disease and include blurred or narrowed vision. In the early stages, ophthalmoscopy helps to detect arterial spasm associated with a decreased ratio of arterioles to venules aperture. Therefore, the classification of arteries and veins in retinal images aims to calculate the ratio of the arterial to venous aperture, which helps in the initial diagnosis of hypertension.

In this paper, we proposed the method to classify the retinal vessels into the arterioles and venules based on improving U-Net and graph cuts. The proposed method includes four steps: pre-processing, improving U-Net architecture, initial arterioles and venules classification, and graph cuts. The RITE dataset [14] and AVRDB dataset [39] are tested to make the comparison between our proposed method and the recent other methods based on accuracy evaluation criteria.

Main contributions of this paper are as follows:

Modifying U-net architecture (MoU-Net) to match arterioles and venules classifiers.

Increasing the efficiency in the classification, MoU-Net is combined with graph cuts to improve the accuracy.

The rest of the paper is organized as the following: the literature review is presented in Section 2. The proposed method for the arterioles and venules classification is presented in Section 3. The experimentation and evaluation results are presented in Section 4. Finally, Section 5 is conclusions and future works.

2 Literature review

In general, the research separated the arterioles and venules segmentation into two categories: graph-based and feature-based methods [16]. In terms of graph-based methods [17, 18], Rothaus [19] presented that the retinal blood vessels were classified into arterioles and venules by using the automated graph separation algorithm. Dashtbozorg [20] proposed an approach including three stages. Firstly, the entire vascular tree is classified based on the kind of every intersection point and secondly, each segmented vessel is assigned by one of two labels. Finally, the graph-based labeling results are combined with a set of intensity features to segment the retinal vessels into arterioles and venules.

In terms of feature-based methods [21, 22], the target pixels are classified by capturing features from fundus image patches centered on the pixels. These methods usually include various common stages: firstly, this stage is to achieve the segmented vasculature tree or to begin with it directly; secondly, the vessel centerlines are made from extracting the vasculature; finally, the vessel segments were considered as arterioles or venules depending on the features extracted from their status or the kind of centerline pixels belonging to them.

In other words, most of the recent researches apply a two steps approach for retinal artery and vein classification: firstly, the vessels are segmented from the image; secondly, these segmented vessels are classified into arteries and veins by applying purely handcrafted features in feature-based techniques or by combining edge information in graph-based techniques. As Welikala [23], arterioles and venules were classified by applying many methods including support vector machines, neural networks, profile-based and region-of-interest-based feature extractions. Zhang [24] proposed improving the arteriovenous segmentation which applied dual-model fundus images and expanded a cascade refined U-net. In the approach, each dual-modal fundus image consists of both of a regular color image and other two monochromic images which were acquired by applying two various wavelengths. These monochromic images provide more useful information on the arteriole and venule. The proposed cascade refined U-net fully used the information and obtained a high performance.

Recently, Deep Learning is more and more developing, some convolutional neural network-based approaches are proposed for combining the vessel segmentation and the artery/vein (A/V) classification. Xu [25] proposed a vessel segmentation method to classify the vessels into arteries and veins simultaneously by enhancing the fully convolutional network (FCN) architecture. AlBadawi [26] FCN architecture is built with an encoder–decoder structure for classifying the vessel into arteries and veins based on pixels. Meyer [27] A/V classification was also employed by using the FCN architecture. The method illustrated high performance on thick vessels. Badawi [28] proposed a fully convolutional neural network with an encode-decoder for grouping the retinal vessels into arterioles and venules, and it did not require any preliminary stages of vessel segmentation. A multiloss function is optimized for learning the pixel-wise and segment-wise retinal vessel labels. Hemelings [29] novel method proposed to not only segment blood vessels but also discriminate A/V simultaneously. Li [30] highly confident prediction related to the peripheral vessels was created by acquiring the structural information among vessels with the post-processing. But, both automatic vessel segmentation and A/V classification have been difficult works because of some below challenges [31]:

The performance of these methods is high for large-scale structures but low for small-scale structures because it is easy to overlook the multiscale structure of blood vessels.

The retinal image consists of a small number of pixels belonging to the vessel.

The blood vessels and choroid are quite similar. So, it is difficult to classify accurately.

In the technical development trends, Deep Learning is considered a major technology in many fields, especially in medical image processing. Evidence of this, many objects in an image were segmented by applying Deep Learning [34, 42]. In fact, convolutional neural networks (CNN) are a typical network structure in the technology. CNN is useful for segmenting objects in an image because it not only reduces the procedures of object feature extraction but also decreases feature selection to make segmentation processing more accurate. Based on the benefits of CNN, Wei Zhang [42] proposed a method to segment iris based on enhanced U-net. The method was built by fully dilated combining with U-Net. To increase the number of extracted global features the original convolution was replaced by dilated convolution.

Recently, most humans attempt to discontinue pandemics. However, these tools utilized to detect novel coronavirus are expensive and consume critical time. To overcome the limitations more and more researchers proposed some methods to classify the suspected subjects in the early stage. Singh [43] proposed an automated coronavirus screening model. The method built based on ensembling the deep transfer models to solve the sensitivity issue thanks to increasing the feature extraction.

Xinghua [44] introduces a deep translation-based change detection network for optical and synthetic aperture radar images. The deep translation maps images from one domain (optical) to another domain (synthetic aperture radar images) via a cyclic structure into the same feature space. After the deep translation, the similar characteristics become comparable to separate the unchanged pixels and changed pixels.

To solve the limitations mentioned above, this study presents the method to classify the retinal blood vessels into arteries and veins based on features by improving U-net architecture and graph cuts.

3 Artery and vein classification of retinal vessel image method

There are two vessel types in a retinal blood vessel image: arterioles and venules. Arterioles are brighter than venules because arterioles transport the blood rich in oxygen to other parts. Artery is usually next to two venules and the other way around, when near the optic disk [1]. The process of the proposed method is presented in Fig. 1. In Fig. 1, the proposed method includes four parts: pre-processing, improving U-shape network architecture, initial arterioles and venules classification, and graph cuts. The pre-processing part is detailed in Section 4.1. In the rest below, we explained the period by period of the proposed method.

Fig. 1

The process of the proposed method.

3.1 Modifying U-shape network Architecture (U-Net)

U-Net is a fully convolutional neural network which has many advances in medical image segmentation [32]. However, the original U-Net has a drawback in extracting some of the complex features. The size of venules and arterioles in the retinal blood vessel is very small and difficult to distinguish them from each other. So, we need to modify the U-Net architecture to separate venules and arterioles. The U-Net architecture includes two paths: encoding path and decoding path. We use the encoding path to extract low-level features and the decoding path to extract high-level features. Figure 2 presented the modifying U-Net architecture (MoU-Net) which we proposed to separate venules and arterioles. The architecture of MoU-Net is like U-Net, it consists of two parts: the encoder and decoder.

Fig. 2

The MoU-Net architecture.

The encoder part includes five blocks, each block consists of three convolution layers with kernel_size 3x3, stride 2×2. Next is the ReLU nonlinear activation layer and a pooling layer consisting of a convolution layer with kernel_size 2×2, stride 2×2. The decoder part includes five blocks, each block consists of three convolutional layers with kernel_size 3x3, stride 2×2, padding 1x1. Next is the ReLU nonlinear activation layer and an Unpooling layer consisting of a convolution layer with kernel_size 2×2, stride 2×2. The layer at the bottom is the middle layer between the layers in the encoder and decoder parts. It uses three convolution layers with a kernel of size 2×2, stride 2×2, and a ReLU nonlinear activation layer. Next is an Unpooling layer with kernel of size 2×2, stride 2×2. The result will be sent to a convolution layer with kernel of size 1x1, stride 1x1, and the sigmoid function used as the activation function.

We proposed the MoU-Net architecture with more depth (5 pooling, 5 unpooling) versus U-Net (4 pooling, 4 unpooling). Moreover, we have modified the Unpooling layer with kernels of size 2×2, stride 2×2 to get output consistent with the graph cuts algorithm to perform the classification. So, the possibility of feature filtering from the input image will be better than U-Net.

Suppose that, in the downsampling path from the encoder, EnC _i represents the ith feature map. In the upsampling path from the decoder, DeC _j represents the jth feature map. We used a bottleneck convolution with kernel of size 2*2 to create an auxiliary output for each feature of the above map. A channels for all the feature maps of all the scales are defined as [15]: $\begin{matrix} F = {{EnC}_{i, k}, {DeC}_{j, k}}, \\ where, i = 1, 2, . . ., m and j = 1, 2, . . ., n \end{matrix}$ (1)

3.2 Training model and initial arterioles and venules classification

To train the MoU-Net model, we used the binary Cross-Entropy loss function [32]. Assume that the dataset training consists of n data points with labels as shown in the equation (2). $\begin{matrix} {(x^{1}, y^{1}), (x^{2}, y^{2}), . . ., (x^{n}, y^{n})}, \\ where, y^{i} = 0 or 1, i = 1, 2, 3, . . . n \end{matrix}$ (2)

Suppose that the symbol θ is the set of all parameters including the convolutional, bias and softmax parameters of the MoU-Net network. For logistic regression, it is necessary to minimize the loss function in the formula (3) by θ:

$\begin{matrix} E (θ) = \frac{1}{n} {\sum_{i = 1}^{i = n} y^{i} {logF}_{θ} (x^{i}) + (1 - y^{i}) \\ \log (1 - F_{θ} (x^{i}))} \end{matrix}$ (3)

The following hyperparameters are used during the model training process, including:

Adam Optimizer: The Adam optimizer is a combination of advantages of two commonly used optimizers including AdaGrad and RMSProp [33]. It is one of the powerful and efficient optimization algorithms commonly used in Deep Learning. Adam has four hyperparameters (α, β₁, β₂, ɛ), where [4], learning rate α (stepsize), β₁, β₂ ∈ [0, 1) are exponential decay rates for the moment estimates. The hyper-parameters β₁, β₂ control the exponential decay rates of moving averages of the gradient (m_t) and the squared gradient (v_t) (the default values of β₁ and β₂ are 0.9 and 0.999). And the value ɛ selected by 10^-8 (ɛ=10^-8)

The learning rate (α) is a hyperparameter used in training neural networks. Its value is a positive number, usually between 0 and 1. The learning rate controls how quickly the model changes the weights to match the classification task. The large learning rate helps the neural network be trained faster, but it can also reduce the accuracy. Here, we ran multiple tests with different learning rates to get the best results. We see that the value α=10^-4 was chosen as the appropriate learning rate to train the model. The remaining three hyperparameters work well with the default values as above 35].

The batch size is a hyperparameter that determines the number of initial training samples to execute the initial parameters before updating the parameters inside the model. In this task, we choose a batch size = 1 to train the model.

The Epochs size establishes the number of times that the training algorithm will be run through the entire training dataset. The epoch size was set to 700 during our model training.

After the MoU-Net model is trained, we perform the initial vascular segmentation on the test dataset to obtain a probability distribution map. This map is the input to the complete arterioles and venules classification step using the graph cuts method.

3.3 Arterioles and venules classification with graph cuts

To further improve the classification results, we use graph cuts algorithm to perform the classification by a global minimization of the built energy function to solve the problems discussed above.

Based on our observations: the larger the vascular area in the image, the more accurate the results can be obtained. Therefore, we choose the image with the largest vascular area as the initial partition image. The graph cuts energy function is shown in the equation (4).

$E (f) = λ R (f_{p}) + B (f)$ (4) where, R(f_p) represents the region term and B(f) represents the boundary term. λ is the equilibrium parameter of the region term relative to the boundary term. In the experiment, we setλ=1. The boundary term of the graph cuts energy function is used to smooth the contour around the vascular region and has little effect on the classification. To determine the region term energy function R(f_p), we use the equation (5), because it gives good results in solving problems of the fuzzy boundary of blood vessels and the distribution of unevenness of luminance [36].

$R_{p_{ij \in P}} (f_{p_{ij}}) = {\begin{matrix} 1 - \frac{1}{1 + \exp (- \frac{p_{ij} - μ}{γ})}, p_{ij} = obj \\ \frac{1}{1 + \exp (- \frac{p_{ij} - μ}{γ})}, p_{ij} = bkg \end{matrix} .$ (5) where, p_ij represents the probability of each pixel in the probability graph P from the output of MoU-Net. The parameters μ and γ were set as μ=0.4 and γ=0.05 in the proposed method, respectively.

The boundary term B(f) is determined by using a classical boundary function [6] shown by the formula (6).

${B (f}_{{p, q}}) \propto \exp (- \frac{{(p - q)}^{2}}{{2 σ}^{2}}) (\frac{1}{dist (p, q})$ (6) where, p and q represent two adjacent pixels in the probability distribution map. σ presented the standard deviation of the difference in intensity of neighboring pixels. Dist(p, q) is the Euclidean distance between pixels p and q.

The position and size of blood vessels in adjacent pixels do not change much. So, these kinds of context information are used to reconstruct the energy function for subsequent blood vessels. Therefore, a new energy function is expressed in the form of formula (7).

$E (f) = λ (R (fP) + R (fI)) . F (f) + B (f)$ (7) where, R(f_P) and R(f_I) represent the region term probability function and the region energy function of the selected blood vessel, respectively. The region term probability function R(f_P) is determined by the probability distribution map from the output of the MoU-Net. The region energy function of the selected vessel R(f_I) is determined by its gray scale information.

To improve the classifier performance and avoid over- or under-partitioning of the vascular region, the vascular grayscale information is added to the definition of the rion term in the equation (8), defined as follows.

$R_{{(f}_{I})} = {\begin{matrix} 1 - \exp (- \frac{{(I}_{i} - \bar{I_{R}})^{2}}{σ_{R}^{2}}), I_{i} = “ {obj}^{″} \\ \exp (- \frac{{(I}_{i} - \bar{I_{R}})^{2}}{σ_{R}^{2}}), I_{i} = “ {bkg}^{″} \end{matrix}$ (8)

where, I_R and σ_R represent the mean brightness and standard deviation of all pixels in the vascular container, respectively.

F(f) represented a position constraint function. It is mainly established based on the vascular position of adjacent pixels which will not change appreciably. It can be defined in the equation (9).

$F (f_{i}) = {\begin{matrix} 1 - \exp (- μ D (I_{i})), I_{i} = “ {obj}^{″} \\ \exp (- μ D (I_{i})), I_{i} = “ {bkg}^{″} \end{matrix}$ (9)

where, D(I_i) is the shortest Euclidean distance between pixel I_i and the previous vascular region. For each pixel I_i in the considered vascular region, D(I_i)=0. And μ controls the search range of the vascular region. This can cause over- or under-partitioning of the vascular area when the μ value is too large or too small. Therefore, the value of μ is determined by many tests based on previous work proposed in [37], the value of μ is possible in the range [0.02, 0.05]. Therefore, we set μ=0.04 in our experiments.

4 Experimentation and evaluation results

4.1 Materials and datasets

Our experiments are developed in Python, with the computer of Intel core i7, 2.9 GHz CPU, 16 GB DDR2 memory. We use publicly available datasets for our experimentation as: RITE (Retinal Images vessel Tree Extraction) dataset [14] and AVRDB (Annotated Dataset for Vessel Segmentation and Calculation of Arteriovenous Ratio) dataset [39].

The RITE database [14] enables comparative studies on the classification of arterioles and venules on retinal fundus images, which is established based on the public available DRIVE database. The dataset contains 40 color retinal fundus images in which every image has the annotations of venules, arterioles, intersections of venules and arterioles by doctors (expert ophthalmologists) [14]. In this dataset, we randomly selected a subset of 30 images for training and the rest 10 images to evaluate the results. The images are cropped to the size of 512*512. And these images augmented the dataset by scaling, random clipping, etc. So, the size of the training dataset is expanded to 2500 images. Each image has been defined under png format. Some of the images of this dataset are presented in Fig. 3. The labeled images in Fig. 3 consist of vessels with arterioles (red color), venules (blur color). The overlapping of arterioles and venules are labelled in green; the vessels which are uncertain are labelled in white [14].

Fig. 3

Some images in RITE dataset. a. Original image b. Annotated image.

The AVRDB dataset consists of 100 fundus retinal images which are annotated with the help of expert ophthalmologists from Armed Forces Institute of Ophthalmology. The vascular network of images in this dataset is classified into arteriolar and venular patterns. These images having the size of 1504 x 1000 contain retinal arteries and veins [39]. In this dataset, we randomly selected a subset of 70 images for training and the rest 30 images to evaluate the results. The images are also cropped to the size of 512*512. And these images augmented the dataset by scaling, random clipping, etc. So, the size of the training dataset is also expanded to 1200 images. Each image has been defined under jpg format. Some of the images of this dataset are presented in Fig. 4. The labeled images in Fig. 4 consist of vessels with arterioles (red color), venules (blur color) [39].

Fig. 4

Some images in AVRDB dataset. a. Original image b. Annotated image.

4.2 Evaluation metrics and experimental results

To evaluate the results of the venules and arterioles classification process, we compare the results of the proposed method to the results of the expert symptom assessment (ground-truth) provided with the above dataset for each retinal image. Now, we used true positive rate (TPR) for arterioles and venules, and accuracy to evaluate the metrics of the results of the proposed method. TPR_arteriols is the ratio of correctly classified artery centerline pixels to all centerline pixels. And TPR_venules is the ratio of correctly classified vein centerline pixels to all centerline pixels. Finally, the accuracy is the ratio of all correctly detected centerline pixels to all centerline pixels with a ground truth label [21]. They are computed as in the equations (10), (11) and (12).

${TPR}_{arterioles} = \frac{{TP}_{arterioles}}{{TP}_{arterioles} + {FP}_{arterioles}}$ (10)

${TPR}_{venules} = \frac{{TP}_{venules}}{{TP}_{venules} + {FP}_{venules}}$ (11)

$\begin{matrix} Accuracy \\ = \frac{{TP}_{arterioles} + {TP}_{venules}}{{TP}_{arterioles} + {FP}_{arterioles} + {TP}_{venules} + {FP}_{venules}} \end{matrix}$ (12) where, TP is true positive and FP is false positive, as described in [11]. The experimentations are implemented with all the images in the above.

We test all images over these two datasets. As an illustration here, with each dataset, we only present two case studies such as in Figs. 5 and 6 for RITE dataset, and Figs. 7 8 for AVRDB dataset.

Fig. 5

Some images results of the proposed method from the RITE test dataset.

Fig. 6

Partially extracted image from segmented image in the RITE test dataset.

Fig. 7

Some images results of the proposed method from the AVRDB test dataset.

Fig. 8

Partially extracted image from segmented image in AVRDB test dataset.

Figures 5 7 presented some images results of the proposed method from the RITE test dataset and AVRDB test dataset, respectively. Figures 6 8 presented the image partially extracted from the segmented image, respectively.

From the above experiments and others, we can see that the proposed method works well and gives reliability results. Table 1 and Table 2 presented the results of the evaluation of the proposed method on the above dataset.

Table 1

Results on individual images of the proposed method from the RITE test dataset

Images number	TPR_arterioles	TPR_venules	Accuracy
1	0.980	0.978	0.975
2	0.955	0.972	0.968
3	0.980	0.976	0.979
4	0.915	0.979	0.974
5	0.908	0.971	0.979
6	0.914	0.979	0.977
7	0.902	0.989	0.978
8	0.925	0.975	0.980
9	0.915	0.981	0.976
10	0.935	0.983	0.974
Overall	0.932	0.978	0.976

Table 2

Results on individual images of the proposed method from the AVRDB test dataset

Images number	TPR_arterioles	TPR_venules	Accuracy
1	0.931	0.962	0.957
2	0.906	0.969	0.967
3	0.978	0.972	0.971
4	0.915	0.979	0.974
5	0.917	0.964	0.961
6	0.927	0.977	0.972
7	0.913	0.981	0.972
8	0.931	0.978	0.974
9	0.917	0.978	0.971
10	0.924	0.981	0.976
11	0.901	0.963	0.947
12	0.925	0.972	0.969
13	0.931	0.975	0.973
14	0.925	0.989	0.967
15	0.928	0.987	0.971
16	0.919	0.979	0.973
17	0.913	0.981	0.972
18	0.921	0.988	0.974
19	0.917	0.968	0.964
20	0.929	0.961	0.956
21	0.909	0.972	0.965
22	0.926	0.975	0.973
23	0.903	0.983	0.979
24	0.927	0.969	0.964
25	0.938	0.987	0.976
26	0.933	0.972	0.963
27	0.921	0.983	0.977
28	0.908	0.968	0.963
29	0.946	0.979	0.975
30	0.919	0.979	0.967
Overall	0.923	0.976	0.969

In Table 1 and Table 2, the proposed method also gives good results. To objectively evaluate the effectiveness of the proposed method, the results of the proposed method are compared to those of Xiayu method [21], AlBadawi method [26], Galdran method [40] and Meng method [38] in RITE dataset by the evaluation as accuracy criteria. The results of the proposed method are also compared to those of Xiayu method [21], AlBadawi method [26] and Meng method [38] in the AVRDB test dataset by the evaluation as accuracy criteria.

Figure 9 presented the result of a test case of the proposed method with Xiayu method [21], AlBadawi method [26], Galdran method [40], and Meng method [38] in RITE dataset. In this case, the result of the proposed method is better than the other methods. Table 3 presented the results of arterioles and venules classification results which compared to the other methods in the RITE dataset.

Fig. 9

A case of the proposed method results compared with other methods in the RITE dataset. (a) Original image (b) Result of expert ophthalmologists (c) Result of Xiayu method [21] (d) Result of AlBadawi method [26] (e) Result of Galdran method [40] (f) Result of Meng method [38] (g) Result of proposed method.

Table 3

The arterioles and venules classification results compared with the other methods in the RITE dataset

Methods	Year	Accuracy (%) (with RITE dataset)
Xiayu method [21]	2017	92.4
AlBadawi method [26]	2018	93.7
Galdran method [40]	2019	89.0
Meng method [38]	2020	96.5
Proposed method	2021	97.6

From Table 3, with RITE dataset, the accuracy of the proposed method is 97.6%while the accuracy of Xiayu method, AlBadawi method, Galdran method, and Meng method are 92.4%, 93.7%, 89.0%and 96.5%, respectively.

Figure 10 presented the result of a test case of the proposed method with Xiayu method [21], AlBadawi method [26] and Meng method [38] in the AVRDB test dataset. In this case, the result of the proposed method is also better than the other methods. Table 4 presented the results of arterioles and venules classification results which compared to the other methods in AVRDB dataset.

Fig. 10

A case of the proposed method result compared with other methods in AVRDB dataset. (a) Original image (b) Result of expert ophthalmologists (c) Result of Xiayu method [21] (d) Result of AlBadawi method [26] (e) Result of Meng method [38] (f) Result of proposed method.

Table 4

The arterioles and venules classification results compared with the other methods in AVRDB dataset

Methods	Year	Accuracy (%) (with AVRDB dataset)
Xiayu method [21]	2017	91.9
AlBadawi method [26]	2018	93.1
Meng method [38]	2020	95.9
Proposed method	2021	96.9

From Table 4, with AVRDB dataset, the accuracy of the proposed method is 96.9%while the accuracy of Xiayu method, AlBadawi method and Meng method are 91.9%, 93.1%and 95.9%, respectively. From Table 3 and Table 4, we see that the results of the proposed method are better than the other methods over the RITE dataset and AVRDB dataset.

As presented in Section 3.1 and Section 3.3, the MoU-Net architecture with more depth (5 pooling, 5 unpooling) versus U-Net (4 pooling, 4 unpooling) makes the feature filtering from the input image better than the U-Net. Moreover, to further improve the classification results, we also use a graph cuts algorithm to perform the classification by the global minimization of the built energy function. While Xiayu method [21] evaluated on the correctly detected vessels, AlBadawi method [26] evaluated on known vessel centerline locations, Galdran method [40] evaluated on providing pixelwise uncertainty estimates, Meng method [38] evaluated on the whole detected vessels, the proposed method is evaluated based on improving the U-Net architecture and graph cuts optimization. These are the reasons why the proposed method gives the better results versus the other methods.

5 Conclusions and future works

The central retinal artery and its branches supply blood to the inner retina. The inner retina is made up of the retinal nerve fiber layer, the ganglion cell layer, and the inner reticular layer. The central retinal artery arises from the ophthalmic artery, which is the first branch of the internal carotid artery in most people. Vascular manifestations in the retina indirectly reflect vascular changes and damage in organs such as the heart, kidneys, and brain because of the similar vascular structure in these organs. Therefore, classifying arterioles and venules in retinal images is a necessary first step as a premise for the diagnosis of diseases such as hypertensive retinopathy, diabetic retinopathy, etc. In this paper, we proposed the method for classifying the retinal vessels into the arterioles and venules based on improving the U-Net architecture and graph cuts with four steps as presented above. The results of the proposed method are better than the other methods on RITE dataset and AVRDB dataset. However, we have not compared the execution times between the methods and on many other datasets. In the future, we will continue to evaluate the execution time, testing on more datasets. Furthermore, we will detect abnormalities in blood vessels to assist in diagnosis.

Footnotes

Acknowledgments

This research is funded by Vietnam National University Ho Chi Minh City (VNU-HCM) under grant number B2019-20-05. We acknowledge the support of time and facilities from Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for this study.

References

Kondermann

, Kondermann

and Yan

, Blood vessel classification into arteries and veins in retinal images,}, 47, Proc. SPIE 6512, Medical Imaging 2007: Image Processing 6512 (2007). DOI:10.1117/12.708469.

Qureshi

T.A.

, Habib

, Hunter

and Al-Diri

, A manually-labeled, artery/vein classified benchmark for the DRIVE dataset, Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems, (2013), 485–488.DOI:10.1109/CBMS.2013.6627847.

Wong

T.Y.

, Amirul

F.M.

, Klein

B.E.K.

, Cotch

M.F.

, Castro

, Sharrett

A.R.

and Shahar

, Retinal vascular caliber, cardiovascular risk factors, and inflammation: the multi-ethnic study of atherosclerosis, Investigative Ophthalmology &Visual Science 47 (2006), 2341–2350. DOI:10.1167/iovs.05-1539.

Wong

T.Y.

, Klein

, Sharrett

A.R.

, et al., Retinal arteriolar narrowing and risk of coronary heart disease in men and women: The Atherosclerosis Risk in Communities Study, JAMA 287(9) (2002), 1153–1159. DOI: jama.287.9.1153.

Nguyen

T.T.

and Wong

T.Y.

, Retinal vascular changes and diabetic retinopathy, Current Diabetes Rep 9 (2009), 277–283.

Joshi

G.D.

, Sivaswamy

and Krishnadas

S.R.

, Optic disk and cup segmentation from monocular color retinal images for glaucoma assessment, in IEEE Transactions on Medical Imaging, 30(6) (2011), 1192–1205. DOI:10.1109/TMI.2011.2106509.

Lim

L.S.

, Mitchell

, Seddon

J.M.

, Holz

F.G.

and Wong

T.Y.

, Agerelated macular degeneration, The Lancet 379(9827) (2012), 1728–1738.

Flamme

, Konieczka

, Bruno

R.M.

, Virdis

, Flammer

A.J.

and Taddei

, The eye and the heart, European Heart Journal 34(17) (2013), 1270–1278.

McGeechan

, Mbiostat , Liew

, et al., Meta-analysis: Retinal vessel caliber and risk for coronary heart disease, Annals of Internal Medicine 151(6) (2009), 404–413.

10.

Grosso

, Veglio

, Porta

, Grignolo

and Wong

, Hypertensive retinopathy revisited: Some answers, more questions, British Journal of Ophthalmology 89(12) (2005), 1646–1654.

11.

Kamisawa

, Egawa

, Nakajima

, Tsuruta

, Okamoto

and Kamata

, Clinical difficulties in the differentiation of autoimmune pancreatitis and pancreatic carcinoma, The American Journal of Gastroenterology 98(12) (2003), 2694–2699.

12.

Dashtbozorg

, Mendonça

A.M.

and Campilho

, An automatic method for the estimation of Arteriolar-to-Venular Ratio in retinal images, Proc 26th IEEE Int Symp Comput Based Med Syst, (2013), 512–513.

13.

Ikram

M.K.

, Jong

F.J.

, Vingerling

J.R.

, et al., Are retinal arteriolar or venular diameters associated with markers for cardiovascular disorders the rotterdam study, Investigative Ophthalmology Vis Sci 45(7) (2004), 2129–2134.

14.

https://medicine.uiowa.edu/eye/rite-dataset (last accessed 03rd Jan 2021)

15.

Boykov

and Kolmogorov

, An experimental comparison of min cut/max-flow algorithms for energy minimization in vision, IEEE Trans on PAMI 26(9) (2004), 1124–1137.

16.

Miri

, Amini

, Rabbani

and Kafieh

, Acomprehensive study of retinal vessel classificationmethods in fundus images, Journal of Medical Signals and Sensors 7(2) (2017), 59–70.

17.

Pellegrini

, Robertson

, MacGillivray

, van Hemert

, Houston

and Trucco

, A Graph Cut Approach to Artery/VeinClassification in Ultra-Widefield Scanning Laser Ophthalmoscopy, in, IEEE Transactions on Medical Imaging 37(2) (2018), 516–526. DOI:10.1109/TMI.2017.2762963.

18.

Srinidhi

C.L.

, Aparna

and Rajan

, Automated method for retinal artery/vein separation via graph search metaheuristic approach, IEEE Trans Image Process 28(6) (2019), 2705–2718. DOI:10.1109/TIP.2018.2889534.

19.

Rothaus

, Jiang

and Rhiem

, Separation of the retinal vascular graph in arteries and veins based upon structural knowledge, Image Vis Comput 27(7) (2009), 864–875.

20.

Dashtbozorg

, Mendonça

A.M.

and Campilho

A.J.

, An automatic graph-based approach for artery/veinclassi_cation in retinal images, in IEEE Trans Image Process 23(3) (2014), 1073–1083. DOI:10.1109/TIP.2013.2263809.

21.

, Ding

, Abràmoff

M.D.

and Cao

, An improved arteriovenous classification method for the earlydiagnostics of various diseases in retinal image, Computer Methods and Programs in Biomedicine 141(2017), 3–9.

22.

Huang

, Dashtbozorg

, Tan

and Romeny

B.M.H.

, Retinal artery/vein classification using genetic-search feature selection, Comput Methods Programs Biomed 161 (2018), 197–207. DOI: 0.1016/j.cmpb.2018.04.016.

23.

Welikala

R.A.

, Foster

P.J.

, Whincup

P.H.

, Rudnicka

A.R.

, Owen

C.G.

, Strachan

D.P.

and Barman

S.A.

, Automated arteriole and venule classi_cation using Deep Learning for retinal images from the UK Biobank cohort, Comput Biol Med 90 (2017), 23–32.

24.

Zhang

, et al., Simultaneous arteriole and venule segmentation of dual-modal fundus images using a multi-task cascade network, IEEE Access 7 (2019), 57561–57573. DOI:10.1109/ACCESS.2019.2914319.

25.

, Wang

, Lv

, Gao

, Li

, Tian

, et al., Simultaneous arteriole and venule segmentation with domain-specific loss function on a new public database, Biomedical Optics Express 9(7) (2018), 3153–3166.

26.

AlBadawi

and Fraz

M.M.

, Arterioles and venules classification in retinal images using fully convolutional deep neural network, Lecture Notes in Computer Science 10882 (2018), 659–668. DOI:10.1007/978-3-319-93000-8_75.

27.

Meyer

M.I.

, Galdran

, Costa

, Mendonça

A.M.

and Campilho

, Deep convolutional artery/veinclassification of retinal vessels, In: Campilho A., Karray F., ter Haar Romeny B. (eds) Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science, Springer, 10882 (2018), 622–630. DOI:10.1007/978-3-319-93000-8_71.

28.

Badawi

S.A.

and Fraz

M.M.

, Multiloss function based deep convolutional neural network for segmentation of retinal vasculature into arterioles and venules, BioMed Research International 2019 (2019), 1–17. DOI:10.1155/2019/4747230.

29.

Hemelings

, Elen

, Stalmans

, Van Keer

, De Boever

and Blaschko

M.B.

, Artery-vein segmentation in fundus images using a fully convolutional network, Comput Med Imaging Graph 76 (2019), 1–12. DOI:10.1016/j.compmedimag.2019.05.004.

30.

, Verma

, Nakashima

, Kawasaki

and Nagahara

, Joint learning of vessel segmentation and artery/vein classification with post-processing, Proceedings of the International Conference on Medical Imaging with Deep Learning 121 (2020), 1–14.

31.

, Wang

, Cao

, Wu

, Jonas

J.B.

, Wang

Y.X.

and Zhang

, Automatic artery/vein classification using a vessel-constraint network formulticenter fundus images, , Front Cell Dev Biol 9 (2021), 1–15. DOI:10.3389/fcell.2021.659941.

32.

Ronneberger

, Fischer

and Brox

, U-Net: convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention, (2015), 234–241.

33.

Duchi

, Hazan

and Singer

, Adaptive subgradient methods for on-line learning and stochastic optimization, The Journal of Machine Learning Research, (2011), 2121–2159.

34.

, Lu

, Zhang

, Zhao

, Yu

, Gao

, Cui

, Wu

and Zhou

, Automatic lung nodule detection using multi-scale dotnodule-enhancement _lter and weighted support vector machines in chest computed tomography, PLoS ONE 14(1) (2019)–10.1371/journal.pone.0210551.

35.

Kingma

D.P.

and Ba

, Adam: a method for stochastic optimization, 3rd International Conference on Learning Representations, San Diego, (2015), 1–15.

36.

Song

, Bai

, Han

, Bhatia

, Sun

, Rockey

and Wu

, Optimal co-segmentation of tumor in PET-CT images with context information, IEEE Transactions on Medical Imaging 32(9) (2013), 1685–1697. DOI:10.1109/TMI.2013.2263388.

37.

Liao

, Zhao

Y.Q.

, Wang

, Zeng

Y.Z.

, Yang

, Shih

F.Y.

and Zou

B.J.

, Efficient liver segmentation in CT images based on graph cuts and bottleneck detection, Physical Medica 32(11) (2016), 1383–1396.

38.

, Zhao

, Zhang

, He

, Zhou

, Jia

, She

, Li

and Zhang

, Automated classification ofarterioles and venules for retina fundus images using dual deeply-supervised network, International Workshop on Multiscale Multimodal Medical Imaging, Lecture Notes in Computer Science 11977 (2020), 59–67. DOI:10.1007/978-3-030-37969-8_8.

39.

http://biomisa.org/index.php/dataset-for-hypertensive-retinopathy/ (last accessed 03rd Jan 2021).

40.

Galdran

, Meyer

, Costa

, MendonÇa and Campilho

, Uncertainty-aware artery/vein classification on retinal images. IEEE 16th International Symposium on Biomedical Imaging, (2019), 556–560. doi: 10.1109/ISBI.2019.8759380.

41.

Morano

, Hervella

, Novo

and Rouco

, Simultaneous segmentation and classification of the retinal arteries and veins from color fundus images, Artificial Intelligence in Medicine 118 (2021), 1–17.

42.

Zhang

, Lu

, Gu

, Liu

, Meng

and Li

, A Robust Iris Segmentation Scheme Based on Improved U-Net, in IEEE Access 7 (2019), 85082–85089. DOI:10.1109/ACCESS.2019.2924464.

43.

Singh

, Kumar

and Kaur

, Densely connected convolutional networks-based COVID-19 screening model, , Appl Intell 51 (2021), 3044–3051. DOI:10.1007/s10489-020-02149-6.

44.

, Du

, Huang

and Tan

, A deep translation (GAN) based change detection network for optical and SARremote sensing images, ISPRS Journal of Photogrammetry and Remote Sensing 179 (2021), 14–34.