Image classification of vaginal microecology detection based on gabor texture and LSTM model

Abstract

BACKGROUND:

Gynecological diseases threaten women’s health, and vaginal microecological testing is a common method for detecting gynecological diseases. Efficient and accurate microecological testing methods have always been the goal pursued by gynecologists.

OBJECTIVE:

In order to automatically identify different types of microbial images in vaginal micromorphology detection, this paper proposes a vaginal microecological image recognition method based on Gabor texture analysis combined with long and short-term memory network (LSTM) model.

METHOD:

Firstly, we denoise the microecological morphological im-ages, which selects the area of interest and sets the label of the microorganism according to the doctors label. Secondly, texture analysis is carried out for the region of interest, which uses Gabor filters with 8 directions and 5 scales to filter the region of interest to extract the texture features on the image. Comparing the differences between different microbial image features, and screening suitable features to reduce the number of features. Then, we design an LSTM model to analyze the relationship of image features in different categories of microorganisms. Finally, we use the full connection layer and Softmax function to realize the automatic recognition of different microbial images.

RESULTS:

The experimental results show that the image classification accuracy of 8 common microorganisms is 81.26%.

CONCLUSION:

Texture analysis combined with LSTM network strategy can identify different kinds of vaginal micro ecological images. Gabor-LSTM model has better classification effect on imbalanced data sets.

Keywords

Microecological detection Gabor LSTM feature extraction image classification

1. Introduction

Gynecological diseases seriously threaten women’s health. Vaginal microecological detection is a common method for gynecological diseases. This method can effectively evaluate women’s health status and guide doctors to treat gynecological diseases [1]. The detection technology of gynecological vaginal microecology includes morphological detection and functional detection. Microecological functional detection mainly relies on equipment and reagents, while morphological detection mainly relies on artificial recognition. There is a certain subjectivity in artificial recognition. Therefore, morphological detection has become a shortcoming that restricts the further promotion of microecological detection technology [2]. How to carry out vaginal morphological examination without interference from external factors has become the focus of vaginal microecological examination.

There are many researches on the detection of microecological morphology. Bai et al. [3] found that there was a close relationship between intestinal microecological environment and gastrointestinal tumors. Therefore, intestinal microecological detection can detect tumors in time and play a key role in clinical diagnosis. Yan et al. [4] analyzed the microecology of nasopharynx. It was found that the microbial species in nasopharyngeal position was more abundant, and this study played a guiding role in the diagnosis of upper respiratory diseases. In the field of vaginal microecology detection, Brotman et al. [5] found that the imbalance of vaginal microecology was significantly associated with HPV infection. Clinicians can determine whether they have HPV infection based on vaginal microecological testing. It can be seen that human microecological detection plays a very important role in disease diagnosis.

Clinical examination based on microecological morphology has obvious advantages, which can detect diseases early and has high clinical value, attracting many people to conduct research [6]. In mainland China alone, there are a number of first-class hospitals such as Beijing Tsinghua Changgung Hospital and Peking University First Hospital for research [7]. However, data collection is difficult due to the large number of microbial categories and uneven distribution. And most of the data are unbalanced, which brings great challenges to microbial classification and identification. Many researchers are also trying to solve the above problems.

The main purpose of this paper is to improve the accuracy in the detection of vaginal microecologic morphology. According to the image data of microecology, the distribution of each colony is very uneven. The imbalance of data sets brings great challenges to the construction of classification model. We propose a classification model to classify 8 microecological images. Our model is composed of Gabor texture feature and LSTM classification model. There are differences in the texture of different microbe images. We use Gabor filter to obtain image texture features and select the features. LSTM method was used to classify the screened features. The classification accuracy, sensitivity, specificity and other indicators are compared, and the parameters are constantly adjusted to build the best classification model. The experimental results show that the proposed model is very effective in distinguishing different types of microecological images, with a classification accuracy of 81.26%. Although the data is very uneven, our model can identify eight microbial characteristics. It shows that our strategy has certain advantages in the identification of micro ecological colonies.

2. Related work

Machine learning and pattern recognition technology have developed rapidly and have been applied in many fields. In particular, in the field of image processing, image feature extraction combined with machine learning algorithm can be used to complete image classification efficiently. There are many methods of feature analysis, including principal component analysis [8], linear discriminant analysis [9], texture analysis [10] and so on. These methods can extract the features of different images and assist the computer to process different types of images.

2.1 Texture analysis of medical images

Texture analysis is widely used in medical image processing. For example, Kondo et al. [11] analyzed the ultrasonic images of liver cancer and used support vector machine model to classify the features. The experimental results show that different types of liver cancer can be classified by image features with a classification accuracy of 84.4%. Liu et al. [12] established a support vector machine classification model which is based on the Gabor wavelet texture analysis to predict the primary central nervous system lymphoma (PCNSL) and glioblastoma multiforme (GBM). The result shows that the model can distinguish different diagnosis categories of tumor images. It shows that the Gabor analysis of MRI can distinguish different types of tumors. Their work shows that texture analysis combined with machine learning algorithms is effective in image classification. In recent years, deep learning has developed rapidly, and many researchers have tried to use neural network models to analyze medical images. Kooi et al. [13] applied a convolutional neural network to the recognition of malignant lesions of breast cancer. This method can achieve better recognition results at low sensitivity in comparison with traditional computer-aided methods, and the accuracy rate of this method will be higher at high sensitivity. This shows that neural network is superior to traditional image recognition methods and deep learning algorithm has great advantages in image processing.

2.2 Medical examination image processing and recognition

Image processing technology is widely used in medical examination image processing. Many researchers have tried to use machine learning algorithms to process microscopic images. Spanho et al. [14] extracted features from the images on the BreaKHis breast pathology data set, and then used a machine learning model to classify the images with an accuracy of 85%. Experiments have shown that this method can distinguish between different pathological findings of breast diseases. However, this method only classifies 6 image features, and the results cannot be well extended to other pathological images. Sharma et al. [15] used a convolutional neural network to classify three grades of malignancy in a small sample of gastric cancer data set. Good results were obtained in the experiment. However, this method only uses convolutional network to extract image features, which is time-consuming and has some limitations. Coudray et al. [16] used an architecture based on Inception network to distinguish pathological images of lung cancer. The experiment worked very well. However, this method requires a lot of computing power and takes a long time. All the above methods have their own advantages and disadvantages [17], so the model needs to be further improved.

It can be seen that the image processing method has been widely used in the recognition of medical examination images and plays a very key role in the diagnosis of many diseases. Because the neural network model can automatically extract image variation features and has high classification accuracy, this model is favored by many researchers. The deep learning model requires a large number of samples to be learned, which requires too much data. Therefore, the advantages of using neural network model to process medical small-sample images are not obvious. There are a lot of studies on microecological morphological image detection, but most of them have problems such as less data and more kinds of microorganisms. How to construct an efficient and accurate classification model is one of the hot issues in current research.

3. Data and methods

In this paper, we propose an image classification model to classify micro ecological images. In this section, we mainly introduce the data set and the model framework.

3.1 Data acquisition

With the help of doctors from Tianjin Cancer Clinical Research Center, we collected 1800 micromorphological images of vaginal microecology with a size of 1200 $\times$ 1800. The microbes on each image were tagged by a professional physician. Including microecological species such as spores, hyphae and other 8 kinds of microbes. The original image and the labeled image are shown in Fig. 1a and b.

Figure 1.

Images of vaginal microecologic detection, (a) Original image, (b) Image after labeling.

As shown in the Fig. 1, a is the original image data, and b is the marked data. The yellow rectangle in Fig. 1b is the name of the marked microorganism. We tagged seven identified microbes. Some of the microbes that we don’t recognize, we’ve tagged them and described them as “other”.

3.2 Model framework

Our model consists of data preprocessing, texture analysis, feature selection, LSTM network and full connection classification. The model structure is shown in Fig. 2.

Figure 2.

Model structure diagram. Our method consists of data preprocessing, feature analysis, feature classification, LSTM network construction, full connection classification, etc.

Firstly, we denoise the original image, obtain the region of interest of the image, and set the region of interest label. Secondly, 8 directions and 5 scales of Gabor wavelet are used to filter the region of interest in order to obtain the texture features of the region of interest. According to the differences between features, feature selection is carried out to reduce the feature dimension. Then, we use LSTM network learning to screen the links between features to speed up model convergence and solve the gradient disappearance problem with lower computing cost. Finally, the automatic classification of different microbial images was completed by using the full connection layer and Softmax functions.

3.3 Image preprocessing

Image preprocessing is an important step before image texture feature. The main purpose of preprocessing is to complete image denoising, obtain the region of interest, and facilitate the construction of classification model. Because the microecological image is the magnified image of the microscope, there will be noise under the influence of the microscope lens and staining reagent. Therefore, the original image should be denoised. Different image noise corresponds to diffeent image denoising methods. Affected by the microscope lens and stain, the original image has many noise points. Our main objective is to remove noise points from the image. Median filter is suitable for denoising the original image.

Median filtering is a nonlinear signal processing technique based on the sorting statistics theory to suppress noise. The basic principle of median filtering is to replace the median value of each point value in the adjacent domain of an image point, so that the surrounding pixel value is close to the true value, thus eliminating the denoising method of isolated noise points. In this method, a two-dimensional sliding template $W$ is used to sort the pixel values of the image in $W$ according to size, and to generate a monotonic ascending (or descending) two-dimensional data sequence. The denoising formula is as follows.

$\displaystyle g(x,y)=\textit{med}\{f(x-k,y-j),(k,j\in W)\}$ (1)

Figure 3.

Median filter denoising, (a) Original image, (b) Denoised image.

Where, $W$ is a two-dimensional template, $f$ is the original image, and $g$ is the processed image. The image denoised by median filtering is shown in Fig. 3. As shown in the Fig. 3, a is the original image, and b is the image after median filtering and denoising. It can be seen from the figure that the fine points in b are reduced compared with those in a. After the image denoising task is completed, the region of interest is selected according to the doctor’s mark. We chose an ROI size of 56 $\times$ 56 and built the data set based on the previous tags. Since there are multiple ROIs on a single image, each ROI corresponds to a different type of microbial tag. We collected a total of 3,416 images of microorganisms. The images and the number of microorganisms are shown in Table 1. It can be seen from the table that there are great differences in the number of microorganisms in different categories. Spores and hyphae account for the most. Therefore, it is very difficult to classify images with an appropriate classification model. In order to solve the problem of data imbalance, we analyze the features of each type of image. We further subdivide the image by comparing the texture differences of each type of image. This paper finds out the differences between different types of images by extracting texture features of images. We try to use these differences to solve the classification of unbalanced image data.

Table 1

Quantity and proportion of various microorganisms

Type	Count	Percent
Spore	1375	40.25%
Hyphae	1375	40.25%
Agglomerated spores	142	4.16%
Blastospore	376	11.01%
Nucleus	47	1.38%
Cells	15	0.44%
Agglomerated hyphae	52	1.52%
Impurity	34	1.00%

Figure 4.

Texture analysis process diagram.

4. Image texture analysis

This section introduces three parts: image texture analysis, feature extraction and feature selection. The process of image texture analysis is shown in Fig. 4. Firstly, Gabor filter is used to analyze the features of 8 kinds of microbial images. Secondly, the filtered image is sampled to obtain the feature sequence. Then, according to the rule that the distance between different types of features is greater than the distance between the same type of features, the features are selected. Finally, three common classifiers are selected to classify the features, and the optimal number of features is found through experimental comparison.

4.1 Image texture analysis based on Gabor wavelet

Image texture is one of the most important features to describe and distinguish different objects. The microecomorphology image is the image under high power microscope, so we use the spectral method to extract the texture features of the image. We mainly use Gabor wavelet to analyze the spectral feature of images. Gabor texture feature is the shortened form of texture feature analyzed with multi-resolution filter based on Gabor wavelet [18]. Gabor wavelets have tunable orientation and radial scales bandwidths, tunable center scales, allowing them to optimally achieve joint resolution in the spatial and frequency domains. Due to the Gabor wavelets capture the local structure corresponding to spatial frequency (scales), spatial localization, and orientation selectivity, they are widely applied in many research areas, such as texture analysis and image segmentation.

The impulse response of Gabor filter can be defined as a cosine wave multiplied by a Gauss function [12]. The sharpness of the filter is controlled through the major axis and minor axis, which is perpendicular to the wave. The filter can be defined as:

$\displaystyle g(x,y)=\left[\frac{1}{2\pi\sigma_{x}\sigma_{y}}\right]\exp\left[% -\frac{1}{2}\left(\frac{x^{2}}{\sigma_{2}^{x}}+\frac{1}{\sigma_{2}^{y}}\right)% +2\pi_{j}\omega_{x}\right]$ (2)

Where, $\omega$ refers to the multiple modulation frequency of Gaussian function. Its Fourier transform, $G(u,v)$ , can be expressed as:

$\displaystyle G(u,v)=\exp\left\{-\frac{1}{2}\left[\frac{(u-\omega)^{2}}{\sigma% _{2}^{u}}+\frac{v^{2}}{\sigma_{2}^{v}}\right]\right\}$ (3)

Where, $\sigma_{u}=1/2\pi\sigma_{x}$ , $\sigma_{v}=1/2\pi\sigma_{y}$ . With $g(x,y)$ as the mother wavelet, a set of asymptotically self-similar filters will be obtained through appropriate scale transform and rotation transformation of $g(x,y)$ , which is referred to as Gabor wavelet. $g_{mn}=a^{-m}g({x}^{\prime},{y}^{\prime})$ , $a>l$ , $m,n\in Z$ There ${x}^{\prime}=a^{-m}(x\cos\theta+y\sin\theta)$ ; ${y}^{\prime}=a^{-m}(x\sin\theta+y\cos\theta)$ ; $\theta=n\pi/k$ ; and $a^{-m}$ are scale factors; and $k$ is the number of directions. A set of filter banks with different directions and scales will be obtained by changing the values of $m$ and $n$ .

Figure 5.

Gabor wavelet function, $a=\sqrt{2}$ ; $k=$ 8; $n=$ 0, 1, 2, 3, 4; $m=$ 0, 1, 2, 37.

In the process of generating Gabor filter banks, the selection of direction and scale is crucial. As shown in Fig. 5, Gabor wavelet functions with five scales and eight directions are selected.

Since microorganisms have no color, in order to facilitate observation in microscopic images, coloring agents are often used to stain the microorganisms for easy observation. In order to reduce the influence of dye color on microbial classification, we convert the original image to grayscale image, and then used Gabor to filter the ROI. The filtering process is shown in Fig. 6.

Figure 6.

Gabor texture filtering process diagram.

4.2 Image feature analysis

After the ROI filtering, the corresponding feature images are obtained and analyzed. 40 groups of feature images can be obtained from each image, with the size of 56 $\times$ 56. We expand the image in turn to obtain the feature sequence of size 1 $\times$ 125440. For different types of image classification, there are many features extracted from each image. Some of these features are helpful for image classification and recognition, but more of them may be invalid. We analyze the features extracted by Gabor and expect to reduce some unnecessary features, which will help to classify different types of images. We propose a method of combining sampling with concrete relations to reduce the dimension of features. Because the pixel values of the adjacent positions of the image feature sequence change very little, it will reduce the operation efficiency if all the pixels are retained. We sample the feature sequence, which can not only improve the transformation difference between features, but also improve the operation efficiency. Figure 7 shows the feature sequence of all images when the sampling scale is 64. We use different colors to compare 8 types of microbial image feature sequences. It shows that Gabor feature can distinguish different kinds of microorganisms.

Figure 7.

Feature sampling graph.

The sampling feature length obtained by sampling the features successively in 64 units is 1960. Since there are many redundant or irrelevant features in the high-dimensional data, we need to select the features and select the key features by analyzing the separability of the features. We mainly judge and sort the separability of features to complete feature selecting. The method of feature selection is as follows.

•

Firstly, we normalized the feature data, because using Gabor filters with different scales to extract features, the difference of feature values is too large, leading to large experimental errors. Specifically, we use the deviation standardization method to map the eigenvalues to between $-$ 1 $\sim$ 1, and the transformation formula is as follows.

$\displaystyle y=2\times\frac{x-x_{\min}}{x_{\max}-x_{\min}}-1$ (4)

Where: $x$ is the original eigenvalue, $x_{\min}$ is the minimum value of all eigenvalues, and $x_{\max}$ is the maximum value of all eigenvalues.

•

Secondly, due to the commonness among samples of the same kind of objects, the attribute distance between samples of the same kind should be smaller than that between samples of different kinds. Therefore, we sort the features according to the inter-class distance, and the calculation formula is as follows.

$\displaystyle D(x,\{a^{i}\})=\sqrt{\frac{\sum_{i=1}^{K}\sum_{k=1}^{n}(x_{k}-a_% {k}^{i})^{2}}{K}}$ (5)

Where, $K$ is the number of samples, $\{a^{i}\}$ is the point set, and $a_{k}^{i}$ is the $k$ component of $i$ points in the point set.

•

Thirdly, in order to select out the key features, we calculate the distance from each attribute point to the point set in turn. And sorting the properties according to the distance value. The sorting formula is as follows.

$\displaystyle\textit{idx}=|1-e^{-\frac{\textit{dist}}{\beta}}|$ (6)

Where: $\beta$ is the set feature weight factor. The weight value is 1 by default and adjusted according to the classification results. dist is the distance from feature point to point set. This value is obtained from Eq. (5). The greater the idx value, the better the attribute classification.

5. Classification model based on LSTM algorithm

The long short-term memory (LSTM) network is a recurrent neural network. The traditional cyclic neural network has the problem of gradient disappearance when dealing with the long-term dependence problem. However, LSTM network model introduces memory unit and forgetting gate. Therefore, the model can determine what information should be remembered based on the state of the input and the previous time. Due to its unique design structure, the LSTM network is well suited for processing and predicting important events with very long intervals and delays in time series. We convert texture features into sequence information and classify them by LSTM model.

Figure 8.

LSTM network structure.

5.1 Principle of LSTM algorithm

LSTM network is very suitable for the prediction of sequence model. The network structure is shown in Fig. 8. As shown in the Fig. 8, $x_{t}$ is the input of characteristic sequence, $h_{t}$ is the output state of hidden layer, and $c_{t}$ is the state of memory unit cell. The specific calculation formula of the network is as follows.

$\displaystyle f_{t}=\sigma_{g}(W_{f}x_{t}+U_{f}h_{t-1}+b_{f})$ (7)

Where, $\sigma$ is the activation function, $W$ and $U$ are the weight matrix of each gate, and $b$ is the bias. $i_{t}$ is the input gate, which is used to control the inflow of information from the memory unit. Its formula is:

$\displaystyle i_{t}=\sigma_{g}(W_{i}x_{i}+U_{i}h_{t-1}+b_{i})$ (8)

Where $o_{t}$ is expressed as an output gate to control the outflow of memory unit information. The expression formula is:

$\displaystyle o_{t}=\sigma_{g}(W_{o}x_{t}+U_{o}h_{t-1}+b_{o})$ (9)

$c_{t}$ formula is as follows.

$\displaystyle\tilde{c_{t}}=\sigma_{g}(W_{c}x_{t}+U_{c}h_{t-1}+b_{c})$ (10) $\displaystyle c_{t}=f_{t}\times c_{t-1}+i_{t}\times\tilde{c_{t}}$ (11)

$h_{t}$ formula is as follows.

$\displaystyle h_{t}=o_{t}\times\sigma_{h}(c_{t})$ (12)

5.2 Train LSTM network model

In the process of feature analysis, we select features and sort them according to their separability. We select different number of features in turn, classify these features, and compare the classification results. According to the classification results, we choose the most suitable number of key features.

We select different number of key features in turn and put them into the constructed classification model as input layer. Because of the special gate structure of LSTM network, it can not only improve gradient disappearance and gradient explosion problems, but also excavate the feature sequence, analyze the deep features among different microbial images and learn and map them. This enables the LSTM network to capture the connections that exist in the feature sequence. At the end of the model, we use the full connection layer and the Softmax function to calculate the prediction probability according to the output of the LSTM unit to complete the classification of different microbial images.

We build the LSTM classification network model of input layer, hidden layer and full connection layer. Combined with the classification accuracy, we constantly adjust the size of each layer to find the best model parameters. The LSTM network model we adopted has a maximum training cycle of 80 and block size of 50. We adopt adam to optimize the network structure. We select different number of feature sequences and different sizes of hidden layers. The experimental results are shown in the Fig. 9. As can be seen from the figure, when the number of key features is 20 $\sim$ 30, the classification accuracy is higher. When the number of hidden layer neurons is 120, the classification effect is better.

Figure 9.

Classification accuracy chart of models with different parameter values.

We transferred the final output of the LSTM network to the full connection layer and used the Softmax activation function to divide the features into eight pre-marked categories. The 8 nodes in the Softmax layer respectively represent the probabilities of 8 image types of microorganisms. The formula for calculating the probability $P_{i}$ is as follows.

$\displaystyle P_{i}=\textit{softmax}(\omega^{x}h+b^{x})$ (13)

where $\omega^{x}$ denotes the weight of Softmax layer, $b^{x}$ denotes the bias of this layer, and $i$ denotes the class of 8 microorganisms. Softmax formula is as follows.

$\displaystyle\textit{softmax}(i)=\frac{e^{i}}{\sum e^{j}}$ (14)

Table 2 shows the structure of LSTM network model after parameter adjustment.

Table 2

LSTM network model parameters

Layer	Quantity	Activation function
Long and short memory layer	21	–
Hidden layer	120	ReLU
Fully connected layer	8	Softmax

6. Results and discussion

In order to verify the accuracy of the model, K-fold cross validation is used to cross-verify the classified data. The K-fold cross validation is the most common cross validation method. The basic idea is to divide the data set into $K$ parts, alternately form the complement set for the verification set and the test set, and keep cycling to get the average result. The general formula of K-fold cross validation is as follows.

$\displaystyle\bar{x}=\frac{1}{k}\sum_{i=1}^{k}x_{i}$ (15)

Where, $x_{i}$ is the result of the $i$ round of cross validation, and $\bar{x}$ is the result of K-fold cross validation. In addition, four evaluation indexes are used to evaluate the performance of the model. Accuracy, sensitivity, specificity and overall accuracy were selected to evaluate the classification effect of the model.

Accuracy refers to the proportion of correctly classified $T P$ and $T N$ in all samples; $T P$ is true positive, indicating the number of correctly classified; $T N$ is true negative, indicating the number of those that are not part of a certain microorganism but are divided into that category. This index is the most intuitive indicator to measure the classification effect, and its calculation formula is as follows.

$\displaystyle\textit{Accuracy}=\frac{TP+TN}{TP+FP+TN+FN}\times 100\%$ (16)

Where, $F P$ and $F N$ are respectively false negative and false positive, indicating the number of a certain microorganism that is misclassified and the number of a certain microorganism that is not classified.

The sensitivity is the percentage of samples with positive detection in the total number of samples. The higher the sensitivity is, the lower the rate of missed diagnosis will be. Its calculation formula is as follows.

$\displaystyle\textit{Sensitivity}=\frac{TP}{TP+FN}\times 100\%$ (17)

Specificity is also called true negative rate, which refers to the ratio between the number of negative samples classified as negative and the actual number of negative samples. The higher the specificity is, the lower the rate of misdiagnosis will be. Its calculation formula is:

$\displaystyle\textit{Specificity}=\frac{TN}{TN+FP}\times 100\%$ (18)

The overall accuracy (OA) is the index to measure the classification ability of the overall model, and its calculation formula is as follows.

$\displaystyle OA=\frac{TP_{i}}{T}\times 100\%$ (19)

Where, $i$ is the species of 8 microorganisms, and $T$ is the number of all samples.

6.1 Results

Our experimental environment is windows10 operating system, matlab2020a. The experimental results are cross-verified by a factor of 10, and the letters S, H, As, B, N, C, Ah and I are used to represent the 8 categories of microorganisms. We use the model to show the classification results. We use the obfuscation matrix to display the classification results, as shown in Fig. 10.

It can be seen from the figure that the model proposed by us has a good effect on the discrimination of 8 microbe images. We calculated the classification index of each category in turn. The experimental results are shown in Table 3. From Table 3, the overall classification accuracy of our proposed model is 81.26%.

Table 3
The experimental results

Model evaluation index	Classification categories
	S	H	As	B	N	C	Ah	I
Accuracy (%)	95.36	95.25	72.73	57.14	25.00	92.06	49.12	38.46
Sensitivity (%)	95.08	95.58	96.30	40.00	25.00	90.63	96.55	83.33
Specificity (%)	95.64	95.01	58.43	40.00	40.00	93.55	32.94	25.00
OA (%)	81.26

Figure 10.

Confusion matrix.

6.2 Results contrast

In order to compare the classification effect of different models, we use Bayesian classifier, k-nearest neighbor classifier and random forest classifier to classify Gabor features. The results are as follows:

Bayesian classification: Bayesian classification is a relatively simple classification algorithm [19]. The basic idea of this algorithm is to first give the options to be classified, and then solve the probability of each option under this condition. Finally, the category of the item is judged according to the probability. The results of Bayesian classification are shown in Fig. 11. As can be seen from the Fig. 11, we screened 6 $\sim$ 400 image features, and the highest classification result was about 58%. Experiments show that Gabor texture features can distinguish different types of microecological images.

Figure 11.

Bayesian classification results.

K-nearest neighbor classification: K nearest neighbor (KNN) classification is a supervised learning algorithm, which determines the category of the test sample by calculating the distance between the test sample and all the training samples [20]. The algorithm has the advantages of few parameters and short training time. We selected different numbers of features in turn and verified the optimal classification accuracy by adjusting the K value of KNN algorithm. The results are shown in Fig. 12. As can be seen from the Fig. 12, when the number of features is 345 and K value is 4, the classification accuracy is the highest 76.9231%.

Figure 12.

K-nearest neighbor classification results.

Random forest classification: Random forest is an algorithm that integrates multiple trees through the idea of integrated learning. Its basic unit is decision tree, and its essence belongs to integrated learning method [21]. We selected different number of image features and different number of trees in turn for classification. The classification results are shown in Fig. 13. It can be seen from the Fig. 13 that the classification accuracy of random forest is between 62% $\sim$ 75% with different selection characteristics.

Table 4

Classification results of different texture features

Fatures	Bayesian classifier	KNN	Random forest
Gabor	58.4135%	76.9231%	72.8365%
LAWs	45.4327%	48.0769%	68.7500%
Tamura	59.1346%	46.1538%	68.0288%
GLCM	53.6058%	44.2308%	49.0385%
PCA	50.0000%	72.1154%	74.0385%

Figure 13.

Results of random forest classification.

We compare the results of common texture feature classification such as Gabor feature, LAWs feature, Tamura feature and GLCM, and the results of PCA method for feature extraction of original images [22]. We use bayesian, KNN, random forest and other classifiers to classify the above characteristics in turn, and the results were shown in Table 4. It can be seen from the classification results that Gabor features have a very good effect on distinguishing different types of microorganisms. KNN classifier is used, and the highest classification accuracy is 76.9231%. In order to verify the influence of LSTM network on feature classification, we input the above features into LSTM network and classify them. The classification results are shown in Fig. 14. As shown in the figure, four features of Gabor, LAWs, Tamura and GLCM were selected and combined with LSTM algorithm and SVM algorithm respectively for experiments. The results show that Gabor combined with LSTM model has the optimal classification accuracy, reaching 81.26%, which proves that the model proposed by us is feasible.

Figure 14.

Classification accuracy map of different texture features combined with LSTM network and SVM.

Figure 15.

Confusion matrix of Gabor-LSTM model and CNN model.

At the same time, we use the current popular deep learning classification method to classify eight categories of images. We use convolution neural network model to classify eight categories of image data, randomly select 3000 images as training data, and the remaining images are classified as test data. The confusion matrix of test set classification results is shown in the Fig. 15. It can be seen from the figure that there is little difference between the two models, but Gabor-LSTM model can recognize multiple small samples. We compare the two models and compare the accuracy of the eight categories and the overall accuracy respectively. The results are shown in the Fig. 16. It can be seen that although the recognition accuracy of CNN model is higher than that of our proposed model, there are a lot of misjudgments and even can not be correctly classified on unbalanced data sets. The strategy of image texture analysis can solve this problem.

Figure 16.

Comparison of accuracy between Gabor-LSTM model and CNN model.

6.3 Discussion

Due to the feature analysis and selection, the number of features is reduced. Therefore, the training time of our model is much shorter than that of deep learning method. In addition, our model has some advantages in dealing with unbalanced data.

It is well known that the detection of vaginal microecologic morphology plays a very important role in the diagnosis and treatment of gynecological diseases in women. Our scheme was able to identify images of eight different types of microbes. Therefore, our model has very important clinical value.

7. Conclusions

For imbalanced data, we propose a texture analysis combined with feature selection scheme, which can effectively improve the classification accuracy. We have proved the feasibility of our scheme through experimental comparison. The images detected by vaginal microecological morphology can be classified and predicted by texture features combined with LSTM model. Different texture features can distinguish different types of microorganisms, and neural network model can be combined to obtain better classification effect. Compared with the traditional method, our model has better accuracy. Compared with deep learning method, our model is more efficient and has more advantages in dealing with unbalanced data.

The microecological image recognition method proposed in this paper can accurately and quickly identify different microbial categories. It can provide guidance for clinicians to predict the disease. Therefore, our method has high clinical value. The experiment also shows that image texture feature analysis can be applied to micro-ecological image recognition, and Gabor feature has great advantages in distinguishing different microbial images.

8. Future prospects

In the follow-up work, we will focus on improving the classification accuracy, and at the same time complete the detection of micro-ecological image targets. By constructing target detection combined with target classification model, we can better assist doctors to evaluate treatment.

References

Ravel

Brotman

. Translating the vaginal microbiome: Gaps and challenges. Genome Medicine. 2016; 8(1): 35. doi: 10.1186/s13073-016-0291-2.

Xiong

Zheng

. Progress in research of relationship between vaginal Lactobacillus and preterm delivery. Chinese Journal of Endemiology. 2017; 38(3): 406. doi: 10.3760/cma.j.issn.0254-6450.2017.03.026.

Bai

Behera

Bruner

. The gut microbiome, symptoms, and targeted interventions in children with cancer: A systematic review. Support Care Cancer. 2017; 26(10): 427. doi: 10.1007/s00520-017-3982-3.

Yan

Pamp

Fukuyama

Hwang

Cho

Holmes

, et al. Nasal microenvironments and interspecific interactions influence nasal microbiota complexity and S. aureus carriage. Cell Host and Microbe. 2013; 14: 631. doi: 10.1016/j.chom.2013.11.005.

Brotman

Shardell

Gajer

Tracy

. Interplay between the temporal dynamics of the vaginal microbiota and human papillomavirus detection. Vaginal Microbiota and HPV. 2014; 210(11): 1723. doi: 10.1093/infdis/jiu330.

Wang

Xiao

Zhang

Wang

Liao

, et al. Characterization of vaginal strains and their potential antagonistic effects on. British Microbiology Research Journal. 2015; 6(4): 185. doi: 10.9734/BMRJ/2015/15116.

Niu

Sun

Liu

Bai

Xiao

. Length-Heterogeneity Polymerase Chain Reaction as a Diagnostic Tool for Bacterial Vaginosis. Chinese Medical Journal. 2018; 131(018): 2242. doi: 10.4103/0366-6999.240801.

Han

Kwitt

Aylward

Bakas

Menze

Asturias

, et al. Brain extraction from normal and pathological images: A joint PCA/image-reconstruction approach. Neuroimage. 2018; 173(3): 431. doi: 10.1016/j.neuroimage.2018.04.073.

Jeon

Kim

. Discovering latent topics with saliency-weighted LDA for image scene understanding. IEEE MultiMedia. 2018; 99: 55. doi: 10.1109/MMUL.2018.2883127.

10.

Wang

Zhou

. Image decomposition and texture analysis via combined bi-dimensional Bedrosian’s principles. IET Image Processing. 2018; 12(2): 262. doi: 10.1049/iet-ipr.2017.0494.

11.

Kondo

Takagi

Nishida

Lwai

Kudo

Ogawa

, et al. Computer-aided diagnosis of focal liver lesions using contrast-enhanced ultrasonography with perflubutane microbubbles. IEEE Transactions on Medical Imaging. 2017; 99: 1427. doi: 10.1109/TMI.2017.2659734.

12.

Liu

Muftah

Das

Robson

Auer

. Classification of MR tumor images based on Gabor wavelet analysis. Journal of Medical and Biological Engineering. 2012; 32(1): 22. doi: 10.5405/jmbe.813.

13.

Kooi

Litjens

Ginneken

Gubern-Merida

Sanchez

Mann

, et al. Large scale deep learning for computer aided detection of mammographic lesions. Medical Image Analysis. 2017; 35: 303. doi: 10.1016/j.media.2016.07.007.

14.

Spanhol

Oliveira

Petitjean

Heutte

. A dataset for breast cancer histopathological image classification. IEEE Transactions on Biomedical Engineering. 2015; 63(7): 1455. doi: 10.1109/TBME.2015.2496264.

15.

Sharma

Zerbe

Klempert

Hellwich

Hufnagl

. Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopatholo histopathology. Computerized Medical Imaging and Graphics. 2017; 61: 2. doi: 10.1016/j.compmedimag.2017.06.001.

16.

Nicolas

Santiago

Theodore

Navneet

Matija

David

, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nature Medicine. 2018; 24: 1559. doi: 10.1038/s41591-018-0177-5.

17.

Lin

Liu

Chen

. Image retrieval and classification using adaptive local binary patterns based on texture features. IET Image Processing. 2012; 6(7): 822. doi: 10.1049/iet-ipr.2011.0445.

18.

Chen

Gao

Zhao

. Single sample description based on Gabor fusion. IET Image Processing. 2020; 13(14): 2840. doi: 10.1049/iet-ipr.2018.6665.

19.

Siamak

Dougherty

Qian

. Optimal bayesian classification with missing values. IEEE Transactions on Signal Processing. 2018; 7(15): 4182. doi: 10.1109/TSP.2018.2847660.

20.

Ruiz

LFC

Guasselli

Caten

. Zanotta

. Iterative K-Nearest Neighbors Algorithm (IKNN) for submeter spatial resolution image classification obtained by Unmanned Aerial Vehicle (UAV). International Journal of Remote Sensing. 2018; 39(2): 16. doi: 10.1080/01431161.2018.1444296.

21.

Paul

Mukherjee

Das

Gangopadhyay

Chintha

Kundu

. Improved Random Forest for Classification. IEEE Transactions on Image Processing. 2018; 4012. doi: 10.1109/TIP.2018.2834830.

22.

Mirunalini

Aravindan

Jaisakthi

. Automatic stenosis detection using SVM from CTA projection images. Multimedia Systems. 2017; 25(2): 83. doi: 10.1007/s00530-017-0578-1.

Image classification of vaginal microecology detection based on gabor texture and LSTM model

Abstract

BACKGROUND:

OBJECTIVE:

METHOD:

RESULTS:

CONCLUSION:

Keywords

1. Introduction

2. Related work

2.1 Texture analysis of medical images

2.2 Medical examination image processing and recognition

3. Data and methods

3.1 Data acquisition

4.1 Image texture analysis based on Gabor wavelet

Table 3 The experimental results

7. Conclusions

8. Future prospects

References

Table 3
The experimental results