Abstract
In order to solve the problems of poor universality, auxiliary algorithm complexity and great limitation in general segmentation algorithms, a new segmentation algorithm for farmland obstacle images using an intuitionistic fuzzy divergence based on threshold techniques was proposed. The original three-dimensional color image was converted to (Z-Y) chromatic aberration grayscale image on XYZ color space as the input images. While using intuitionistic fuzzy divergence, a modified Wu’s membership function and Sugeno’s intuitionistic fuzzy generator were used to find the membership and non-membership functions respectively. A new exponential intuitionistic fuzzy divergence based entropy formulas has been proposed and the optimum threshold value has been obtained by minimizing intuitionistic fuzzy divergence. The experimental results indicated that the proposed algorithm could clearly detect all types of obstacles and overcome the influence of unstructured environments well such as uneven illumination, shadow, weather and so on. The results inspire us explore further applications of intuitionistic fuzzy sets in the segmented images that contain a high degree of uncertain information. Furthermore, this proposed method can be used for the agricultural robots vision navigation accurately.
Keywords
Introduction
Farmland image processing is an important aspect of precision agriculture research. It has been widely used in the fields of intelligent mechanical weeding [14, 30], precision spraying [40], robot picking [4, 21], disease recognition [5, 17] and so on. Obstacle image segmentation is an important step of farmland image processing which is used for agricultural robots vision navigation system to acquire object information in robot walking regions such as person, vehicle, house, pole and so on. Then the robot can sense the agriculture environment and keep away from the obstacles. It is known that agriculture environment for robot vision navigation is complex and can be easily affected by the unstructured features such as lighting, shadow, weather and so forth. Therefore, how to separate the farmland obstacle from the images has become an important research field.
Over the past few years, extensive researches have been conducted in this field and many types of segmentation techniques have been proposed in literatures, each one of them based on a certain methodology to segment farmland objects. Meyer and Neto [23] have improved an unsupervised vegetation index, Excess Green minus Excess Red (ExG–ExR) compared to the commonly used Excess Green (ExG), and the normalized difference indices (NDI) to separate plants and backgrounds. Song et al. [33] have introduced the Otsu muti-threshold method into the minimum error Bayesian decision theory in order to detect the obstacles. Zhang et al. [42] segmented the farmland images by analyzing the distribution of pixel on the scanning line and extracted the obstacle object region after arrangement. Zhao et al. [43] proposed an agricultural image segmentation method based on the particle swarm optimization and the K-means clustering. Geng et al. [8] have applied a C-V model based on level set and prior information to segment weed, wheat and apple images. Montalvo et al. [25] applied a vegetation index to discriminate vegetation and non-vegetation, the double thresholding to achieve the separation of weeds and crops. Montalvo et al. [24] proposed new automatic expert systems for image segmentation in maize fields, which was based on three consecutive stages where the main underlying idea was the successive application of automatic image processing tasks mapping the expert knowledge. Godec et al. [10] presented an out-of-the-box segmentation algorithm to allow for a rough per-pixel separation of the object and the background, which enabled for a more robust training of the classifier. Ji et al. [19] proposed an iterative threshold segmentation of apple branch images based on contrast-limited adaptive histogram equalization (CLAHE). Huo et al. [16] proposed the partitioning and parallel K-means algorithm based on CUDA (Compute Unified Device Architecture) to raise the running speed. Meng et al. [22] used the fuzzy C-means clustering method (FCM) based on two-dimensional histogram and YCgCr color model to identify the crops. Han et al. [13] developed wavelet multi-resolution decomposition to detect some obstacles in farmland. Hu et al. [15] presented the improved two-dimensional entropy segmentation algorithm to position obstacles. Liu and Shi [20] combined Otsu segmentation in Lab color space and weighted fuzzy entropy segmentation in YUV color space to segment farmland images.
However, because of the complex background and different obstacle shapes in the real farmland environment, general segmentation algorithms have some problems such as poor universality, auxiliary algorithm complexity, large calculation and great limitation. In addition, there is hardly any work using intuitionistic fuzzy set to segment farmland images, even though it has been widely used in other fields [1, 29]. Intuitionistic fuzzy set theory considering two uncertainties (membership and non-membership degrees) can overcome the aboveproblems.
In the present work, a new segmentation algorithm for farmland obstacle images using an intuitionistic fuzzy divergence based on threshold techniques is proposed. A new divergence based on entropy formula has been constructed. The formulation has been used to find the optimal threshold value for (Z-Y) chromatic aberration grayscale image on the XYZ color space. While using intuitionistic fuzzy divergence, a modified Wu’s membership function is used to find the membership function of the image. And then Sugeno’s intuitionistic fuzzy generator is applied to find the non-membership function. Experiments are conducted on several farmland obstacle images to verify the effectiveness of the proposed algorithm.
Intuitionistic fuzzy divergence method
Intuitionistic fuzzy sets (IFS)
The theory of fuzzy sets (FSs) was proposed by Zadeh [41] in 1965. A fuzzy set (FS) A in a finite set X ={ x1, x2, …, x n } can be mathematically defined as a set of ordered pairs A ={ (x, μ A (x)) |x ∈ X }, where, the function μ A (x) : X → [0, 1] represents the belongingness or membership degree of an element x in X. Therefore, the non-belongingness or non-membership degree of an element x in X is 1 - μ A (x). But in real world, this may not be true as the sum of membership and non-membership degrees are equal to 1. In fact, there are often some uncertainty or hesitation.
The concept of the intuitionistic fuzzy set (IFS), a more generalized fuzzy set was introduced by Atanassov [2, 3] in 1983. An IFS A in X can be represented as A ={ (x, μ A (x) , ν A (x)) |x ∈ X }, where, the functions μ A (x) , ν A (x) : X → [0, 1] represent the membership and non-membership degrees of an element x in X respectively with the necessary condition 0 ≤ μ A (x) + ν A (x) ≤1. A third function π A (x) =1 - μ A (x) - ν A (x) is called hesitation degree (or intuitionistic fuzzy index), namely due to the lack of knowledge of whether x belongs to A or not. It is obvious that 0 ≤ π A (x) ≤1 for each x ∈ X.
It is easily observed that every fuzzy set is a particular case of IFS and in the case of π A (x) =0 for each x ∈ X.
Intuitionistic fuzzy set has been proved to be more general than fuzzy set along with the membership and non-membership degrees which can be able to represent the lack of knowledge more accurately. As a result, intuitionistic fuzzy set is able to model a lot of situations where the classical fuzzy set fails to use all the available information. This is precisely the idea behind the usage of intuitionistic fuzzy set for the purpose of farmland obstacle segmentation.
Intuitionistic fuzzy divergence (IFD)
The existing distance measure
The intuitionistic fuzzy distance is a measure of the difference between two intuitionistic fuzzy sets. Various distance (metric) measures have been proposed. For example, Szmidt and Kacprzyk [34] extended from fuzzy sets to intuitionistic fuzzy set and proposed modified intuitionistic fuzzy distances by adding membership degrees μ A (x) and μ B (x), non-membership degrees ν A (x) and ν B (x), and the hesitation degrees π A (x) and π B (x) respectively. Hamming distance formula is as follows:
On an attempt to take the advantages of Hausdorff metric, Grzegorzewski [12] proposed intuitionistic fuzzy distance based on Hausdorff metric. Later, Yang and Chiclana [39] proposed a generalization of the Grzegorzewski’s distances, which takes into account the hesitant part. Hamming distance formula is as follows:
In this work, a new intuitionistic fuzzy distance measure called intuitionistic fuzzy divergence (IFD) based on exponential intuitionistic fuzzy entropy is proposed. There are three parameters considered as follows, namely, the membership degree, the non-membership degree and the hesitation degree.
Let A ={ (x, μ
A
(x) , ν
A
(x)) |x ∈ X } and B ={ (x, μ
B
(x) , ν
B
(x)) |x∈ X } be two intuitionistic fuzzy sets, where μ
A
(x) and μ
B
(x), ν
A
(x) and ν
B
(x), π
A
(x) =1 - μ
A
(x) - ν
A
(x) and π
B
(x) =1 - μ
B
(x) - ν
B
(x) are membership, non-membership and hesitation degrees respectively. Considering the hesitation degree including certain, negative and hesitation information, the hesitation degrees can be defined into three parts: μ
A
(x) π
A
(x) and μ
B
(x) π
B
(x), ν
A
(x) π
A
(x) and ν
B
(x) π
B
(x), and and , which denote the proportion of tending to be certain, negative and hesitation information. Therefore, the intuitionistic fuzzy sets A and B can be converted into the following formula:
In an image A of size M × N with L distinct gray levels having the probabilities p0, p1, …, pL-1, the exponential entropy is defined as . The exponential fuzzy entropy of image A is defined as [7]:
Based on this entropy, a fuzzy divergence formula [9, 26] is given as:
Corresponding to the above formula, the divergence between images A and B due to the new membership degrees is given as:
Thus, the overall intuitionistic fuzzy divergence (IFD) between images A and B by adding Equations (7) and (8) is defined as:
Now, if A is the actual threshold image and B is the ideally thresholded image, then μ
B
(b
ij
) =1, ν
B
(b
ij
) =0 and π
B
(b
ij
) =0. Hence, Equation (9) becomes
The IFD will be minimized according to the Eq. (10) for the proposed algorithm to find the optimal threshold value.
In grayscale image A of size M × N (vertical ×horizontal) pixels with L distinct gray levels, the number of pixels with gray level g (g = 0, 1, 2, …, L - 1) is denoted as p (g). Farmland obstacle image segmentation algorithm was introduced using IFD according to the Equation (10) in order to separate the obstacles from the farmland obstacle image considered.
Capturing the original images
The experimental images were respectively captured under natural and farm conditions using a digital camera (Coolpix L16, Nikon, Tokyo, Japan) in Maduwang Village Xiwang Residential Baqiao District Xi’an City, Northwest A&F University Yangling, and Yangji Town Baiyin City. In this study, the farmland images included several objects such as person, pole, house, vehicle and so on. It should be noted that the path was not considered. The farmland images were transmitted to PC using a USB interface. The original images captured by the digital camera were 2594×1944 pixels in JPEG format. There were totally 1500 imagescaptured.
Converting into (Z-Y) chromatic aberration grayscale image A
In order to test the running speed of the algorithm, all the original images were normalized to 640×480 (vertical×horizontal) pixels. Next, these images were converted from RGB into XYZ color space. The conversion function was as follows:
For a particular threshold gray level t (t = 0, 1, 2, …, L - 1), the mean of the object (or, foreground) and background regions were computed respectively and written as:
Each one of these intuitionistic fuzzy sets is associated with a gray level g of the grayscale L used in the image A. Intuitionistic fuzzy sets A
O
and A
B
was constructed to associate with the obstacle object and the background of the image A respectively.
(i) Membership degree
Considering the gray levels of object and background mainly concentrated on the means of object and background class, the membership degree functions of object and background were computed using a modified Wu’s membership function [35] as follows:
(ii) Non-membership degree
Intuitionistic fuzzy generators were used to elicit the non-membership degree function. There are many generators such as Sugeno, Yager’s generator in literatures [11, 36–38]. In this paper, Sugeno’s generator was used. The non-membership degree functions of object and background were written as follows:
(iii) Hesitation degree
The hesitation degree functions of object and background were computed as:
A is the actual grayscale images and B is the assumed ideally thresholded images. Then IFD between A and B was calculated by Equation (10).
Selecting the best threshold T
The gray level of minimizing IFD associated with the intuitionistic fuzzy set A O and A B was chosen as the best threshold T.
Segmenting the chromatic aberration grayscale images A using the best threshold T
To separate the obstacle object from the background, it is necessary to accurately determine the property that fulfilling g ≥ T belongs to the object, otherwise to the background.
Results and discussion
In order to test the performance of the proposed approach, all the experimental images were segmented using the calculated threshold value. The relevant parameters of the algorithm are as follows: k = 3, β = 0.5, lm = 4, C = (the maximum gray value minus the minimum gray value). Results of the segmented images are showed in Fig. 1 along with other existing methods. The classical Otsu’s method [28] to choose optimal thresholds by maximizing the between class variance was applied to the test images in order to have non-fuzzy references. And the existing intuitionistic fuzzy distance methods such as Szmidt’s Hamming distance [34], Yang’s Hamming distance [39] and Jati’s intuitionistic fuzzy divergence [18] were also applied to the images so as to have intuitionistic fuzzy references. Threshold values were calculated for all the test images and then used for the image segmentation. In this paper, it shows that these 8 images contain different kinds of farmland obstacles.
MATLAB V7.11.0 (The Mathworks, Natick, Mass.) was used to perform all the procedures involved in this study. The codes were run on a PC with an Inter(R) Core(TM) i5-3317U CPU @ 1.70 GHz processor, 4.00GB of RAM, and Windows 7.
Validation indices were used to evaluate the performance of the proposed algorithm in a quantitative way [32]. Only a few indices that were intuitively plausible, easy to implement and included in the standard literature [31] on image segementation have been taken into account. Two indices were listed because of space constraints. One was accuracy which is commonly used; the other one was the misclassification error (ME) which is characterized by its capability to distinguish small segmentation degradation. Both of them are defined as follows:
Figure 1 shows the obstacles on farmland images such as person, pole, farmhouse and vehicle. In Fig. 1, the first column shows the original images considered for thresholding which are quoted as numbers like original image (1, 2, …, 8). Column (a) shows the results of the test images thresholded by Otsu’s method. It is observed that the resulting Otsu segmented image contains much more noise and the shape of obstacle is also not distinguished from the background. Using Szmidt and Yang’s methods, images are segmented and the results are portrayed in column (b) and (c) of Fig. 1. It is seen that both Szmidt and Yang’s methods give almost similar and little bit better results than the Otsu’s method except original image 2 even though the shape of obstacle is also not clear and much noise is present around the obstacles. These 8 test images were segmented by Jati’s method and the output images are shown in column (d). The threshold images using Jati’s method are much better than Szmidt and Yang’s methods because the shape of obstacle is clear and less noise around the obstacles. Experimental images were segmented using the values obtained from the proposed method. And the resulting images are showed in column (e) in Fig. 1. The proposed method using IFD, it is observed that the obstacles are segmented better with less noise than Otsu’s method and the existing intuitionistic fuzzy distance methods. In addition, it can also detect the edge of farmland obstacle showing in originalimage 6.
It indicated that farmland obstacle image segmentation with the proposed algorithm can overcome the influence of complex environments well such as ununiform lighting. Besides, the farmland obstacle objects can be separated from the background well. In the original images segmentation using the proposed algorithm, the objects are the clearest having the smallest noise fragments which are the most comprehensive compared with other methods. Therefore, the proposed algorithm is feasible and effective for farmland obstacle image segmentation.
After farmland image segmentation, extracting the bounding boxes of farmland obstacles are shown in Fig. 2 for the purpose of preventing agriculture robot collision. This can lay a good foundation for realizing the agricultural robots vision navigation.
Table 1 shows the threshold of the images using different segmentation methods. It is observed that the threshold values of Otsu’s method are much bigger than the intuitionistic fuzzy distance methods which get almost closer threshold values. Threshold level of the proposed intuitionistic fuzzy divergence method is a little bigger than the intuitionistic fuzzy method.
The accuracy of the various thresholding methods along with the proposed method was evaluated. The accuracy values for all the 8 images are given in Table 2. Table 2 reveals that the image segmentation processed by the proposed method surpasses other four methods.
The misclassification errors for the tested images are listed in Table 3. It is clear that the ME values of Otsu’s method are the largest of all the methods. And the ME values of the proposed algorithm is the smallest for a perfect segmented image.
The effective and least effective segmentation number and rate are listed in Table 4. For the proposed algorithm, the effective segmentation number is the largest of all; the least effective segmentation number is the smallest of all; the effective segmentation rate is more than 95%; and the least effective segmentation rate is only about 9%. Therefore, the proposed algorithm to segment farmland obstacle images obtained the most satisfied result.
Overall, the threshold images and the indices show the effectiveness of the proposed algorithm for the segmentation of the farmland obstacle images. It is useful for farmland obstacle detection. And it can lay a good foundation for realizing the agricultural robots vision navigation.
In this paper, a novel obstacle segmentation scheme from farmland images using intuitionistic fuzzy divergence was proposed. The algorithm was proved to be highly effective. The methodology involves minimization of intuitionistic fuzzy divergence between the actual image and the ideally thresholded image. While using intuitionistic fuzzy divergence, a modified Wu membership function was used to find the membership function of the image. And then, Sugeno’s intuitionistic fuzzy generator was used to find the non membership function. A new exponential intuitionistic fuzzy divergence based on entropy formula was proposed and used to find the optimum threshold value on (Z-Y) chromatic aberration grayscale images. Experiments were conducted on several farmland obstacle images. Comparing with Otsu’s method and the existing intuitionistic fuzzy methods for obstacle segmentation, it was observed that all types of obstacles were detected clearly with little noise using the proposed method. The results inspire us explore further applications of intuitionistic fuzzy sets in the segmented images that contain a high degree of uncertain information. Furthermore, this proposed method can be used for the agricultural robots vision navigation accurately. The proposed intuitionistic fuzzy divergence pays more attention to the hesitation degree containing certain, negative and hesitation information better than the existing intuitionistic fuzzysets.
In the future work, the farmland obstacle images with path region should be considered. And next plan is to investigate the path planning and collision avoidance of the facilitate robot. Additionally, machine learning methods would be beneficial to improve the performance and robusticity further.
Footnotes
Acknowledgments
This work is supported by the National Key Research Program of China (High efficiency spraying technologies and intelligent equipments of Ground and aviation, Grant No.SQ2016ZY06002827) and Shaanxi Provincial Science & Technology Innovation and Key Projects (Obstacle detection and key technology in agricultural navigation, Grant No. 2016NY-186). The authors also gratefully acknowledge the helpful comments and suggestions of the reviewers which have improved the presentation.