A study on plant recognition using conventional image processing and deep learning approaches

Abstract

Plant species recognition from images or videos is challenging due to a large diversity of plants, variation in orientation, viewpoint, background clutter, etc. In this paper, plant species recognition is carried out using two approaches, namely, traditional method and deep learning approach. In traditional method, feature extraction is carried out using Hu moments (shape features), Haralick texture, local binary pattern (LBP) (texture features) and color channel statistics (color features). The extracted features are classified using different classifiers (linear discriminant analysis, logistic regression, classification and regression tree, naïve Bayes, k-nearest neighbor, random forest and bagging classifier). Also, different deep learning architectures are tested in the context of plant species recognition. Three standard datasets (Folio, Swedish leaf and Flavia) and one real-time dataset (Leaf12) is used. It is observed that, in traditional method, feature vector obtained by the combination of color channel statistics+LBP+Hu+Haralick with Random Forest classifier for Leaf12 dataset resulted in a plant recognition accuracy (rank-1) of 82.38%. VGG 16 Convolutional Neural Network (CNN) architecture with logistic regression resulted in an accuracy of 97.14% for Leaf12 dataset. An accuracy of 96.53%, 96.25% and 99.41% is obtained for Folio, Flavia and Swedish leaf datasets using VGG 19 CNN architecture with logistic regression as a classifier. It is also observed that the VGG (Very large Convolutional Neural Network) CNN models provided a higher accuracy rate compared to traditional methods.

Keywords

Plant species recognition deep learning convolutional neural network machine learning classification

1 Introduction

In agriculture, plant species identification is used for weed detection [5], growth estimation, and plant disease classification [1]. Also, plants are used as medicines providing solutions to diabetes [4] and cardiovascular diseases [12]. In plant species recognition [2], leaf plays an important role compared to other parts like flower, seeds and stem. Computer vision techniques are utilized in automatic plant identification and recognition. Numerous mobile applications such as Pl@ntNet [11], leafsnap [13] are also developed.

Shape is one of the main characteristics to classify objects. Hu et al. [9] proposed a shape descriptor known as Multiscale distance matrix (MDM). MDM is a global based contour approach. MDM is invariant to rotation, translation, scaling, and bilateral symmetry. Decomposed Newton’s Method (DNM) and Maximum Margin Criterion (MMC) are applied for dimensionality reduction and further nearest neighbor (1NN) classifier is used. This method is tested for two datasets namely, Swedish Leaf Dataset and ICL (Intelligent Computing Laboratory) leaf dataset.

Zhao et al. [29] proposed a counting based shape descriptor to recognize simple and compound leaves. Independent-Inner Distance Shape Context (I-IDSC) measures the count of active shape pattern rather than considering matching features. Nearest neighbor classifier is used for classification. I-IDSC descriptor is tested over five datasets namely, Swedish leaf, ICL, Smithsonian, Plumbers Island, and their own leaf dataset formed from 54 species of Hong Kong.

Naresh et al. [18] proposed a modified Local Binary Pattern (LBP) for feature extraction and nearest neighbor classifier for medicinal plant classification. This method is tested with several standard leaf datasets and a collected dataset from Mysore, India. Tomar et al. [27] observed that the directed acyclic graph multi-class least square support vector machine (DAG-MLSTSVM) classifier performed better than artificial neural network and support vector machine. Prior to classification, leaf features are extracted based on shape and texture (21-d). Further, Hybrid Feature Selection (HFS) is carried out to identify the best features.

Ghazi et al. [6] applied transfer learning over LifeCLEF plant dataset with the help of pre-trained models like AlexNet, GoogleNet and VGGNet. For all these deep convolutional neural networks, fine tuning is performed and various parameters are analyzed after data augmentation. Parameters like batch size and number of iterations are analyzed. Lee et al. [14] discussed that Convolutional Neural Network (CNN) is used to learn leaf features and further gain knowledge based on selective features using Deconvolution Network (DN) approach. The authors observed that leaf veins help in more accurate plant identification than leaf shape. Learning features from hybrid local-global methods with deep learning performs better recognition than other techniques.

Sun et al. [23] proposed a 26-layer ResNet (Residual Network) model for plant identification. BJFU100 dataset is used and it consists of 10000 images of 100 ornamental plant species found in Beijing Forestry University campus. For experimental analysis, BJFU100 and Flavia datasets are utilized. In deep residual networks, 18, 26, 34, and 50 layers are considered. Amongst the four set of layers considered, ResNet26 outperformed the other three models. For experimental training, learning rate is set to 0.001. Flavia dataset accuracy (99.65%) result is compared with other approaches like Radial Basis Probabilistic Neural Network (RBPNN), Support Vector Machine (SVM), Deep Belief Network with dropout (DBN) and ResNet26. ResNet26 architecture produced an accuracy of 91.78% recognition rate for BJFU100 dataset. Barre et al. [3] developed a LeafNet, a CNN-based plant identification system. The LeafNet consisted of five sets of 2 convolutional layers and 1 max-pooling layer followed by 1 convolution, 1 max-pooling layer and 3 fully connected layers. LeafNet is tested over Leafsnap, Foliage and Flavia datasets.

Based on the extensive literature survey, it is identified that the reported work on plant recognition over Indian plant species are sparse. Also, numerous research works are carried out using the features such as shape, texture, color, morphological or physiological features. Reported works on plant species recognition using deep learning architecture are limited. Hence in this paper, an investigation is performed using traditional methods and deep learning architectures in order to achieve higher plant recognition rate.

2 Methodology

Two approaches are used for plant species recognition, namely, traditional method and deep learning methods as depicted in Fig. 1. The conventional image classification steps include preprocessing, feature extraction and classification using the machine learning classifiers. Feature extraction includes extraction of shape features, texture features and color features from leaves of plant images. These features are known as handcrafted features. Local Binary Pattern (LBP) [19] and Haralick texture features [7] are used to extract the texture information. Hu moments [10] are used for shape extraction and color channel statistics (mean and standard deviation of three color channels) for color information. To the extracted features, classification is carried out using machine learning techniques such as Logistic Regression (LR), K- Nearest Neighbour (KNN), Classification and Regression Tree (CART), Random Forest classifier (RF), linear discriminant analysis (LDA), Bagging Classifier (BC) and Naïve Bayes (NB) classifier [15, 16].

Fig.1

Plant classification methods.

In deep learning method, CNN based pre-trained models such as VGG 16, VGG 19, Inception-v3 and Inception-ResNet-v2 is used for feature extraction. These models have been trained on ImageNet dataset and their weights are free to use. ImageNet dataset contains about 1.2 million images. Pre-trained model weights are used as initial weights in deep learning architectures to extract the features from the input image. To the features extracted from pre-trained model, machine learning classification techniques is applied.

The Convolutional Neural Network [3] consists of several convolution layers, max pooling layer and a fully connected layer (FCL) as shown in Fig. 2. Convolutional layer is used as a feature extractor. Max pooling layer is used to reduce the dimension of the extracted feature vector. The fully connected layer is used to convert the feature map to 1-D feature vector and also, it is used as a classification layer. The deep learning models such as VGG 16, VGG 19, Inception-v3, and Inception-ResNet-v2 are developed based on the CNN principle.

Fig.2

Convolutional neural network.

VGG 16 [21] has 16 weight layers containing two sets of two convolution layers with max pooling, two sets of three convolution layers with max pooling followed by three fully connected layers. Similar to VGG 16, VGG 19 [21] has two sets of two convolution layers with max pooling, three sets of four convolution layers with max pooling followed by three fully connected layers. There are 138 and 144 million parameters in VGG 16 and VGG 19, respectively. In both the architectures, the width of layers starts from 64 and increases by 2 times after max pooling till 512 is reached. Also, both the models use ReLU as activation function in all layers and uses softmax for the final fully connected layer. Computational complexity of VGGNet is greater compared to other models.

‘Feature pooling’ is a concept used in Inception-v3 [25] that uses 1×1, 3×3 or 5×5 convolutions to collect maximum feature from each convolution. Inception-v3 uses stem (contains few convolutional layers and max pooling layers), few sets of filter concatenation and fully connected layers.

Inception-ResNet-v2 [24] merges both the concepts of Inception-v3 and ResNet [8] architectures. Inception-ResNet-v2 has stem as in Inception-v3 and Residual blocks as in ResNet model. But, inside every residual block, filter concatenation is carried out and their filter size varies for these residual blocks.

3 Datasets

Three standard datasets (Folio [17], Swedish leaf [22], Flavia [28]) and one real-time dataset are used in the experimental studies. Swedish leaf dataset is a standard dataset prominent because of its clarity. Swedish leaf dataset contains 15 different classes with 75 images in each class. Totally, there are 1125 images in this dataset.

Flavia dataset contains 32 classes of leaves and 1907 images. Folio database has 637 images in 32 classes. These two datasets have uneven number of images in each class. Hence, for Flavia dataset 50 images are used in each class (1600 images). Similarly, for folio dataset 18 images in each class (576 images) are used. The Real-time dataset is named as Leaf12 dataset. Twelve plant species images are collected and each class contains 320 images. It is photographed under different illumination conditions, color backgrounds, viewpoints and orientations using a portable camera. The list of plants in Leaf12 dataset and their sample images are shown in Fig. 3.

Fig.3

Leaf12 samples.

4 Results and discussion

Plant species recognition rate is determined using traditional methods and deep learning methods for three standard datasets (Folio, Flavia and Swedish leaf) and one real-time dataset (Leaf12). The implementation is carried out using Python language with the help of OpenCV package. For neural network, Keras package with theano as backend is used. Train and test size for analysis of results is set as 70% and 30%, respectively. The results of various datasets including Leaf12 dataset for traditional methods and pre-trained models are discussed in this section.

4.1 Folio dataset

Accuracies obtained using conventional technique and pre-trained neural networks for Folio dataset are summarized in Table 1. For most of the handcrafted features, either LDA or RF performed better compared to other classifiers. In traditional methods, handcrafted features (color channel statistics, Hu, LBP, Haralick) with LDA obtained an accuracy of 79.77%. Usage of pre-trained models (VGG 16, VGG 19, Inception-v3, Inception-ResNet-v2) resulted in the improvement of accuracy compared to conventional methods. VGG 19 with LR classifier outperformed other pre-trained models with an accuracy of 96.53%. Pawara et al. reported plant species recognition accuracy of 96.35% and 95% for AlexNet and GoogleNet architecture [20], respectively. An improvement by a factor of 0.18% is achieved with VGG 19+Logistic regression classifier.

Table 1
Accuracies of Folio Dataset (%)

Classifiers

Handcrafted Features LDA LR NB KNN CART RF BC

Color Channel 51.45 32.37 51.45 53.18 45.66 54.91 53.76

Hu 29.48 13.29 34.1 34.68 35.84 42.2 36.42

LBP 52.02 38.15 38.15 38.73 30.06 48.55 42.77

Haralick 63.58 35.26 56.07 52.02 49.13 60.69 59.54

Color Channel+Hu 60.12 34.1 57.8 60.12 55.49 71.1 61.85

Color Channel+LBP 62.43 51.45 53.18 50.29 46.24 72.25 63.01

Color Channel+Haralick 73.99 46.24 63.01 67.63 57.8 65.9 64.16

LBP+Hu 57.8 41.62 46.82 39.31 36.42 61.85 49.13

Haralick+Hu 68.79 35.84 57.23 61.85 51.45 69.94 63.58

LBP+Haralick 72.83 48.55 51.45 45.66 47.4 68.79 57.23

Color Channel+LBP+Hu 68.79 52.6 59.54 56.07 54.34 75.14 65.32

Color Channel + LBP + Haralick 76.3 60.12 58.96 59.54 56.65 72.25 67.05

Color Channel + LBP + Hu + Haralick 79.77 60.12 60.12 60.69 45.66 78.03 71.68

Pre-Trained Models As Feature Extractors

VGG 16 63.01 93.64 82.66 88.44 68.21 91.33 79.19

VGG 19 54.91 96.53 85.55 89.02 58.38 90.17 77.46

Inception-v3 83.24 86.71 60.12 72.83 42.2 80.35 58.38

Inception-ResNet-v2 80.35 84.39 48.55 69.94 42.2 76.88 58.38

	Classifiers
Color Channel	51.45	32.37	51.45	53.18	45.66	54.91	53.76
Hu	29.48	13.29	34.1	34.68	35.84	42.2	36.42
LBP	52.02	38.15	38.15	38.73	30.06	48.55	42.77
Haralick	63.58	35.26	56.07	52.02	49.13	60.69	59.54
Color Channel+Hu	60.12	34.1	57.8	60.12	55.49	71.1	61.85
Color Channel+LBP	62.43	51.45	53.18	50.29	46.24	72.25	63.01
Color Channel+Haralick	73.99	46.24	63.01	67.63	57.8	65.9	64.16
LBP+Hu	57.8	41.62	46.82	39.31	36.42	61.85	49.13
Haralick+Hu	68.79	35.84	57.23	61.85	51.45	69.94	63.58
LBP+Haralick	72.83	48.55	51.45	45.66	47.4	68.79	57.23
Color Channel+LBP+Hu	68.79	52.6	59.54	56.07	54.34	75.14	65.32
Color Channel + LBP + Haralick	76.3	60.12	58.96	59.54	56.65	72.25	67.05
Color Channel + LBP + Hu + Haralick	79.77	60.12	60.12	60.69	45.66	78.03	71.68
Pre-Trained Models As Feature Extractors
VGG 16	63.01	93.64	82.66	88.44	68.21	91.33	79.19
VGG 19	54.91	96.53	85.55	89.02	58.38	90.17	77.46
Inception-v3	83.24	86.71	60.12	72.83	42.2	80.35	58.38
Inception-ResNet-v2	80.35	84.39	48.55	69.94	42.2	76.88	58.38

4.2 Swedish leaf dataset

The results for Swedish leaf dataset are listed in Table 2. In traditional methods, LDA and RF classifier holds good whereas, LR is the best classifier for pre-trained models. Handcrafted features (Haralick, Hu) with LDA classifier resulted in an accuracy of 92.01%. In deep learning methods, VGG 19 with LR classifier produced an accuracy of 99.41%. The accuracy of VGG 16 CNN architecture with LR is higher than Alexnet (97.81%) and GoogleNet (98.24%) models as reported by Pawara et al. [20].

Table 2
Accuracies of Swedish Leaf Dataset (%)

Classifiers

Handcrafted Features LDA LR NB KNN CART RFC BC

Color Channel 68.34 47.34 54.44 51.18 50 60.06 57.4

Hu 58.88 36.69 50.59 63.31 52.96 66.86 61.83

LBP 75.15 72.78 61.24 61.24 50 70.41 64.5

Haralick 86.69 63.31 61.83 73.67 62.72 76.04 72.78

Color Channel+Hu 83.14 62.43 60.95 72.49 66.27 77.51 74.56

Color channel+LBP 85.8 79.29 68.05 73.08 63.91 77.81 73.37

Color Channel+Haralick 88.76 71.3 71.6 77.22 75.15 82.54 79.29

LBP+Hu 83.14 77.22 66.27 67.46 67.16 79.88 73.08

Haralick+Hu 92.01 74.56 67.46 82.84 76.63 83.14 79.59

LBP+Haralick 83.73 79.59 68.05 73.37 65.38 81.07 77.22

Color channel+LBP+Hu 90.53 83.73 70.41 77.81 70.41 83.73 79.29

Color Channel+LBP+Haralick 86.39 83.73 72.49 79.29 68.64 86.09 79.29

Color Channel+LBP+Hu+Haralick 88.46 85.8 73.96 81.07 80.18 87.87 85.5

Pre-Trained Models As Feature Extractors

VGG16 89.05 98.52 93.79 96.75 81.95 96.45 93.79

VGG19 86.09 99.41 91.72 95.27 79.59 96.75 90.53

Inception V3 86.39 94.67 68.93 89.05 68.05 88.17 80.18

Inception ResNetV2 78.99 96.15 68.34 83.43 71.01 89.05 86.09

	Classifiers
Color Channel	68.34	47.34	54.44	51.18	50	60.06	57.4
Hu	58.88	36.69	50.59	63.31	52.96	66.86	61.83
LBP	75.15	72.78	61.24	61.24	50	70.41	64.5
Haralick	86.69	63.31	61.83	73.67	62.72	76.04	72.78
Color Channel+Hu	83.14	62.43	60.95	72.49	66.27	77.51	74.56
Color channel+LBP	85.8	79.29	68.05	73.08	63.91	77.81	73.37
Color Channel+Haralick	88.76	71.3	71.6	77.22	75.15	82.54	79.29
LBP+Hu	83.14	77.22	66.27	67.46	67.16	79.88	73.08
Haralick+Hu	92.01	74.56	67.46	82.84	76.63	83.14	79.59
LBP+Haralick	83.73	79.59	68.05	73.37	65.38	81.07	77.22
Color channel+LBP+Hu	90.53	83.73	70.41	77.81	70.41	83.73	79.29
Color Channel+LBP+Haralick	86.39	83.73	72.49	79.29	68.64	86.09	79.29
Color Channel+LBP+Hu+Haralick	88.46	85.8	73.96	81.07	80.18	87.87	85.5
Pre-Trained Models As Feature Extractors
VGG16	89.05	98.52	93.79	96.75	81.95	96.45	93.79
VGG19	86.09	99.41	91.72	95.27	79.59	96.75	90.53
Inception V3	86.39	94.67	68.93	89.05	68.05	88.17	80.18
Inception ResNetV2	78.99	96.15	68.34	83.43	71.01	89.05	86.09

4.3 Flavia dataset

The results of Flavia dataset are tabulated in Table 3. Handcrafted features (color channel statistics, LBP, Hu, Haralick) with LDA classifier resulted in an accuracy of 89.17%. With respect to pre-trained models, VGG 19 with LR classifier produces a plant species recognition rate of 96.25%. Thanh et al. [26] reported an accuracy of 95.11% using CNN for Flavia dataset. VGG 19 + LR has an improvement of the order of 1.14% to CNN model.

Table 3
Accuracies of Flavia Dataset (%)

Classifiers

Handcrafted Features LDA LR NB KNN CART RF BC

Color Channel 49.38 31.25 47.29 51.67 47.29 57.29 56.88

Hu 35.21 16.25 33.33 44.38 42.5 56.25 53.33

LBP 66.88 58.75 53.75 56.04 38.75 57.92 50.42

Haralick 73.54 40.83 56.67 61.25 53.75 64.79 60

Color Channel+Hu 66.67 47.5 56.25 67.71 59.17 77.71 70.62

Color channel+LBP 78.54 72.08 63.75 69.38 54.17 74.58 67.71

Color Channel+Haralick 80.21 55.21 59.79 71.46 65 75.21 71.25

LBP+Hu 76.67 66.04 62.92 62.92 52.5 74.79 67.08

Haralick+Hu 79.58 56.88 60 72.08 63.75 78.54 73.75

LBP+Haralick 83.75 70 65 63.75 55 74.58 66.67

Color channel+LBP+Hu 82.29 75.42 67.29 71.46 59.58 82.08 72.29

Color Channel + LBP + Haralick 86.88 76.04 68.33 72.71 62.5 81.04 78.75

Color Channel + LBP + Hu + Haralick 89.17 78.96 70.21 75.83 65.62 85.62 77.5

Pre-Trained Models As Feature Extractors

VGG 16 71.88 95 86.04 92.71 70.62 93.12 84.38

VGG 19 76.67 96.25 87.5 88.96 67.71 93.54 85.83

Inception-v3 56.46 92.5 58.13 82.29 55.83 87.71 77.5

Inception-ResNet-v2 60 92.71 64.38 80.83 53.33 82.71 72.5

	Classifiers
Color Channel	49.38	31.25	47.29	51.67	47.29	57.29	56.88
Hu	35.21	16.25	33.33	44.38	42.5	56.25	53.33
LBP	66.88	58.75	53.75	56.04	38.75	57.92	50.42
Haralick	73.54	40.83	56.67	61.25	53.75	64.79	60
Color Channel+Hu	66.67	47.5	56.25	67.71	59.17	77.71	70.62
Color channel+LBP	78.54	72.08	63.75	69.38	54.17	74.58	67.71
Color Channel+Haralick	80.21	55.21	59.79	71.46	65	75.21	71.25
LBP+Hu	76.67	66.04	62.92	62.92	52.5	74.79	67.08
Haralick+Hu	79.58	56.88	60	72.08	63.75	78.54	73.75
LBP+Haralick	83.75	70	65	63.75	55	74.58	66.67
Color channel+LBP+Hu	82.29	75.42	67.29	71.46	59.58	82.08	72.29
Color Channel + LBP + Haralick	86.88	76.04	68.33	72.71	62.5	81.04	78.75
Color Channel + LBP + Hu + Haralick	89.17	78.96	70.21	75.83	65.62	85.62	77.5
Pre-Trained Models As Feature Extractors
VGG 16	71.88	95	86.04	92.71	70.62	93.12	84.38
VGG 19	76.67	96.25	87.5	88.96	67.71	93.54	85.83
Inception-v3	56.46	92.5	58.13	82.29	55.83	87.71	77.5
Inception-ResNet-v2	60	92.71	64.38	80.83	53.33	82.71	72.5

4.4 Leaf12

Random Forest (RF) classifier performed well for traditional classification. LR classifier acts as the best classifier for all pre-trained models considered for analysis. The pre-trained model, VGG 16 with LR resulted in an accuracy of 97.14% and the results are tabulated in Table 4. It is also observed that the pre-trained models using deep learning architecture yield higher accuracy compared to traditional methods.

Table 4
Accuracies of Leaf12 Dataset (%)

Classifiers

Handcrafted Features LDA LR NB KNN CART RF BC

Color Channel 32.38 32.81 35.5 81.08 68.14 80.99 73.18

Hu 13.8 10.68 12.76 27.43 35.42 48.44 40.89

LBP 41.41 41.23 28.82 45.23 30.21 51.74 43.75

Haralick 33.68 23.61 22.14 57.73 43.92 57.64 53.91

Color Channel+Hu 40.1 35.68 17.27 81.86 65.62 82.11 74.57

Color channel+LBP 51.22 52.08 36.72 73.78 57.64 78.12 68.84

Color Channel+Haralick 47.31 45.57 34.98 79.08 67.19 82.38 75.69

LBP+Hu 44.53 42.53 21.96 47.74 35.76 64.58 52.43

Haralick+Hu 36.11 29.08 18.06 63.98 51.74 69.97 63.19

LBP+Haralick 53.91 47.83 35.24 62.85 44.88 69.53 58.77

Color channel+LBP+Hu 54.95 53.56 28.3 73.7 59.72 79.95 71.09

Color Channel + LBP + Haralick 62.24 57.9 40.54 77.43 59.9 81.42 73.35

Color Channel + LBP + Hu + Haralick 63.72 58.77 31.86 77.86 62.5 82.38 74.22

Pre-Trained Models As Feature Extractors

VGG 16 80.21 97.14 66.67 90.89 63.45 92.27 80.47

VGG 19 78.12 96.53 62.33 90.8 61.37 93.49 78.21

Inception-v3 70.14 90.28 45.92 81.16 41.41 83.77 63.02

Inception-ResNet-v2 88.19 93.32 59.29 84.11 48.7 84.55 70.14

	Classifiers
Color Channel	32.38	32.81	35.5	81.08	68.14	80.99	73.18
Hu	13.8	10.68	12.76	27.43	35.42	48.44	40.89
LBP	41.41	41.23	28.82	45.23	30.21	51.74	43.75
Haralick	33.68	23.61	22.14	57.73	43.92	57.64	53.91
Color Channel+Hu	40.1	35.68	17.27	81.86	65.62	82.11	74.57
Color channel+LBP	51.22	52.08	36.72	73.78	57.64	78.12	68.84
Color Channel+Haralick	47.31	45.57	34.98	79.08	67.19	82.38	75.69
LBP+Hu	44.53	42.53	21.96	47.74	35.76	64.58	52.43
Haralick+Hu	36.11	29.08	18.06	63.98	51.74	69.97	63.19
LBP+Haralick	53.91	47.83	35.24	62.85	44.88	69.53	58.77
Color channel+LBP+Hu	54.95	53.56	28.3	73.7	59.72	79.95	71.09
Color Channel + LBP + Haralick	62.24	57.9	40.54	77.43	59.9	81.42	73.35
Color Channel + LBP + Hu + Haralick	63.72	58.77	31.86	77.86	62.5	82.38	74.22
Pre-Trained Models As Feature Extractors
VGG 16	80.21	97.14	66.67	90.89	63.45	92.27	80.47
VGG 19	78.12	96.53	62.33	90.8	61.37	93.49	78.21
Inception-v3	70.14	90.28	45.92	81.16	41.41	83.77	63.02
Inception-ResNet-v2	88.19	93.32	59.29	84.11	48.7	84.55	70.14

4.5 Performance analysis

Table 5 shows the performance metrics of various datasets. Precision, recall, F1-score, Rank-1 and Rank-5 accuracies are the performance measures taken into consideration. It is noticeable that the performance of VGG based models is best for all the four datasets, standard as well as for real-time dataset. Rank-5 accuracies for Swedish leaf and Leaf12 datasets is 100%. Also, the performance of pre-trained models used for feature extraction is higher than conventional methods.

Table 5
Performance Metrics of leaf datasets

Dataset Method Precision Recall F1-Score Rank-1 Accuracy(%) Rank-5 Accuracy(%)

Folio VGG 19 + LR 0.97 0.97 0.97 96.53% 99.42%

Swedish leaf VGG 19 + LR 0.99 0.99 0.99 99.41% 100%

Flavia VGG 19 + LR 0.96 0.96 0.96 96.25% 98.75%

Leaf12 VGG 16 + LR 0.97 0.97 0.97 97.14% 100%

Dataset	Method	Precision	Recall	F1-Score	Rank-1 Accuracy(%)	Rank-5 Accuracy(%)
Folio	VGG 19 + LR	0.97	0.97	0.97	96.53%	99.42%
Swedish leaf	VGG 19 + LR	0.99	0.99	0.99	99.41%	100%
Flavia	VGG 19 + LR	0.96	0.96	0.96	96.25%	98.75%
Leaf12	VGG 16 + LR	0.97	0.97	0.97	97.14%	100%

5 Conclusion

Plant species recognition is carried out by two approaches namely, traditional methods (feature extraction followed by classifier) and deep learning method (pre-trained models with machine learning classifiers). Four different datasets (Folio, Swedish, Flavia and Leaf12) are considered in this studies. From the experimental investigation, it is observed that the deep learning model yielded a higher accuracy compared to that of conventional methods for all datasets considered. Logistic regression classifier with pre-trained models resulted in an improved accuracy compared to that of other classifiers with pre-trained models. VGG 16 or 19 deep learning architectures with LR classifier resulted in higher accuracy compared with Inception-v3 and Inception-ResNet-v2. Maximum plant recognition rate obtained for different datasets are listed in Table 6 given below. Further, the accuracies can be improved by increasing the number of images using data augmentation methods.

Table 6

Rank-1 Accuracies of leaf datasets

Dataset	Architecture	Prediction Rate
Folio	VGG 19 + LR	96.53%
Swedish leaf	VGG 19 + LR	99.41%
Flavia	VGG 19 + LR	96.25%
Leaf12	VGG 16 + LR	97.14%

References

Aakif

, Khan

, Automatic classification of plants based on their leaves, Biosystems Engineering 139 (2015), 66–75.

Backes

, Casanova

, Bruno

, Plant leaf identification based on volumetric fractal dimension, International Journal of Pattern Recognition and Artificial Intelligence 23(6) (2009), 1145–1160.

Barré

, Stöver

, Müller

, Steinhage

, LeafNet: A computer vision system for automatic plant species identification, Ecological Informatics 40 (2017), 50–56.

Chowdhury

, Diabetes reversal by plant-based diet, Journal of Metabolic Syndrome 06(04) (2017).

Dyrmann

, Karstoft

, Midtiby

, Plant species classification using deep convolutional neural network, Biosystems Engineering 151 (2016), 72–80.

Ghazi

M.M.

, Yanikoglu

, Aptoula

, Plant identification using deep neural networks via optimization of transfer learning parameters, Neurocomputing 235 (2017), 228–235.

Haralick

, Shanmugam

, Dinstein

, Textural features for image classification, IEEE Transactions on Systems, Man, and Cybernetics 3(6) (1973), 610–621.

, Zhang

, Ren

, Sun

, Deep residual learning for image recognition, In: IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

, Jia

, Ling

, Huang

, Multiscale distance matrix for fast plant leaf recognition, IEEE Transactions on Image Processing 21(11) (2012), 4667–4672.

10.

M.K.

, Visual pattern recognition by moment invariants, IRE Transactions on information Theory 8(2) (1962), 179–187.

11.

Joly

, Goeau

, Bonnet

, Bakic

, Barbe

, Selmi

, Yahiaoui

, Carre

, Mouysset

, Molino

J.F.

, Boujemaa

, Interactive plant identification based on social image data, Ecological Informatics 23 (2014), 22–34.

12.

Katare

, Saxena

, Agrawal

, Joseph

, Subramani

, Yadav

, Lipid-lowering and antioxidant functions of bottle gourd (Lagenaria siceraria) extract in human dyslipidemia, Journal of Evidence-Based Complementary & Alternative Medicine 19 (2014), 112–118. doi: 10.1177/2156587214524229.

13.

Kumar

, Belhumeur

P.N.

, Biswas

, Jacobs

D.W.

, Kress

W.J.

, Lopez

I.C.

, Soares

J.V.

, Leafsnap: A computer vision system for automatic plant species identification, In: Computer vision–ECCV, Springer, Berlin, Heidelberg, 2012, pp. 502–516.

14.

Lee

S.H.

, Chan

C.S.

, Mayo

S.J.

, Remagnino

, How deep learning extracts and learns leaf features for plant classification, Pattern Recognition 71 (2017), 1–13.

15.

Marsland

, Machine learning: An algorithmic perspective, CRC Press, 2015.

16.

Mitchell

T.M.

, Machine learning, Burr Ridge, IL: McGraw Hill, 45(37), 1997.

17.

Munisami

, Ramsurn

, Kishnah

, Pudaruth

, Plant leaf recognition using shape features and colour histogram with k-nearest neighbour classifiers, In: Procedia Computer Science (Elsevier) Journal 58 (2015), 740–747.

18.

Naresh

Y.G.

, Nagendraswamy

H.S.

, Classification of medicinal plants: An approach using modified LBP with symbolic representation, Neurocomputing 173 (2016), 1789–1797.

19.

Ojala

, Pietikäinen

, Harwood

, A comparative study of texture measures with classification based on featured distributions, Pattern recognition 29(1) (1996), 51–59.

20.

Pawara

, Okafor

, Schomaker

, Wiering

, Data augmentation for plant classification, In: International Conference on Advanced Concepts for Intelligent Vision Systems Springer, Cham, 2017, pp. 615–626.

21.

Simonyan

, Zisserman

, Very deep convolutional networks for large-scale image recognition, (2014). arXiv preprint arXiv:1409.1556.

22.

Soderkvist

, Computer vision classification of leaves from swedish trees. Master’s Thesis, Linkoping University, 2001.

23.

Sun

, Liu

, Wang

, Zhang

, Deep learning for plant identification in natural environment, Computational Intelligence and Neuroscience (2017), 1–6. https://doi.org/10.1155/2017/7361042.

24.

Szegedy

, Ioffe

, Vanhoucke

, Alemi

A.A.

, Inception-v4, inception-resnet and the impact of residual connections on learning, In: AAAI, Vol. 4, 2017, p. 12.

25.

Szegedy

, Vanhoucke

, Ioffe

, Shlens

, Wojna

, Rethinking the inception architecture for computer vision, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), 2818–2826.

26.

Thanh

T.K.N.

, Truong

Q.B.

, Truong

Q.D.

, Xuan

H.H.

, Depth Learning with Convolutional Neural Network for Leaves Classifier Based on Shape of Leaf Vein, In: Asian Conference on Intelligent Information and Database Systems Springer, Cham, 2018, pp. 565–575.

27.

Tomar

, Agarwal

, Leaf recognition for plant classification using direct acyclic graph based multi-class least squares twin support vector machine, International Journal of Image and Graphics 16(03) (2016), p. 1650012.

28.

S.G.

, Bao

F.S.

, Xu

E.Y.

, Wang

Y.X.

, Chang

Y.F.

, Xiang

Q.L.

, A leaf recognition algorithm for plant classification using probabilistic neural network, In: 2007 IEEE International Symposium on Signal Processing and Information Technology, 2007, pp. 11–16.

29.

Zhao

, Chan

S.S.

, Cham

W.K.

, Chu

L.M.

, Plant identification using leaf shapes— A pattern counting approach, Pattern Recognition 48(10) (2015), 3203–3215.

	Classifiers
Handcrafted Features	LDA	LR	NB	KNN	CART	RF	BC
Color Channel	51.45	32.37	51.45	53.18	45.66	54.91	53.76
Hu	29.48	13.29	34.1	34.68	35.84	42.2	36.42
LBP	52.02	38.15	38.15	38.73	30.06	48.55	42.77
Haralick	63.58	35.26	56.07	52.02	49.13	60.69	59.54
Color Channel+Hu	60.12	34.1	57.8	60.12	55.49	71.1	61.85
Color Channel+LBP	62.43	51.45	53.18	50.29	46.24	72.25	63.01
Color Channel+Haralick	73.99	46.24	63.01	67.63	57.8	65.9	64.16
LBP+Hu	57.8	41.62	46.82	39.31	36.42	61.85	49.13
Haralick+Hu	68.79	35.84	57.23	61.85	51.45	69.94	63.58
LBP+Haralick	72.83	48.55	51.45	45.66	47.4	68.79	57.23
Color Channel+LBP+Hu	68.79	52.6	59.54	56.07	54.34	75.14	65.32
Color Channel + LBP + Haralick	76.3	60.12	58.96	59.54	56.65	72.25	67.05
Color Channel + LBP + Hu + Haralick	79.77	60.12	60.12	60.69	45.66	78.03	71.68
Pre-Trained Models As Feature Extractors
VGG 16	63.01	93.64	82.66	88.44	68.21	91.33	79.19
VGG 19	54.91	96.53	85.55	89.02	58.38	90.17	77.46
Inception-v3	83.24	86.71	60.12	72.83	42.2	80.35	58.38
Inception-ResNet-v2	80.35	84.39	48.55	69.94	42.2	76.88	58.38