Learning Discriminative Salient LBP for Cloud Classification in Wireless Sensor Networks

Abstract

We focus on the issue of ground-based cloud classification in wireless sensor networks (WSN) and propose a novel feature learning algorithm named discriminative salient local binary pattern (DSLBP) to tackle this issue. The proposed method is a two-layer model for learning discriminative patterns. The first layer is designed to learn the most salient and robust patterns from each class, and the second layer is used to obtain features with discriminative power and representation capability. Based on this strategy, discriminative patterns are obtained according to the characteristics of training cloud data from different sensor nodes, which can adapt variant cloud images. The experimental results show that the proposed algorithm achieves better results than other state-of-the-art cloud classification algorithms in WSN.

1. Introduction

Clouds have an important effect on the earth's energy balance and climate change due to interacting with solar and terrestrial radiation [1]. Therefore, cloud analysis has a guiding role in flight planning and aviation [2] and provides better warnings of future disasters, such as tornados [3]. Ground-based cloud classification, as an important cloud observation technique, has received more and more attention from research fields. It is because successful cloud classification can improve the precision of weather prediction and help to understand climatic conditions. Until now, clouds are still classified by human observers who have received professional training in meteorological station [4]. Despite their unquestionable usefulness, this method takes high cost of human resources and different observers may obtain discrepant classification results. Therefore, automatic ground-based cloud classification technique is urgently required in this field.

To address the cloud classification, a number of ground-based imaging devices have been developed for capturing cloud images, which provides hardware supporting for automatic cloud image classification. The cloud images can be obtained by the devices, such as the whole sky imager (WSI) [5–7], total sky imager (TSI) [8, 9], infrared cloud imager (ICI) [10], and all-sky imager (ASI) [11]. Traditionally, the cloud classification techniques handle cloud images captured from only one image sensor.

Meanwhile, wireless sensor networks (WSN) have gained worldwide attention in recent years, particularly with the development of smart sensors [12, 13]. WSN has important applications such as remote environmental monitoring and target tracking. When each image sensor serves as a sensor node, WSN can be employed to classify clouds. Cloud classification in WSN has two advantages over traditional cloud classification. First, each image sensor should allocate a computing device in traditional cloud classification, while all image sensors in WSN share one computing device in the task manager node which is able to reduce costs. Second, cloud classification in WSN can obtain more complete cloud observation data than traditional cloud classification due to deploying sensor nodes at different locations. However, cloud classification in WSN is a challenging task due to the extreme appearance variations under different atmospheric conditions. Furthermore, the sensor nodes in WSN are in the different locations, and therefore the cloud images captured from different sensor nodes are heterogeneous. Intuitively, if the trained classifier can adapt according to the training data, the above challenging will be overcome.

In this paper, we focus on the issue of cloud classification in WSN. A novel feature learning algorithm named discriminative salient local binary pattern (DSLBP) is proposed. Based on this strategy, discriminative patterns are obtained according to the characteristics of training cloud data from different sensor nodes, which can adapt variant cloud images. The proposed method is a two-layer model for learning discriminative patterns. In the first layer, the most salient and robust patterns are learnt from each class; then in the second layer, representation capability of features is maximized by forming a global set of discriminative patterns based on patterns obtained in the first layer. The experimental results show that the proposed algorithm achieves better results than state-of-the-art cloud classification algorithms in different cloud image databases.

The remainder of this paper is organized as follows. Section 2 details the proposed algorithm. Section 3 presents the experimental results which show the super performance of our algorithm. Finally, we conclude the paper in Section 4.

1.1. Related Work

To the best of our knowledge, our work is the first to study cloud classification in WSN, and therefore we only introduce the traditional cloud classification algorithms. Over the past few years, a number of methods have been proposed for ground-based cloud classification. Existing cloud classification techniques are generally based on the characteristics of structure and texture in cloud images, which are inspired by image or video classification [14]. The algorithms based on structure features include cloud fraction and edge sharpness [15] and Fourier transformation [16], while the algorithms based on texture features include Law's measures [17] and cooccurrence and autocorrelation matrices [18]. In addition, several algorithms [4, 7] are proposed to fuse these two characteristics as the final representation. Although these above works have been done on this topic, they can not effectively explore useful information of cloud images. It is because cloud images, as one kind of nature texture, usually possess very large intraclass variations due to the large variation on illumination, climate, and deformation [19]. Therefore, it demands a more powerful algorithm to extract discriminative information of cloud images.

Fortunately, local binary pattern (LBP) and its extension have been proposed as effective feature extraction methods for texture and image classification [20, 21]. The properties of LBP are rotation invariance, low time complexity, and robustness against monotonic illumination change. The predefinition “uniform” patterns of LBP (the uniform LBP for short) effectively improve the performance of classification. The basic assumption is that the uniform LBP patterns occupy the salient proportion among all patterns (approximately 80%) and they are sufficient to reflect textural structures. However, uniform LBP patterns are not the salient patterns in two cases. The one is that the cloud images captured from different sensor nodes are heterogeneous, while the uniform LBP patterns are constant. The other one is that the ground-based cloud images are with irregular edges and shapes. Figure 1(a) illustrates that the proportions of uniform LBP patterns occupy all rotation invariant patterns in a randomly selected sample of 50 ground-based cloud images from different sensor nodes. From Figure 1(a), we can observe that the uniform LBP patterns do not account for a high proportion of the patterns in cloud images; therefore they can not capture the fundamental properties of these images. Although Liao et al. [22] proposed dominant LBP (DLBP) as an improved strategy to solve this problem, the method based on DLBP only considered the pattern occurrences of salient patterns, while the type of pattern information is lost. To address this limitation, we propose an adaptive feature learning algorithm named discriminative salient local binary pattern (DSLBP) for ground-based cloud image classification in WSN. The proposed algorithm can obtain discriminative features based on the characteristics of cloud database from different sensor nodes, as a result of improving the performance.

Figure 1

(a) The proportions of uniform LBP patterns occupy all rotation invariant patterns in ground-based cloud images; (b) the proportions of salient patterns occupy all rotation invariant ones in the same ground-based cloud images as (a).

2. The Proposed Algorithm

In this section, we first briefly review LBP and then describe the proposed DSLBP algorithm.

2.1. Review LBP

The LBP is a gray-scale texture operator that describes the spatial structure of the local image texture. The LBP operator labels each pixel in the image by computing the sign of the difference between the values of that pixel and its neighboring pixels. The result is a decimal number computed by the obtained binary string. Then the image can be represented by the histogram of these decimal numbers. The LBP value for the central pixel is computed as

\begin{matrix} {LBP}_{P, R}^{ri} = \min_{0 \leq l < P} \{\sum_{p = 0}^{P - 1} s (g_{p} - g_{c}) \times 2^{[(p + l) \mod P]}\}, \end{matrix}

(1)

where P is the total number of involved neighbor pixels, R is the radius of the circle which determines the distance between the neighbors and the central pixel, and

g_{c}

and

g_{p}

are the gray intensities of the central pixel and neighbor pixels, respectively. The step function

s (x)

is described with

s (x) = 1

x \geq 0

and

s (x) = 0

otherwise. The minimum value in (1) denotes the label of the rotation invariant LBP at the central pixel.

Let N denote the total number of rotation invariant LBP patterns. According to the definition in (1), the value of N is determined by neighboring samples P. For example, when $P = 8$ , $N = 36$ . In order to reduce the interference of noise, Ojala et al. [20] defined the U value at each pixel as the number of bitwise transitions between 0 and 1 in the LBP:

\begin{array}{l} U ({LBP}_{P, R}) = |s (g_{P - 1} - g_{c}) - s (g_{0} - g_{c})| \\ + \sum_{p = 1}^{P - 1} |s (g_{P} - g_{c}) - s (g_{p - 1} - g_{c})| . \end{array}

(2)

The uniform LBP only includes those pixels with no more than two transitions (i.e.,

U \leq 2

) in the adjacent binary presentation. For example, the LBP pattern “00000000” has U value of 0 and “00010000” has U value of 2, while the pattern “11010111” would not be included among the uniform LBP because it has four transitions. Accordingly, a uniform rotation invariant pattern is defined as

\begin{matrix} {LBP}_{P, R}^{riu 2} = \{\begin{cases} \sum_{p = 1}^{P - 1} s (g_{p} - g_{c}), & U ({LBP}_{P, R}) \leq 2, \\ P + 1, & otherwise, \end{cases} \end{matrix}

(3)

where the superscript

riu 2

means uniform rotation invariant patterns with

U \leq 2

2.2. Discriminative Salient LBP

The uniform patterns in LBP, which indicate the structural information with straight or low curvature edges, are the fundamental properties of local image texture. The basic assumption is that the uniform LBP patterns occupy the salient proportion among all patterns (approximately 80%) and they are sufficient to reflect textural structures. However, when applying LBP algorithm for ground-based cloud classification in WSN, there are two disadvantages reducing the performance of classification. First, ground-based cloud images, as a kind of natural texture, often contain several complicated structures, such as curvature edges, crossing boundaries, and corners. Extraction of the uniform patterns from cloud images may result in imprecise representation because the uniform LBP patterns are not the fundamental properties of texture in cloud images. Second, the cloud databases from different sensor nodes in WSN are heterogeneous due to different parameters of the sensor nodes. If we utilize the same predefined patterns to extract features without considering the characteristics of sensor nodes in WSN, the performance will decline sharply owing to lack of generalization.

To overcome the above drawbacks, we propose an adaptive feature learning algorithm named discriminative salient local binary pattern (DSLBP) for ground-based cloud image classification. Based on this strategy, discriminative patterns are obtained according to the characteristics of training cloud data from different sensor nodes, which can adapt variant cloud images. The proposed method is a two-layer model for learning discriminative patterns. In the first layer, the most salient and robust patterns are learnt from each class; then in the second layer, representation capability of features is maximized by forming a global set of discriminative patterns based on patterns obtained in the first layer. In this section, we introduce the two layers in detail.

2.2.1. The First Layer

The robustness of features generally includes rotation invariant, resistance to noise, and illumination changes, while the robustness to noise is associated with pattern occurrences of cloud images. In other words, if a pattern rarely occurs in a cloud image, it is probably sensitive to noise. To ensure the robustness of feature representation, the most salient and robust patterns from each class are learnt in the first layer. First, we build rotation invariant LBP histogram by (1) for every cloud image from each class and then accumulate all of these histograms into a single histogram. Finally, we sort the histogram in descending order. The first several patterns in this sorted histogram are the most frequently occurring patterns in this class of cloud images which are as the salient patterns. The minimum value k of determining the salient patterns is calculated by

\begin{matrix} k = \arg \min_{S} (\frac{\sum_{j = 0}^{k - 1} H [j]}{\sum_{j} H [j]}) \geq T . \end{matrix}

(4)

Here,

H [1,2, \dots]

denotes the sorted histogram of all rotation invariant patterns, and T is a threshold determining the proportion of salient patterns. We empirically set

T = 80 %

. The salient patterns of class i by solving (4) are denoted as

S [i]

. Figure 1(b) shows the proportions of salient patterns using the same cloud images in Figure 1(a). Compared to uniform patterns, the salient patterns account for a higher proportion of all patterns. The pseudocode for determining the salient patterns for each class is presented in Algorithm 1.

Algorithm 1: Determining the salient patterns for the class i.

Input: M train images from class i; the parameters P and R.

Output: the salient patterns $S [i]$ for class i.

Step 1. Select an image $I_{i}$ in the M training samples;

Step 2. Initialize the pattern histogram $H_{i} [0,1, \dots] = 0$ ;

Step 3. Compute the pattern label for each center pixel $g_{c} \in$ image $I_{i}$ using $LB P_{P, R}^{ri}$ of (1);

Step 4. Increase the count in the corresponding bin by 1, that is, $H_{i} [LB P_{P, R}^{ri}]$ ++;

Step 5. Accumulate the histograms from all M training samples into a single histogram;

Step 6. Sort the single histogram by counts in descending order, denote it as $H [0,1, 2, \dots]$ ;

Step 7. Calculate the number k of salient patterns by (4)

Step 8. The patterns corresponding to $H [0]$ , $H [1], \dots, H [k - 1]$ are the salient patterns $S_{0}; S_{1}; \dots; S_{k - 1}$ ;

Step 9. Return $S_{0}; S_{1}; \dots; S_{k - 1}$ ;

2.2.2. The Second Layer

In the first layer, we obtain robust features for each class. This layer is designed to strengthen the ability of feature description from different class, which can achieve discriminative representation. The pattern set $S [i]$ represents salient patterns of class i; however, individual $S [i]$ can not describe textural structure of the whole database. We take the union of all pattern sets $S [i]$ 's $(i = 1,2, \dots, C)$ to cover the information of all classes, which is formulated as

\begin{matrix} S_{G} = S [1] \cup S [2] \cup \dots \cup S [C], \end{matrix}

(5)

where

S_{G}

denotes the union representation of all classes and serves as the final feature for classification.

In summary, the whole procedure of the proposed learning model can be summarized in Figure 2. Compared to uniform LBP used in WSN, several advantages of our proposed DSLBP algorithm can be concluded. (1)

The DSLBP extracts salient patterns based on the characteristics of cloud images by using statistical analysis. This kind of strategy can effectively overcome the interference of the noise.

(2)

The LBP utilizes predefined uniform patterns for all databases without considering the differences among cloud images captured from sensor nodes in WSN, which lacks generalization. Instead, the DSLBP learns discriminative patterns based on the characteristics of cloud images captured from different sensor nodes. As a result, improved performances are achieved.

(3)

The LBP predefined uniform patterns as the feature representation from all training images without considering interclass information, and therefore uniform LBP lacks the discriminative power and representation capability, while the proposed DSLBP algorithm takes the union of all salient pattern sets to cover the information of all classes, where the discriminative power and representation capability are well considered.

Figure 2

A visualized picture of the proposed DSLBP model.

3. Experimental Results

3.1. Databases

To evaluate the effectiveness of our algorithm, we carry out a series of experiments on the Kiel database and IapCAS database, which are from different sensor nodes in WSN.

3.1.1. Kiel Database

The Kiel database derives from a sensor node in WSN. The core equipment of this node is a digital camera equipped with a fisheye lens, which provides a field of view larger than $18 0^{°}$ . The camera is set to capture one cloud image per 15 seconds. More information about the camera can be found in [23]. The Kiel database covers a wide range of sky conditions and solar zenith angles. According to the international cloud classification system published in WMO, the database is divided into seven classes. The sample number of each class is different and the total number is 1500. Figure 3(a) shows the samples from each class.

Figure 3

The cloud samples from (a) the Kiel database and (b) the IapCAS database.

3.1.2. IapCAS Database

The second database is the IapCAS database, which derives from another sensor node in WSN. The images in the IapCAS database are more challenging. The sample number of each class is also different and the total number is 860. Figure 3(b) shows the samples from each class.

3.2. Methods in Comparison Study

The experimental goal is to compare the proposed DSLBP algorithm with four current state-of-the-art methods, which are identified as follows. (1)

The fusion of texture and structure (FUS1) [7]: Heinle et al. [7] extract spectral and simple textural features as the final representation.

(2)

The fusion of texture and structure (FUS2) [4]: Zhuo et al. [4] extract textural and structural features as the final representation for each cloud image.

(3)

Local binary pattern (LBP) [20]: LBP has been introduced in Section 2.1, where the uniform patterns are considered.

(4)

Dominant LBP (DLBP) [22]: Liao et al. [22] proposed DLBP as an improved strategy to solve the problem in uniform LBP [20], where the pattern occurrences of salient patterns are considered.

3.3. The Experimental Setup and Result Analysis

For the fair comparison, we use the same experimental setup for all the experiments. First, each cloud image is converted to gray scale and normalized to have zero mean and unit standard deviation. Then each extracted patch feature is normalized via Weber's law [24]. Finally, $χ^{2}$ distance and nearest neighbor classifier are used. In each experiment, one-third of samples are randomly chosen from each class as training data while the remaining images are used for testing. The partition process is implemented 100 times independently. The average accuracy over these 100 random splits is reported as the final result for reliability.

The experimental results of different algorithms on the Kiel and IapCAS databases are listed in Table 1. Meanwhile, we also utilize the confusion matrices to show the detailed performance of each class using the proposed DSLBP method for the two databases, which are shown in Figures 4 and 5.

Table 1

The comparison of our method with the state-of-the-art methods on the Kiel database and IapCAS database.

	Kiel database (%)	IapCAS database (%)
Heinle et al. [7]	83.69	72.12
Zhuo et al. [4]	87.62	76.51
LBP [20]	86.54	76.36
DLBP [22]	90.31	81.32
DSLBP	93.22	84.76

Figure 4

Confusion table of our method on the Kiel database (%). The element of row i, column j in confusion table means the percentage of the ith class being recognized as the jth class.

Figure 5

Confusion table of our method on the IapCAS database (%).

From the results, several conclusions can be drawn. First, the proposed DSLBP algorithm achieves the highest classification accuracies in the two databases. Second, the performance of our DSLBP method is over 6% better than that of LBP method in the Kiel database and over 8% in the IapCAS database. It is because our DSLBP learns the discriminative salient patterns. Third, the classification results of DSLBP are better than that of DLBP, indicating that only pattern occurrences of salient patterns are considered by DLBP failing to fully characterize useful information.

The above two databases derive from different sensor nodes in WSN, and therefore the cloud images in WSN are captured with various hardware settings, under varying illumination and noise conditions, and postprocessed with different image techniques. The proposed DSLBP learns the discriminative patterns from training samples, which can adapt different sensor nodes in WSN.

4. Conclusions

In this paper, a novel feature learning algorithm named discriminative salient local binary pattern (DSLBP) is proposed for ground-based cloud classification in WSN. The proposed method is a two-layer model for learning discriminative patterns based on local binary pattern (LBP). The first layer is designed to learn the most salient and robust patterns from each class, and the second layer is used to obtain features with discriminative power and representation capability. Based on this strategy, discriminative patterns are obtained according to the characteristics of training cloud data from different sensor nodes, which can adapt variant cloud images. The experimental results show that our method achieves better results than other state-of-the-art cloud classification algorithms in WSN.

Footnotes

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grant no. 61401309 and the Doctoral Fund of Tianjin Normal University under Grant no. 52XB1405 and no. 5RL134.

References

Kazantzidis

Tzoumanikas

Bais

A. F.

Fotopoulos

Economou

Cloud detection and classification with the use of whole-sky ground-based images

Atmospheric Research 2012 113 80 88

10.1016/j.atmosres.2012.05.005

2-s2.0-84861554997

Taravat

Del

F. F.

Cornaro

Vergari

Neural networks and support vector machine algorithms for automatic cloud classification of whole-sky ground-based images

IEEE Geoscience and Remote Sensing Letters 2014 12 3 666 670

Agee

E. M.

A revised tornado definition and changes in tornado taxonomy

Weather and Forecasting 2014 29 5 1256 1258

Zhuo

Cao

Z. G.

Xiao

Cloud classification of ground-based images using texture-structure features

Journal of Atmospheric and Oceanic Technology 2014 31 1 79 92

10.1175/jtech-d-13-00048.1

2-s2.0-84892459412

Shield

J. E.

Johnson

R. W.

Karr

M. E.

Wertz

J. L.

Automated day/night whole sky imagers for field assessment of cloud cover distributions and radiance distributions

Proceedings of the 10th Symposium on Meteorological Observations and Instrumentation of the American Meteorological Society

1998

11 16

Shields

J. E.

Johnson

R. W.

Karr

M. E.

Burden

A. R.

Baker

J. G.

Daylight visible/NIR whole-sky imagers for cloud and radiance monitoring in support of UV research programs

Ultraviolet Ground- and Space-based Measurements, Models, and Effects III

November 2003

International Society for Optical Engineering

155 166 Proceedings of SPIE

Heinle

Macke

Srivastav

Automatic cloud classification of whole sky images

Atmospheric Measurement Techniques 2010 3 3 557 567

10.5194/amt-3-557-2010

2-s2.0-80053965495

Long

C. N.

Slater

D. W.

Tooman

Total Sky Imager Model 880 status and testing results

2001

Pacific Northwest National Laboratory

Long

C. N.

Sabburg

J. M.

Calbó

Pagès

Retrieving cloud characteristics from ground-based daytime color all-sky images

Journal of Atmospheric and Oceanic Technology 2006 23 5 633 652

10.1175/jtech1875.1

2-s2.0-33745036199

10.

Shaw

J. A.

Thurairajah

Short-term Arctic cloud statistics at NSA from the infrared cloud imager

Proceedings of the 13th ARM Science Team Meeting

2003

11.

Cazorla

Olmo

F. J.

Alados-Arbotedas

Development of a sky imager for cloud cover assessment

Journal of the Optical Society of America A: Optics and Image Science, and Vision 2008 25 1 29 39

10.1364/josaa.25.000029

2-s2.0-40149093392

12.

Liang

Cheng

Huang

S. C.

Chen

Opportunistic sensing in wireless sensor networks: theory and application

IEEE Transactions on Computers 2014 63 8 2002 2010

10.1109/tc.2013.85

MR3246633

13.

Liang

Radar sensor wireless channel modeling in foliage environment: UWB versus narrowband

IEEE Sensors Journal 2011 11 6 1448 1457

10.1109/JSEN.2010.2097586

2-s2.0-79955414672

14.

Zhang

Wang

Xiao

Zhou

Liu

Attribute regularization based human action recognition

IEEE Transactions on Information Forensics and Security 2013 8 10 1600 1609

10.1109/tifs.2013.2258152

2-s2.0-84884513342

15.

Liu

Sun

X. J.

Gao

T. C.

Zhao

S. J.

Comparison of cloud properties from ground-based infrared cloud measurement and visual observations

Journal of Atmospheric and Oceanic Technology 2013 30 6 1171 1179

10.1175/jtech-d-12-00157.1

2-s2.0-84880726521

16.

Calbó

Sabburg

Feature extraction from Whole-sky ground-based images for cloud-type recognition

Journal of Atmospheric and Oceanic Technology 2008 25 1 3 14

10.1175/2007JTECHA959.1

2-s2.0-38949213355

17.

Buch

K. A.

Sun

C. H.

Thorne

L. R.

Cloud classification using whole-sky imager data

Proceedings of the 9th Symposium on Meteorological Observations and Instrumentation

1995

Charlotte, NC, USA

353 358

18.

Singh

Glennen

Automated ground-based cloud recognition

Pattern Analysis and Applications 2005 8 3 258 271

10.1007/s10044-005-0007-5

2-s2.0-28344434791

MR2221958

19.

Liu

Wang

Xiao

Zhang

Cao

Tensor ensemble of ground-based cloud sequences: its modeling, classification, and synthesis

IEEE Geoscience and Remote Sensing Letters 2013 10 5 1190 1194

10.1109/lgrs.2012.2236073

2-s2.0-84879907728

20.

Ojala

Pietikäinen

Mäenpää

Multiresolution gray-scale and rotation invariant texture classification with local binary patterns

IEEE Transactions on Pattern Analysis and Machine Intelligence 2002 24 7 971 987

10.1109/tpami.2002.1017623

2-s2.0-0036647193

21.

Guo

Z. H.

Zhang

A completed modeling of local binary pattern operator for texture classification

IEEE Transactions on Image Processing 2010 19 6 1657 1663

10.1109/tip.2010.2044957

2-s2.0-77952607168

MR2814628

22.

Liao

Law

M. W. K.

Chung

A. C. S.

Dominant local binary patterns for texture classification

IEEE Transactions on Image Processing 2009 18 5 1107 1118

10.1109/tip.2009.2015682

2-s2.0-65149098349

MR3029990

23.

Kalisch

Macke

Estimation of the total cloud cover with high temporal resolution and parametrization of short-term fluctuations of sea surface insolation

Meteorologische Zeitschrift 2008 17 5 603 611

10.1127/0941-2948/2008/0321

2-s2.0-63649117140

24.

Liu

Fieguth

P. W.

Texture classification from random features

IEEE Transactions on Pattern Analysis and Machine Intelligence 2012 34 3 574 586

10.1109/tpami.2011.145

2-s2.0-84862908425