Adaptive cutoff distance: Clustering by fast search and find of density peaks

Abstract

Clustering by fast search and find of density peaks (CFSFDP) was proposed to create clusters by finding high-density peaks, quickly. CFSFDP mainly based on two rules: 1) a cluster center has a high dense point and 2) a cluster center lies at a large distance from other clusters centers. The effectiveness of CFSFDP highly depends upon the cutoff distance (C_d), which is used to estimate the density of each data point. However, there is a need to provide the predefined C_d. In this paper, we propose an adaptive way to estimate the accurate C_d by using the characteristics of Improved Sheather-Jones (ISJ) method named as IJS-CFSFDP. ISJ method provides the best estimation for C_d to measure accurate density of each data point. We perform a number of experiments on standard benchmark clustering datasets and real academic dataset of students. The evaluated clustering results on education dataset validate the IJS-CFSFDP can be used to make intelligent contents delivery system based on the capability and intelligence of the student. The experimental results on synthetic datasets show that the proposed adaptive C_d method creates better clusters as compare to the CFSFDP, mean shift, affinity propagation and k-means.

Keywords

Density based clustering kernel density estimation optimal cutoff distance selection Improved Sheather-Jones (ISJ) method

1 Introduction

Clustering algorithms are aimed to analyze data by organizing data into a set of disjoint categories, called clusters. Clustering has been successfully applied in different fields such as pattern recognition [1 –3],astronomy [4], cyber security [5], health care [6], bioinformatics [7, 8], social networks [9], education [10, 11], and image processing [12, 13] etc. The potential efforts are also required to apply clustering in emerging fields such as big data [14, 15], IoT [16 –20], and Virtual Reality [21]. Clusteringalgorithms can be categorized into different types such as, density-based [22 –28], model-based [29, 30], grid-based [31, 32], hierarchical [8 , 34], and partitioning [35 , 39].

The k-means [35] clustering is a state-of-the-art partitioning algorithm. In k-means, data is partitioned into k clusters and these clusters are iteratively optimized. The k-means creates spherical clusters and cannot detects the outliers. The accuracy of k-means is subjected to appropriate knowledge to the number of clusters and initial selection of centroids.

Affinity propagation (AP) [36] clustering is an effective clustering algorithm that outperforms traditional and classical clustering algorithm. AP gives promising results in different applications, such as face clustering, gene detection, and air-travel routines. However, the time complexity of AP is much higher than k-means and mean shift. For clusters of arbitrary shape, AP does not perform well.

The mean shift [37] is well known kernel density estimation based clustering algorithm, which is successfully used for image segmentation, visual tracking, space analysis, and mode seeking. In mean shift there is no need of domain knowledge to do clustering. However, the mean shift depends upon the window size.

Density based clustering is a primary approach to create clusters of arbitrary shapes and to identify noise from spatial dataset. The clusters are characterized as dense regions and some points have isolated densities declared as noise or outliers. Clusters of arbitrary shapes are created by connecting densities with maximum set of density-connected points [23]. For density based clustering approaches there is only need of minimum domain knowledge to cluster the datasets [22].

DBSCAN [25] is a popular density based clustering method used to create clusters of arbitrary shapes. DBSCAN is robust to noise, well scale to large datasets, and requires minimum number of input parameter. However, it is not completely deterministic for border points, and does not perform well on highly overlapped dense regions. Correctly estimate the input parameters is also a difficult task. To overcome the problems of DBSCAN, a new density based clustering method was proposed, abbreviated as OPTICS [22]. OPTICS can create more accurate clusters in data of varying density. However, DBSCAN, and OPTICS still cannot identify some boarder points. A number of DBSCAN variants have been proposed, such as VDBSCAN [27], DVBSCAN [24], ST-DBSCAN [26], and DBCLASD [24] to improve the performance of DBSCAN.

Alex et al. proposed a density based clustering method named as clustering by fast search and find of density peaks (CFSFDP) [40]. CFSFDP has two basic rules for clusters centers: 1) cluster center has high-density as compare to its neighbors, and 2) cluster center is positioned at high distance from other clusters centers. The effectiveness of algorithm depends upon cutoff distance (C_d), which is an essential parameter to estimate better density at each point. In CFSFDP C_d is selected based on the heuristic that the average number of neighbors is around 1 to 2% of the whole dataset [40]. To minimize the statistical errors of CFSFDP for estimating the densities and to identify the border points, there is a need to estimate C_d adaptively instead of using 1 to 2% heuristic of neighbor points.

To overcome aforementioned problem, we propose an adaptive way to estimate C_d by using the phenomena of Improved Sheather-Jones (ISJ) method [41] named as IJS-CFSFDP. ISJ provides the better estimation of data points to determine the C_d. The C_d can be used to 1) estimate the densities of underlaying dataset and 2)to detect the noise, and 3) refine the overlapping border regions of the clusters. Mean integrated squared error (MISE) is minimized to estimate the C_d by ISJ method. Experimental results on standard benchmark clustering datasets validate the effectiveness of the proposed adaptive C_d method.

The rest of this paper is organized as follows. The background knowledge is presented in Section 2. Section 3 describes the proposed method to select C_d by using Improved Sheather-Jones (ISJ) method in detail. Experimental results are presented and discussed in Section 4, and finally, the concluding remarks are presented in Section 5.

2 Background knowledge

In this section we have presented a brief introduction of CFSFDP.

2.1 CFSFDP

CFSFDP deals with two basic properties of data points: 1) local density ρ_i, and 2) distance of data point i to closest higher dense point represented as δ_i. For a data point i, local density ρ_i is calculated as follows: $ρ_{i} = \sum_{j} X (d_{ij} - C_{d}),$ (1) where $X (x) = {\begin{matrix} 1 & x < 0 \\ 0 & otherwise \end{matrix}$

C_d represents the cutoff distance and d_ij is distance between point i and j. The ρ_i is the number of points that are closer than C_d to point i. The δ_i is minimum distance of point i with a nearest high-density point. The δ_i can be calculated by Equation 2. $δ_{i} = {\begin{matrix} min_{j : ρ_{j} > ρ_{i}} (d_{ij}) & if \exists j s . t . ρ_{j} > ρ_{i} \\ max_{j : ρ_{j} > ρ_{i}} (d_{ij}) & otherwise . \end{matrix}$ (2)

The values of δ are much larger for those points, which have the maximum density globally or locally. So the centers of the clusters are points which have the maximum values of ρ and δ as shown in Fig. 1.

Figure 1(A) contains 28 data elements plotted in a 2D space with decreasing density. The decision graph in Fig. 1(B) illustrates the points 1 and 10 are the maximum density points that lies at large distance from other points, so these can be identified as cluster centers. Points 26, 27 and 28 have high values of δ and have low values of ρ. These are isolated points and hence identified as outliers or noisepoints.

With the help of decision graph one can identify clusters centers successfully. After the identification of clusters centers, other points are assigned to clusters in a single round. The rest of data elements are assigned to the nearest cluster with respect to defined clusters centers. Furthermore, for border point’s conditions, a border region for each cluster is identified. The border region of a cluster contains a set of data points that are part of the underlying cluster and also fall in the C_d radius of other cluster points. The next step is to find the maximum density in the border region of cluster, which is expressed as ρ_j . The points with higher-density than ρ_j are identified as cluster core points and other points are identified as halo point of the cluster (considered as outliers ornoise).

3 Proposed method

In this section we have presented a kernel density estimation and adaptive cutoff distance by Improved Sheather-Jones (ISJ) method as follows.

3.1 Kernel density estimation

The nonparametric density estimation is an important tool in statistical analysis of data. It can be used to evaluate the degree of symmetry in a dataset, multimodality, discriminant analysis, summarization of Bayesian posteriors and classification [41, 42]. The nonparametric density estimation is an alternative to parametric estimation of density. Unlike the classical approach, the nonparametric density estimation is not affected by specification bias [41], which makes it more flexible for modeling of data. The kernel density estimation (KDE) is most common density estimation approach [37, 43]. The standard method to estimate the density is introduced by a narrow Gaussian kernel (or alternative) ${\hat{f}}_{h} (d_{i})$ at each point d_i and compute the integral of all kernel values over the entire dataset [43, 45]. For KDE, identical and independent distributed samples are drawn {x₁, x₂, x₃, . . . , x_n} with an unknown probability density function is given as follows: ${\hat{f}}_{h} (x; C_{d}) = \frac{1}{n} \sum_{i = 1}^{n} K_{C_{d}} (x - x_{i}),$ (3) where the K_{C
_d} represents the kernel function and C_d denotes the bandwidth of the kernel function. The kernel function is scaled by C_d. Gaussian kernel is most commonly used kernel to estimate the density, which is as follows: $K (x, x_{i}; C_{d}) = \frac{1}{\sqrt{2 π C_{d}}} e^{- \frac{{(x - x_{i})}^{2}}{2 C_{d}}}$

Generally, kernel density estimator places the small kernel with bandwidth K_{C
_d} at each sample point x_i. Alex et al. proposed to use Equation 1 or Equation 3 in CFSFDP to compute the density. However, the effectiveness of KDE depends upon the appropriate choice of C_d. Mean integrated squared error (MISE) is well-studied criteria for optimal choice of smoothing parameter for quality estimation of KDE [41], which is given below:

${MISE {\hat{f}} (C_{d})}^{} = 𝔼_{f} \int [\hat{f} (x; C_{d}) - f (x)]^{2} dx$ (4)

Equation 4 can be decompose into two components: 1) integrated squared bias and 2) integrated variance. The decomposed equation is given as follow:

$\begin{matrix} MISE {\hat{f}} (C_{d}) & = & \int \underset{pointwise bias of f}{\underset{︸}{(𝔼_{f} [\hat{f} (x; C_{d})] - f (x))^{2}}} dx \\ + \int \underset{point variance of f}{\underset{︸}{({Var}_{f} [\hat{f} (x; C_{d})]}} dx, \end{matrix}$ (5) where variance and expectation operators apply to random data samples. However another alternative criteria to compute MISE is expected l1 error, $E_{f} \int | \hat{f} (x; C_{d}) - f (x) | dx,$ which is scale invariant and has better theoretical properties [44]. Computational cost of expected l1 error method is not traceable as compare to MISE. For the Gaussian kernel estimation, MISE first order asymptotic approximation is given in Equation 6.

${AMISE {\hat{f}} (C_{d})}^{} = \frac{1}{4} {C_{d}}^{2} {∥ f^{"} ∥}^{2} + \frac{1}{2 N \sqrt{C_{d}}}$ (6)

Where ∥f^" ∥ ² is equal to ∫ (f^" (x)) ² dx. The asymptotically optimized value of C_d is a minimizer of the AMISE as follows: $*^{C_{d}} = {(\frac{1}{2 N {\sqrt{∥ f^{"} ∥}}^{2}})}^{2 / 5}$ (7)

To compute the optimal $*^{C_{d}}$ from Equation 7, one needs to estimate the ∥f^" ∥ ². Consider the problem of estimating function ∥f^(k) ∥ ² for any arbitrary integer k ≥ 1. The identity ${∥ f^{(k)} ∥}^{2} = {(- 1)}^{(k)} 𝔼_{f} (f^{(2 k)} (X))$ suggests two plug-in estimators [41, 46], which are given as follow:

${(- 1)}^{(k)} 𝔼_{f} (f^{(2 k)} (X)) = \frac{{(- 1)}^{(k)}}{N^{2}} \sum_{j = 1}^{N} \sum_{m = 1}^{N} \emptyset^{2 k} (X_{j}, X_{m}; C_{d_{k}})$

${∥ \hat{f^{(k)}} ∥}^{2} = \frac{{(- 1)}^{(k)}}{N^{2}} \sum_{j = 1}^{N} \sum_{m = 1}^{N} \emptyset^{2 k} (X_{j}, X_{m}; 2 C_{d_{k}})$

The both estimators ${(- 1)}^{(k)} 𝔼_{f} (f^{(2 k)} (X))$ and ${∥ \hat{f^{(k)}} ∥}^{2}$ estimates the same quantity for any given smoothing parameter. To make both estimators asymptotically equivalent in sense of mean squared error, the $*^{C_{d_{k}}}$ can be selected as follows:

$\begin{matrix} *^{{C_{d}}_{k}} \\ = {(\frac{1 + 1 / 2^{k + 1 / 2} * 1 * 3 * 5 *, . . ., * (2 k - 1)}{N \sqrt{/ 2} {∥ f^{(k + 1)} ∥}^{2}})}^{2 / (3 + 2 k)} \end{matrix}$ (8)

In Equation 8 the computation of $*^{{C_{d}}_{k}}$ requires the estimation of ∥f^(k+1) ∥ ² which is unknown. So, each $*^{{C_{d}}_{k}}$ is estimated by using following expression:

$\begin{matrix} *^{{\hat{C_{d}}}_{k}} \\ = {(\frac{1 + 1 / 2^{k + 1 / 2} * 1 * 3 * 5 *, . . ., * (2 k - 1)}{N \sqrt{/ 2} {∥ \hat{f^{(k + 1)}} ∥}^{2}})}^{2 / (3 + 2 k)} \end{matrix}$ (9)

The computation of ${∥ \hat{f^{(k + 1)}} ∥}^{2}$ needs the estimation of $*^{{\hat{C_{d}}}_{k + 1}}$ itself, which in turn requires the estimation of $*^{{\hat{C_{d}}}_{k + 2}}$ , and so on, as seen from Equation 9. The problem in computation of ${∥ \hat{f^{(k + 1)}} ∥}^{2}$ is to estimate the infinite sequence { $*^{{\hat{C_{d}}}_{k + n}}, n \geq 1}$ . However, if $*^{{\hat{C_{d}}}_{k + 1}}$ is given for some l >0, then all { $*^{{\hat{C_{d}}}_{k}}, 1 \leq k \leq 1}$ can be estimated recursively. The l-stage direct plug-in bandwidth selector [46] uses this idea to find optimal bandwidth. For a given integer l > 0, the l-stage plug-in method select C_d by computing the following equation: $*^{{\hat{C_{d}}}_{}} = ξ γ^{[l]} (*^{C_{d_{l + 1}}}),$ (10)

where the $*^{C_{d_{l + 1}}}$ is estimated via ∥ f^(l+1) ∥ ² computing by assuming that f is a normal density with variance and mean estimated from the data [41, 46]. Because of assumptions used in l-stage plug-in method can lead to arbitrarily bad estimation of $*^{C_{d}}$ , for example, when true f is far from being Gaussian. Botev et al. [41] proposed a unique solution of the nonlinear equation: $C_{d} = ξ γ^{[l]} (C_{d})$ (11)

For higher value of l, Equation 11 can either be solved with Newton’s method or fixed point iteration while considering with initial condition C_d=0 [38, 41]. Equation 11 can be solved with fixed point iteration as shown in the following algorithm:

3.2 Proposed method for density estimation

For given l > 2, initialized with $ɛ_{0} = ɛ$ , where ɛ is machine precision, and n=0;

Set $ɛ_{n + 1} = ξ γ^{[l]} (ɛ_{n})$ ;

If $| ɛ_{n + 1} = ɛ_{n} | < ɛ$ , then stop and set $*^{\hat{C_{d}}} = ɛ_{n + 1}$ ; otherwise, set n = n + 1 and repeat from step 2;

Deliver the Gaussian kernel density estimator in Equation 3 evaluated at $*^{\hat{C_{d}}}$ as the final estimator of f, and $*^{\hat{C_{d_{2}}}} = γ^{[l]} (ɛ_{n + 1})$ as the smoothing parameter for optimal estimation of ${‖ f^{″} ‖}^{2}$ .

The improved Sheather-Jones uses O (n) operations to get an optimal bandwidth.

4 Results and discussion

To validate the robustness of IJS-CFSFDP, four standard synthetic clustering datasets (aggregation [47], flame [48], spiral [49], and toys problem [50]) are used in our experiments and compared with CFSFDP, mean shift, affinity propagation and k-means.

Flame dataset consists of 240 data points and comprises of 2 clusters. Figure 2(A) shows ground truth of flame dataset. Figure 2(B) shows clusters, which are formed by using C_d as 2% of whole dataset. CFSFDP identify most of the core points as noise. In Fig. 2(C) clusters are formed by considering 1% neighbors exist around a point in dataset. However, the effect to decrease in C_d cannot create good clusters as shown in Fig. 2(C). In Fig. 2(C) CFSFDP identified some of core points as noise points and also could not identify the compact relationship among the density connected points. Figure 2(D) shows clusters created by IJS-CFSFDP in which an optimal adaptive C_d is used to estimate the accurate density of dataset. By estimating accurate density with optimal choice of C_d, IJS-CFSFDP can creates better clusters and discovers density connected points in dataset, successfully.

We have also compared the effectiveness of IJS-CFSFDP on toys problem datasets. The datasets consist of 300, 1500 points having 2 clusters. Figure 4(A) shows ground truth of toys problem dataset having 1500 data points. CFSFDP identified some of the clusters core and boarder points as halo points as shown in Fig. 4(B). Furthermore, CFSFDP cannot discover a maximum density connective points to integrate different parts of a cluster. Figure 4(C) shows clusters formed by calculating the ρ of each object using IJS-CFSFDP which produces accurate clusters successfully and provides strong foundation for density connectivity. Figure 4(D) shows the toys problem dataset having 300 data points with comparatively more disperse densities. Figure 4(E) shows clusters created by CFSFDP. CFSFDP has detected more core points as noise and also could not detect effective relationship among connected densities. The appropriate selection of C_d is important to discover connected densities and to detect border points. Figure 4(F) shows the accurate clusters created by IJS-CFSFDP.

Path-based spiral dataset consist of 312 data points with separation of 3 clusters spread over 2 dimensional space. CFSFDP creates same clusters as compared to IJS-CFSFDP because of spiral dataset have not overlapping densities. In our experiments we have used the same parameters used by mean shift, AP, k-means, and CFSFDP their experiments. Figure 5(A) shows the visual results on aggregation dataset. Aggregation is 2 dimensional dataset consists of 788 data points and comprises of 7 clusters. Figure 5(B) shows the clusters created by mean shift with window size of 4.6. After a number of iteration at window size of 4.6 it creates better clusters. However, the accuracy of mean shift is subject to the accurate selection of window size. The Fig. 5(C) shows seven clusters created by k-means. The k-means is effective when there is no overlapping in dataset. However, it could not create good clusters when data distribution is highly complex and overlapped. Figure 5(D) shows the clusters created by AP. The AP separates the flame dataset into 18 clusters. Figure 5(E) shows the visualization of resultant clusters on aggregation dataset using CFSFDP while Fig. 5(F) shows the visual result of clusters create by IJS-CFSFDP. CFSFDP has identified some boarder points as outliers, which are actually part of clusters. Where IJS-CFSFDP identified the border points as part of cluster, accurately.

However, the CFSFDP results as shown in Fig. 5(E) misclassify 82 points and declared as noise points at C_d = 1.860108 calculated by given method in CFSFDP. The ISJ based cutoff distance in CFSFDP method adaptively calculates the optimal cutoff distance for CFSFDP, which resultant to the better clusters as compare to k-means, mean shift, AP and CFSFDP. The mean shift is also use kernel density estimator to estimate the density of dataset like CFSFDP. However, both have deficiency to select optimal smoothing parameter to estimate accurate densities and to discover boarder points as well. The optimal C_d and window size is hard to estimate for users. However, the IJS-CFSFDP gives more robust methodology to estimate adaptively optimal C_d in order to get robust performance of CFSFDP.

After a numerous experiments we conclude that IJS-CFSFDP works well in small size datasets. However, in compound dataset [51] IJS-CFSFDP could not separate two red clusters as shown in Fig. 6 (B). IJS-CFSFDP merged inner small cluster into outer red cluster because its density was very low and it was very near to high dense region of outer cluster. In future, we will consider to refine this issue.

Recently, the incorporation of web based systems, social networks, ubiquitous computing, and IoT in education system has changed the entire education systems. The web based education systems are virtual form of instructions and beyond the geographical limitations. The modern education systems are generating the massive amount of data that can be used to extract the hidden intrinsic pattern. The clustering is a major technique to analyze data intelligently. Mostly, the existing educational system are static and could not satisfy the diversity of students. Clustering technique can be utilize to make the existing education systems smarter and intelligent [10, 11].

To improve the academic performance of student, we utilize the IJS-CFSFDP to group the students into unique clusters based on their GPA obtained in previous semester. With the special attention of teachers and counseling toward the weak group of students, their performance can be improve. The proposed method can also be utilize to make intelligent contents delivery system based on the capability and intelligence of the student.

Table 1 shows the comparison of C_d and identified clusters points between the standard CFSFDP and IJS-CFSFDP on different datasets. The CFSFDP fails to identify most of the core points and declare them as noise points. However, in propose adaptive estimation of C_d, IJS-CFSFDP can identify the core points, correctly. The accuracy is achieved because of propose adaptive estimation of C_d better express the density of dataset.

5 Conclusion

In this paper, ISJ based adaptive optimal cutoff distance method is proposed for CFSFDP named as IJS-CFSFDP. ISJ method is an adaptive way to calculate C_d of a dataset, which is further used in kernel density estimation and identify the border points in CFSFDP. ISJ gives most accurate estimation of smoothing parameter because it does not use normal references rules as compare to other state-of-the-art methods used to calculate smoothing parameter in KDE. ISJ provides minimum value of AMISE, which effectively used as C_d in CFSFDP to cluster datasets adaptively. So, the problem of C_d in CFSFDP is solved by proposed method. Experiments results on four benchmark datasets shows the robustness and effectiveness of the IJS-CFSFDP as compared to CFSFDP.

Footnotes

Acknowledgments

This research is sponsored by National Natural Science Foundation of China (Nos. 61171014, 61371185, 61401029, 61472044, 61472403, 61571049) and the Fundamental Research Funds for the Central Universities (Nos. 2014KJJCB32, 2013NT57) and by SRF for ROCS, SEM.

References

, Feng

, Zhang

and Jiao

, SAR image segmentation based on quantum-inspired multiobjective evolutionary clustering algorithm, Information Processing Letters114(6) (2014), 287–293.

and Tan

Y.-P.

, Uncorrelated discriminant simplex analysis for view-invariant gaitsignal computing, Pattern Recognition Letters31(5) (2010), 382–393.

and Zhang

, Gait recognition for human identification based on ICA and fuzzy SVM through multiple views fusion, Pattern Recognition Letters28(16) (2007), 2401–2411.

Jones Kristen

and Lacy

, Measuring the clustering around normal and dust-obscured quasars at 2 in the spitzer extragalactic representative volume survey (SERVS),, In American Astronomical Society Meeting Abstracts223(223) (2014).

Yan

, Qian

, Sharif

and Tipper

, A survey on cyber security for smart grid communications, IEEE Communications Surveys and Tutorials14(4) (2012), 998–1010.

Lefevre

, Rondet

, Parizot

and Chauvin

, Applying multivariate clustering techniques to health data: The 4 types of healthcare utilization in the paris metropolitan area, PloS One9(12) (2014).

and Su

, Identification of cell types from single-cell transcriptomes using a novel clustering method, Bioinformatics37(10) (2015), 2041–2256.

Jaeger

, Barth

, Niehues

and Fufezan

, pyGCluster, a novel hierarchical clustering approach, Bioinformatics30(6) (2014), 896–898.

Chang

M.-S.

, Chen

L.-H.

, Hung

L.-J.

, Rossmanith

and Wu

G.-H.

, Exact algorithms for problems related to the densest k-set problem, Information Processing Letters114(9) (2014), 510–513.

10.

Shovon

Md.H.I.

and Haque

, An Approach of Improving Student’s Academic Performance by using K-means clustering algorithm and Decision tree, International Journal of Advanced Computer Science and Applications3(8) (2012).

11.

Markowska-Kaczmar

, Kwasnicka

and Paradowski

, Intelligent techniques in personalization of learning in e-learning systems, In Computational Intelligence for Technology Enhanced Learning, SpringerBerlin Heidelberg, 2010, pp. 1–23.

12.

, Liong

V.E.

, Zhou

and Zhou

, Learning compact binary face descriptor for face recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)37(10) (2015), 2041–2256.

13.

, Zhou

, Tan

Y.-P.

, Shang

and Zhou

, Neighborhood repulsed metric learning for kinship verification, IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI)36(2) (2014), 331–345.

14.

et al., Managing big city information based on webVRGIS, IEEE Access4 (2016), 407–415.

15.

Sun

, Bie

and Zhang

, Measuring semantic-based structural similarity in multi-relational networks, International Journal of Data Warehousing and Mining12(1) (2016), 20–33.

16.

Lin

, Yang

, Lv

et al., A self-assessment stereo capture model applicable to the internet of things[J], Sensors15(8) (2015), 20925–20944.

17.

Mehmood

, Shaikh

M.U.

and Bie

, amd H. Dawood, Iot enable web warehouse architecture: A secure approach, Personal and Ubiquitous19(7) (2015), 1157–1167.

18.

Sun

and Jara

A.J.

, An extensible and active semantic model of information organizing for the Internet of Things, Personal and Ubiquitous18(8) (2014), 1821–1833.

19.

Sun

, Yan

, Lu

, Bie

and Zhou

, Constructing the web of events from raw data in the web of things, Mobile Information Systems11(1) (2014), 105–125.

20.

Sun

, Yan

, Zhang

, Xia

, Wang

, Bie

and Tian

, Organizing and querying the big sensing data with event-linked network in the internet of things, International Journal of Distributed Sensor Networks2014 (2014).

21.

, Yin

, Han

, Chen

et al., WebVR–web virtual reality engine based on P2P network[J], Journal of Networks6(7) (2011), 990–998.

22.

Lovely Sharma

and Ramya

K.A.

, Review on density based clustering algorithms for very large datasets, International Journal of Emerging Technology and Advanced Engineering3(12) (2013), 398–403.

23.

Shah

G.H.

, Bhensdadia

C.K.

and Ganatra

A.P.

, An empirical evaluation of density-based clustering techniques, International Journal of Soft Computing and Engineering (IJSCE) (2012), 2231–2307.

24.

Parimala

, Lopez

and Senthilkumar

N.C.

, A survey on density based clustering algorithms for mining large spatial databases, International Journal of Advanced Science and Technology31(1) (2011), 59–66.

25.

Ester

, Kriegel

H.-P.

, Sander

and Xu

, A density-based algorithm for discovering clusters in large spatial databases with noise, In Kdd96(34) (1996), 226–231.

26.

Birant

and Kut

, ST-DBSCAN: An algorithm for clustering spatial-temporal data, Data and Knowledge Engineering60(1) (2007), 208–221.

27.

Liu

, Zhou

and Wu

, VDBSCAN: Varied density based spatial clustering of applications with noise, In Procedding: Service Systems and Service Management, 2007, pp. 1–4.

28.

Mehmood

, Zhanga

, Biea

, Dawooda

and Ahmad

, Clustering by fast search and find of density peaks via heat-diffusion, Neurocomputing, Special Issue on Bridging the Semantic Gap in Multimedia Understanding, 2016 (Forthcoming).

29.

Sheather

S.J.

and Jones

M.C.

, A reliable data-based bandwidth selection method for kernel density estimation, Journal of the Royal Statistical Society Series b-Methodological53(3) (1991), 683–690.

30.

Franti

and Virmajoki

, Iterative shrinking method for clustering problems, Pattern Recognition39(5) (2006), 761–775.

31.

Bhatnagar

, Kaur

and Chakravarthy

, Clustering data streams using grid-based synopsis, Knowledge and Information Systems41(1) (2014), 127–152.

32.

Sun

and Lu

, A grid-based subspace clustering algorithm for high-dimensional data streams, In: Proceeding of 1st International Conference on Advanced Data Mining and Applications Location: Wuhan, PEOPLES R CHINA, 2005.

33.

Chen

, Xu

Z.-S.

and Xia

M.-M.

, Hierarchical hesitant fuzzy K-means clustering algorithm, Applied Mathematics-A Journal of Chinese Universities29(1) (2014), 1–17.

34.

Jacques

and Preda

, Functional data clustering: A survey, Advances in Data Analysis and Classification8(3) (2014), 231–255.

35.

MacQueen

, Some methods for classification and analysis of multivariate observations, In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability1(14) (1967), 281–297.

36.

Frey

B.J.

and Dueck

, Clustering by passing messages between data points, Science315(5814) (2007), 972–976.

37.

Cheng

, Mean shift, mode seeking, and clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence17(8) (1995), 790–799.

38.

Krishnaswamy

, Matthew

H.S.

, Mingueneau

, Bendall

S.C.

, Litvin

, Stone

, Pe’er

and Nolan

G.P.

, Conditional density-based analysis of T cell signaling in single-cell data, Science346(6213) (2014), 1250689.

39.

Lahari

, Ramakrishna Murty

and Satapathy

S.C.

, Partition based clustering using genetic algorithm and teaching learning based optimization: Performance analysis, Advances in Intelligent Systems and Computing338 (2015), 191–200.

40.

Rodriguez

and Laio

, Clustering by fast search and find of density peaks, Science344(6191), 1492–1496.

41.

Botev

Z.I.

, Grotowski

J.F.

and Kroese

D.P.

, Kernel density estimation via diffusion, The Annals of Statistics38(5) (2010), 2916–2957.

42.

Scott

D.W.

, Multivariate Density Estimation. Theory, Practice and Visu-alization, Wiley, New York, MR1191168, 1992.

43.

Lehmann

E.L.

, Model specification: The views of Fisher and Neyman, and later develop-ments, Statistical Science5(2) (1990), 160–168.

44.

Felber

, Kohler

and Krzyrak

, Adaptive Density Estimation from Data Containing Bounded Measurement Errors, In Stochastic Models, Statistics and Their Applications, 2015, pp. 67–74.

45.

Rosenblatt

, Remarks on some nonparametric estimates of a density-function, Annals of Mathematical Statistics27(3) (1956), 832–837.

46.

Fukunaga

and Hostetler

, The estimation of the gradient of a density function, with applications in pattern recognition, IEEE Transactions on Information Theory21 (1975), 32–40.

47.

Gionis

, Mannila

and Tsaparas

, Clustering aggregation, ACM Transactions on Knowledge Discovery from Data (TKDD)1(1) (2007), 1–30.

48.

and Medico

, FLAME, a novel fuzzy clustering method for the analysis of DNA microarray data, BMC Bioinformatics8(3) (2007), artical.

49.

Chang

and Yeung

D.Y.

, Robust path-based spectral clustering, Pattern Recognition41(2) (2008), 191–203.

50.

Pedregosa

, Varoquaux

, Gramfort

, Michel

, Thirion

, Grisel

and Duchesnay

, Scikit-learn: Machine learning in Python, The Journal of Machine Learning Research12 (2011), 2825–2830.

51.

Zahn

C.T.

, Graph-theoretical methods for detecting and describing gestalt clusters, IEEE Transactions on Computers100(1) (1971), 68–86.