Real-time classifier based on adaptive competitive self-organizing algorithm

Abstract

This article introduces a novel adaptive competitive self-organizing (ACS) model, with applicability for real-time clustering and vector quantization. An important feature of this model is its dynamic structure and self-adjusting parameters that also offers a solution to the problem of parasitic limit points and consequently in more accurate label assignments. This unsupervised classifier is free of any external control mechanism. Its self-organizing (SO) dynamic is governed by the gradient descent (GD) theory in cooperation with a competition mechanism based on Lotka–Volterra competitive exclusion. The core algorithm of this classifier is based on developing an energy function, where its minima or equilibrium points correspond to the centroid of similar input patterns. Since this energy function is a form of Lyapunov function, it guarantees stabilization of the dynamical trajectories of labels in finite numbers of isolated equilibrium points. This energy function along with other control parameter functions, then, will be the base for the set of ordinary differential equations (ODEs) describing the overall dynamic of our system. Finally, the effectiveness of the proposed ACS model is demonstrated by implementing it on both real and artificial data sets as well as comparing with other well-known clustering methods. ACS method showed a better clustering performance in some categories and an overall comparable rendition.

Keywords

Real-time classifier self-organizing gradient descent competition mechanism equilibrium points parasitic limit points

Get full access to this article

View all access options for this article.

References

Amorim

D. G.

Delgado

M. F.

Ameneiro

S. B.

(2007). Polytope ARTMAP: Pattern classification without vigilance based on general geometry categories. IEEE Transactions on Neural Networks, 18, 1306–1325.

Anzai

(2012). Pattern recognition & machine learning. Elsevier.

Athinarayanan

Sayeh

M. R.

Wood

D. A.

(2002). Adaptive competitive self-organizing associative memory. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 32, 461–471.

Bahaa

E. S.

Malvin

C. T.

(1991). Fundamentals of photonics. New York, NY: Wiley.

Banerjee

Bovolo

Bhattacharya

Bruzzone

Chaudhuri

Mohan

B. K.

(2015). A new self-training-based unsupervised satellite image classification technique using cluster ensemble strategy. IEEE Geoscience and Remote Sensing Letters, 12, 741–745.

Ben-Hur

Horn

Siegelmann

H. T.

Vapnik

(2001). Support vector clustering. Journal of Machine Learning Research, 2, 125–137.

Carlsson

Mémoli

Ribeiro

Segarra

(2016). Excisive Hierarchical Clustering Methods for Network Data. ArXiv Preprint. arXiv:1607.06339.

Carpenter

G. A.

Grossberg

(1988). The ART of adaptive pattern recognition by a self-organizing neural network. Computer, 21, 77–88.

Celebi

M. E.

(Ed.). (2015). Partitional clustering algorithms. Cham, Switzerland: Springer.

10.

Cheng

Sayeh

M. R.

Zargham

M. R.

Cheng

(2011). Real-time vector quantization and clustering based on ordinary differential equations. IEEE Transactions on Neural Networks, 22, 2143–2148.

11.

Cox

T. F.

Cox

M. A. A.

(2001). Multidimensional scaling (2nd ed.). Boca Raton, FL: Chapman and Hall.

12.

Deboeck

Kohonen

(Eds.). (2013). Visual explorations in finance: With self-organizing maps. Springer.

13.

Everitt

B. S.

Landau

Leese

(2001). Cluster analysis. London, England: Edward Arnold.

14.

Fisher

R. A.

(1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 179–188.

15.

Grossberg

(2013). Adaptive resonance theory. Scholarpedia, 8(5), 1569.

16.

Hofmann

Buhmann

(1995). Multidimensional scaling and data clustering. (1995). In Tesauro

Touretzky

D. S.

Leen

T. K.

, (Eds.), Advances in neural information processing systems 7 (pp. 459–466). Cambridge, MA: MIT Press.

17.

Hopfield

J. J.

(1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79, 2554–2558.

18.

Horn

Gottlieb

(2001). Algorithm for data clustering in pattern recognition problems based on quantum mechanics. Physical Review Letters, 88(1), Article 018702.

19.

Horn

Gottlieb

(2002). The method of quantum clustering. In Advances in neural information processing systems (NIPS) (Vol. 14, pp. 769–776).

20.

Kaden

Lange

Nebel

Riedel

Geweniger

Villmann

(2014). Aspects in classification learning: Review of recent developments in learning vector quantization. Foundations of Computing and Decision Sciences, 39, 79–105.

21.

Kohonen

(1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 59–69.

22.

Kohonen

(2013). Essentials of the self-organizing map. Neural Networks, 37, 52–65.

23.

H. H.

(2009). Generalized competitive learning of Gaussian mixture models. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 39, 901–909.

24.

Marasigan

Z. A.

Dionisio

Solano

(2015, July). Microarray data clustering and visualization tool using self-organizing maps. In Information, intelligence, systems and applications (IISA), 2015 6th international conference on (pp. 1–5). IEEE.

25.

Meyer-Base

Thummler

(2008). Local and global stability analysis of an unsupervised competitive neural network. IEEE Transactions on Neural Networks, 19, 346–351.

26.

Pang

Jia

Zhang

Huang

Yin

(2015). Unsupervised web topic detection using a ranked clustering-like pattern across similarity cascades. IEEE Transactions on Multimedia, 17, 843–853.

27.

Príncipe

J. C.

Miikkulainen

(2009). Advances in self-organizing maps (Lecture Notes in Computer Science). Berlin, Germany: Springer.

28.

Rego

R. L. M. E.

Araújo

A. F. R.

de Lima Neto

F. B.

(2010). Growing self-reconstruction maps. IEEE Transactions on Neural Networks, 21, 211–223.

29.

Ripley

B. D.

(1996) Pattern recognition and neural networks. Cambridge, UK: Cambridge University Press.

30.

Roberts

S. J.

(1997). Parametric and non-parametric unsupervised cluster analysis. Pattern Recognition, 30, 261–272.

31.

Rokach

Maimon

(2005). Clustering methods. In Maimon

Rokach

(Eds.), Data mining and knowledge discovery handbook (pp. 321–352). New York, NY: Springer.

32.

Sayeh

M. R.

Athinarayanan

Zargham

M. R.

(1994). Operation of an associative memory. Pattern Recognition, 27, 1815–1821.

33.

Silva

T. C.

Zhao

(2012). Stochastic competitive learning in complex networks. IEEE Transactions on Neural Networks and Learning Systems, 23, 385–398.

34.

Sutskever

Martens

Dahl

G. E.

Hinton

G. E.

(2013). On the importance of initialization and momentum in deep learning. Proceedings of Machine Learning Research, 28, 1139–1147.

35.

Wunsch

(2005). Survey of clustering algorithms. IEEE Transactions on Neural Networks, 16, 645–678.