Manifold distance-based particle swarm optimization for classification

Abstract

Classification is one of the most important research topics in data mining. In this paper, a manifold distance-based particle swarm optimization (MDPSO) for classification is proposed. Firstly, a new selection strategy is designed to maintain information of some good solutions as well as enhance the diversity of swarm. Furthermore, a manifold distance is utilized to measure the similarity of two samples, which measures the geodesic distance along the manifold and can better reflect the global consistency of the given problems. MDPSO is applied to solve 10 benchmark classification problems selected from UCI (University of California Irvine) datasets and 10 artificial data sets. The experimental results show that MDPSO outperforms the typical C4.5, nearest neighbour, and Michigan particle swarm optimization on most of the testing problems used in this paper. In addition, MDPSO is applied to a real-world application problem, namely handwritten digit recognition, with a good performance obtained.

Keywords

Classification manifold distance particle swarm selection operator swarm intelligence

Get full access to this article

View all access options for this article.

References

Angeline

(1998) Using selection to improve particle swarm optimization. In: Proceedings of the IEEE International Conference on Evolutionary Computation, Anchorage, AK, USA, pp. 84–89.

Bandyopadhyay

Maulik

(2001) Nonparametric genetic clustering: comparison validity indices. IEEE Transactions on Systems, Man and Cybernetics, Part C 31(1): 120–125.

Bandyopadhyay

Maulik

(2002) Genetic clustering for automatic evolution of clusters and application to image classification. Pattern Recognition 35: 1197–1208.

Burges

CJC

(1998) A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2: 121–167.

Caponetto

Fortuna

Grazion

Xibilia

(1993) Genetic algorithms and applications in system engineering: a survey. Transactions of the Institute of Measurement and Control 15(3): 143–156.

Cervantes

Isasi

Galván

(2005) Binary particle swarm optimization in classification. Neural Networks World 15(3): 229–241.

Cervantes

Galván

Isasi

(2007) Building nearest prototype classifiers using a Michigan approach PSO. Proceeding of the 2007 IEEE Swarm Intelligence Symposium, pp. 135–140.

Cervantes

Galván

Isasi

(2009) Michigan particle swarm optimization for prototype reduction in classification problems. New Generation Computing 27: 239–257.

Chatterjee

Siarry

(2006) Nonlinear inertia weight variation for dynamic adaptation in particle swarm optimization. Computers and Operations Research 33(3): 859–871.

10.

Chekina

Gutfreund

Kontorovich

. (2012) Exploiting label dependencies for improved sample complexity. Machine Learning. DOI 10.1007/s10994-012-5312-9

11.

Clerc

Kennedy

(2002) The particle swarm-explosion, stability and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6(1): 58–73.

12.

Dong

Krzyzak

Suen

(2002) Local learning framework or handwritten character recognition. Engineering Applications of Artificial Intelligence 5(2): 603–618.

13.

Esmin

AAA

(2007) Generating fuzzy rules from examples using the particle swarm optimization algorithm. In Proceedings of the HIS, pp. 340–343.

14.

Falco

Della Cioppa

Tarantion

(2006) Evaluation of Particle Swarm Optimizaiton Effectiveness in Classification. Berlin: Springer-Verlag.

15.

Garcia

Molina

Lozano

Herrera

(2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization. Journal of Heuristics 15: 617–644.

16.

Gong

Jiao

Wang

Zhang

(2008) Image texture classification using a manifold distance based evolutionary clustering method. Optical Engineering 47(7): 1–10.

17.

Islam

Jonathan Wu

Ahmadi

. (2007) Investigating the performance of naive-Bayes classifiers and k-nearest neighbor classifiers. In: International Conference on Convergence Information Technology, pp. 1541–1546. DOI 10.1109/ICCIT.2007.148.

18.

Kennedy

Eberhart

(1995) Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Network, Perth, Australia, Vol. 4, pp. 1942–1948.

19.

Quinlan

(1986) Induction of decision trees. Machine Learning 1: 81–106.

20.

Shi

Eberhart

(1998a) A modified particle swarm optimizer. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 69–73.

21.

Shi

Eberhart

(1998b) Parameter selection in particle swarm optimization. In: Porto

Saravanan

Waagen

. (eds) Lecture Notes in Computer Science 1447: 591–600.

22.

UCI Machine Learning Repository (2007). Available at: http://archive.ics.uci.edu/ml/ (accessed 8 January 2013).

23.

Wang

Sun

Zhang

(2006) Classification Rule Mining Based on Particle Swarm Optimization, Vol. 4062/2006. Berlin: Springer-Verlag.

24.

Wang

Sun

Zhang

(2007) A PSO-Based Classification Rule Mining Algorithm, Vol. 4682/2007. Berlin: Springer-Verlag.

25.

Zhang

(2000) Neural networks for classification: a survey. IEEE Transactions on Systems, Man, and Cybernetics – Part C: Applications and Reviews 30(4): 451–462.