Parallel simulation of neural networks

Abstract

We simulate five neural networks on a vector multiprocessor. The training time can be reduced significantly especially when the training data size is large. These five neural networks are: 1) the ART1 network, 2) the ART 2 network, 3) the feedforward network, 4) the recurrent network, and 5) the Hopfield network. The training algorithms are programmed in such a way to best utilize 1) the inherent parallelism in neural computing, and 2) the vector and concur rent operations available on the parallel machine. To prove the correctness of parallelized training algorithms, each neural network is trained to perform a specific function. ART 1 and ART 2 are trained to recognize binary and analog patterns. The feedforward network is trained to perform the Fourier transform, the recurrent network is trained to predict the solution of a delay differential equation, the Hopfield network is trained to solve the traveling salesman problem. The machine we experiment with is the Alliant FX/80.

Get full access to this article

View all access options for this article.

References

Waibel, A. , Hanazawa, T. , Hinton, G. , Shikano, K. , and Lang, K. "Phoneme recognition: neural networks vv. hidden Markov models," Proceedings of Int'1 Conference on ASSP, 1988.

Alan Lapedes and Robert Farber , 'Nonlinear signal processing using neural networks: prediction and system modeling," technical report, LA-UR-87-2662, Los Alamos.

Jorgensen, C.C. "Neural network representation for sensor graphs in autonomous robot path planning," Proceedings of the IEEE First Int'l Conf. on Neural Networks, pp. 507-515, San Diego, CA.1987.

Mesrobian, E. and Skrzypek, J. , "Discrimination of natural textures: A neural network architecture," Proceedings of the IEEE First Int'1 Conf. on Neural Networks, pp. 247-258, San Diego, CA . 1987.

McClelland, J.L. and Kawamoto , A.H. ,"Mechanisms of sentence processing: Assigning roles to constituents of sentences," in Parallel Distributed Processing pp. 272-325, MIT Press, Cambridge, MA.

G.A. Carpenter and S. Grossberg , "A massively parallel architecture for a self-organizing neural pattern recognition machine," Computer Vision, Graphics, and Image Processing, 37, 54-115, 1987.

G.A. Carpenter and S. Grossberg , "ART2: Self-organization of stable category recognition codes for analog input patterns," Applied Optics, 26, 4919-4930,1987.

G.A. Carpenter and S. Grossberg , "ART3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures ," Neural Networks, 3,129-152,1990.

S. Grossberg , "Nonlinear neural networks: principles, mechanisms, and architectures," Neural Networks, 1, pp 17-61, 1988.

10.

Lippmann, R.P. ,"An introduction to computing with neural nets ," IEEE ASSP magazine, pp. 4-22, April, 1987.

11.

Alliant Computer Systems Corporation , FX/FORTRAN Language Manual, July 1988.

12.

Pineda, F.J. ," Generalization of back-propagation to recurrent neural networks," Physical Review Letters, Vol. 59, No. 19, 9 November 1987.

13.

Hopfield, J.J. and Tank, D.W. ,"Neural computation of decisions in optimization problem," Biol. Cybern. 52,141-152 (1985).

14.

Wilson, G.V. and Pawley, G.S. ,"On the stability of the travelling salesman problem algorithm of Hopfield and Tank," Biol. Cybern. 58,63-70 (1988).

15.

Szu, H. ," Fast TSP algorithm based on binary neuron output and analog neuron input using the zero-diagonal interconnect matrix and necessary and sufficient constraints of the permutation matrix," Proceedings of IEEE Int'l Conf. on Neural Networks, Vol. II pp. 259-266,1988.

16.

Sabochick, M.J. and Richlin, D.L. ,"New global energy-minimization method," Physical Review B, Vol. 37, No.18, 846-850,15 June, 1988.