'pMATLAB Parallel MATLAB Library'

Abstract

MATLAB® has emerged as one of the languages most commonly used by scientists and engineers for technical computing, with approximately one million users worldwide. The primary benefits of MATLAB are reduced code development time via high levels of abstractions (e.g. first class multi-dimensional arrays and thousands of built in functions), interpretive, interactive programming, and powerful mathematical graphics. The compute intensive nature of technical computing means that many MATLAB users have codes that can significantly benefit from the increased performance offered by parallel computing. pMatlab provides this capability by implementing parallel global array semantics using standard operator overloading techniques. The core data structure in pMatlab is a distributed numerical array whose distribution onto multiple processors is specified with a “map” construct. Communication operations between distributed arrays are abstracted away from the user and pMatlab transparently supports redistribution between any block-cyclic-overlapped distributions up to four dimensions. pMatlab is built on top of the MatlabMPI communication library and runs on any combination of heterogeneous systems that support MATLAB, which includes Windows, Linux, MacOS X, and SunOS. This paper describes the overall design and architecture of the pMatlab implementation. Performance is validated by implementing the HPC Challenge benchmark suite and comparing pMatlab performance with the equivalent C+MPI codes. These results indicate that pMatlab can often achieve comparable performance to C+MPI, usually at one tenth the code size. Finally, we present implementation data collected from a sample of real pMatlab applications drawn from the approximately one hundred users at MIT Lincoln Laboratory. These data indicate that users are typically able to go from a serial code to an efficient pMatlab code in about 3 hours while changing less than 1% of their code.

Keywords

parallel computing parallel programming models parallel MATLAB HPC challenge

Get full access to this article

View all access options for this article.

References

Abadi, M. , Cardelli, L. , Pierce, B. , and Plotkin, G. (1991). Dynamic typing in a statically typed language , ACM Transactions on Programming Languages and Systems, 13(2): 237—268 , ACM Press .

Bikshandi, G. , Guo, J. , Hoeflinger, D. , Almasi, G. , Fraguela, B. , Garzaran, M. , Padua, D. , and von Praun, C. (2006). Programming for parallelism and locality with hierarchically tiled arrays , in Proceedings of the Eleventh ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP06), New York, March.

Choy, R. (2003). Parallel Matlab survey, http://supertech.lcs.mit.edu/~cly/survey.html

Choy, R. and Edelman, A. (2005). Parallel MATLAB: doing it right . Proceedings of the IEEE, 93(2).

CMTM Cornell multitask toolbox for MATLAB, http://www.cs.cornell.edu/Info/People/lnt/multimatlab.html

Cummings, J.C. , Crotinger, J.A. , Haney, S.W. , Humphrey, W.F. , Karmesin, S.R. , Reynders, J.V. , Smith, S.A. , and Williams, T.J. (1998). Rapid application development and enhanced code interoperability using the POOMA framework , in Proceedings of the SIAM Workshop on Object-Oriented Methods and Code Interoperability in Scientific and Engineering Computing (OO98), Yorktown Heights, NY, October.

Dean, L. , Grad-Freilich, S. , Kepner, J. , and Reuther, A. (2005). Distributed and parallel computing with MATLAB, tutorial presented at Supercomputing 2005, November 12, Seattle, WA .

DeLuca, C.M. , Heisey, C.W. , Bond, R.A. , and Daly, J.M. (1997). A portable object-based parallel library and layered framework for real-time radar signal processing , in Proceedings of the First Conference on International Scientific Computing in Object-Oriented Parallel Environments (ISCOPE '97), Marina Del Rey, CA, pp. 241— 248.

Dongarra, J. , van de Geijn, R. , and Walker, D. (1994). Scalability issues affecting the design of a dense linear algebra library , Journal of Parallel and Distributed Computing, 22: 523—537 .

10.

El-Ghazawi, T. , Carlson, W. , Sterling, T. , and Yelick, K. (2005). UPC: Distributed shared memory programming, John Wiley and Sons, Hobohen, NJ .

11.

Falcon. Falcon project: Fast array language computation, http:// www.csrd.uiuc.edu/falcon/falcon.html

12.

Funk, A. , Kepner, J. , Basili, V. , and Hochstein, L. (2005). A relative development time productivity metric for HPC systems , Proceedings of the High Performance Embedded Computing Workshop (HPEC2005), Lexington, MA, September 20—22.

13.

Johnson, P. (ed.) (2004). Proceedings of 26th International Conference on Software Engineering (ICSE 2004), Edinburgh, Scotland, May 23—28.

14.

Johnson, P. (ed.) (2005). Proceedings of 27th International Conference on Software Engineering (ICSE 2005), St. Louis, Missouri, May 15—21.

15.

Kepner, J. (ed.) (2004). Special issue on HPC productivity , International Journal of High Performance Computing Applications, 18(4).

16.

Kepner, J. and Ahalt, S. (2004). MatlabMPI, Journal of Parallel and Distributed Computing, 64(8): 997—1005 .

17.

Koelbel, C. (1994). The high performance Fortran handbook, Cambridge, MA: MIT Press .

18.

Lebak, J. , Kepner, J. , Hoffmann, H. , and Rutledge, E. (2005). Parallel VSIPL++: An open standard software library for high-performance parallel signal processing , Proceedings of the IEEE, 93(2).

19.

Loveman, D.B. (1993). High performance Fortran, Parallel and Distributed Technology: Systems and Applications, IEEE , 1(1).

20.

Luszczek, P. , Dongarra, J. , Koester, D. , Rabenseifner, R. , Lucas, B. , Kepner, J. , McCalpin, J. , Bailey, D. , and Takahashi, D. (2005). Introduction to the HPC Challenge benchmark suite , April, http://repositories.cdlib.org/lbnl/LBNL-57493/

21.

MathWorks Inc. (2005). Distributed computing toolbox user's guide, http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/

22.

McCalpin, J. (2005). STREAM: Sustainable memory bandwidth in high performance computers, http://www.cs.virginia.edu/stream/

23.

Morrow, G. and van de Geijn, R. (1998). A parallel linear algebra server for Matlab-like environments , in Proceedings of Supercomputing 1998, Orlando, FL, November.

24.

Numrich, R. and Reid, J. (1998). Co-array Fortran for parallel programming , ACM SIGPLAN Fortran Forum, 17(2): 1—31 .

25.

Nieplocha, J. , Harrison, R. , Krishnan, M. , Palmer, B. , and Tipparaju, V. (2002). Combining shared and distributed memory models: Evolution and recent advancements of the Global Array Toolkit , Proceedings POOHL'2002 Workshop of ICS-2002, New York City.

26.

Ramaswamy, S. and Banerjee, P. (1995). Automatic generation of efficient array redistribution routines for distributed memory multicomputers . Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation (Frontiers '95), McClean, VA, February 6—9.

27.

Reuther, A. , Currie, T. , Kepner, J. , Kim, H. , McCabe, A. , Moore, M. , and Travinin, N. (2004). LLgrid: Enabling on-demand grid computing with gridMatlab and pMatlab , in Proceedings of High Performance Embedded Computing Workshop (HPEC2004), Lexington, MA 28—30 September.

28.

RTExpress. Parallel MATLAB development for high performance computing with RTExpress, http://www.rtexpress.com/

29.

Zosel, M.E. (1993). High performance Fortran: an overview, Compcon Spring '93 , Digest of Papers, San Francisco, CA , February 22—26.