Parallel Programmability and the Chapel Language

Abstract

In this paper we consider productivity challenges for parallel programmers and explore ways that parallel language design might help improve end-user productivity. We offer a candidate list of desirable qualities for a parallel programming language, and describe how these qualities are addressed in the design of the Chapel language. In doing so, we provide an overview of Chapel's features and how they help address parallel productivity. We also survey current techniques for parallel programming and describe ways in which we consider them to fall short of our idealized productive programming model.

Keywords

parallel languages productivity parallel programming programming models Chapel

Get full access to this article

View all access options for this article.

References

Adl-Tabatabai, A.-R. , Lewis, B.T. , Menon, V. , Murhpy, B.R. , Saha, B. , and Shpeisman, T. (2006). Compiler and runtime support for efficient software transactional memory , in PLDI '06: Proceedings of the 2006 ACM SIGPLAN Conference on Programming Language Design and Implementation , pp. 26—37, ACM Press, New York, NY.

Albert, E. , Knobe, K. , Lukas, J.D. , and Steele, Jr., G.L. (1988). Compiling Fortran 8x array features for the Connection Machine computer system , in PPEALS `88: Proceedings of the ACM/SIGPLAN Conference on Parallel Programming: Experience with Applications, Languages, and Systems, pp. 42—56, ACM Press, New York, NY.

Alverson, G. , Kahan, S. , Korry, R. , McCann, C. , and Smith, B. (1995). Scheduling on the Tera MTA, in D. G. Feitelson , and L. Rudolph , editors, Job scheduling strategies for parallel processing, vol. 949 of Lecture Notes in Computer Science, pp. 19—44, Berlin: Springer Verlag .

Barriuso, R. and Knies, A. (1994). SHMEM user's guide for C, Technical Report, Cray Research Inc.

Blumofe, R.D. and Leiserson, C.E. (1994). Scheduling multithreaded computations by work stealing , in Proceedings of the 35th Annual Symposium on Foundations of Computer Science (FOCS `94), pp. 356—368, New Mexico.

Bonachea, D. (2002). GASNet specification v1.1, Technical Report UCB/CSD-02-1207, U.C. Berkeley. (newer versions also available at http://gasnet.cs.berkeley.edu)

Carlson, B. , El-Ghazawi, T. , Numerich, R. , and Yelick, K. (2003). Programming in the partitioned global address space model, Tutorial at Supercomputing 2003. (notes available at http://upc.gwu.edu)

Carlson, W.W. , Draper, J.M. , Culler, D.E. , Yelick, K. , Brooks, E. , and Warren, K. (1999). Introduction to UPC and language specification , Technical Report CCS-TR-99-157, Center for Computing Sciences, Bowie, MD .

Chamberlain, B.L. (2001). The design and implementation of a region-based parallel language, Ph.D. thesis, University of Washington .

10.

Chamberlain, B.L. , Choi, S.-E. , Lewis, E.C. , Lin, C. , Snyder, L. , and Weathersby, W.D. (1998). ZPL's WYSIWYG performance model , in Proceedings of the Third International Workshop on High-Level Programming Models and Supportive Environments, pp. 50—61, IEEE Computer Society Press.

11.

Chamberlain, B.L. , Lewis, E.C. , Lin, C. , and Snyder, L. (1999). Regions: An abstraction for expressing array computation , in ACM/SIGAPL International Conference on Array Programming Languages, pp. 41—49.

12.

Chamberlain, B.L. and Snyder, L. (2001). Array language support for parallel sparse computation , in Proceedings of the 2001 International Conference on Supercomputing, pp. 133—145, ACM SIGARCH.

13.

Chandra, R. , Menon, R. , Dagum, L. , Kohr, D. , Maydan, D. , and McDonald, J. (2000). Parallel programming in OpenMP, Morgan Kaufmann, San Francisco, CA.

14.

Chapman, B.M. , Mehrotra, P. , and Zima, H.P. (1992). Programming in Vienna Fortran , Scientific Programming, 1(1): 31—50 .

15.

Chen, W.-Y. , Bonachea, D. , Duell, J. , Husbands, P. , Iancu, C. , and Yelick, K. (2003). A performance analysis of the Berkeley UPC compiler , in Proceedings of the International Conference of Supercomputing (ICS), San Francisco, CA.

16.

Chen, W.-Y. , Iancu, C. , and Yelick, K. (2005). Communication optimizations for fine-grained UPC applications , in 14th International Conference on Parallel Architectures and Compilation Techniques (PACT), St. Louis, MO.

17.

Cray Inc. (2005). Chapel specification , Cray Inc., Seattle, WA , 0.4 edition. (http://chapel.cs.washington.edu ).

18.

Dagum, L. and Menon, R. (1998). OpenMP: An industry-standard API for shared-memory programming , IEEE Computational Science and Engineering, 5(1): 46—55 .

19.

Dahlgren, T. , Epperly, T. , Kumfert, G. , and Leek, J. (2006). Babel users' guide, Lawrence Livermore National Laboratory , 0.11.0 edition.

20.

Deitz, S.J. (2005). High-Level programming language abstractions for advanced and dynamic parallel computations, Ph.D. thesis, University of Washington .

21.

Deitz, S.J. , Callahan, D. , Chamberlain, B.L. , and Snyder, L. (2006). Global-view abstractions for user-defined reductions and scans , in PPoPP `06: Proceedings of the 11th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 40—47, ACM Press, New York, NY.

22.

Diaconescu, R. and Zima, H.P. (2007). An approach to data distributions in Chapel , International Journal of High Performance Computing Applications, 21(3): 313—334 .

23.

Dotsenko, Y. , Coarfa, C. , and Mellor-Crummey, J. (2004). A multi-platform Co-Array Fortran compiler , in Proceedings of the 13th International Conference of Parallel Architectures and Compilation Techniques (PACT 2004) , Antibes Juan-les-Pins, France.

24.

El-Ghazawi, T. , Carlson, W. , Sterling, T. , and Yelick, K. (2005). UPC: Distributed shared-memory programming, Wiley-Interscience, Hobohen, NJ .

25.

Fox, G. , Hiranandani, S. , Kennedy, K. , Koelbel, C. , Kremer, U. , Tseng, C.-W. , and Wu, M.-Y. (1990). Fortran D language specification, Technical Report CRPC-TR 90079, Rice University, Center for Research on Parallel Computation .

26.

Fraser, K. (2003). Practical lock-freedom, Ph.D. thesis, King's College, University of Cambridge .

27.

Geist, A. , Beguelin, A. , Dongarra, J. , Jiang, W. , Manchek, R. , and Sunderam, V. (1994a). PVM 3 user's guide and reference manual, Technical Report ORNL/TM-12187, Oak Ridge National Laboratory .

28.

Geist, A. , Beguelin, A. , Dongarra, J. , Jiang, W. , Manchek, R. , and Sunderam, V. (1994b). PVM: Parallel virtual machine, a user's guide and tutorial for networked parallel computing, Scientific and Engineering Computation, Cambridge, MA: MIT Press .

29.

Gilbert, J.R. , Moler, C. , and Schreiber, R. (1992). Sparse matrices in MATLAB: Design and implementation , SIMAX, 13(1): 333—356 .

30.

Gropp, W. , Huss-Lederman, S. , Lumsdaine, A. , Lusk, E. , Nitzberg, B. , Saphir, W. , and Snir, M. (1998). MPI: The complete reference, volume 2, Scientific and Engineering Computation, Cambridge, MA: MIT Press .

31.

Harris, T. , Marlow, S. , Jones, S.P. , and Herlihy, M. (2005). Composable memory transactions , in ACM SIGPLAN Symposium on Principles and Practices of Parallel Programming, Chicago, IC.

32.

Harris, T. , Plesko, M. , Shinnar, A. , and Tarditi, D. (2006). Optimizing memory transactions , in PLDI '06: Proceedings of the 2006 ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 14—25, ACM Press , New York, NY.

33.

High Performance Fortran Forum (1993). High Performance Fortran language specification , Scientific Programming, 2(1—2): 1—170 .

34.

High Performance Fortran Forum (1997). High Performance Fortran Language Specification Version 2.0.

35.

Hilfinger, P.N. , Bonachea, D.O. , Datta, K. , Gay, D. , Graham, S.L. , Liblit, B.R. , Pike, G. , Su, J.Z. , and Yelick, K.A. (2005). Titanium language reference manual, Technical Report UCB/EECS-2005-15, Electrical Engineering and Computer Sciences, University of California at Berkeley .

36.

Joerg, C.F. (1996). The Cilk system for parallel multithreaded computing, Ph.D. thesis, MIT Department of Electrical Engineering and Computer Science .

37.

Koelbel, C.H. , Loveman, D.B. , Schreiber, R.S. , Guy L. Steele, Jr. , and Zosel, M.E. (1996). The High Performance Fortran handbook, Scientific and Engineering Computation, Cambridge, MA: MIT Press .

38.

Liskov, B. , Snyder, A. , Atkinson, R. , and Schaffert, C. (1977). Abstraction mechanisms in CLU , ACM SIGPLAN, 12(7): 75—81 .

39.

Message Passing Interface Forum (1994). MPI: A message passing interface standard , International Journal of Supercomputing Applications, 8(3/4): 169—416 .

40.

Message Passing Interface Forum (1997). MPI-2: Extensions to the message-passing interface.

41.

Ngo, T.A. , Snyder, L. , and Chamberlain, B.L. (1997). Portable performance of data parallel languages , in Proceedings of SC97: High Performance Networking and Computing, San Jose, CA .

42.

Nieplocha, J. and Carpenter, B. (1999). ARMCI: A portable remote memory copy library for distributed array libraries and compiler run-time systems, in J. Rolim et al., editors, Proceedings of the 3rd Workshop on Runtime Systems for Parallel Programming, Lecture Notes in Computer Science vol. 1586, pp. 533—546, San Juan, Puerto Rico , Berlin: Springer Verlag .

43.

Numerich, R.W. and Reid, J. (1998). Co-array Fortran for parallel programming , SIGPLAN Fortran Forum, 17(2): 1—31 .

44.

Numerich, R.W. and Reid, J. (2005). Co-arrays in the next Fortran standard , SIGPLAN Fortran Forum, 24(2): 4—17 .

45.

Parzyszek, K. , Nieplocha, J. , and Kendall, R.A. (2000). A generalized portable SHMEM library for high performance computing , in M. Guizani, and Z. Shen, editors, Twelfth IASTED International Conference on Parallel and Distributed Computing and Systems, pp. 401—406 , Las Vegas, Nevada .

46.

Randall, K.H. (1998). Cilk: Efficient multithreaded computing, Ph.D. thesis, MIT Department of Electrical Engineering and Computer Science .

47.

Snir, M. , Otto, S. , Huss-Lederman, S. , Walker, D. , and Dongarra, J. (1998). MPI: The complete reference, volume 1, Scientific and Engineering Computation, 2nd edition, Cambridge, MA: MIT Press .

48.

Snyder, L. (1999). The ZPL programmer's guide, Scientific and Engineering Computation, Cambridge, MA: MIT Press .

49.

Su, J. and Yelick, K. (2005). Automatic support for irregular computations in a high-level language , in 19th International Parallel and Distributed Processing Symposium (IPDPS), Denver, CO .

50.

Supercomputing Technologies Group (2001). Cilk 5.3.2 Reference manual, MIT Laboratory for Computer Science .

51.

Ujaldon, M. , Zapata, E.L. , Chapman, B.M. , and Zima, H.P. (1997). Vienna-Fortran/HPF extensions for sparse and irregular problems and their compilation , IEEE Transactions on Parallel and Distributed Systems, 8(10).

52.

UPC Consortium (2005). UPC Language specification (v 1.2). (available at http://upc.gwu.edu)

53.

von Eicken, T. , Culler, D.E. , Goldstein, S.C. , and Schauser, K.E. (1992). Active Messages: A mechanism for integrated communication and computation , in 19th International Symposium on Computer Architecture, Gold Coast, Australia, pp. 256—266.

54.

Yelick, K. , Semenzato, L. , Pike, G. , Miyamato, C. , Liblit, B. , Krishnamurthy, A. , Hilfinger, P. , Graham, S. , Gay, D. , Colella, P. , and Aiken, A. (1998). Titanium: A high-performance Java dialect , Concurrency: Practice and Experience, 10(11—13): 825—836 .

55.

Zima, H. , Brezany, P. , Chapman, B. , Mehrotra, P. , and Schwald, A. (1992). Vienna Fortran—a language specification version 1.1, Technical Report NASA-CR-189629/ICASE-IR-21, Institute for Computer Applications in Science and Engineering .