Understanding Application Performance via Micro-benchmarks on Three Large Supercomputers: Intrepid,Ranger and Jaguar

Abstract

The emergence of new parallel architectures presents new challenges for application developers. Supercomputers vary in processor speed, network topology, interconnect communication characteristics and memory subsystems. This paper presents a performance comparison of three of the fastest machines in the world: IBM’s Blue Gene/P installation at ANL (Intrepid), the SUN-Infiniband cluster at TACC (Ranger) and Cray’s XT4 installation at ORNL (Jaguar). Comparisons are based on three applications selected by NSF for the Track 1 proposal to benchmark the Blue Waters system: NAMD, MILC and a turbulence code, DNS. We present a comprehensive overview of the architectural details of each of these machines and a comparison of their basic performance parameters. Application performance is presented for multiple problem sizes and the relative performance on the selected machines is explained through micro-benchmarking results. We hope that insights from this work will be useful to managers making buying decisions for supercomputers and application users trying to decide on a machine to run on. Based on the performance analysis techniques used in the paper, we also suggest a step-by-step procedure for estimating the suitability of a given architecture for a highly parallel application.

Keywords

performance analysis micro-benchmarking scientific applications supercomputers architecture

Get full access to this article

View all access options for this article.

References

Alam, S. , Barrett, R. , Bast, M. , Fahey, M.R. , Kuehn, J. , McCurdy, C. , Rogers, J. , Roth, P. , Sankaran, R. , Vetter, J.S. , Worley, P. and Yu, W. ( 2008) Early evaluation of IBM Blue Gene/P. SC 08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing, pp. 1-12. Piscataway, NJ: IEEE Press.

Alam, S.R. , Kuehn, J.A. , Barrett, R.F. , Larkin, J.M. , Fahey, M.R. , Sankaran, R. and Worley, P.H. ( 2007) Cray XT4: An early evaluation for petascale scientific simulation. SC ‘07: Proceedings of the 2007 ACM/IEEE conference on Supercomputing, pp. 1-12. New York: ACM Press.

Bernard, C. , Burch, T. , DeGrand, T.A. , DeTar, C. , Gottlieb, S. , Heller, U.M. , Hetrick, J.E. , Orginos, K. , Sugar, B. and Toussaint, D. ( 2000) Scaling tests of the improved Kogut-Susskind quark action . Phys. Rev. D 61.

Bhatele, A. and Kale, L.V. ( 2009) An evaluation of the effect of interconnect topologies on message latencies in large supercomputers. Proceedings of Workshop on Large-Scale Parallel Processing (IPDPS ‘09).

Bhatele, A. , Kumar, S. , Mei, C. , Phillips, J.C. , Zheng, G. and Kale, L.V. ( 2008) Overcoming scaling challenges in biomolecular simulations across multiple platforms. Proceedings of IEEE International Parallel and Distributed Processing Symposium 2008.

Dongarra, J. and Luszczek, P. ( 2005) Introduction to the HPC Challenge Benchmark Suite. Technical Report UT-CS-05-544, University of Tennessee, Department of Computer Science.

Hoisie, A. , Johnson, G. , Kerbyson, D.J. , Lang, M. and Pakin, S. ( 2006) A performance comparison through benchmarking and modeling of three leading supercomputers: Blue Gene/L, Red Storm, and Purple. SC ‘06: Proceedings of the 2006 ACM/IEEE conference on Supercomputing . New York: ACM Press, p. 74.

IBM Blue Gene Team (2008) Overview of the IBM Blue Gene/P project. IBM J. Res. Develop . 52(1-2).

IBM System Blue Gene Solution (2008) Blue Gene/P Application Development Redbook. http://www.redbooks.ibm.com/abstracts/sg247287.html

10.

Kalé, L. and Krishnan, S. ( 1993) CHARM++: A Portable Concurrent Object Oriented System Based on C++. Proceedings of OOPSLA’ 93, Paepcke, A. (Ed.), New York: ACM Press, pp. 91-108.

11.

Kufrin, R. ( 2005) Perfsuite: an accessible, open source performance analysis environment for Linux. Proceedings of the Linux Cluster Conference.

12.

Kumar, S. , Dozsa, G. , Berg, J. , Cernohous, B. , Miller, D. , Ratterman, J. , Smith, B. and Heidelberger, P. ( 2008) Architecture of the component collective messaging interface . Proceedings of the 15th European PVM/MPI Users’ Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface. Berlin: Springer, pp. 23-32.

13.

Oliker, L. , Canning, A. , Carter, J. , Iancu, C. , Lijewski, M. , Kamil, S. , Shalf, J. , Shan, H. , Strohmaier, E. , Ethier, S. and Goodale, T. ( 2007) Scientific application performance on candidate PetaScale platforms. Proceedings of IEEE Parallel and Distributed Processing Symposium (IPDPS).

14.

Phillips, J.C. , Braun, R. , Wang, W. , Gumbart, J. , Tajkhorshid, E. , Villa, E. , Chipot, C. , Skeel, R.D. , Kalé, L. and Schulten, K. ( 2005) Scalable molecular dynamics with NAMD. J. Comput. Chem. 26(16): 1781-1802.

15.

Taylor, M.A. , Kurien, S. and Eyink, G.L. ( 2003) Recovering isotropic statistics in turbulence simulations: The Kolmogorov 4/ 5th law. Phys. Rev. E 68.

16.

Williams, S. , Waterman, A. and Patterson, D. (2009) Roofline: an insightful visual performance model for multicore architectures . Commun. ACM 52(4): 65-76.

17.

Zheng, G. , Kakulapati, G. and Kalé, L.V. ( 2004) Bigsim: a parallel simulator for performance prediction of extremely large parallel machines. 18th International Parallel and Distributed Processing Symposium (IPDPS), Santa Fe, NM, p. 78.