We present an end-to-end simulation framework that is capable of simulating High-Performance Computing (HPC) systems with hundreds of thousands of interconnected processors. The tool applies discrete event simulation and is driven by real-world application traces. It provides a semantically correct replay of MPI application traces and maintains reasonable simulation details of both the processors in general and the interconnection network in particular. Among other things, it features several interconnection network topologies, flexible routing schemes, arbitrary application task placement, point-to-point statistics collection, and data visualization. With a few case studies, we demonstrate the usefulness of this tool for assisting high-level system design as well as for performance projection and application tuning of future HPC systems.
Peterson, J.L., P.J. Bohrer, L. Chen et al. 2006. Application of Full-System Simulation in Exploratory System Design and Development . IBM Journal of Research and Development50(2/3), 321- 332.
2.
Bohrer, P. , J. Petersen, M. Elnozahy et al. 2004. Mambo: A Full System Simulator for the PowerPC Architecture. ACM SIGMET-RICS Performance Evaluation Review Archive31(4), 8-12.
3.
Magnusson, P.S., M. Christensson, J. Eskilson et al. 2002. Simics: A Full System Simulation Platform. IEEE Computer35(2), 50- 58.
4.
Varga, A.2001. The OMNeT++ Discrete Event Simulation System. In Proceedings of the European Simulation Multiconference (ESM’ 01), Prague, Czech Republic.
5.
Varga, A. and R. Hornig.2008. An Overview of the OMNeT++ Simulation Environment . In Proceedings of First International Conference on Simulation Tools and Techniques for Communications, Networks and Systems (SIMUTools’08), Marseille, France.
6.
Gropp, W., E. Lusk, and A. Skjellum.1999. Using MPI, 2nd edition. MIT Press, Cambridge, MA.
7.
Luszczek, P. , D. Bailey, J. Dongarra et al. 2006. The HPC Challenge (HPCC) Benchmark Suite. In Proceedings of the 2006 ACM/IEEE Conference on Supercomputing (SC’06), Tampa, FL, USA.
8.
Cormen, T.H. , C.E. Leiserson, R.L. Rivest et al. 2001. Introduction to Algorithms, 2nd edition. MIT Press, Cambridge, MA and McGraw-Hill.
9.
Duato, J., S. Yalamanchili, and L. Ni.2003. Interconnection Networks: An Engineering Approach. Morgan Kaufmann, Amsterdam .
10.
Vetter, J.S. , N. Bhatia, E.M. Grobelny et al. 2005. Capturing Petascale Application Characteristics with the Sequoia Toolkit. In Proceedings of the International Parallel Computing Conference (ParCo ‘05) , Malaga, Spain.
11.
Shende, S.S.2006. The TAU Parallel Performance System. International Journal of High Performance Computing Applications20(2), 287-311.
12.
Pillet, V., J. Labarta, T. Cortes et al. 1995. PARAVER: A Tool to Visualise and Analyze Parallel Code. In Proceedings of WoTUG-18: Transputer and Occam Developments, IOS Press, Amsterdam, 44, 17-31.
13.
Chung, I-H. , R.E. Walkup, H.-F. Wen et al. 2006. MPI Performance Analysis Tools on Blue Gene/L In Proceedings of the 2006 ACM/IEEE Conference on Supercomputing (SC’06), Tampa, FL.
14.
Halliwell, G. , R. Bleck, and E. Chassignet.1998. Atlantic Ocean Simulations Performed using a New Hybrid-Coordinate Ocean Model. EOS Transactions . American Geophysical Union (AGU), Fall 1998 Meeting , San Francisco, CA.
15.
Smith, R.D. , J.K. Dukowicz, and R.C. Malone.1992. Parallel Ocean General Circulation Modeling. Physica D60(1-4), 38-61.
16.
Michalakes, J. , J. Dudhia, D. Gill et al. 1998. Design of a Next-Generation Regional Weather Research And Forecast Model. In Towards Teracomputing: Proceedings of the Eighth Workshop on the Use of Parallel Processors in Meteorology , Reading, UK , 16- 20.
17.
Hertel, E.S. , R.L. Bell, M.G. Elrick et al. 1993. CTH: A Software Family for Multi-Dimensional Shock Physics Analysis. In Proceedings of the 19th International Symposium on Shock Waves, Marseille, France, 377-382.
18.
Case, D.A. , T.E. Cheatham III, T. Darden et al. 2005. The Amber Biomolecular Simulation Programs. J. Computationa Chemistry26(16), 1668-1688.
19.
Andreoni, W. and A. Curioni.2000. New Advances in Chemistry and Material Science with CPMD and Parallel Computing. Parallel Computing26(7), 819-842.
20.
Plimpton, S.J.1995. Fast Parallel Algorithms for Short-Range Molecular Dynamics. Journal of Computational Physics117, 1- 19.
21.
Fahey, M.R. and J. Candy.2004. GYRO: A 5-D gyrokinetic-Maxwell Solver. In Proceedings of the 2006 ACM/IEEE Conference on Supercomputing (SC’04), Pittsburgh, PA, USA, 389- 396.
22.
Mirin, A.A. , R.H. Cohen, B.C. Curtis et al. 1999. Very High Resolution Simulation of Compressible Turbulence on the IBM-SP System. In Proceedings of the 1999 ACM/IEEE Conference on Supercomputing (SC’99), Portland, OR, USA.
23.
Hoisie, A., O. Lubeck, and H. Wasserman.1999. Performance Analysis of Wavefront Algorithms on Very-Large Scale Distributed Systems. In Lecture Notes in Control and Information Sciences 249, Springer Berlin, Heidelberg, 171-187.
24.
Mathis, M.M. and D.J. Kerbyson.2004. Performance Modeling of Unstructured Mesh Particle Transport Computations. In Proceedings of 18th International Parallel and Distributed Processing Symposium (IPDPS), Santa Fe, NM, USA, IEEE , 245-252.
25.
Sekercioglu, Y.A., A. Varga, and G.K. Egan.2003. Parallel Simulation Made Easy with OMNeT++. In Proceedings of the European Simulation Symposium (ESS ’03), Delft, The Netherlands.
26.
Varga, A., Y.A. Sekercioglu, and G.K. Egan.2003. A Practical Efficiency Criterion for the Null Message Algorithm. In Proceedings of the European Simulation Symposium (ESS ’03), Delft, The Netherlands.
27.
Sulistio, A. , C.S. Yeo , and R. Buyya.2004. A Taxonomy of Computer-Based Simulations and its Mapping to Parallel and Distributed Systems Simulation Tools. Software - Practice and Experience34(7), 653-673.
28.
Fujimoto, R.M.1993. Parallel and Distributed Discrete Event Simulation: Algorithms and Applications. In Proceedings of the 25th ACM Winter Simulation Conference (WSC ’93), Los Angeles, CA, USA, 106-114.
29.
Tropper, C.2002. Parallel and Distributed Discrete Event Simulation. Nova Publishers, USA.
30.
Ferscha, A. and S.K. Tripathi.1994. Parallel and Distributed Simulation of Discrete Event Systems. Technical Report. University of Maryland, College Park, MD.
31.
Adiga, N.R. , M.A. Blumrich, D. Chen et al. 2005. Blue Gene/L Torus Interconnection Network. IBM Journal of Research and Development49(2/3), 265-276.
32.
Choudhury, N. , Y. Mehta, T.L. Wilmarth et al. 2005. Scaling an Optimistic Parallel Simulation Of Large-Scale Interconnection Networks . In Proceedings of the 37th ACM Winter Simulation Conference (WSC ’05), Orlando, FL, USA, 591-600.
33.
Badia, R.M. , J. Labarta, J. Gimenez et al. 2003. Dimemas: Predicting MPI Applications Behavior in Grid Environments. In Proceedings of the Workshop on Grid Applications and Programming Tools (GGF ’03), Seattle, WA, USA.
34.
Sharapov, I. , R. Kroeger, G. Delamarter et al. 2006. A Case Study in Top-Down Performance Estimation for a Large-Scale Parallel Application . In Proceedings of the 2006 ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP ’06), New York, NY, USA, 81-89.
35.
Prakash, S. and R.L. Bagrodia.1998. MPI-SIM: Using Parallel Simulation to Evaluate MPI Programs . In Proceedings of the 30th ACM Winter Simulation Conference (WSC ’98), Washington DC, USA, 467-474.
36.
Riesen, R.2006. A Hybrid MPI Simulator. In Proceedings of 2006 International Conference on Cluster Computing, Barcelona, Spain, 1-9.
37.
Ricciulli, L. , P. Lincoln, and J. Meseguer.1996. Execution-Driven Distributed Simulation of Parallel Architectures. Technical Report, Computer Science Laboratory, SRI International , Menlo Park, CA.
38.
Schwetman, H.1986. CSIM: A C-based, Process-Oriented Simulation Language. In Proceedings of the 18th ACM Winter Simulation Conference (WSC ’86), 387-396.
39.
Bagrodia, R., E. Deeljman, S. Docy et al. 1999. Performance Prediction of Large Parallel Applications using Parallel Simulations . ACM SIGPLAN Notices34(8), 151-162.
