A Performance Model of the Parallel Ocean Program

Abstract

In this paper we describe a performance model of the Parallel Ocean Program (POP). In particular, the latest version of POP (v2.0) is considered, which has similarities and differences to the earlier version (v1.4.3) as commonly used in climate simulations. The performance model encapsulates an understanding of POP’s data decomposition, processing flow, and scaling characteristics. The model is parametrized in many of the main input parameters to POP as well as characteristics of a processing system such as network latency and bandwidth. The performance model has been validated to date on a medium-sized (128 processor) AlphaServer ES40 system with the QsNet-1 interconnection network, and also on a larger scale (2048 processor) Blue Gene/Light system. The accuracy of the performance model is high when using two standard benchmark configurations, one of which represents a realistic configuration similar to that used in Community Climate System Model coupled climate simulations. The performance model is also used to explore the performance of POP after possible optimizations to the code, and different task to processor assignment strategies, whose performance cannot be currently measured.

Keywords

Performance modeling large-scale systems performance analysis ocean modeling

Get full access to this article

View all access options for this article.

References

Almasi, G. et al. 2004. Unlocking the performance of the Blue-Gene/L supercomputer . Proceedings of the IEEE/ACM Supercomputing Conference, Pittsburgh, PA.

Blackmon, M. et al. 2001. The Community Climate System Model . Bulletin of the American Meteorological Society 82: 2357-2376 .

Bryan K. 1969. A numerical method for the study of the circulation of the world ocean . Journal of Computational Physics 4: 347 .

Carrington, L. , Snavely, A. , and Wolter, N. 2005. A performance prediction framework for scientific applications. Future Generation Computer Systems. Special issue on large-scale system performance modeling and analysis, Elsevier, Amsterdam , in press.

Chervin, R. M. and Semtner, A. J. 1988. An ocean modeling system for supercomputer architectures of the 1990s . Proceedings of the NATO Advanced Research Workshop on Climate-Ocean Interaction, M. Schlesinger , editor, Kluwer, Dordrecht.

Cox, M. D. 1984. A primitive equation, three-dimensional model of the ocean. GFDL Ocean Group Technical Report No. 1, GFDL/NOAA, Princeton, NJ .

D’Azevedo, E. F. , Eijkhout, V. L. , and Romine, C. H. 1993. Conjugate gradient algorithms with reduced synchronization overhead on distributed memory multiprocessors. Computer Science Technical Report CS-93-185, University of Tennessee, Knoxville .

Dongarra, J. and Eijkhout, V. 2003. Finite-choice algorithm optimization in conjugate gradients. Computer Science Technical Report UT-CS-03-502, University of Tennessee, Knoxville .

Dukowicz, J. K. and Smith, R. D. 1994. Implicit free-surface method for the Bryan-Cox-Semtner ocean model . Journal of Geophysical Research 99: 7991-8014 .

10.

Dunnigan, T. H. , Fahey, M. R. , White, J. B. , and Worley, P. H. 2003. Early evaluation of the Cray X1 . Proceedings of the IEEE/ACM Supercomputing Conference, Phoenix, AZ.

11.

Fonlupt, C. , Marquet, P. , and Dekeyser, J.-L. 1998. Data-parallel load balancing strategies . Parallel Computing 24: 1665-1684

12.

Hoisie, A. , Lubeck, O. , and Wasserman, H. J. 2000. Performance and scalability analysis of teraflop-scale parallel architectures using multidimensional wavefront applications . International Journal of High Performance Computing Applications 14(4): 330-346 .

13.

Jones, P. W. , Worley, P. H. , Yoshida, Y. , White, J. B. III , and Levesque, J. 2005. Practical performance portability in the Parallel Ocean Program (POP) . Concurrency and Computation: Practice and Experience 17(10): 1317-1327 .

14.

Kerbyson, D. J. , Alme, H. J. , Hoisie, A. , Petrini, F. , Wasserman, H. J. , and Gittings, M. 2001. Predictive performance and scalability modeling of a large-scale application . Proceedings of the IEEE/ACM Supercomputing Conference, Denver, CO.

15.

Kerbyson, D. J , Hoisie, A. , and Wasserman, H. J. 2002. Exploring advanced architectures using performance prediction. Innovative Architecture for Future Generation High-Performance Processors and Systems, IEEE Computer Society Press, Los Alamitos, CA , pp. 27-37.

16.

Kerbyson, D. J , Hoisie, A. , and Pautz, S. 2003. Performance modeling of deterministic transport computations. Performance Analysis and Grid Computing, V. Getov , M. Gerndt , A. Hoisie , A. Malony , and B. Miller , editors, Kluwer, Dordrecht , pp. 21-39.

17.

Kerbyson, D. J. , Hoisie, A. , and Wasserman, H. J. 2005. A performance comparison between the Earth Simulator and other terascale systems on a characteristic ASCI work-load . Concurrency and Computation: Practice and Experience 17: 1219-1238 .

18.

Maltrud, M. E. and McClean, J. L. 2005. An eddy-resolving global 1/10 degree ocean simulation . Ocean Modelling 8: 31-54 .

19.

Mathis, M. M. and Kerbyson, D. J. 2005. Performance modeling of unstructured mesh particle transport computations . International Journal of Supercomputing in press.

20.

Mathis, M. M. , Kerbyson, D. J. , and Hoisie, A. 2005. A performance model of non-deterministic particle transport on large-scale systems. Future Generation Computer Systems. Special issue on large-scale system performance modeling and analysis, Elsevier, Amsterdam , in press.

21.

Murray, R. J. 1996. Explicit generation of orthogonal grids for ocean models . Journal of Computational Physics 126: 251 .

22.

Petrini, F. , Feng, W. C. , Hoisie, A. , Coll, S. , and Frachtenberg, E. 2002. The Quadrics network: high performance clustering technology . IEEE Micro 22(1): 46-57 .

23.

Petrini, F. , Kerbyson, D. J. , and Pakin, S. 2003. The case of the missing supercomputer performance: achieving optimal performance on the 8192 processors of ASCI Q . Proceedings Of the IEEE/ACM Supercomputing Conference, Phoenix, AZ (awarded best paper).

24.

Semtner, A. J. Jr. 1986. Finite-difference formulation of a world ocean model. Advanced Physical Oceanographic Numerical Modeling, J. J. O’Brien , editor, Reidel, Dordrecht .

25.

Smith, R. D. and Gent, P. 2002. Reference manual for the Parallel Ocean Program (POP). Los Alamos Unclassified Report LA-UR-02-2484.

26.

Smith, R. D. and Kortas, S. 1995. Curvilinear coordinates for global ocean models. Los Alamos Unclassified Report LA-UR-95-1146.

27.

Smith, R. D. , Maltrud, M. E. , Bryan, F. O. , and Hecht, M. W. 2000. Numerical simulation of the North Atlantic ocean at 1/10 degree . Journal of Physical Oceanography 30: 1532-1561 .