The Model Coupling Toolkit: A New Fortran90 Toolkit for Building Multiphysics Parallel Coupled Models

Abstract

Many problems in science and engineering are best simulated as a set of mutually interacting models, resulting in a coupled or multiphysics model. These models present challenges stemming from their interdisciplinary nature and from their computational and algorithmic complexities. The computational complexity of individual models, combined with the popularity of the distributed-memory parallel programming model used on commodity micro-processor-based clusters, results in a parallel coupling problem when building a coupled model. We define and elucidate this problem and how it results in a set of requirements for software capable of simplifying the construction of parallel coupled models. We describe the package, the Model Coupling Toolkit (MCT), which we have developed to meet these general requirements and the specific requirements of a parallel climate model. We present the MCT programming model with illustrative code examples. We present representative results that measure MCT’s scalability, performance portability, and a proxy for coupling overhead.

Keywords

Parallel computing multiphysics parallel coupling coupled models

Get full access to this article

View all access options for this article.

References

Baum, J. D. , Luo, H. , Mestreau, E. L. , Sharov, D. , Loehner, R. , Pelessone, D. , and Charman, C. 2001. Recent developments of a coupled CFD/CSD methodology . Proceedings of the International Conference on Computational Science, San Francisco, CA, May, Lecture Notes in Computer Science, Vol. 2073, Springer-Verlag, Berlin, pp. 1087-1097 .

Bertrand, F. , Bramley, R. , Bernholdt, D. E. , Kohl, J. A. , Sussman, A. , Larson, J. W. , and Damevski, K. B. 2005. Data redistribution and remote method invocation in parallel component architectures . Proceedings of the International Parallel Distributed Processes Symposium, Denver, CO.

Bettge, T. , Craig, A. , James, R. , Wayland, V. , and Strand, G. 2001. The DOE Parallel Climate Model (PCM): the computational highway and backroads . Proceedings of the International Conference on Computational Science (ICCS) 2001, San Francisco, CA, May, Lecture Notes in Computer Science, Vol. 2073, Springer-Verlag, Berlin, pp. 148-156 .

Boville, B. A. and Gent, P. R. 1998. The NCAR Climate System Model, Version One . Journal of Climate 11: 1115-1130 .

Bryan, F. O. , Kauffman, B. G. , Large, W. G. , and Gent, P. R. 1996. The NCAR CSM ux coupler. NCAR Technical Note 424, NCAR, Boulder, CO .

Craig, A. P. , Kaufmann, B. , Jacob, R. , Bettge, T. , Larson, J. , Ong, E. , Ding, C. , and He, H. 2005. cpl6: the new extensible high performance parallel coupler for the Community Climate System Model . International Journal of High Performance Computing Applications 19(3).

Dahlgren, T. , Epperly, T. , and Kumfert, G. 2004. Babel User’s Guide (version 0.9.0 ed). CASC, Lawrence Livermore National Laboratory .

Decyk, V. K. , Norton, C. D. , and Syzmanski, B. K. 1996. Introduction to object-oriented concepts Using Fortran90. See http://www.cs.rpi.edu/%7Eszymansk/OOF90/F90_Objects.html.

Decyk, V. K. , Norton, C. D. , and Syzmanski, B. K. 1997. Expressing object-oriented concepts in Fortran90 . ACM Fortran Forum 16(1): 13-18 .

10.

Edjlali, G. , Sussman, A. , and Saltz, J. 1997. Interoperability of data-parallel run-time Libraries . International Parallel Processing Symposium, Geneva, Switzerland.

11.

He, Y. and Ding, C. 2005. Coupling multicomponent models by MPH on distributed memory computer architectures . International Journal of High Performance Computing Applications 19(3).

12.

Heath, M. T. and Dick, W. A. 1998. Virtual rocketry: rocket science meets computer science . IEEE Computational Science and Engineering 5(1): 16-26 .

13.

Hill, C. , DeLuca, C. , Balaji, V. , Suarez, M. , da Silva, A. , and the ESMF Joint Specification Team . 2004. The architecture of the Earth System Modeling Framework . Computing in Science and Engineering 6: 18-28 .

14.

Jacob, R. , Schafer, C. , Foster, I. , Tobis, M. , and Anderson, J. 2001. Computational design and performance of the fast ocean atmosphere model . Proceedings of the International Conference on Computational Science, San Francisco, CA, May, Lecture Notes in Computer Science, Vol. 2073, Springer-Verlag, Berlin, pp. 175-184 .

15.

Jacob, R. , Larson, J. , and Ong, E. 2005. M × N communication and parallel interpolation in CCSM3 using the Model Coupling Tookit . International Journal of High Performance Computing Applications 19(3).

16.

Jones, P. W. 1999. First- and second-order conservative remapping schemes for grids in spherical coordinates . Monthly Weather Review 127: 2204-2210 .

17.

Kiehl, J. and Gent, P. R. 2004. The Community Climate System Model, Version Two . Journal of Climate 17: 3666-3682 .

18.

Larson, J. W. , Guo, J. , Gaspari, G. , da Silva, A. , and Lyster, P. M. 1998. Documentation of the Physical-Space Statistical Analysis System (PSAS) Part III: the software implementation. DAO Office Note 98-05, NASA/Goddard Space Flight Center, Greenbelt, MD .

19.

Lefantzi, S. and Ray, J. 2003. A component-based scientific toolkit for reacting flows . Proceedings of the 2nd MIT Conference on Computational Fluid and Solid Mechanics, Cambridge, MA, June 17-20, Vol. 2, pp. 1401-1405 .

20.

MPI Forum . 1994. MPI: a message passing interface standard . International Journal of Supercomputer Applications and High Performance Computing 8(3/4): 159-416 .

21.

Musser, D. R. and Saini, A. 1996. STL Tutorial and Reference Guide: C++ Programming with the Standard Template Library, Addison-Wesley, Reading, MA .

22.

Ranganathan, M. , Acharya, A. , Edjlali, G. , Sussman, A. , and Saltz, J. 1996. Run-time coupling of data-parallel programs . Proceedings of the International Conference on Supercomputing, Philadelphia, PA.

23.

Toth, G. et al. 2004. A physics-based software framework for Sun-Earth connection modeling . Multiscale Coupling of Sun-Earth Processes, Proceedings of the Conference on the Sun-Earth Connection, Kona, HI , A. T. Y. Lui , Y. Kamide , and G. Consolini , editors, Elsevier, Amsterdam, pp. 383-397 .

24.

Valcke, S. , Caubel, A. , Vogelsang, R. , and Declat, D. 2004. OASIS3: Ocean Atmosphere Sea Ice Soil User’s Guide. Technical Report TR/CMGC/04/68, CERFACS, Toulouse, France .