Efficient parallelization of a regional ocean model for the western Mediterranean Sea

Abstract

This paper focuses on the parallelization of an ocean model applying current multicore processor-based cluster architectures to an irregular computational mesh. The aim is to maximize the efficiency of the computational resources used. To make the best use of the resources offered by these architectures, this parallelization has been addressed at all the hardware levels of modern supercomputers: firstly, exploiting the internal parallelism of the CPU through vectorization; secondly, taking advantage of the multiple cores of each node using OpenMP; and finally, using the cluster nodes to distribute the computational mesh, using MPI for communication within the nodes. The speedup obtained with each parallelization technique as well as the combined overall speedup have been measured for the western Mediterranean Sea for different cluster configurations, achieving a speedup factor of 73.3 using 256 processors. The results also show the efficiency achieved in the different cluster nodes and the advantages obtained by combining OpenMP and MPI versus using only OpenMP or MPI. Finally, the scalability of the model has been analysed by examining computation and communication times as well as the communication and synchronization overhead due to parallelization.

Keywords

Message passing shared memory hybrid programming multicore clusters ocean model finite element method

Get full access to this article

View all access options for this article.

References

Allievi

Bermejo

(1997) A generalized particle search-locate algorithm for arbitrary grids. Journal of Computational Physics132: 157–166.

Bermejo

Conde

(2002) A conservative quasi-monotone semi-Lagrangian scheme. Monthly Weather Review130: 423–430.

Bermejo

Saavedra

(2012) Modified Lagrange–Galerkin methods of first and second order in time for convection–diffusion problem. Numerical Mathematics120: 601–638.

Bermejo

Galán del Sastre

Saavedra

(2012) A second order in time modified Lagrange-Galerkin-finite element method for the incompressible Navier-Stokes equations. SIAM Journal on Numerical Analysis50: 3084–3109.

Chorin

(1968) Numerical solution of the Navier-Stokes equations. Mathematics of Computation22: 745–762.

Cowles

(2008) Parallelization of the FVCOM coastal ocean model. International Journal of High Performance Computing Applications22(2): 177–193.

Crainic

Toulouse

(2010) Parallel meta-heuristics. In: Gendreau

Potvin

J-Y

(eds) Handbook of Metaheuristics (International Series in Operations Research & Management Science, vol. 146). New York: Springer, pp. 497–541.

Douglas

Russell

(1982) Numerical methods for convection-dominated diffusion problems based on combining the method of characteristics with finite element or finite difference procedures. SIAM Journal on Numerical Analysis19: 871–885.

Fringer

Gerritsen

Street

(2006) An unstructured-grid, finite-volume, nonhydrostatic, parallel coastal ocean simulator. Ocean Modelling14: 139–173.

10.

Gallardo-Andrés

Galán del Sastre

Bermejo

(2010) PROMES-MOSLEF: An atmosphere-ocean coupled regional model. Coupling and preliminary results over the Mediterranean basin. In: 4th HYMeX workshop.

11.

Garrido

Arias

Cazorla

. (2009) PROMESPAR: A parallel implementation of the regional atmospheric model PROMES. In: Proceedings of the world congress on engineering, London, UK, 1–3July.

12.

González Gutiérrez

Bermejo

(2005) A semi-Lagrangian level set method for incompressible Navier-Stokes equations with free surface. International Journal for Numerical Methods in Fluids49: 1111–1146.

13.

Guermond

Minev

Shen

(2006) An overview of projection methods for incompressible flows. Computer Methods in Applied Mechanics and Engineering195: 611–645.

14.

Henshawa

Schwendemanb

(2008) Parallel computation of three-dimensional flows using overlapping grids with adaptive mesh refinement. Journal of Computational Physics227(16): 7469–7502.

15.

KCachegrind (2013) Profile data visualization. Available at: http://kcachegrind.sourceforge.net(accessed 7 November 2013).

16.

Kernighan

Lin

(1970) An efficient heuristic procedure for partitioning graphs. Bell Systems Technical Journal49: 291–307.

17.

Luong

Breshears

(2004) Application of multiblock grid and dual-level parallelism in coastal ocean circulation modeling. Journal of Scientific Computing20(2): 257–275.

18.

MacQueen

(1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley symposium on mathematical statistics and probability, pp. 281–297.

19.

Malevsky

Thomas

(1997) Parallel algorithms for semi-Lagrangian advection. International Journal for Numerical Methods in Fluids25: 455–473.

20.

METIS (2013) Serial graph partitioning and fill-reducing matrix ordering. Available at: http://glaros.dtc.umn.edu/gkhome/metis/metis/overview (accessed 7 November 2013).

21.

MPI (2012) MPI forum. Available at: http://www.mpi-forum.org/ (accessed 7 November 2013).

22.

OpenMP (2013) OpenMP specifications. Available at: http://openmp.org/wp/openmp-specifications/ (accessed 7 November 2013).

23.

ParMETIS (2013) ParMETIS – Parallel graph partitioning and fill-reducing matrix ordering. Available at: http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview.

24.

Pironneau

(1982) On the transport-diffusion algorithm and its applications to the Navier-Stokes equations. Numerische Mathematik38: 309–332.

25.

Rabenseifner

Hager

Jost

(2009) Hybrid MPI/OpenMP parallel programming on clusters of multi-core SMP nodes. In: 17th Euromicro international conference on parallel, distributed and network-based processing.

26.

Rivera

Henichea

Glowinskib

. (2010) Parallel finite element simulations of incompressible viscous fluid flow by domain decomposition with Lagrange multipliers. Journal of Computational Physics229(13): 5123–5143.

27.

Sannino

Artale

Lanucara

(2001) An hybrid OpenMP-MPI parallelization of the Princeton ocean model. In: Parallel computing: Advances and current issues, pp. 222–229.

28.

Shuttleworth

Maliassov

Zhou

(2009) Partitioners for parallelizing reservoir simulations. In: Society of Petroleum Engineers reservoir simulation symposium, The Woodlands, Texas, 2–4February.

29.

Temam

(1969) Sur l’approximation de la solution des équations de Navier-Stokes par la method des pas fractionnaires (II). Archive for Rational Mechanics and Analysis33: 377–385.

30.

Temam

Ziane

(2004) Some mathematical problems in geophysical fluid dynamics. In: Friedlander

Serre

(eds). Handbook of Mathematical Fluid Dynamics. North-Holland: Elsevier.

31.

Tseng

Chien

(2011) Parallel domain-decomposed Taiwan multi-scale community ocean model. Computers & Fluids45(1): 77–83.

32.

Valgrind (2012) Instrumentation framework for building dynamic analysis tools. Available at: http://www.valgrind.org/ (accessed 7 November 2013).

33.

Wang

Qiao

Xia

(2010) Parallelization of a coupled wave-circulation model and its application. Ocean Dynamics60(2): 331–339.

34.

Xiu

Karniadakis

(2001) A semi-Lagrangian high order method for Navier-Stokes equations. Journal of Computational Physics172: 658–684.