The TheLMA project: Multi-GPU implementation of the lattice Boltzmann method

Abstract

In this paper, we describe the implementation of a multi-graphical processing unit (GPU) fluid flow solver based on the lattice Boltzmann method (LBM). The LBM is a novel approach in computational fluid dynamics, with numerous interesting features from a computational, numerical, and physical standpoint. Our program is based on CUDA and uses POSIX threads to manage multiple computation devices. Using recently released hardware, our solver may therefore run eight GPUs in parallel, which allows us to perform simulations at a rather large scale. Performance and scalability are excellent, the speedup over sequential implementations being at least of two orders of magnitude. In addition, we discuss tiling and communication issues for present and forthcoming implementations.

Keywords

computational fluid dynamics CUDA GPU computing lattice Boltzmann method

Get full access to this article

View all access options for this article.

References

Agullo E. , Demmel J. , Dongarra J. , Hadri B. , Kurzak J. , Langou J. , et al. (2009) Numerical linear algebra on emerging architectures: The PLASMA and MAGMA projects. J Phys Conf Ser 180: 012037.

Broquedis F. , Clet-Ortega J. , Moreaud S. , Furmento N. , Goglin B. , Mercier G. , et al. (2010) hwloc: a generic framework for managing hardware affinities in HPC applications. In 18th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), 2010. Piscataway, NJ: IEEE Press, pp. 180-186.

d’ Humières D. ( 1994) Generalized lattice-Boltzmann equations. Rarefied gas dynamics - Theory and simulations; Proceedings of the 18th International Symposium on Rarefied Gas Dynamics. (26-30 July 1992) University of British Columbia: Vancouver, Canada.

d’Humières D. , Ginzburg I. , Krafczyk M. , Lallemand P. and Luo L-S. ( 2002) Multiple-relaxation-time lattice Boltzmann models in three dimensions. Phil Trans Math Phys Eng Sci, 360: 437-451.

Dongarra J. , Moore S. , Peterson G. , Tomov S. , Allred J. , Natoli V. , et al. (2008) Exploring new architectures in accelerating CFD for Air Force applications. In Proceedings of HPCMP Users Group Conference, pp. 14-17.

Fan Z. , Qiu F. , Kaufman A. and Yoakum-Stover S. ( 2004) GPU cluster for high performance computing. In Proceedings of the 2004 ACM/IEEE conference on Supercomputing . Los Alamitos, CA: IEEE Computer Society Press, p. 47.

Frisch U. , Hasslacher B. and Pomeau Y. ( 1986) Lattice-gas automata for the Navier-Stokes equation. Phys Rev Lett 56: 1505-1508.

Kuznik F. , Obrecht C. , Rusaouën G. and Roux J-J. ( 2009). LBM Based Flow Simulation Using GPU Computing Processor . Computers and Mathematics with Applications (27).

McNamara GR and Zanetti G. ( 1988) Use of the Boltzmann Equation to Simulate Lattice-Gas Automata. Phys Rev Lett 61: 2332-2335.

10.

Nvidia (2010) Compute Unified Device Architecture Programming Guide version 3.1.1.

11.

Obrecht C. , Kuznik F. , Tourancheau B. and Roux J-J. ( 2011) A new approach to the lattice Boltzmann method for graphics processing units. Comput Math Appl (in press).

12.

Obrecht C. , Kuznik F. , Tourancheau B. and Roux J-J. ( 2010) Global memory access modelling for efficient implementation of the LBM on GPUs. In High Performance Computing for Computational Science - VECPAR2010 (Lecture Notes in Computer Science, vol. 61). Berlin: Springer.

13.

Papadopoulou M. , Sadooghi-Alvandi M. and Wong H. ( 2009). Micro-benchmarking the GT200 GPU. Technical report. Computer Group, ECE, University of Toronto.

14.

Riegel E. , Indinger T and Adams N (2009) Implementation of a Lattice-Boltzmann method for numerical fluid mechanics using the nVIDIA CUDA technology. Comput Sci Res Development 23: 241-247.

15.

Ruetsch G. and Micikevicius P. ( 2009) Optimizing matrix transpose in CUDA. NVIDIA CUDA SDK Application Note.

16.

Tölke J. ( 2008) Implementation of a lattice Boltzmann kernel using the compute unified device architecture developed by nVIDIA. Comput Visualiz Sci 13: 1-11.

17.

Tölke J. and Krafczyk M. ( 2008) TeraFLOP computing on a desktop PC with GPUs for 3D CFD . Int J Comput Fluid Dyn 22: 443-456.