Performance portability in reverse time migration and seismic modelling via OpenACC

Abstract

Heterogeneity among the computational resources within a single machine has significantly increased in high performance computing to exploit the tremendous potential of graphics processing units (GPUs). Portability in terms of code development and performance has been a challenge due to major differences between GPU programming and memory models from one side and conventional central processing units (CPUs) from another side. Performance characteristics of compilers and processors also vary between machines. Emerging high-level directive-based programming models such as OpenACC has been proposed to target this challenge. In this work, we develop OpenACC implementations for both seismic modelling and reverse time migration algorithms that solve the isotropic, acoustic, and elastic wave equations. We employ OpenACC to take advantage of the computational power of two Nvidia GPU cards: (1) M2090 and (2) K40, residing in IBM and CRAY XC30 clusters respectively. We also explore the main aspects of hybridization seismic modelling and reverse time migration by implementing an Message Passing Interface (MPI)+OpenACC approach. We expose various mapping techniques to develop a portable code that maximizes performance regardless of compiler or platform. Depending on the intensity of the computations, different propagators exhibited different speedup behaviours against a full socket CPU MPI implementation. A performance enhancement of ~10× was obtained, when the acoustic model was ported to a single GPU, compared with a 1.7× speedup obtained using the isotropic model. Our MPI+OpenACC implementation of reverse time migration and seismic modelling shows promising scaling when multiple GPUs were used.

Keywords

OpenACC seismic imaging reverse time migration General-Purpose Graphics Processing Units accelerator Message Passing Interface

Get full access to this article

View all access options for this article.

References

Abdelkhalek

Calandra

Coulaud

et al . (2009) Fast seismic modeling and reverse time migration on a GPU cluster. In: International conference on high performance computing and simulation, Leipzig, Germany, June 2009, pp.36–43. USA: IEEE. Available at: http://ieeexplore.ieee.org

Advisory Review Board (ARB) O (2014) The OpenMP API specification for parallel programming. Available at: http://openmp.org/wp/.

Baysal

Kosloff

Sherwood

(1983) Reverse time migration. Geophysics 48(11): 1514–1524.

Berenger

(1994) A perfectly matched layer for the absorption of electromagnetic waves. Journal of Computational Physics 114(2): 185–200.

Bland

Wells

Messer

et al . (2012) Titan: Early experience with the cray xk6 at oak ridge national laboratory. In: Proceedings of cray user group conference (CUG 2012).

Carcione

Herman

Ten Kroode

APE

(2002) Seismic modeling. Geophysics 67(4): 1304–1325.

Chang

McMechan

(1986) Reverse-time migration of offset vertical seismic profiling data using the excitation-time imaging condition. Geophysics 51(1): 67–84.

CUDA N (2007) Compute Unified Device Architecture Programming Guide. Available at: http://developer.download.nvidia.com/compute/cuda/1.0/NVIDIA_CUDA_Programming_Guide_1.0.pdf

CAPS Enterprise (2010) HMPP directives. Available at: https://www.olcf.ornl.gov/wp-content/uploads/2012/02/HMPPWorkbench-3.0_HMPP_Directives_ReferenceManual.pdf.

10.

Feki

Al–Jarro

Bagci

(2013) Multi-GPU-based acceleration of the explicit time domain volume integral equation solver using MPI-openacc. In: Radio science meeting (joint with AP-S symposium), Egypt, 2013, pp.90–90. USA : IEEE. Available at: http://ieeexplore.ieee.org

11.

Ghosh

Liao

Calandra

et al . (2012) Experiences with OpenMP, PGI, HMPP and OpenACC directives on ISO/TTI kernels. In: High Performance Computing, Networking, Storage and Analysis. USA: IEEE, pp.691–700. Available at: http://ieeexplore.ieee.org

12.

Herdman

Gaudin

McIntosh–Smith

et al . (2012) Accelerating hydrocodes with openacc, opecl and cuda. In: High Performance Computing, Networking, Storage and Analysis. USA: IEEE, pp. 465–471. Available at: http://ieeexplore.ieee.org

13.

Levesque

Sankaran

Grout

(2012) Hybridizing s3d into an exascale application using openacc: An approach for moving to multi-petaflops and beyond. In: Proceedings of the international conference on high performance computing, networking, storage and analysis, 2012, p.15. USA: IEEE Computer Society Press.

14.

Munshi

Gaster

Mattson

et al . (2011) OpenCL Programming Guide. Pearson Education. Available at: http://ptgmedia.pearsoncmg.com/images/9780321749642/samplepages/0321749642.pdf

15.

OpenACC-Standardorg (2013) The OpenACC application programming interface. Available at: http://www.openacc.org/sites/default/files/OpenACC.2.0a_1.pdf.

16.

Qawasmeh

Chapman

Hugues

et al . (2015) GPU technology applied to reverse time migration and seismic modeling via openacc. In: Proceedings of the 6th international workshop on programming models and applications for multicores and manycores, 2015, pp.75–85. USA: ACM.

17.

Siddiqui

Feki

(2014) Predictive performance tuning of OpenACC accelerated applications. In: Supercomputing. Springer, p.511.

18.

Suh

Wang

(2011) Expanding domain methods in GPU based TTI reverse time migration. In: SEG Technical Program Expanded Abstracts. pp.3460–3464. USA: Society of exploration geophysicists.

19.

The Portland Group (2010) PGI accelerator programming model for Fortran & C. Available at: http://www.pgroup.com/lit/whitepapers/pgi_accel_prog_model_1.3.pdf.

20.

Wolfe

(2010) Implementing the PGI accelerator model. In: Proceedings of the 3rd workshop on general-purpose computation on graphics processing units, 2010, pp.43–50. USA: ACM.