Grid-computing environments are becoming increasingly popular for scientific computing due to the significant increase in capacity that they represent when compared to a single computational resource (e.g., a single cluster), their ubiquitous availability and advances in grid middleware components. Several such specialized grid environments are now available for users in the commercial and research sectors. One such effort for research is the TeraGrid, consisting of a collection of geographically distributed heterogeneous supercomputer resources including data storage resources. Parallel implementations for these environments are inherently multilevel and obtaining efficient mapping of work to processors can be eXtremely challenging. This paper eXtends an eXisting MPI application to the grid via the use of grid-enabled MPI libraries. The application uses a simulation-optimization framework involving coarse-grained parallelism in the optimizer and fine-grained parallelism in the finite-element-based simulator. Using parallelism at both these levels is essential for problems involving computationally intensive simulation steps. A hierarchical grid architecture consisting of a collection of supercomputers is ideally suited for these types of problems as a good application-to-architecture mapping can be obtained with a proper implementation. This paper presents the performance results of our implementation on the TeraGrid network consisting of three geographically distributed supercomputers.
Abramson, D., Foster, I., Giddy, J., Lewis, A., Sosic, R., Sutherst, R., and White, N.1997. The Nimrod computational workbench: A case study in desktop metacomputing. In Proceedings of Australian Computer Science Conference (ACSC 97), Macquarie University, Sydney, Australia, 1997.
2.
Allen, G., Dramlitsch, T., Foster, I., Karonis, N., Ripeanu, M., Seidel, E., and Toonen, B.2001. Supporting efficient execution in heterogeneous distributed computing environments with Cactus and Globus. Proceedings of SC2001 Conference, Denver, CO, November 10—16, 2001.
3.
Bangerth, W., Klie, H., Matossian, V., Parashar, M., and Wheeler, M.2005. An autonomic reservoir framework for the stochastic optimization of well placement. Cluster Computing8:255—269.
4.
Bear, J.1972. Dynamics of Fluids in Porous Media. Environmental Science Series. New York: American Elsevier .
5.
Benger W., Foster, I., Novotny, J., Seidel, E., Shalf, J., Smith, W., and Walker, P.1999. Numerical relativity in a distributed environment. In Proceedings of Ninth SIAM Conference on Parallel Processing for Scientific Computing, 1999.
6.
Boghosian B., Coveney, P., Dong, S., Finn, L., Jha, S., Karniadakis, G., and Karonis, N.2006. Nektar, SPICE, and Vortonics: Using federated grid for large scale scientific applications”. Challenges of Large Applications in Distributed Environments (CLADE) 2006:34—43.
7.
Casanova H. , and Dongarra, J.1997. Netsolve: A network-enabled server for solving computational science problems. 1997. International Journal of Supercomputer Applications and High Performance Computing11(3):212—223.
Dong S., Karniadakis G., and Karonis, N.2005. Cross-site computations on the Teragrid. Computing in Science & Engineering (CiSE) Magazine7(5) Sept/Oct 2005:14—23.
11.
Dong, S., Insley, J., Karonis, N., Papka, M., Binns, J., and Karniadakis, G.2006. Simulating and visualizing the human arterial system on the TeraGrid. Future Generation of Computer Systems (FGCS)22(8):1011—1017.
12.
Dong, S., Karonis, N., and Karniadakis, G.2006. Grid solutions for biological and physical cross-site simulations on the TeraGrid. IEEE International Parallel & Distributed Processing Symposium (IPDPS), 2006 .
13.
European CommissionEnabling Grids for E-sciencE: http://public.eu-egee.org/
14.
Foster I., and Kesselman, C.1999. The Grid, Blueprint for a New Computing Infrastructure . San Francisco: Morgan Kauffman Publishers.
15.
Foster, I. , and Kesselman, C.1997. Globus. A metacomputing infrastructure toolkit. International Journal of Supercomputing Applications11(2):115—128.
16.
Globus: http://www.globus.org/
17.
Goldberg, D.1989. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley.
18.
Grama, A., Gupta A., Karypis G., and Kumar V.2003. Introduction to Parallel Computing. 2nd edition. Addison Wesley.
19.
Grimshaw, A.S. , and Wulf, W.A.1997. The Legion vision of a world wide virtual computer. Communications of the ACM 40(1):39—45.
20.
Gropp, W., Lusk W., and Skjellum, A.1999. Using MPI: Portable Parallel Programming with the Message-Passing Interface. Cambridge, MA: The MIT Press.
21.
Huyakorn, P.S. , and Pinder, G.F.1983. Computational Methods in Subsurface Flow. Academic Press.
22.
Insley, J. , Papka, M., Dong, S., Karniadakis, G., and Karonis, N.2007. Runtime visualization of the human arterial tree. IEEE Transactions of Visualization and Computer Graphics (to appear).
23.
Japan National Research Grid Initiative [NAREGI]: http://www.naregi.org/index_e.html
24.
Ferńandeza, J.-J., Garćıaa , I., Carazob, J.-M., and Marabini, R.2007. Electron tomography of complex biological specimens on the grid. Future Generation Computer Systems23:435—446.
25.
Karonis, N., Toonen, B., and Foster, I.2003. MPICH-G2: A grid enabled implementation of the message passing interface, Journal of Parallel and Distributed Computing (JPDC)63(5):551— 563.
26.
Karonis, N., Papka, M., Binns, J., Bresnahan, J., Insley, J., Jones, D., and Link, J.2003. High-resolution remote rendering of large datasets in a collaborative environment. Future Generation of Computer Systems (FGCS)19(6):909—917.
27.
Kere, P., and Lento, J.2005. Design optimization of laminated composite structures using distributed grid resources. Composite Structures71:435—438.
28.
Lagana, D. , Legato, P., Pisacane, O., and Vocaturo, F.2006. Solving simulation optimization problems on grid computing systems. Parallel Computing32:688—700.
29.
Luna, F., Nebro, A., and Alba, E.2006. Observations in using grid-enabled technologies for solving multi-objective optimization problems. Parallel Computing 32:377—393.
30.
Mahinthakumar, G., and Saied, F.2002. A hybrid MPI-OpenMP implementation of an implicit finite-element code on parallel architectures. International Journal of High Performance Computing Applications16(4) Winter: 371—393.
31.
Mahinthakumar, G., and Saied, F.1999. Implementation and performance analysis of a parallel multicomponent groundwater transport code. Proceedings of the SIAM Parallel Processing Meeting, San Antonio, TX, March 1999.
32.
Mahinthakumar, G., Hoffman F. M., Hargrove, W.W., Karonis, N.1999. Multivariate geographic clustering in a metacomputing environment using Globus. Proceedings of SC99 Conference on High Performance Networking and Computing , Portland, Oregon, Nov 12—18, 1999.
33.
Mahinthakumar, G.1999. PGREM3D: Massively Parallel Codes for Groundwater Flow and Transport, Online User's guide: http://www.4.ncsu.edu/∼gmkumar/pgrem3d.pdf
34.
Mahinthakumar, G., Saied, F., and Valocchi, A.J.1997. Comparison of some parallel Krylov solvers for large scale contaminant transport simulations. High Performance Computing. Society for Computer Simulation International , 1997:134—139.
35.
Melab, N. , Cahon, S., and Talbi, E.-G.2006. Grid computing for parallel bioinspired algorithms. Journal of Parallel and Distributed Computing66(8):1052—1061.
36.
Michalewicz, Z.1996. Genetic Algorithms + Data Structures = Evolution Programs. 3 rd edition. New York: Springer-Verlag .
37.
Morrison, R.D.2000. Application of forensic techniques for age dating and source identification in environmental litigation . Environmental Forensics1:131—153.
38.
MPICH-G2: http://www.niu.edu/mpi/
39.
MPICH-VMI: http://vmi.ncsa.uiuc.edu/
40.
NASA's Information Power Grid: http://www.nas.nasa.gov/About/IPG/ipg.html
41.
Pakin, S., and Pant, A.2000. VMI 2.0: A dynamically reconfigurable messaging layer for availability, usability, and management. 8th InternationalSymposium on High-Performance Computer Architecture, 2000.
42.
Parashar, M., Muralidhar, R., Lee, W., Arnold, D., Dongarra, J., and Wheeler, M.2005. Enabling interactive and collaborative oil reservoir simulations on the grid. Concurrency and Computation: Practice and Experience17:1387—1414.
43.
Poeter, E.P. , and Hill, M.C.1997. Inverse models: A necessary next step in groundwater modeling. Groundwater35(2) Mar—Apr 1997:250—260.
44.
Saied, F., and Mahinthakumar, G.1998. Efficient parallel multigrid based solvers for large scale groundwater flow simulations. Computers Mathematics and Applications35(7):45—54.
45.
Sayeed, M., and Mahinthakumar, G.2003. Hybrid optimization approaches for solving groundwater inverse problems in a parallel computing environment . Proceedings of ASCE World Water Congress, Philadelphia, PA, June 22—26.
46.
Schreiber, A. , Metsch, T., and Kersken, H.2005. A problem solving environment for multidisciplinary coupled simulations in computational grids. Future Generation Computer Systems21(6):942— 952.
47.
Sudholt, W., Baldridgea, K., Abramson, D., Enticott, C., Garicc, S., Kondricb, C., and Nguyenb, D.2005. Application of grid computing to parameter sweeps and optimizations in molecular modeling. Future Generation Computer Systems21(1):27—35.
48.
Sun, N.-Z.1994. Inverse Problems in Groundwater Modeling, Theory and Applications of Transport in Porous Media. Kluwer Academic Publishers , 6,337 pp.
US National Science Foundation TeraGrid [TG]: http://www.teragrid.org
51.
US Department of Energy Earth System Grid [ESG]: http://www.earthsystemgrid.org
52.
US Department of Energy Open Science Grid [OSG]: http://www.opensciencegrid.org
53.
Yang, X., Hayes, M., Jenkins, K., and Cant, S.2005. The Cambridge CFD grid for large-scale distributed CFD applications. Future Generation Computer Systems21:45—51.
