Component-based approach for programming and running scientific applications on grids and clouds

Abstract

This paper presents an approach to programming and running scientific applications on grid and cloud infrastructures based on two principles: the first one is to follow a component-based programming model, the second is to apply a flexible technology which allows for virtualization of the underlying infrastructure. The solutions described in this paper include high-level composition and deployment consisting of a scripting-based environment and a manager system based on an architecture description language (ADL), a dynamically managed pool of component containers, and interoperability with other component models such as Grid Component Model (GCM). We demonstrate how the proposed methodology can be implemented by combining the unique features of the Common Component Architecture (CCA) model together with the H2O resource sharing platform, resulting in the MOCCA component framework. Applications and tests include data mining using the Weka library, Monte Carlo simulation of the formation of clusters of gold atoms, as well as a set of synthetic benchmarks. The conclusion is that the component approach to scientific applications can be successfully applied to both grid and cloud infrastructures.

Keywords

cloud computing component programming distributed application grid computing programming model

Get full access to this article

View all access options for this article.

References

Altunay

Avery

Blackburn

Bockelman

Ernst

Fraser

. (2011) A science driven production cyberinfrastructure – the Open Science Grid. Journal of Grid Computing 9: 201–218.

Armstrong

. (2006) The CCA component model for high-performance scientific computing. Concurrency and Computation: Practice and Experience 18: 215–229.

Baduel

Baude

Caromel

(2005) Object-Oriented SPMD. In IEEE International Symposium on Cluster Computing and the Grid, Vol. 2, pp. 824–831.

Baduel

. (2006) Programming, deploying, composing, for the grid. In: Cunha

Rana

(eds) Grid Computing: Software Environments and Tools. Heidelberg: Springer.

Baude

Caromel

Dalmasso

Danelutto

Getov

Henrio L et

al.

(2009) GCM: a grid extension to fractal for autonomous distributed components. Annales des Télécommunications 64: 5–24.

Baude

Caromel

Henrio

Morel

(2007) Collective interfaces for distributed components. In Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2007) 14–17 May 2007, Rio de Janeiro, Brazil. IEEE Computer Society, pp. 599–610.

Beckman

(2005) Building the TeraGrid. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 363: 1715–1728.

Beisiegel

. (2007) SCA Service Component Architecture Assembly Model Specification, Version 1.0. http://osoa.org/display/Main/Service+Component+Architecture+Home.

Berthold

Takada

(2002) Java Astrodynamics Toolkit (JAT). http://jat.sourceforge.net/.

10.

Bertrand

Bramley

Sussman

Bernholdt

Kohl

Larson JW et

al.

(2005) Data redistribution and remote method invocation in parallel component architectures. 19th International Parallel and Distributed Processing Symposium (IPDPS 2005), CD-ROM / Abstracts Proceedings, 4–8 April 2005, Denver, CO. IEEE Computer Society.

11.

Bishop

Warren

(2003) JavaSpaces in Practice. Boston, MA: Addison-Wesley.

12.

Bolze

Cappello

Caron

Daydé

Desprez

Jeannot E et

al.

(2006) Grid'5000: A large scale and highly reconfigurable experimental grid testbed. International Journal of High Performance Computing Applications 20: 481–494.

13.

Bouziane

Pérez

Priol

(2008) A software component model with spatial and temporal compositions for grid infrastructures. In: Luque

Margalef

Benitez

(eds) Proceedings Euro-Par 2008 – Parallel Processing, 14th International Euro-Par Conference, Las Palmas de Gran Canaria, Spain, 26–29 August 2008 (Lecture Notes in Computer Science, vol. 5168). Berlin: Springer, pp. 698–708.

14.

Bruneton

. (2006) The fractal component model and its support in Java. Software: Practice and Experience 36: 1257–1284.

15.

Bubak

Gubała

Kapałka

Malawski

Rycerz

(2005) Workflow composer and service registry for grid applications. Future Generation Computer Systems 21: 79–86.

16.

Bubak

Malawski

Gubala

Kasztelnik

Nowakowski

Harezlak

(2009) Virtual Laboratory for Collaborative Applications. IGI Global, Chapter 27: 531–551.

17.

Bubak

Malawski

Zajac

(2003) Architecture of the grid for interactive applications. In: Sloot PMA et

al.

(eds) Proceedings of Computational Science – ICCS 2003, International Conference, Melbourne, Australia and St. Petersburg, Russia, June 2003 (Lecture Notes in Computer Science , vol. 2657). Berlin: Springer, pp. 207–213.

18.

Caromel

Henrio

(2005) A Theory of Distributed Objects. Berlin: Springer.

19.

Ciepiela

Harezlak

Kocot

Bartynski

Kasztelnik

Nowakowski P et

al.

(2010) Exploratory programming in the Virtual Laboratory Proceedings of the International Multiconference on Computer Science and Information Technology, Wisla, Poland. IEEE Press, pp. 621–628.

20.

Coppola

Danelutto

Lacour

Pérez

Priol

Tonellotto N et

al.

(2005) Towards a common deployment model for grid systems. In: Gorlatch

Danelutto

(eds) Proceedings of the Integrated Research in Grid Computing Workshop. Pisa, Italy: Università di Pisa, Dipartimento di Informatica, TR-05-22: 31–40.

21.

Danelutto

Aldinucci

(2006) Algorithmic skeletons meeting grids. Parallel Computing 32: 449–462.

22.

Dean

Ghemawat

(2008) MapReduce: Simplified data processing on large clusters. Communication of the ACM 51: 107–113.

23.

Deelman

(2010) Grids and clouds: Making workflow applications work in heterogeneous distributed environments. International Journal of High Performance Computing Applications 24: 284–298.

24.

Dünnweber

Gorlatch

(2009) Higher-Order Components for Grid Programming – Making Grids More Usable. Berlin: Springer.

25.

Dyrda

Malawski

Bubak

Naqvi

(2009) Providing security for MOCCA component environment. In 23rd IEEE International Symposium on Parallel and Distributed Processing, IPDPS 2009, Rome, Italy, 23–29 May 2009. IEEE Press, pp. 1–7.

26.

Falcone

Chopard

Hoekstra

(2010) MML: Towards a Multiscale Modeling Language. Procedia CS 1: 819–826.

27.

Foster

Kesselman

(2006) Scaling System-Level Science: Scientific exploration and IT implications. Computer 39: 31–39.

28.

Foster

(2006) Globus Toolkit Version 4: Software for service-oriented systems. Journal of Computer Science Technology 21: 513–520.

29.

Foster

Kesselman

Tsudik

Tuecke

(1998) A security architecture for computational grids. In ACM Conference on Computer and Communications Security, pp. 83–92.

30.

Gabriel

Fagg

Bosilca

Angskun

Dongarra

Squyres JM et

al.

(2004) Open MPI: Goals, concept, and design of a next generation MPI implementation. In Proceedings 11th European PVM/MPI Users' Group Meeting. Budapest, Hungary, pp. 97–104.

31.

Gentzsch

Girou

Kennedy

Lederer

Reetz

Riedel M et

al.

(2011) DEISA – Distributed European Infrastructure for Supercomputing Applications. Journal of Grid Computing 9: 259–277.

32.

Gil

Deelman

Ellisman

Fahringer

Fox

Gannon D et

al.

(2007) Examining the challenges of scientific workflows. Computer 40: 24–32.

33.

Gorissen

Wendykier

Kurzyniec

Sunderam

(2005) Integrating grid information services using JNDI. 6th IEEE/ACM International Workshop on Grid Computing (Grid 2005). Seattle, Washington, USA.

34.

Govindaraju

. (2003) Merging the CCA component model with the OGSI framework. In CCGRID ‘03: Proceedings of the 3rd International Symposium on Cluster Computing and the Grid. Washington, DC: IEEE Computer Society, p. 182.

35.

Gubała

Herezlak

Bubak

Malawski

(2006) Semantic composition of scientific workflows based on the Petri nets formalism. In Second IEEE International Conference on e-Science and Grid Computing (e-Science'06), Amsterdam, The Netherlands. IEEE Computer Society, p. 12.

36.

Henderson

(1995) Job scheduling under the portable batch system. In Feitelson

Rudolph

(eds), Job Scheduling Strategies for Parallel Processing (Lecture Notes in Computer Science , vol. 949). Berlin:Springer, pp. 279–294.

37.

Henning

(2008) The rise and fall of CORBA. Communications of the ACM 51: 52–57.

38.

Herráez

(2006) Biomolecules in the computer: Jmol to the rescue. Biochemistry and Molecular Biology Education 34: 255–261.

39.

HLA (2010) IEEE standard for modeling and simulation (M and S) high level architecture (HLA) – framework and rules. Technical report IEEE Standard 1516-2010 (Revision of IEEE Std 1516-2000).

40.

Jurczyk

Golenia

Malawski

Kurzyniec

Bubak

Sunderam

(2006) Enabling remote method invocations in peer-to-peer environments: RMIX over JXTA. In Wyrzykowski

Dongarra

Meyer

Wasniewski

(eds), Parallel Processing and Applied Mathematics, 6th International Conference, PPAM 2005, Poznan, Poland, September 11–14, 2005, Revised Selected Papers (Lecture Notes in Computer Science, vol. 3911). Berlin: Springer, pp. 667–674.

41.

Karonis

Toonen

Foster

(2003) MPICH-G2: A grid-enabled implementation of the message passing interface. Journal of Parallel and Distributed Computing 63: 551–563.

42.

Kohn

et al. (2001) Divorcing language dependencies from a scientific software library. In Proceedings of the 10th SIAM Conference on Parallel Processing for Scientific Computation. Portsmouth, VA: SIAM.

43.

Kranzlmüller

Lucas

Öster

(2010) The European Grid Initiative (EGI). In Davoli

Meyer

Pugliese

Zappatore

(eds), Remote Instrumentation and Virtual Laboratories. Boston, MA: Springer US, Chapter 6, 61–66.

44.

Krishnan

Gannon

(2004) XCAT3: A framework for CCA components as OGSA services. In Proceedings of the International Workshop on High-Level Parallel Programming Models and Supportive Environments (HIPS), Santa Fe, NM, pp. 90–97.

45.

Kumfert

Leek

Epperly

(2007) Babel remote method invocation. In: 21st International Parallel and Distributed Processing Symposium (IPDPS 2007), Proceedings, 26–30 March 2007, Long Beach, CA. IEEE, pp. 1–10.

46.

Kurzyniec

Wrzosek

Drzewiecki

Sunderam

(2003a) Towards self-organizing distributed computing frameworks: The H2O approach. Parallel Processing Letters 13: 273–290.

47.

Kurzyniec

Wrzosek

Sunderam

Slomiński

(2003b) RMIX: A multiprotocol RMI framework for Java. In Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS'03), Nice, France. IEEE Computer Society, pp. 140—146.

48.

Malawski

Bartyński

Bubak

(2010) Invocation of operations from script-based grid applications. Future Generation Computer Systems 26: 138–146.

49.

Malawski

Bartynski

Ciepiela

Kocot

Pelczar

Bubak

(2006a) An ADL-based support for CCA components on the Grid. In CoreGRID Workshop on Grid Systems, Tools and Environments, Sophia-Antipolis, France.

50.

Malawski

Bartynski

Ciepiela

Kocot

Pelczar

Bubak

(2007a) A new approach to supporting component applications on grid. In Bubak

Turala

Wiatr

(eds), Proceedings of Cracow Grid Workshop – CGW'06, 15–18 October 2006. Krakow, Poland: ACC-Cyfronet AGH, 328–336.

51.

Malawski

Bubak

Baude

Caromel

Henrio

Morel

(2007b) Interoperability of grid component models: GCM and CCA case study. In CoreGRID Symposium (CoreGRID Series). Berlin: Springer, pp. 95–106.

52.

Malawski

Bubak

Placek

Kurzyniec

Sunderam

(2006b) Experiments with distributed component computing across grid boundaries. In: Proceedings of the HPC-GECO/CompFrame workshop in conjunction with HPDC 2006, Paris, France.

53.

Malawski

Gubala

Kasztelnik

Bartynski

Bubak

Baude F et

al.

(2008a) High-level scripting approach for building component-based applications on the grid. In Danelutto

Fragopoulou

Getov

(eds) Making Grids Work: CoreGRID Workshop, Heraklion, Crete. Berlin: Springer, pp. 307–320.

54.

Malawski

Harezlak

Bubak

(2006c) Towards multiprotocol and multilanguage interoperability: Experiments with Babel and RMIX. In Bubak

Turala

Wiatr

(eds), Proceedings of Cracow Grid Workshop – CGW'05, 20–23 November 2005. Krakow, Poland: ACC-Cyfronet AGH, 266–278.

55.

Malawski

Kocot

Ryszka

Bubak

Wieczorek

Fahringer

(2008b) Optimization of application execution in the GridSpace environment. In Gorlatch

Fragopoulou

Priol

(eds), CoreGRID Integration Workshop 2008 – Integrated Research in Grid Computing, pp. 395–405.

56.

Malawski

Kurzyniec

Sunderam

(2005) MOCCA – towards a distributed CCA framework for metacomputing. In Proceedings of the 10th International Workshop on High-Level Parallel Programming Models and Supportive Environments (HIPS2005) in conjunction with International Parallel and Distributed Processing Symposium (IPDPS 2005). IEEE Computer Society.

57.

Malawski

Meizner

Bubak

Gepner

(2011) Component approach to computational applications on clouds. Procedia Computer Science 4: 432–441.

58.

Mayer

McGough

Furmento

Lee

Newhouse

Darlington

(2003) ICENI dataflow and workflow: Composition and scheduling in space and time. In UK e-Science All Hands Meeting, Nottingham, UK, pp. 627–634.

59.

Meizner

Malawski

Ciepiela

Kasztelnik

Harezlak

Nowakowski P et

al.

(2009) Virolab security and virtual organization infrastructure. In Dou

Gruber

Joller

(eds), Proceedings Advanced Parallel Processing Technologies, 8th International Symposium, APPT 2009, Rapperswil, Switzerland, 24–25 August 2009 (Lecture Notes in Computer Science , vol. 5737). Berlin: Springer, pp. 230–245.

60.

Morgan

Cantor

Carmody

Hoehn

Klingenstein

(2004) Federated security: The Shibboleth approach. EDUCAUSE Quarterly 27: 12–17.

61.

Mougin

Barriolade

(2001) Web Services, Business Objects and Component Models. Technical Report, Orchestra Networks.

62.

Murty

(2008) Programming Amazon Web Services – S3, EC2, SQS, FPS, and SimpleDB. Sebastopol, CA: O'Reilly.

63.

NGG Group (2004) Next Generation Grids 2 requirements and options for European Grids research 2005–2010 and beyond. Technical report.

64.

Object Management Group, Inc (2004) Common Request Broker Architecture Specification, Version 3.0.3. Available at: http://www.omg.org/cgi-bin/doc?formal/04-03-01.

65.

Object Management Group, Inc (2006a) CORBA Component Model, υ4.0. http://www.omg.org/technology/documents/formal/components.htm.

66.

Object Management Group, Inc (2006b) Deployment and configuration of component-based distributed applications specification, version 4.0. http://www.omg.org/docs/formal/06-04-02.pdf.

67.

Ostermann

Iosup

Yigitbasi

Prodan

Fahringer

Epema

(2010) A performance analysis of EC2 cloud computing services for scientific computing. In Akan

Bellavista

Cao

Dressler

Ferrari

Gerla M et

al.

(eds), Cloud Computing (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering , vol. 34(9)). Berlin: Springer, pp. 115–131.

68.

Ousterhout

(1998) Scripting: Higher-level programming for the 21st century. Computer 31: 23–30.

69.

Parlavantzas

Morel

Getov

Baude

Caromel

(2007) Performance and scalability of a Component-Based grid application. In Proceedings of IPDPS 2007, pp. 1–8.

70.

Perez

Priol

Ribes

(2003) A parallel CORBA component model for numerical code coupling. The International Journal of High Performance Computing Applications (IJHPCA) 17: 417–429.

71.

Rellermeyer

Alonso

Roscoe

(2007) R-OSGi: distributed applications through software modularization. In Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware, New York, NY. Berlin: Springer, pp. 1–20.

72.

Schatz

(2009) CloudBurst: highly sensitive read mapping with MapReduce. Bioinformatics 25: 1363–1369.

73.

Schwiegelshohn

Badia

Bubak

Danelutto

Dustdar

Gagliardi F et

al.

(2010) Perspectives on grid computing. Future Generation Computer Systems 26: 1104–1115.

74.

Sloot

Tirado-Ramos

Altintas

Bubak

Boucher

(2006) From molecule to man: Decision support in individualized e-Health. Computer 39: 40–46.

75.

Sotomayor

Keahey

Foster

(2008) Combining batch execution and leasing using virtual machines. In Parashar

Schwan

Weissman

Laforenza

(eds) HPDC ‘08: Proceedings of the 17th International Symposium on High Performance Distributed Computing. New York: ACM Press, pp. 87–96.

76.

Streit

Erwin

Lippert

Mallmann

Menday

Rambadt M et

al.

(2005) Unicore – From project results to production grids. Advances in Parallel Computing 14: 357–376.

77.

Thain

Tannenbaum

Livny

(2005) Distributed computing in practice: the Condor experience. Concurrency and Computation – Practice and Experience 17: 323–356.

78.

van Nieuwpoort

Maassen

Wrzesinska

Hofman

RFH

Jacobs

CJH

Kielmann T et

al.

(2005) Ibis: a flexible and efficient Java-based grid programming environment. Concurrency – Practice and Experience 17: 1079–1107.

79.

Van Nieuwpoort

Wrzesińska

Jacobs

CJH

Bal

(2010) Satin: A high-level and efficient grid programming model. ACM Transactions on Programming Languages and Systems 32: 3.

80.

Vecchiola

Pandey

Buyya

(2009) High-performance cloud computing: A view of scientific applications. In 2009 10th International Symposium on Pervasive Systems, Algorithms, and Networks (ISPAN), pp. 4–16.

81.

Vogels

(2003) Web services are not distributed objects. IEEE Internet Computing 7: 59–66.

82.

Wilde

Foster

Iskra

Beckman

Zhang

Espinosa A et

al.

(2009) Parallel scripting for applications at the petascale and beyond. Computer 42: 50–60.

83.

Wilson

Johnston

(2000) Modelling gold clusters with an empirical many-body potential. European Physical Journal D 12: 161–169.

84.

Witten

Frank

(2005) Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems). Waltham, MA: Morgan Kaufmann.

85.

Zhao

Belloum

Bubak

(2009) Special section on workflow systems and applications in e-Science. Future Generation Computer Systems 25: 525–527.