Pricing for Utility-Driven Resource Management and Allocation in Clusters

Abstract

Users perceive varying levels of utility for each different job completed by the cluster. Therefore, there is a need for existing cluster resource management systems (RMS) to provide a means for the user to express its perceived utility during job submission. The cluster RMS can then obtain and consider these user-centric needs such as Quality-of-Service requirements in order to achieve utility-driven resource management and allocation. We advocate the use of computational economy for this purpose. In this paper, we describe an architectural framework for a utility-driven cluster RMS. We present a user-level job submission specification for soliciting user-centric information that is used by the cluster RMS for making better resource allocation decisions. In addition, we propose a dynamic pricing function that the cluster owner can use to determine the level of sharing within a cluster. Finally, we define two user-centric performance evaluation metrics: Job QoS Satisfaction and Cluster Profitability for measuring the effectiveness of the proposed pricing function in realizing utility-driven resource management and allocation.

Keywords

utility computing market model service pricing resource management cluster computing

Get full access to this article

View all access options for this article.

References

Altair Grid Technologies (2000). OpenPBS Release 2. 3 Administrator Guide, Altair Grid Technologies, August.

Buyya, R. (ed.) (1999). High Performance Cluster Computing: Architectures and Systems, Prentice Hall PTR, Upper Saddle River, NJ .

Buyya, R. , Abramson, D. , and Giddy, J. (2000). An economy driven resource management architecture for global computational power grids , in Proc. of International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 2000), Las Vegas, NV, June.

Buyya, R. , Abramson, D. , and Giddy, J. (2001). A case for economy grid architecture for service oriented grid computing , in Proc. of 10th International Heterogeneous Computing Workshop (HCW 2001), San Francisco , CA, April.

Buyya, R. , Cortes, T. , and Jin, H. (2001). single system image , The International Journal of High Performance Computing Applications, 15(2): 124—135 .

Buyya, R. , Giddy, J. , and Abramson, D. (2000). An evaluation of economy-based resource trading and scheduling on computational power grids for parameter sweep applications , in Proc. of 2nd Annual Workshop on Active Middle-ware Services (AMS 2000), Pittsburgh, PA, August.

Buyya, R. , Murshed, M. , and Abramson, D. (2002). A deadline and budget constrained cost-time optimization algorithm for scheduling task farming applications on global grids , in Proc. of International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 2002), Las Vegas, NV, June.

Chun, B.N. and Culler, D.E. (2000a). Market-based proportional resource sharing for clusters, University of California at Berkeley , Computer Science Division , Technical Report CSD-1092, January.

Chun, B.N. and Culler, D.E. (2000b). REXEC: A decentralized, secure remote execution environment for clusters , in Proc. of 4th Workshop on Communication, Architecture, and Applications for Network-based Parallel Computing (CANPC '00), Toulouse, France, January.

10.

Chun, B.N. and Culler, D.E. (2002). User-centric performance analysis of market-based cluster batch schedulers , in Proc. of 2nd IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2002), Berlin, Germany, May.

11.

Feitelson, D.G. (2006). Parallel Workloads Archive, Oct. [Online]. Available: http://www.cs.huji.ac.il/labs/parallel/workload

12.

Ibm (2003). LoadLeveler for AIX 5L Version 3.2 Using and Administering, SA22-7881-01, IBM, October.

13.

Irwin, D.E. , Grit, L.E. , and Chase, J.S. (2004). Balancing risk and reward in a market-based task service , in Proc. of 13th International Symposium of High Performance Distributed Computing (HPDC13), Honolulu, HI, June.

14.

Lifka, D. (1995). The ANL/IBM SP scheduling system , in Proc. of 1st Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 1995), Santa Barbara, CA, April.

15.

Mu'alem, A.W. and Feitelson, D.G. (2001). Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling , IEEE Trans. Parallel and Distributed Systems, 12(6): 529—543 .

16.

Pfister, G.F. (1998). In Search of Clusters, 2nd ed., Prentice Hall PTR, Upper Saddle River, NJ .

17.

Platform Computing (2001). LSF Version 4.1 Administrator's Guide, Platform Computing .

18.

Stonebraker, M. , Devine, R. , Kornacker, M. , Litwin, W. , Pfeffer, A. , Sah, A. , and Staelin, C. (1994). An economic paradigm for query processing and data migration in Mariposa . in Proc. of 3rd International Conference on Parallel and Distributed Information Systems (PDIS '94), Austin, TX, September.

19.

Sherwani, J. , Ali, N. , Lotia, N. , Hayat, Z. , and Buyya, R. (2004). Libra: a computational economy-based job scheduling system for clusters , Software: Practice and Experience, 34(6): 573—590 .

20.

Sulistio, A. , Poduval, G. , Buyya, R. , and Tham, C.-K. (2005). Constructing a grid simulation with differentiated network service using GridSim , in Proc. of 6th International Conference on Internet Computing (ICOMP 2005), Las Vegas, NV, June.

21.

Sun Microsystems (2002). Sun ONE Grid Engine , Administration and User's Guide, Sun Microsystems , October.

22.

Supercluster Research and Development Group ( 2004). Maui Scheduler Version 3.2 Administrator's Guide, Supercluster Research and Development Group . [Online]. Available: http://www.supercluster.org/mauidocs/mauiadmin.shtml

23.

University of Wisconsin-Madison (2004). Condor Version 6.7.1 Manual, University of Wisconsin-Madison . [Online]. Available: http://www.cs.wisc.edu/condor/manual/v6.7

24.

Waldspurger, C.A. , Hogg, T. , Huberman, B.A. , Kephart, J.O. , and Stornetta, W.S. (1992). Spawn: A distributed computational economy , IEEE Trans. Software Eng., 18(2): 103— 117 .