The analysis of large sensor datasets for structural and functional features has applications in many domains, including weather and climate modeling, characterization of subsurface reservoirs, and biomedicine. The vast amount of data obtained from state-of-the-art sensors and the computational cost of analysis operations create a barrier to such analyses. In this paper, we describe middleware system support to take advantage of large clusters of hybrid CPU–GPU nodes to address the data and compute-intensive requirements of feature-based analyses of large spatio-temporal datasets.
AbbasiHWolfMEisenhauerG (2010) DataStager: scalable data staging services for petascale applications. Cluster Computing13: 277–290.
2.
Arpaci-DusseauRAndersonETreuhaftN (1999) Cluster I/O with River: making the fast case common. In: IOPADS '99: Input/output for parallel and distributed systems, Atlanta, GA, 5 May 1999, pp. 10–22.
3.
AugonnetCThibaultSNamystR (2011) StarPU: a unified platform for task scheduling on heterogeneous multicore architectures. Concurrency and Computation: Practice and Experience23: 187–198.
4.
BeynonMKurcTCatalyurekU (2001) Distributed processing of very large datasets with DataCutter. Parallel Computing27: 1457–2478.
5.
BosilcaGBouteillerADanalisA. (2012) DAGuE: A generic distributed DAG engine for high performance computing. Parallel Computing38(1–2): 37–51.
6.
ChandolaVVatsavaiRR (2011) A scalable Gaussian process analysis algorithm for biomass monitoring. Statistical Analysis and Data Mining4: 430–445.
7.
ChangCKurcTSussmanA (2000) Optimizing retrievel and processing of multi-dimensional scientific datasets. In: Proceedings of the international parallel and distributed processing symposium (IPDPS 2000), Cancun, Mexico, 1–5 May 2000, pp. 405–410.
8.
CooperLKongJWangF (2011) Morphological signatures and genomic correlates in glioblastoma. In: IEEE international symposium on biomedical imaging: from nano to macro, Chicago, Illinois, USA, 30 March–2 April 2011, pp. 1624–1627.
9.
CooperLAKongJGutmanDA (2010) An integrative approach for in silico glioma research. IEEE Transactions on Biomedical Engineering57: 2617–2621.
10.
DeanJGhemawatS (2010) MapReduce: a flexible data processing tool. Communications of the ACM53: 72–77.
11.
DocanCParasharMKlaskyS (2010) DataSpaces: an interaction and coordination framework for coupled simulations workflows. In: Proceedings of the 19th international symposium on high performance distributed computing (HPDC’10)Chicago, Illinois, USA, 20–25, June 2010, pp. 25–36.
12.
HartleyTCatalyurekURuizA (2008) Biomedical image analysis on a cooperative cluster of GPUs and multicores. In: ICS'08: Proceedings of the 2008 ACM international conference on supercomputing, Island of Kos, Greece, 7–12 June 2008, pp. 15–25.
13.
KohnSRBadenSB (1995) A parallel software infrastructure for structured adaptive mesh methods. In: ACM/IEEE conference on supercomputing, San Diego, California, USA, 4–8December 1995, pp. Article no. 36. New York: ACM Press.
14.
LeQRanzatoMMongaR (2012) Building high-level features using large scale unsupervised learning. In: Proceedings of the 29th international conference on machine learning, Edinburgh, Scotland, UK, 26 June– 1 July 2012.
15.
LeeHBMacqueenJB (1980) A K-means cluster-analysis computer-program with cross-tabulations and next-nearest-neighbor analysis. Educational and Psychological Measurement40: 133–138.
16.
LiuYHillDJAbdelzaherT (2008) Virtual sensor–powered spatiotemporal aggregation and transformation: a case study analyzing near-real-time NEXRAD and precipitation gauge data in a digital watershed. In: Proceedings of the environmental information management conference 2008 (EIM-2008), Albuquerque, NM, 10–11 September 2008.
17.
LofsteadJKlaskySSchwanK (2008) Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS). In: CLADE 2008 at high performance distributed computing conference, Boston, Massachusetts, 23 June 2008, pp. 15–24.
18.
MeredithJS (2012) EAVL: The extreme-scale analysis and visualization library. In: Eurographics symposium on parallel graphics and visualization (EGPGV), Cagliari, Italy, 13–14May 2012, pp. 21–30. Cabliari: Eurographics Association.
19.
MillerMC (2001) Enabling interoperation of high performance, scientific computing applications: modeling scientific data with the sets and fields (SAF) modeling system. In: International conference on computational science, San Francisco, CA, USA, 28–30 May 2001, pp. 158–170. London: Springer-Verlag.
20.
MontiSTamayoPMesirovJ (2003) Consensus clustering: a resampling-based method for class discovery and visualization of gene expression microarray data. Machine Learning52: 91–118.
21.
MoranP (2001) Field model: an object-oriented data model for fields. Technical Report No. 20010066513, NASA Ames Research Center.
22.
PlaleBSchwanK (2000) dQUOB: managing large data flows using dynamic embedded queries. In: IEEE international high performance distributed computing (HPDC), Pittsburgh, Pennsylvania, USA, 1–4August 2000, pp. 263–270.
23.
TeodoroGHartleyTCatalyurekU (2010) Run-time optimizations for replicated dataflows on heterogeneous environments. In: Proceedings of the 19th ACM international symposium on high performance distributed computing (HPDC), Chicago, Illinois, USA, 20–25 June 2010, pp. 13–24.
24.
TeodoroGKurcTPanT (2012a) Accelerating large scale image analyses on parallel, CPU–GPU equipped systems. In: 26th IEEE international parallel and distributed processing symposium (IPDPS), Shanghai, China, 21–25 May 2012, pp. 1093–1104.
25.
TeodoroGPanT (2013) High-throughput analysis of large microscopy image datasets on CPU–GPU cluster platforms. In: The 27th IEEE international parallel and distributed processing symposium (IPDPS' 13), Boston, Massachusetts, USA, 20–24 May 2013. USA: IEEE Society.
26.
TeodoroGPanTKurcT (2012b) High-throughput execution of hierarchical analysis pipelines on hybrid cluster platforms. Technical report no. CCI-TR-2012-4. Center for Comprehensive Informatics, Emory University, USA.
27.
VetterJSGlassbrookRDongarraJ (2011) Keeneland: bringing heterogeneous GPU computing to the computational science community. Computing in Science & Engineering13: 90–95.
