Challenges in exascale radio astronomy: Can the SKA ride the technology wave?

Abstract

The Square Kilometre Array (SKA) will be the most sensitive radio telescope in the world. This unprecedented sensitivity will be achieved by combining and analyzing signals from 262,144 antennas and 350 dishes at a raw datarate of petabits per second. The processing pipeline to create useful astronomical data will require hundreds of peta-operations per second, at a very limited power budget. We analyze the compute, memory and bandwidth requirements for the key algorithms used in the SKA. By studying their implementation on existing platforms, we show that most algorithms have properties that map inefficiently on current hardware, such as a low compute–bandwidth ratio and complex arithmetic. In addition, we estimate the power breakdown on CPUs and GPUs, analyze the cache behavior on CPUs, and discuss possible improvements. This work is complemented with an analysis of supercomputer trends, which demonstrates that current efforts to use commercial off-the-shelf accelerators results in a two to three times smaller improvement in compute capabilities and power efficiency than custom built machines. We conclude that waiting for new technology to arrive will not give us the instruments currently planned in 2018: one or two orders of magnitude better power efficiency and compute capabilities are required. Novel hardware and system architectures, to match the needs and features of this unique project, must be developed.

Keywords

Exascale Square Kilometre Array supercomputers power efficiency astronomy

Get full access to this article

View all access options for this article.

References

ARM (n.d.) big.LITTLE processing. Available at: http://www.arm.com/products/processors/technologies/biglittleprocessing.php.

Bakhoda

Yuan

Fung

WWL

. (2009) Analyzing CUDA workloads using a detailed GPU simulator. In: Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS, Boston, 26-28 April 2009), pp.163–174. doi:10.1109/ISPASS.2009.4919648.

Binkert

Beckmann

Black

. (2011) The gem5 simulator. SIGARCH Computer Architecture News 39(2): 1–7. doi:10.1145/2024716.2024718.

Bridle

Schwab

(1989) Wide field imaging I: Bandwidth and time-average smearing. Synthesis Imaging in Radio Astronomy 6: 247.

Clark

Plante

PCL

Greenhill

(2011) Accelerating radio astronomy cross-correlation with graphics processing units. Computing Research Repository abs/1107.4264.

Convey Computer^™ (n.d.) Convey Computer website. Available at: http://www.conveycomputer.com.

Cook

(2012) CUDA Programming: A Developer’s Guide to Parallel Computing with GPUs. 1st edition. San Francisco, CA: Morgan Kaufmann Publishers.

Cornwell

Golap

Bhatnagar

(2008) The non-coplanar baselines effect in radio interferometry: The W-Projection algorithm. IEEE Journal of Selected Topics in Signal Processing 2(5): 647-657.

Cornwell

Voronkov

Humphreys

(2012) Wide field imaging for the square kilometre array. Proceedings of SPIE 8500, Image Reconstruction from Incomplete Data VII 8500L: 1–12. doi:10.1117/12.929336.

10.

De Souza

Bunton

Campbell-Wilson

. (2007) A radio astronomy correlator optimized for the Xilinx Virtex-4 SX FPGA. In: Proceedings of the international conference on field programmable logic and applications, Amsterdam, 27-29 August, pp.62–67. doi:10.1109/FPL.2007.4380626.

11.

Dennard

Gaensslen

(2007) Design of ion-implanted MOSFET’s with very small physical dimensions. IEEE Solid-State Circuits Society Newsletter 12(1): 38–50. doi:10.1109/N-SSC.2007.4785543.

12.

Dongarra

(n.d.) HPCG benchmarking. Available at: http://www.sandia.gov/maherou/docs/HPCG-Benchmark.pdf.

13.

Esmaeilzadeh

Blem

St Amant

. (2011) Dark silicon and the end of multicore scaling. SIGARCH Computer Architecture News 39(3): 365–376. doi:10.1145/2024723.2000108.

14.

Frigo

Johnson

(1998) FFTW: An adaptive software architecture for the FFT. In: Proceedings of the 1998 IEEE international conference on acoustics speech and signal processing, Vol. 3, Seattle, 12–15 May, pp.1381–1384.

15.

Google® (2014) Custom motherboard announcement. Available at: https://plus.google.com/111282580643669107165/posts/Uwh9W3XiZTQ.

16.

Green 500 (n.d.) Green500 website. Available at: http://www.green500.org/.

17.

Hennessy

Patterson

(2006) Computer Architecture: A Quantitative Approach. 4th edition. San Francisco, CA: Morgan Kaufmann Publishers.

18.

Högbom

(1974) Aperture synthesis with a non-regular distribution of interferometer baselines. Astronomy and Astrophysics Supplement 15: 417.

19.

Humphreys

Cornwell

(2011) Analysis of convolutional resampling algorithm performance. Available at: http://www.skatelescope.org/uploaded/59116_132_memo_humphreys.pdf.

20.

Intel (n.d.(a)) Intel random number generator. Available at: http://software.intel.com/sites/default/files/m/d/4/1/d/8/441_Intel_R__DRNG_Software_Implementation_Guide_final_Aug7.pdf.

21.

Intel (n.d.(b)) Intel SSE and AVX extensions. Available at: http://software.intel.com/en-us/intel-isa-extensions.

22.

Intel (2011) Signal processing on Intel^® architecture: Performance analysis using Intel^® performance primitives. Technical report, Intel^®. Available at: http://www.intel.nl/content/dam/doc/white-paper/signal-processing-on-intel-architecture.pdf.

23.

Jeffs

(n.d.) Beamforming presentation. Available at: http://ens.ewi.tudelft.nl/Education/courses/et4235/Beamforming.pdf.

24.

Jongerius

Corporaal

Broekema

. (2012) Analyzing LOFAR station processing on multi-core platforms. In: Proceedings of the ICT Open 2012, 22–23 October 2012, pp. 71-76. Rotterdam, The Netherlands,. Available at: http://www.ictopen2013.nl/content/proceedings+2012.

25.

Jongerius

Wijnholds

Nijboer

. (2014) An End-to-End Computing Model for the Square Kilometre Array. IEEE Computer 47(9).

26.

Kamil

Shalf

Strohmaier

(2008) Power efficiency in high performance computing. In: Proceedings of the IEEE international symposium on parallel and distributed processing, Miami, 14–18 April, pp.1–8. doi:10.1109/IPDPS.2008.4536223.

27.

Keckler

Dally

Khailany

. (2011) GPUs and the future of parallel computing. IEEE Micro 31(5): 7–17. doi:10.1109/MM.2011.89.

28.

Lee

Kim

Chhugani

. (2010) Debunking the 100 × GPU vs. CPU myth: An evaluation of throughput computing on CPU and GPU. SIGARCH Computer Architecture News 38(3): 451–460. doi:10.1145/1816038.1816021.

29.

Leng

Hetherington

ElTantawy

. (2013) GPUWattch: Enabling energy optimizations in GPGPUs. In: Proceedings of the 40th annual international symposium on computer architecture, Tel-Aviv, 23–27 June. doi:10.1145/2485922.2485964.

30.

Ahn

Strong

. (2009) McPAT: An integrated power, area, and timing modeling framework for multicore and manycore architectures. In: Proceedings of the 42nd annual international symposium on microarchitecture, New York, 12–16 December, pp.469–480.

31.

Lobeiras

Amor

Doallo

(2011) FFT implementation on a streaming architecture. In: Proceedings of the 19th Euromicro international conference on parallel, distributed and network-based processing (PDP), Ayia Napa, 9–11 February, pp.119–126. doi:10.1109/PDP.2011.31.

32.

Nieuwpoort

Romein

(2011) Correlating radio astronomy signals with many-core hardware. International Journal of Parallel Programming 39(1): 88–114. doi:10.1007/s10766-010-0144-3.

33.

Offringa

de Bruyn

Zaroubi

. (2010) A LOFAR RFI detection pipeline and its first results. In: Groningen

(ed.) RFI mitigation workshop—RFI 2010, Groningen. Proceedings of Science, POS(RFI2010)036. SISSA.

34.

Patterson

(2005) Latency lags bandwidth. In: Proceedings of the 2005 international conference on computer design., San Jose, 2–5 October. pp.3–6. doi:10.1109/ICCD.2005.67.

35.

Perley

RAE

(1984) A proposal for a large, low frequency array located at the VLA site. VLA Scientific Memorandum: 146.

36.

Putnam

Caulfield

Chung

. (2014) A reconfigurable fabric for accelerating large-scale datacenter services. In: Proceedings of the 41st annual international symposium on computer architecture, Minneapolis, 14–18 June, pp. 13–24. DOI: 10.1109/ISCA.2014.6853195. Available at http://research.microsoft.com/apps/pubs/default.aspx?id=212001.

37.

Romein

(2013) Signal processing on GPUs for radio telescopes. In: Proceedings of the GPU technology conference. San Jose, 18-21 March.

38.

Romein

(2012) An efficient work-distribution strategy for gridding radio-telescope data on GPUs. In: Proceedings of the ACM international conference on supercomputing, Venice, Italy, 25–29 June pp.321–330.

39.

Romein

Broekema

Mol

. (2010) The LOFAR correlator: Implementation and performance analysis. In: Proceedings of the 15th ACM SIGPLAN symposium on principles and practice of parallel programming, Bangalore, 9–14 January, pp.169–178. doi:10.1145/1693453.1693477.

40.

Schwab

(1984) Relaxing the isoplanatism assumption in self-calibration; Applications to low-frequency radio interferometry. Astronomical Journal 89: 1076–1081.

41.

Shahbahrami

Juurlink

Vassiliadis

(2005) Efficient vectorization of the FIR filter. In: Proceedings of the 16th annual workshop on circuits, systems and signal processing, Veldhoven, 17–18 November, pp.432–437.

42.

SKA Organisation (n.d.) Square kilometer array. Available at: http://www.skatelescope.org/.

43.

SKA Organisation (2013) SKA1 Baseline design. Available at: https://www.skatelescope.org/wp-content/uploads/2012/07/SKA-TEL-SKO-DD-001-1_BaselineDesign1.pdf.

44.

SKA Organisation (2014) SKA phase 1 science (level 0) requirements specification. Available at: https://www.skatelescope.org/wp-content/uploads/2014/03/SKA1-Level0-Requirements.pdf.

45.

Szomoru

(2011) The UniBoard: A multi-purpose scalable high-performance computing platform for radio-astronomical applications. In: Proceedings of the URSI General Assembly and Scientific Symposium, Istanbul, 13–20 August, pp.1–4. doi:10.1109/URSIGASS.2011.6051281.

46.

Tasse

van der Tol

van Zwieten

. (2012) Applying full polarization A-Projection to very wide field of view instruments: An imager for LOFAR. Astronomy & Astrophysics, Article number A105, Volume 553, May 2013. Available at: http://arxiv.org/abs/1212.6178.

47.

Taylor

Carilli

Perley

(eds) (1999) Synthesis Imaging in Radio Astronomy II Astronomical Society of the Pacific Conference Series, Vol. 180). (San Francisco, CA: Astronomical Society of the Pacific.

48.

Thompson

Moran

Swenson

(2001) Interferometry and Synthesis in Radio Astronomy. 2nd edition. Weinheim, Germany: Wiley-VCH.

49.

Top500 (n.d.) Top500 website. Available at: http://www.top500.org/.

50.

Van Amesfoort

Varbanescu

Sips

. (2009) Evaluating multi-core platforms for HPC data-intensive kernels. In: Proceedings of the 6th ACM conference on computing frontiers, Ischia, 18–20 May, pp.207–216. doi:10.1145/1531743.1531777.

51.

Van Haarlem

Wise

Gunst

. (2013) LOFAR: The LOw-Frequency ARray. Astronomy and Astrophysics. 556(August 2013). Article number A2.

52.

Varbanescu

van Amesfoort

Cornwell

. (2009) Building high-resolution sky images using the Cell/B.E. Science of Computer Programming 17(1–2): 113–134. Available at: http://dl.acm.org/citation.cfm?id=1507443.1507454.

53.

Vassiliadis

Wong

Gaydadjiev

. (2004) The MOLEN polymorphic processor. IEEE Transactions on Computers 53(11): 1363–1375. doi:10.1109/TC.2004.104.

54.

Venkatesh

Sampson

Goulding

. (2010) Conservation cores: Reducing the energy of mature computations. SIGARCH Computer Architecture News 38(1): 205–218. doi:10.1145/1735970.1736044.

55.

Woods

(2010) Accelerating software radio astronomy FX correlation with GPU and FPGA co-processors. Master’s Thesis, University of Cape Town, South Africa. http://books.google.nl/books?id=ANHKXwAACAAJ.

56.

Wulf

McKee

(1995) Hitting the memory wall: Implications of the obvious. SIGARCH Computer Architecture News 23(1): 20–24. doi:10.1145/216585.216588.

57.

Yan

Shunying

(2011) A high performance FFT library with single instruction multiple data (SIMD) architecture. In: Proceedings of the international conference on electronics, communications and control (ICECC), Ningbo, 9–11 September, pp.630–633. doi:10.1109/ICECC.2011.6066463.