On Monday 11 February, we acquired the new state of the art ‘brains’ for the Low Frequency Array (LOFAR). We are now the proud owners of the COBALT2.0 GPU cluster. COBALT2.0 will execute the next generation correlator and beam forming application of LOFAR.
Published by the editorial team, 15 February 2019
Bossers & Cnossen supplied the cluster in partnership with Dell Technologies. COBALT2.0 is a state-of-the-art cluster and its design is future proof. COBALT2.0 is amongst the most energy efficient (greenest) HPC correlators.
LOFAR Mega Mode
LOFAR is the world’s largest radio telescope. LOFAR has the unique ability to address key astronomical questions. This makes it one of the scientific cornerstones of international radio astronomy. Thanks to COBALT2.0, LOFAR will be able to carry out simultaneous observations for several science cases. This so-called multi-mode for LOFAR is known as the LOFAR Mega Mode (LMM). The LMM is jointly funded by ASTRON and NWO via the NWO LMM grant proposal, led by Jason Hessels (ASTRON & UvA). The science cases range from all-sky imaging surveys to discovery and high cadence monitoring of exotic pulsars. To make this happen, COBALT2.0 will simultaneously process data streams from LOFAR stations across countries in Europe. Processing of the data happens in real-time for both the ‘imaging’ and the ‘beam-forming’ modes.
Technical specifications COBALT 2.0
Hardware:
- 26 Intel Xeon Gold Skylake-SP 6140 CPUs
- 26 NVIDIA Tesla V100 GPUs
- COBALT2.0 is equipped with 100 Gbps EDR InfiniBand Technology
Compute Power:
- 63 TFLOPS of CPU compute power
- 360 TFLOPS of GPU compute power
COBALT2.0 can receive more than 1 Terabits/s i.e. more than 30 billion samples per second from the LOFAR stations and process them in real-time. It offers significant extra capacity (compared to the LMM requirements) both in terms of network bandwidth and compute power. The extra capacity will be used for data quality improvements. An example of this is online high time and frequency resolution radio frequency interference excision. The additional capacity of COBALT2.0 will also play an important role during the planned DUPPLO and LOFAR 2.0 upgrades, for instanceparallel observations with Low Band Antenna’s and High Band Antenna’s. Interesting enough, the availability of Tensor Cores in Volta V100, which can further speed up performance for half precision computations, opens up the possibility of exploring new astronomical observing modes which may be carried out with lower precision.
The COBALT2.0 team is looking forward to the exciting challenge of implementing the full LOFAR Mega Mode, and exploiting its new scientific capabilities. Subsequently COBALT2.0 will replace the existing COBALT1.0 correlator.
Scientific capabilities of LOFAR
COBALT2.0 will play a pivotal role in defining the scientific capabilities of LOFAR. With its induction, scientists and engineers at ASTRON are further pushing the envelope to turn LOFAR into a true multitasking radio telescope without changing its receiving elements. The excellent success of this project is possible due to the pioneering modern digital design of the LOFAR telescope. Besides, ASTRON’s inherent team strength consisting of end to end expertise starting from laying down of the science case up to the final technical implementation made this a success. And most important, the resulting increased efficiency and capabilities of LOFAR will lead to new discoveries to help understand, and unravel more scientific mysteries of the universe in the years to come.
COBALT2.0 team
The COBALT2.0 team comprises of V.N. Pandey (ASTRON, Kapteyn Institute), Jan David Mol, Cees Bassa, Jason Hessels, Rene Kaptijn, Chris Broekema, Jasmin Klipic, Reinoud Bokhorst, Klaas Stuurwold, Jorrit Schaap, Arthur Coolen from ASTRON and Hopko Meijering, Arjen Koers, Wietze Albers from University of Groningen (RUG).
The COBALT2.0 team would like to acknowledge the referees: Paul Boven (JIVE), John Romein (ASTRON) and Alexander van Amesfoort for their help and support. The team also thanks Kapil Khandelwal (Univ. of Notre Dame, USA), Akhil Chandra (Cetera Group, USA), Eite Tiesinga (RUG), Bram Veenboer (ASTRON) and Yan Grange (ASTRON) for discussions and suggestions.
For more information, please see [(1), (2), (3), (4), (5), (6), (7)] and for further queries please contact: V.N. Pandey, Jan David Mol or Rene Kaptijn.