Chinese Supercomputer Tianhe-2. Image Credit: Prof. Yutong Lu.

SKA Science Data Processor consortium concludes work

Published: Thu, 09/05/2019
The P3 (platform performance prototype) at the University of Cambridge, used to test the design for SDP. Credit: SDP Consortium

SKA Global Headquarters, 9 May 2019 – The SKA’s Science Data Processor (SDP) consortium has concluded its engineering design work, marking the end of five years’ work to design one of two supercomputers that will process the enormous amounts of data produced by the SKA’s telescopes.

The international consortium, led by the University of Cambridge in the UK, has designed the elements that will together form the “brain of the SKA”. In total, close to 40 institutions in 11 countries took part[1]. SDP is the second stage of processing for the masses of digitised astronomical signals collected by the telescope’s receivers, following the correlation and beamforming that takes place in the Central Signal Processor (CSP).

“It’s been a real pleasure to work with such an international team of experts, from radio astronomy but also the High-Performance Computing industry” said Maurizio Miccolis, SDP’s Project Manager for the SKA Organisation. “We’ve worked with almost every SKA country to make this happen, which goes to show how hard what we’re trying to do is.”

The role of the consortium was to design the computing hardware platforms, software, and algorithms needed to process science data from CSP into science data products.

“SDP is where data becomes information” said Rosie Bolton, Data Centre Scientist for the SKA Organisation “This is where we start making sense of the data and produce detailed astronomical images of the sky.”

To do this, SDP will need to ingest the data and move it through data reduction pipelines at staggering speeds, to then form data packages that will be copied and distributed to a global network of regional centres where it will be accessed by scientists around the world.

SDP itself will be composed of two supercomputers, one located in Cape Town, South Africa to process data from SKA-mid and one in Perth, Western Australia, to process data from SKA-low.

“We estimate SDP’s total compute power to be around 250 PFlops – that’s 25% faster than IBM’s Summit, the current fastest supercomputer in the world,” said Maurizio. “In total, up to 600 PB of data will be distributed around the world every year from SDP – that’s enough to fill more than a million average laptops.”

Additionally, because of the sheer quantity of data flowing into SDP – some 5 Tb/s, or 100,000 times faster than the projected global average broadband speed in 2022 – it will need to make decisions on its own in almost real-time about what is noise and what is worthwhile data to keep.

The team also designed SDP so that it can detect and remove manmade radio frequency interference (RFI) – for example from satellites and other sources – from the data.

“By pushing what’s technologically feasible and developing new software and architecture for our HPC needs, we also create opportunities to develop applications in other fields” added Maurizio.

High-Performance Computing plays an increasingly vital role in enabling research in fields such as weather forecasting, climate research, drug development and many others where cutting-edge modelling and simulations are essential.

Prof. Paul Alexander, Consortium Lead at the University of Cambridge concluded “I’d like to thank everyone involved in the consortium for their hard work over the years. Designing this supercomputer wouldn’t have been possible without such an international collaboration behind it.”

Image in header: Chinese Supercomputer Tianhe-2. Image Credit: Prof. Yutong Lu.

Image in text: The P3 (platform performance prototype) at the University of Cambridge, used to test the design for SDP. Credit: SDP Consortium


[1]The SDP consortium was led by the University of Cambridge and involved the following institutes: Barcelona Supercomputing Center (BSC) - Centre Nacional de Supercomputació, Spain, Beijing Normal University (BNU), China, Centre for High Performance Computing (CHPC), South Africa, Centro Supercomputación Castilla y León (FCSCL), Spain, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia, Forschungszentrum Jülich GmbH (Jülich), Germany, Fudan University (FDU), China, Guangzhou University (GZHU), China, International Centre for Radio Astronomy Research (ICRAR), Australia, Inspur Incorporation (INSPUR), China, Institute of Computing Technology (ICT), Chinese Academy of Sciences, China, Instituto de Astrofísica de Andalucía-CSIC (IAA), Spain, Institute of Space Science and Astronomy, University of Malta (ISSA), Malta, Kunming University of Science and Technology (KMUST), China, Max Planck Institute for Radio-astronomy (MPIfR), Germany, National Astronomical Observatories of China (NAOC), China, National Research Council Canada (NRC), Canada, Netherlands Institute for Radio Astronomy (ASTRON), the Netherlands, New Zealand Alliance (NZA), New Zealand, Pawsey Supercomputing Centre, Australia, Portuguese ENGAGE SKA Consortium, Portugal, Science and Technology Facilities Council (STFC), UK, Shanghai Astronomical Observatory (SHAO), China, Shanghai Advanced Research Institute (SARI), Chinese Academy of Sciences, China, Shanghai Jiao Tong University (SJTU), China, South African Radio Astronomy Observatory (SARAO), South Africa, University of Calgary, Canada, University of Cambridge, UK, University of Cape Town, South Africa, University College London (UCL), UK, University of Manchester, UK, University of Oxford (OeRC), UK, Yunnan Astronomical Observatory (YAO), China, Victoria University of Wellington (VUW), New Zealand

[2]Globally, the average broadband speed per household in 2022 should reach ~75Mbps. Source: CISCO

Find out more about SDP’s work, including photos and videos.

About the SKA

The Square Kilometre Array (SKA) project is an international effort to build the world’s largest radio telescope, led by the SKA Organisation based at the Jodrell Bank Observatory near Manchester, UK. The SKA will conduct transformational science to improve our understanding of the Universe and the laws of fundamental physics, monitoring the sky in unprecedented detail and mapping it hundreds of times faster than any current facility.

The SKA is not a single telescope, but a collection of telescopes, called an array, to be spread over long distances. The SKA will be constructed in Australia and South Africa; with a later expansion in both countries and into other African countries.

Already supported by 12 countries – Australia, Canada, China, France, India, Italy, the Netherlands, New Zealand, South Africa, Spain, Sweden and the United Kingdom – the SKA Organisation has brought together some of the world’s finest scientists, engineers and policy makers and more than 100 companies and research institutions in the design and development of the telescope.

Read more about the SKA’s Critical Design Reviews

Frequently Asked Questions

Read the complete press release here