Published by the editorial team, 3 May 2018

The new era of radio astronomy we are entering involves solving the most pressing data challenges in IT. With massive radio telescopes such as the Square Kilometre Array (SKA) coming online, we need new ways to store, transport, explore, and access massive amounts of data. During a two-day Hackathon (23-24 April 2018) at ASTRON, 34 data-experts from academia and industry gathered to address some of these challenges.

The Hackathon opened on 23 April with a talk by Michael Wise, the head of ASTRON’s Astronomy Group. He stressed how astronomy is evolving to be increasingly data-intensive and that the only way we can cope with this is to go beyond standard pipelines as employed in current radio astronomy telescopes. He highlighted the necessity for clever classification of astronomical objects and updated archives to ensure enriching science.

Next, Rob Lyon, a current post-doctoral researcher in the SKA group at University of Manchester, talked about the current SKA design and the challenges expected for every design component. He used an example of pulsar searches at exa-scale (10^18 bytes) to illustrate how the SKA era would breach machine learning frontiers given a pressing imbalance issue. Both these talks can be accessed here: https://bit.ly/2Fqyqop.

During the Hackathon, participants were given complete independence to work on a project of their choosing. The projects were proposed in advance and proposers outlined brief idea, required skills, etc. This set the stage for various collaborative teams to be formed on the day.

Teams of 4-5 people worked together on several different projects including topics such as machine learning classifiers, gold standard test vectors in singularity, cloud computing and scalability for astronomy dataset and Apertif/LOFAR Machine learning for transients.

This event served as the first platform to bring together astronomy community and industry experts to identify and address data issues expected in the intensely data-driven radio astronomy era. Participants have started thinking about alternatives to current state-of-the-art GPU driven computing. For example, Gema Parreno and her team could successfully demonstrate Google Cloud Platform scalability for storing, accessing, analyzing and effectively visualising the radio astronomy data.

Discussions ensued during the two-day event about various machine learning classifiers and how to choose ones that would suit our unique needs. Another interesting project was led by Bart Scheers from Centrum Wiskunde Informatica and Matthieu Marseille from Thales group. Here, they used SQL queries in integration with python (HealPy) to rapidly scour large astronomy datasets. They observed that such query combination drastically altered the computational time to only 1.3%.

The final results of the ASTRON Hackathon projects are outlined in a collaborative Google document which can be accessed here: https://bit.ly/2vYINAj.

The final presentations from some of the teams can be found here: https://bit.ly/2Fqyqop.

Another way to follow up on developments during the session is to search Twitter for #astronhackathon.

Sources
Related

Latest tweets

Daily image of the week

On June 13-17, the LOFAR Family Meeting took place in Cologne. After two years LOFAR researchers could finally meet in person again. The meeting brings together LOFAR users and researchers to share new scientific results.
https://www.astron.nl/dailyimage/main.php?date=20220621

Our renewed ‘Melkwegpad’ (Milky Way Path) is finished! The new signs have texts in Dutch on the one side and in English on the other side. The signs concerning planets have a small, 3D printed model of that planet in their centre.
https://www.astron.nl/dailyimage/
#Melkwegpad @RTVDrenthe

Daily image of the week

The background drawing shows how the subband correlator calculates the array correlation matrix. In the upper left the 4 UniBoard2s we used. The two ACM plots in the picture show that the phase differences of the visibilities vary from 0 to 360 degrees.

Daily image of the week: Testing with the Dwingeloo Test Station (DTS)
One of the key specifications of LOFAR2.0 is measuring using the low- and the highband antenna at the same time. For this measurement we used 9 lowband antenna and 3 HBA tiles.
https://www.astron.nl/dailyimage/main.php?date=20220607

searchtwitter-squarelinkedin-squarebarsyoutube-playinstagramfacebook-officialcrosschevron-right