LOFAR2.0 Newsletter December 2018

After several years of loyal duty, it is time to replace the COBALT 1.0 cluster (the heart and brains of the LOFAR telescope where the data from all the stations comes together and is correlated and beamformed). With only a few months to go, COBALT 2.0 is almost ready for use. After a successful tender, the hardware order has been placed. In this interview René Kaptijn will tell you all you need to know.

Could you tell a bit about the tender?
The COBALT2.0 project is running for quite a while now, I got actively involved in December last year. At that point we knew we had to make a requirements document for the tender. It was a challenging job to make this document, but we are very happy with the end result. At first, we planned to publish the tender earlier. But despite the hard work of the team and because of the fact that a crucial team member was not available, we had a delay of a few months. But then the team could make the needed adjustments to get the tender ready for publication.

In July the tender was ready for publication on Tendernet. In the meantime, we had also approached some potential suppliers. In August we received questions that the tenders submitted. It quickly became clear that we had to downscale our demands a little bit, otherwise it was not possible to offer this within the budget.

In October we had two offers, both were good. But we had to decide and the offer from Bossers en Cnossen in Groningen scored much higher on our tender rating. That is why the tender was awarded to them.

How is the cooperation with Bossers en Cnossen until now?
The cooperation with Bossers en Cnossen runs very smoothly. We are happy with them. They are the preferred supplier of the RuG and they already supplied systems to SURFsara and Nikhef. They are also specialised in High Performance Computing. We are confident that it will be a success with the delivery of COBALT 2.0.

What is the planning for COBALT 2.0?
The project is divided into two phases. In the first phase we will replace the hardware. In the second phase the software will be adapted to be able to use the actual power of the new cluster. This includes implementing the LOFAR Mega Mode (LMM) with parallel observations. We are now in phase 1 and everything seems to go as planned. If everything goes according to plan, the cluster will be installed on 8 January. Then, a couple of days later, on 11 January, the acceptance of the cluster is planned. At that point we will be the official owner of COBALT 2.0. And in May 2019 we expect to go live with COBALT 2.0.

What is happening between January and May?
Once the cluster is installed we have planned two months to fine tune COBALT 2.0 and then two months to commission COBALT 2.0. It’s good to know that we can run COBALT 1.0 and COBALT 2.0 parallel. So, while we are testing COBALT 2.0, COBALT 1.0 is still operational. This also allows us to test COBALT 2.0 sufficiently.

A big wish we had was that the new hardware for COBALT 2.0 could run the software that is used for COBALT 1.0 with minimal adaptations. This makes it possible to easily transfer the COBALT 1.0 software and allows us fine tuning the software instead of writing completely new software.

What will happen to COBALT 1.0?
Fortunately, we do not have to dispose of COBALT 1.0. There are already some ideas to get COBALT 1.0 to Dwingeloo and keep using it. Of course, we already have some colleagues who have the greatest plans for COBALT 1.0. We will see what will happen.

What are you most proud of in this project?
I’m really proud of how smooth the tender and project went and all the hard work the team has done. They have put in a lot of effort to make this happen. Because of the good people and their work, the project runs very smoothly. I look forward to the rest of this project with the great team!

LOFAR was developed in the years 2001 to 2008 and the first stations were rolled-out more than 10 years ago. After almost 10 years of successful operations, LOFAR is reaching its end-of-life time and LOFAR2.0 will take over. Many original LOFAR components such as the HBA structure, cabinets, cables and more will be re-used for LOFAR2.0 and facing an extended lifetime.

To find out the condition and state in which the LOFAR components are after 10 years being in all kind of weather, the LOCAL project was started to inspect the most critical components and to find out if and how these components are ready to serve another long operational period for LOFAR2.0.

LOCAL stands for LOFAR’s Condition And Lifecycle and the project carried out an assessment of the field installed LOFAR components during the warm summer months this year. Two teams visited a representative number of remote- and core stations to inspect various items such as HBA’s, LBA’s, electronic boards, cabling, ground-anchoring etc. This information is collected and further analyst and the LOCAL report will be presented soon. We found out that some components are, sometimes to our surprise still in a very good condition while others require extra maintenance or need to be replaced completely. Not only the weather has been creating a hostile condition for LOFAR but small rodents and birds also caused serious damage at specific stations.

The following pictures showing Henri Meulman inspecting the HBA element and Jan-Pieter de Reijer checking one of the LBA ground-anchors.

In November, lab test of the White Rabbit clock distribution system have commenced in the ASTRON radiolab. LOFAR 2.0 Stage 1 will enable the distribution of a single-clock signal to all Dutch LOFAR stations, which greatly increases the robustness and precision of ionospheric calibration by separating these effects from clock errors. But also, for earth lightning studies a single clock is critical to remove a major source of systematic error: Simultaneous LBA+HBA data will allow for a more accurate determination of the time-structure of the pulses. This time structure is telling about the flow of charge during a single step, a currently unexplored area.

Currently, the LOFAR stations outside the core operate with separate clocks that are synchronized via GPS. The accuracy of this clock synchronization is limited and therefore the clocks drift over +/- 15ns on timescales of tens of minutes. The new clock distribution system is based on White Rabbit technology. White Rabbit is a fully deterministic Ethernet-based network for general-purpose data transfer and sub-nanosecond accuracy time transfer. For LOFAR the application of White Rabbit will lead to a 10- to 100-fold enhancement in clock synchronization between stations – i.e. with respect to the currently applied Rubidium/GPS and Syncoptics systems. Additionally, this technology enables the use of a central clock for all Dutch LOFAR stations, also the ones that are not connected via the concentrator node.

Before rolling-out the White Rabbit system in the field, its stability is characterised in the lab to ensure it meets the LOFAR 2.0 requirements. These tests are currently ongoing in the Radiolab. The first results are positive. After completing the lab tests, the next step will be to install and evaluate a White Rabbit link between Groningen and the concentrator node in Exloo/Buinen.

The Polarion® requirements management tool has been introduced to support the LOFAR 2.0 systems engineering approach. Polarion is used to define and store requirements at all levels of the system, and to ensure traceability. Such a tool is indispensable for a complex system like LOFAR 2.0. Polarion is now up and running, and the team has been trained. Most of the LOFAR 2.0 Stage 1 use-cases, science and system requirements have been imported into Polarion and links have been established between them. From now on, Polarion is the central location for all LOFAR 2.0 requirements.