LOFAR Development Newsletter December 2024

Carla Baldovin

Production Test Stations (PTS) have been fully verified at station level (Figure 1).

All the designs of the LOFAR2.0 products have been released and are currently in production. The UniBoards2 are being delivered to ASTRON on an (almost) weekly basis. The expectation is to start receiving the remaining components in mid-January. At that point, the activity will concentrate in the Integration Test Facility (ITF) in Westerbork, where the Antenna Processing Subracks will be integrated and tested.

The AIV team is now preparing a more detailed plan for the rollout activities, including the dismantling of the LOFAR1 stations (in the Netherlands). The plan needs to consider various factors such as:

• storage needed;

• storage capability in Westerbork;

• production throughput: the LOFAR products are made by different manufacturers, sometimes a single product includes a component made by a second company. Also, the time needed to complete production is different for different products;

• availability of people to join the activities;

• weather conditions.

In collaboration with our industrial partners, we are preparing a delivery schedule that will help us to manage efficiently the hardware deliveries.

AIV is about the installation of the hardware in the field, until its verification at single station level. The main processes are shown in Figure 2, the procedures for the installation of the hardware are based on the experience gained in the installation of PTS. Agreements with Telescope Operations and the Commissioning team are put in place to achieve a smooth transition between AIV and Commissioning.

In November the team had a review of the full set of drawings that describe the cabinet configuration. These include, among many others: APS, RCU assembly, UniB2 assembly, network and power cable routing, etc. The first review focused on the drawings for the stations in the Netherlands. A second review will focus on the drawings for the international stations, this is expected to take place in early January.

A high-level planning of the AIV timeline is presented in Figure 3.

Arno Schoenmakers

Telescope Manager & CEP

The Telescope Manager is an important  component in the product breakdown of LOFAR2.0. The Telescope Manager comprises all software and infrastructure related to central control, monitoring, specification, and scheduling of LOFAR2.0. The TMSS services are part of this, but also the central monitoring system and the Grafana based GUIs developed by the Operators. As we are shifting our attention towards the rollout of many more LOFAR2.0 stations, this component becomes more important in the system as a whole.

We improved the reporting functions of TMSS. It was not always clear what the reported numbers exactly meant, and whether the underlying calculations were in line with that definition. We have now improved that, so that reports are more clear and factually correct. Next in line is the function to create station-based reports, which is especially valuable for reports of the usage of international stations.

Furthermore, we delivered new functionality for the Operators to keep track of reservations and to connect that easily to events or issues within the telescope. This saves them quite some time to handle all this.

Although we are now less involved in the development of the new proposal tool, we stay in close collaboration with that team. We discuss, a.o., the authentication and authorization mechanisms, and the exchange of data and templates between TMSS and the new proposal tool.

The Operators continued to develop new Grafana based monitoring screens for the stations and the system as a whole. Some support was provided to them but not to the level they really needed. Still, they have made good progress in visualizing the current state and issues with the PTS LOFAR2.0 stations.

Together with Telescope Operations we are investigating options for renewing the infrastructure that runs the central services and stores our monitoring and statistics data. The data throughput numbers that we have from PTS allow us to estimate our needs for a full LOFAR2.0 system. We are now in the process to translate these needs that to an affordable and workable hardware solution.

We also helped with the downscaling of the existing CEP hardware (COBALT and CEP4). This activity was planned and executed in early September. Together with this, the operating systems had to be updated on many existing hosts. This added quite a bit of work for the team, especially for the COBALT and CEP4 systems. Unfortunately, this was time that we could not plan well due to dependencies on other teams, and we could not spend it on LOFAR2.0 development.

Stations

Most of our effort has gone into supporting the testing of the PTS stations, most notable CS032 and RS307. This shows the value of these test stations, as we already found and solved many smaller and larger issues in the software.

Meanwhile we are making preparations for the rollout of all LOFAR2.0 stations. This requires a higher level of automation than we needed for PTS, if we do not want to be overwhelmed with manual labour. Together with the AIV team, and Operations, we are investigating what needs to be more automated and prepared. We are on tight schedule, so this is getting high priority.

For the coming months, our priority will remain with the preparations for the rollout of all LOFAR2.0 stations.

Roberto Pizzo    John Swinbank

A LOFAR2.0 Data Processing and Management Busy Week took place at ASTRON between 21-25 October to discuss various aspects of the realization of the LOFAR2.0 observing program. A combination of Large Programs PIs, ASTRON staff, and expert LOFAR users participated to the event. The meeting featured several discussions about the LOFAR2.0 Observatory, specifically:

1. its structure and processes, to establish how the various part of the Observatory will work together to generate science-ready data products;
2. supported observing modes and their timeline for delivery to production;
3. the data management plans of the various large programmes, including their expected data products and plans for returning final data products to the LTA;
4. pipeline development processes, to establish standards and responsibilities for developing code used to generate the final science-ready data products.

All the discussions served to build a shared view of how LOFAR2.0 will run. The conclusions have been captured in a document summarizing various recommendations to the LOFAR-ERIC director and council regarding the operations and governance of LOFAR2.0. The report will ultimately become public.

Carla Baldovin and DANTE team

DANTE Critical Design Review is scheduled to take place on December 3rd. The goal of this milestone is to confirm that the design is mature enough and production at large scale can start. The AHBAFE (Advanced HBA Front-End) will be used in the new stations in Italy and Bulgaria, each of them will have ~1600 front ends.

At the previous newsletter we reported the testing of 20 prototypes manufactured externally. The next step was the production of the remaining 60 boards. Due to problems with the availability of PCBs, and to reduce further delays, an alternative material was used. The change of PCB material poses a certain degree of risk to the performance of the AHBAFE, therefore we decided to produce a small batch (24) first, test it and then produce with the remaining. In October, we installed 16 AHBAFE in a tile in CS001 for a test campaign. The results were encouraging, with the DANTE tile showing 10% higher sensitivity at 150MHz compared to the LOFAR1 HBAFE (Figure 4). The AHBAFE are still in CS001 and are being used in the regular test observations of LOFAR2.0.

To this date, all 80 boards have been produced and were approved using the production test setup developed in parallel to the AHBAFE. This was indeed the moment to put the test setup to proof, since it will be used at the manufacturing facility to test the thousands of boards during production (Figure 5). Any failures need to be detected in a short time; a single board is tested in ~3 minutes and we expect to reduce this time further.

The final tests will be done once all boards are potted (ongoing). At that point, we will have 5 tiles with DANTE boards in CS001.

Arno Schoenmakers

There are no new developments for the LIFT Transient Buffer implementation. The FPGA firmware implementation is planned to start soon. Once the firmware has been delivered, we still need to implement the software side of LIFT and execute all the necessary tests to fully deliver the Transient Buffer functionality of LOFAR2.0.

Boudewijn Hut    André Gunst

The LOFAR Development Program runs multiple projects, just a few:
– to deliver the Advanced HBA Front-End;
– to integrate the Station into the Telescope;
– to upgrade the Central Processing subsystem;
– to deliver monitoring and control of the Telescope.

Together, these projects deliver new capabilities to LOFAR, such that it ever improves the observed data.

Requirements are written up to prepare the development. However, depending on the project, this occurs in different locations and sometimes in different formats. There are even cases that they only exist in people’s heads. All making it very hard for an outsider to understand what can be expected from the development.

In our requirements tool, Polarion, we now introduce an easier structure of products, functions, and requirements. The products are defined, and functions are assigned to them. Combined, this represents a unique place to capture requirements: new requirements can easily be added to the correct product and function. Moreover, it becomes clear where conflicts may be, and they can be addressed upfront.