LOFAR Development Newsletter May 2023

André Gunst    Boudewijn Hut

The LOFAR (Low Frequency Array) Telescope is an instrument that can be used to investigate lightning flashes in a thunderstorm at a unique level of detail. With the distribution of stations over a large area, it becomes possible to record radio waves originating in a single lightning flash from different angles (Figure 1). When upgrading to LOFAR2.0, we want to ensure that LOFAR2.0 remains a world-leading instrument for lightning science.

However, the list of possible instrument adjustments is long. In their quest to answer fundamental questions, scientists need to have the best instruments. Figure 2 (left) shows how the feature list traces back to the lightning science case. But the available budget allows only for a limited number of adjustments to the basic LOFAR2.0 design.

Therefore, we adapt our engineering processes to become more iterative and concurrent. With an iterative approach, we focus on what adds most value to the end user. The high-priority features lead to the Minimal Viable Product (MVP) and the system description. With the concurrent approach, we organise the work such that all disciplines work in parallel and use a single model to share and update information (see Figure 2, right). That is different compared to the sequential approach, which has the disadvantage of added overhead of handing over information from one group to the other. Note that also the science users are at the table, providing them access to all expertise they need, and giving them full control during the process. Adapting our processes will save a lot of time and money in making the optimal instrument for lightning researchers. In turn, more and better data can be collected with LOFAR2.0, resulting in fascinating new insights into the meter-scale structure of lightning initiation and propagation.

Ágnes Mika

It is an exciting time in the Station project. With the rollout of L2TS phase 2 (L2TS-2) only a few weeks away, we are waiting for the last boards (RCU2) to arrive. The RCU2s will first be tested in the lab using our recently built test setup (see Fig. 1). We are then going to populate three subracks and test them thoroughly (individually and connected to each other) in the lab, before moving them to CS001 in Exloo.

A lot of preparation went into getting us to this stage. The subracks were designed and the parts ordered (see Fig. 2), PCBs were designed and produced in house (e.g., for controlling the fans and signal distribution to all 32 RCU2s, Fig. 3), issues with the Uniboard2s were solved and the RCU2 test setup was completed. The verification and validation plan for testing the subracks in the lab is ready. Modifications have been made to the central processing facility COBALT to accommodate L2TS-2 data, allowing us to assess the signal quality in astronomical observations.

While working on getting everything ready for the rollout of L2TS-2, we have not forgotten about the longer-term tasks, either. Discussions with KIWA were held to clarify which standards we need to comply with to obtain a CE certification for the new LOFAR2.0 subracks. We are now deriving a list of tests from those standards that we will need to perform and document. The L2TS-2 verification and validation plan are being worked out. These tests will prove if L2TS-2 complies with the requirements (in other words: “have we built the right thing”?).

The Station project ends after the verification and validation tests have been executed with L2TS-2 at CS001 and the results evaluated. A formal review late this summer will determine whether the new station is ready to be produced and installed on a large scale.

Nico Ebbendorf

With L2TS on its way, procurement of the PTS (Production Test Station) and rollout hardware is ongoing. We mentioned in the previous newsletter that we have placed an order for all the electronic components of LOFAR2.0. This order is carried out by four Electronic Manufacturing Services (EMS) who were selected during the European tender. The challenge is to acquire all the components within the planned period (and a low price) so that the board assembly can start on time for the PTS and later the rollout of LOFAR2.0. We are dealing with a number of hard to get and some obsolete components. Our designers are discussing the issues with the EMSs and working towards a solution so that production can start in time. Our planning is that the production of PTS will start in January next year. Production for the rollout will follow after verification of PTS in the second halve next year.

Meanwhile the European tender for the subracks is in preparation. The design is using as much as possible standardised 19-inch components. However, the rack cooling system require custom made parts which has been designed and tested for an optimum cooling of the (for LOFAR2.0 increased) power dissipation of the electronics.

Currently two items are still in preparation for procurement, these are the station network equipment and the power supplies. We expect that this will be clarified soon.

André Offringa

Development of LOFAR2.0 pipelines is progressing quickly! First of all, a lot of progress is being made to make the LINC pipeline (formerly known as “Prefactor”) ready for LBA processing, and this pipeline is expected to be ready within 6 months. Additionally, the Rapthor-HBA-NL pipeline that performs direction-dependent calibration will be delivered within 3 months. This pipeline will form the basis for a production ready LOFAR2.0 pipeline and serves also as a development and test platform for future extensions to imaging and calibration components. Improvements to these components again help expert users to develop new processing strategies.

Finally, work is underway to implement a LOFAR VLBI pipeline that produces high-resolution images of individual sources.

All these pipelines are written such that they can easily be integrated in an SDC framework, and active synergy with SKA development teams improves our momentum and helps us to produce stable and future-proof software products.

Carla Baldovin

The Advanced HBA front end board (AHBAFE) consists of a series of blocks; its complexity can be appreciated in Figure 1.

Progress during the past period:

• True time delay (TTD) board potted and tested (Figure 2).
• an updated version of the low noise amplifier (LNA) board optimized, assembled, and tested (Fig. 2).
• The layout of the HBAFE is now completed. After several iterations and reviews (see previous Newsletter), we now have the complete layout of the front-end board ready for prototyping (Fig. 2).
• In consultation with Telescope Operations an LED is now placed in the front-end board (seen on a 3D printed model of the front-end board in the image). This will facilitate diagnostics during maintenance activities (Fig. 2).
• Started a series of tests to characterize the frontend’s performance as a function of temperature (Fig. 3).

In the coming weeks, we will produce prototypes for a small number of boards. These will be then tested thoroughly in the laboratory and only if the tests are satisfactory, will we proceed to the production of a sufficient number of boards to integrate one tile.

At the same time, thanks to the knowledge the team has gained during this period, an update of the plan is being prepared and will soon be shared with the stakeholders.

Arno Schoenmakers

Team Ruby

Personnel 

The constituency of Team Ruby has changed a bit over the past months. First of all, we welcomed Rene Lourens, who is our new scrum master. The role of scrum master was formerly taken by Arno Schoenmakers, who became the Product Owner for the team in January, and by Corné Lukken, who served as a Scrum Master ad interim between February 1st and April 1st. We are happy that we have been able to find a permanent and dedicated scrum master for the team, to help organise and improve our way of working within the scrum context. René will serve the team for half his time; the other half is dedicated to team Rainbow in the Science Data Center (SDC) of ASTRON.

Apart from Rene, we have two other additions to the team: Fanna Lautenbach, formerly part of team Rainbow and broadly experienced in, a.o., web frontend technology, and Leonardo Pelonero from INAF, who is an INAF in-kind contribution to the ILT. With Fanna and Leonardo, we hope to have gained two developers to contribute to the TMSS frontend and LOFAR2.0 Monitoring and Alarming system. Unfortunately, Arthur had to decide to leave the team, he will be working for Telescope Operations in the coming period.

Station Control 

The Station Control efforts focus mostly on the preparations for the next test phase of the LOFAR2.0 test station L2TS. As part of the second phase of tests, during which the station is fully equipped with LOFAR2.0 hardware, we need to process the beamlet data at the COBALT correlator, for autocorrelations of the L2TS station itself, but also to compare the station data with that of LOFAR1 stations. To allow this we have added a translation layer to the COBALT code so that COBALT can now process both LOFAR1 and LOFAR2.0 Station beamlet data. This work has been done together with the HPC team of the Smart Backend group at ASTRON.

Furthermore, we finished work related to the station calibration data format. In LOFAR1 the station calibration data was in a custom-formatted binary file format. For LOFAR2.0 we decided to use only HDF5 formatted files for calibration data, but also for statistics data such as SST, XST. This allows our users to use a huge collection of open-source third–party applications to inspect, visualize and adapt or export these files.

Finally, together with the Tango-Controls community, we have managed to improve and upgrade the Tango-Controls Python interface (PyTango) which we use in the core of our system, and which also enables archiving of our monitor data in a Prometheus database system.

Many more small improvements have been made as well, certainly worth mentioning but too long a list for this contribution.

TMSS

TMSS was moved to a new and faster server, and we see a substantial improvement in the responsiveness of the system.

Commissioning of the observing modes is progressing, and we expect to switch all production observations to TMSS soon, but certainly before Cycle 20 start in June.

Further commissioning is ongoing for expected observing setups for cycle 20. Among these is commissioning for Dynamic Scheduling, which we want to have ready for cycle 20. Currently we are testing this with ~60 pulsars for pulsar timing observations. Next on the list is to test the responsive telescope functionality.

The LOFAR Cycle 20 is intended to run fully with TMSS, only. Some administrative functionality may only be commissioned after the cycle starts on June 1st, as well as final observing strategies that can run later in the cycle.

In the LIFT project, we have finalised the definition of the Minimal Viable Product. Based on this definition, Eric Kooistra is now designing the required firmware functionality for LIFT. Together with Eric, Jan David is looking into the monitoring and control software needs for Station Control. We expect a first version of both to be ready in early May.

There have been some contacts and meetings with representatives of the Cosmic Ray group to investigate possible synergetic opportunities. These will be followed-up the coming weeks. A first brief inventory learns that both groups, LIFT, and the CR group, will benefit from working together to create a more advanced transient buffer functionality in LOFAR.