A waterfall of exciting developments, that’s how I would characterize the last three months: The LOFAR2.0 Test Station (L2TS) works like a charm, L2TS data is arriving at COBALT, the first pulsar has been observed, all timing distributor hardware has been delivered, we can control the L2TS through TMSS, and so on.
At the time of writing, we are a hair’s breadth away from measuring the first light with L2TS, i.e. the measurement of a correlated sky signal between a LOFAR2.0 and a LOFAR1.0 station on COBALT. In the background, preparations for mass production continued. Many electronic parts have been delivered and are ready to be processed once the board layouts are approved.
Last, but not least, the documents to tender the purchase of new central network switches are ready for publication. Our new backbone will make the LOFAR network ready for LOFAR2.0 and brings us a step closer to the ambition to connect all stations to 100 Gbit/s in the future!
Happy reading!
Station – The proof of the pudding
The first week of July marked a major milestone in the Station project: the rollout of the LOFAR2 Test Station Phase 2 (L2TS-2). The three subracks (two low- and one high-band) containing all the new equipment were moved to station CS001 in Exloo. After a few days spent on checking if everything was installed correctly, the team kicked off the validation period with a highly successful busy week. During this week, astronomers and telescope engineers joined the development team and learnt how to use the system.
Over the summer, major successes were achieved by the extended team. Figure 1 shows the first simultaneous LBA and HBA observations, achieving one of the major goals of the Station upgrade project.
Next to proving that the new station complies with the requirements, big steps were made in making it more user-friendly. Numerous dashboards were developed with Grafana for controlling and monitoring the system (Figure 2).
During the validation process several bugs were discovered and addressed by the team. There were no major problems found, proving once more the value of the staged development and testing approach taken by the project.
Procurement LOFAR2.0 hardware
Just before the summer holiday break we have finished the European procurement process for the LOFAR2.0 Subracks (Antenna Processing Subracks, APS). This tender involved six Subracks for PTS and all the Subracks required for the Dutch and international stations. The order is awarded to the company Comdes who represents the Subrack manufacturer Apra. In the development phase (DTS and L2TS) we have used a different manufacturer for the Subracks. For PTS and the rollout of LOFAR2.0 we will be using a different Subrack. However, Apra is a very well-known manufacturer. The Apra Subracks has been used in the current LOFAR for over 10 years and proofed to be very reliable and low EMI and good EMC performances. Before Comdes will supply the six Subracks for PTS, we will work together with the Apra engineers on a prototype rack in order to minimize the risks for PTS.
Our PCB manufactures has confirmed that more than 90% of all electronic components are available now and board production for PTS (two stations plus spares) is planned to start in January. This means that we need to finalize the PCB layout so that the manufactures can start to order the bare boards. Lead time for the bare boards is approximately 10 weeks and we need to have the latest L2TS modifications included in the layouts of the station boards. Although we have ordered electronic components for all LOFAR stations to ensure the availability and reducing financial risk, we will order the PCBs for the rollout after verification of PTS giving us the opportunity for a last PCB layout update.
So far, we have spent a large amount of the hardware budget but there are still some procurement activities ongoing like for the station power supply and CEP6. Our next activity is to plan for accepting the hardware as soon production has started. There will be a large amount of hardware coming to us which need to be carefully organized for Assembly, Integration, and verification (AIV). The rollout team is currently working on the AIV planning and will be presented later. The following picture presents some of the planning highlights.
LOFAR2.0 pipeline developments
LOFAR’s dense network of long baselines up to approximately 2000 km allow for angular resolutions in the sub-arcsecond regime. With the LOFAR VLBI pipeline, producing high resolution images of radio sources becomes possible, paving the road towards high-resolution post-processing of LoTSS and future widefield imaging work. Over the past 21 months much effort was put in porting the pipeline to the Common Workflow Language (CWL).
Last July, the long baseline working group and interested community members came together in Paris for another successful “busy week”, working on various science and technical topics. Last month, the generic pipeline version of the LOFAR VLBI pipeline saw its final release. Now the final preparations are being made for a first release of the CWL implementation for others to start testing. The side-by-side illustrates the difference in resolving power between the Dutch and the international array.
DANTE
The main activity during the last months has been the production of 16 boards, done entirely at ASTRON. This is a time-consuming process since it can be automatized only partially. The board contains ~1000 components; up to ~700 can be placed using a pick and place machine – this is due to the limitations of the machine itself, and the rest are placed manually.
The first step was to produce a single board that was tested against the current LOFAR1 front end on 7 distinct aspects such as: gain, noise, linearity, beamforming accuracy. The results from the comparison are very encouraging; the DANTE AHBAFE shows a performance equal to the LOFAR1 FE in 2 aspects, and better for the remainder 5 parameters.
With this encouraging result, production continued during the summer, and we are now close to finalizing it. The boards are being tested in the EMC room, assembled on a 4-element array. Measurements are testing control, power, RF stability, RF coupling, sensitivity, passband, and power noise.
The next iteration of DANTE foresees the production of 80 boards at an external facility, with the goal of testing the manufacturability of the design. Discussions with companies during the summer concluded with the selection of the one that will produce this first batch. In preparation, the first version of the test fixture is now being used (see image lower-left panel).
All this progress has only been possible in such a brief time because of the hard work and dedication of the team of engineers: Albert van Duin, Lesley Goudbeek, Paulus Kruger, Henri Meulman, Mark Olijve, Mark Ruiter.
The work on DANTE will continue towards finalizing the testing on the lab to then move to the Dwingeloo test field, where the boards will be assembled on a tile. The boards will be potted, to test the possible impact of the coating on the performance. A series of tests will be dedicated to evaluating the new boards’ reliability and lifetime. The goal is to have a robust design ready in November so production of the next batch of prototypes can start shortly after.
LOFAR Software Development
TMSS
The Dynamic Scheduler is now in full production mode for cycle 20 and we are expanding its options and versatility as the need arises. So far, the operational staff is very happy with the functionality and the support. The next challenge is to prepare a full cycle; if the system has shown to be able to do that, we consider this phase of the dynamic scheduler development done. Some functionality that has been added lately are the ability to enable parallel observations and reservations, which was necessary to run the IDOLS project, and the ‘LST-pressure’ plots which help to pinpoint potential issues with requested observations.
A lot of work is being put in improving the TMSS frontend code. We are updating the older visualization and rendering libraries, improving performance by optimization of code, such as removing unnecessary loops, and we are refactoring the code for better readability and maintainability. And while this is happening, we are using input from the Operators and Science Support to improve the overall user experience and usability. This work is showing good progress and will continue in the coming sprints.
LIFT
We have reviewed and accepted the firmware design for the LIFT functionality. Also, we have decided what memory modules need to be ordered to be placed on the Uniboards for LIFT. Our next step is to inventory the software needs based on the LIFT minimal product definition and make an estimate of the amount of work for that, as well. We can then judge and decide on how to fit the necessary activities into the tight budget we have, so that we can start planning for the implementation when LOFAR2.0 is being rolled out.
An interesting news item (in Dutch) on lightning research has been published by the Dutch weather service KNMI: https://www.knmi.nl/over-het-knmi/nieuws/ontdekking-van-kleine-bliksemontladingen
It shows that LOFAR with LIFT has good visibility in the weather and lightning community.
Network and CEntral Processor upgrade
In the July newsletter it was reported that we are working on a big upgrade to the old LOFAR network equipment and CEntral Processing (CEP) clusters. The first step in this upgrade is the replacement of the network spine from the LOFAR Concentrator to CEP and the Core network equipment at the CEP location itself. At the end of September we have published a tender for two main new switches to form the new spine of this network. Our goal is to obtain a network that allows up to 400 Gbit/s speeds between all systems. A second goal is to simplify the network by going to a solution based only on the Ethernet protocol while we used a hybrid solution with Infiniband before. We aim for a solution that should last at least 5 years and hopefully will enable all kinds of new science well past 2030. Our main challenge is the distance between the LOFAR Core and CEP of 64 km, as this is only supported by a limited set of networking systems.
The diagram below shows where the new Concentrator switch and Core switch are planned as part of the LOFAR2.0 development.
SEE ALSO: