This LOFAR2.0 newsletter is bursting at the seams with highlights. It's fantastic how we continue to improve LOFAR together, and how enthusiastic the users are to start using those improvements! Two of those users, Harish Vedantham and Brian Hare received prestigious ERC Starting Grants for their research using LOFAR. Congratulations Brian and Harish! Brian is convinced that LOFAR is the best instrument in the world to study lightning, and the investments into the LOFAR2.0 transient buffer enabled by his award will make it even better. More good news that will boost LOFAR’s capabilities, particularly in the Low-band, is that the ILT decided to implement a connection between the NenuFAR antennas and the French LOFAR station. This will enable LOFAR users to benefit from the huge collecting area of NenuFAR. The technology development on LOFAR2.0 continues in full swing, as you will see from the contributions in this Newsletter.
Perhaps not as exciting, but nevertheless important is that ASTRON is moving to an instrument centric approach. For LOFAR this means that the LOFAR2.0 program has evolved into the “LOFAR development program” since January this year. The scope of the LOFAR2.0 program was (only) to deliver the LOFAR2.0 upgrade. The new program coordinates all LOFAR related development activities. Its scope entails everything from the antennas to the delivery of L1 data products to the Science Data Centre. The program “new-style” is helping to oversee and manage the overall LOFAR design, the development and deployment of upgrades, and the LOFAR instrument roadmap.
At the December 3rd, 2021 deadline we received 20 EoIs, spanning a wide range of techniques and scientific topics, including: lightning; cosmic dawn and the epoch of reionization; radio recombination lines; solar and space weather studies; cosmic rays; exoplanets; pulsars and transients; and deep surveys at the highest-possible angular resolution and the lowest achievable radio frequencies. The EoIs represent the combined scientific plans of over 500 unique authors, spanning all the current LOFAR partner countries, and beyond. The LOFAR2.0 project scientists are busy digesting the roughly 200 pages of submitted plans in order to distill an overview of the requested observations. It is clear that the oversubscription on LOFAR2.0's expected available observing time is high, so one of the primary goals is to find ways to run projects in a commensal way. This includes running simultaneous LBA+HBA observations, simultaneous imaging and beam-formed observations, and commensal triggers on cosmic rays. As intended, LOFAR2.0 makes it possible to do this, and hence to multiplex the telescope’s science output. At the same time, it is also important to take stock of the requested sky areas and depths in order to see how a global observing programme might be scheduled in practice. The LOFAR2.0 project scientists are preparing an overview document summarising the EoIs for the submitters and ILT Board. This will serve to help tailor an EoI workshop to be held with EoI submitters at the end of March, 2022, which in turn is geared towards helping the various teams prepare their full observing proposals.
Hairy Network Adventures
The upgrade of LOFAR to LOFAR2.0 will result in much more scientific data to be explored, but all that data also needs to be transported from the stations to the central systems. The data rate increment of the stations will be a factor two, except for the core stations, which will produce 1.5x more data. In principal the data can still be transported over the existing 10 Gbps links, except for the core stations. For these stations the bandwidth will be doubled in case that is necessary.
The problem arises at some junctions in the network topology where stations are combined. This is the case for a number of trajectories of some stations. Furthermore, we need to equip each Dutch station with a Timing Distribution signal (White Rabbit) as well. In case dark fiber is used for the connections this can all be solved ourselves. However, some of the remote station trajectories are delivered by service providers. Upgrading those trajectories is a challenge by itself, because the service we need does not fit well by the service they normally deliver.
So far, we have had a number of discussions with those service providers, however some trajectories follow a very complex route which makes changes costly. Although we are and have been looking for many creative solutions, till now, we don’t have the “golden solution” for the right price (say the inverse of “gold”) for all the remote station trajectories.
Less issues are expected for the junctions to the stations outside The Netherlands, because they run via national research infrastructures (similar as SurfNet for the Netherlands). Research infrastructures seem to be the inverse of commercial service providers and are eager to deliver challenging non-standard services and loads of bandwidth.
To be continued …
Since our last report, we have realized some important developments.
First of all, in early December we have done the first temperature resilience tests of the new subrack cooling design. To test this, with limited hardware available, we have used dummy RCU2L boards, that are boards that dissipate an equal amount of power as the real RCU2H boards, and roughly at the same location in the board. By equipping a subrack with many dummy boards and a few real boards we could run a test in which the power dissipation within the subrack is comparable to a real LOFAR2 subrack. The tests showed that the temperatures of all equipment inside the subrack remained below the upper threshold even in the extremely high ambient temperature of +55 degrees Celsius. The only real issue, still under investigation, is that one of the Uniboard FPGAs became significantly warmer than the others, by 10 degrees Celsius. Although its temperature remained below the upper limit set by the manufacturer, the cause of this still has to be identified. Apart from this, all seemed well and under control.
Early January, we finally received the RCU2H boards from the manufacturer. Unfortunately, there were quite some assembly errors on these boards and we had to return all but one to the manufacturer for repairs. We hope to receive them back again, soon. The single RCU2H board that remained at ASTRON has been fixed by ourselves, so that at least we can start testing the real-life behavior of the RCU2H.
Meanwhile, our colleagues at INAF haven been working hard to get the documentation package for the RF-design of the RCU2H ready for review. For the RCU2L this documentation package was already delivered and approved in September last year. We have received the documentation and will be reviewing it half February. Once passed, INAF has delivered all promised work to the project, a milestone that calls for a celebration of some sort. Stay put for that…!
Before the Christmas holidays, we set up a long-duration test of the current DTS subrack. We switched on all boards, and started monitoring and archiving of the monitor data with the intention to leave it unsupervised and see what happens. It was truly encouraging that, during the 10 days we left system running, it actually kept running in a stable matter without unforeseen failures or overheating issues. All monitor data has been stored in a database and was accessible through the Grafana user interface. This result has given the engineering team quite a boost in confidence.
After testing the temperature behavior of the subrack, we are currently testing the EMC behavior of the subrack with RCU2Ls. All front panels, needed for shielding the insides of the subrack from the outside RFI-rich environment, have been produced in the mechanical workshop. Early tests in January have revealed some interesting but solvable issues related to the grounding of the power supplies on the APSPU boards. Further testing will have to show whether the current design meets the overall requirements.
While testing is continuing, a lot of work is being put in the preparation of the documents for the tendering of the manufacturing of the hardware. We plan to have the tender for all newly developed boards published by the end of February. Getting there requires many documents and design drawings to be updated, consistent and reviewed, and to be centrally available and versioned. This is quite an effort, but we are well organized to deliver all of this, in time.
Last November our team installed White Rabbit (WR) equipment in one remote station (RS208) and started a pilot program. The idea was to use exactly the same configuration planned for the final rollout and keep the station operational with the purpose of monitoring the behavior of the system over a long period of time and spot possible malfunctions. As part of the program, we did a test observation of 8 hours. The figure shows a series of plots for 4 different stations; three of them (RS106, RS305 and RS306) with the current LOFAR clock system and one with WR, our pilot RS208. The graphs show the clock offset with respect to the core stations during the 8-hour period. If you note the scale of the offset (y-axis) in the different plots, the improvement achieved with WR becomes immediately evident, with an offset smaller than 0.6 ns during the 8-hour test. For comparison, the current LOFAR system shows typical variations of 10-20 ns per 8 hours. This result gives us further confidence in that we will improve the clock accuracy for the LOFAR stations in the Netherlands thanks to WR.
At the same time, the production of the pulseshaper - a level and pulse length adjustment circuit to adapt the White Rabbit output to the station's internal clock distribution - is in full swing. The picture shows one of the panels with 20 printed circuit boards which will be assembled and soldered on our in-house production line during February.
The rollout of TD in the Dutch remote stations is planned for 2022. We will keep you updated about the process via this newsletter, so stay tuned!
Now the DTS development is nearing its completion and testing is in a full swing, we have to move on to the next face. This next step (PTS) is to refurbish a complete LOFAR station with LOFAR2.0 station hardware and will be followed by the final LOFAR2.0 Rollout. A combined PTS and Rollout European procurement process will be carried out for all the electronic boards (PCBA’s). Writing the procurement documentation is currently ongoing and design engineers are updating all the technical documentation to complete the procurement documentation. We are expecting to announce the LOFAR2.0 procurement very soon. It will take almost six months until the selection can take place and starting the order process.
At this moment, an intermediate step, LOFAR2.0 Test Station (L2TS) is considered in order to reduce risk and save time later in the project. Due to the extreme long lead time in acquiring components, in my current planning I included a 14 months component lead time, we may order additional hardware based on the current DTS design. Because of the small number of boards for one station, we can start the orders quickly without an EU procurement process. The L2TS orders can run in parallel with the EU procurement process and we are expecting to start AIV activities of L2TS by the end of this year long before the start of the PTS hardware production. The picture below illustrates a part of the procurement planning showing how the L2TS and EU procurements progressing.
The LOFAR4SW project is getting closer to its official closure in February 2022. The main goal of the project, implemented under Horizon 2020 as a design study, was to develop an upgrade to the existing LOFAR infrastructure so that it would be possible to use the instrument's capabilities for space weather purposes.
In the frame of the project, the team prepared a conceptual and technical study that included the definition of scientific use cases, software development, and the design and building of prototype hardware components for dual-beam HBA antenna. A test station was built in Chilbolton, UK, which served as a testbed for the software developed by the project. The CDR review panel evaluated LOFAR4SW as mature and ambitious, also during the stakeholders meeting project collected positive feedback.
On 14 February the Project team is organising an online Final Demo Session. The aim of the meeting is to present the outcomes of the LOFAR4SW project to a wider audience. The meeting will focus on highlighting new opportunities that a fully implemented LOFAR4SW infrastructure can open up in the context of space weather, and discuss the possibilities of future implementation of the developed solutions.
Participation at the meeting is possible for invited users and to a limited number (based on first come first served) of interested community members who can apply for participation by filling out the registration form available on the event website (http://lofar4sw.eu/wp/?p=1185).
Including the international stations (IS) opens up new possibilities for detailed studies of the morphological and spectral properties of radio sources at low frequencies. The vast majority of objects, unresolved in the 6" (Dutch-only) images are resolved in the ~0.3"-0.5" images obtained using the IS. Moreover, the unique characteristics of LOFAR allow to image with astonishing details extended sources at sub-arcsec resolution. The figure shows one radio galaxy, out of the tens of thousands objects detected in the NEP region using LOFAR, whose bright lobes are separated by ~100 arcsec. These observations, once completed, will complement the extremely deep optical and near-infrared data obtained by ground telescopes and the Euclid satellite to investigate the formation and evolution of galaxies.
The image on the left is obtained from a 72 hours observation at 6" resolution, the middle and right image, both at 0".55 resolution, are obtained from 8 and 32 hours, respectively. The sub-arcsec images were obtained using the long baseline pipeline (Morabito et al. 2021) and the selfcal facet script (van Weeren et al. in prep).
We are busy preparing the start of a new project to develop a drop-in replacement of the HBA frontend. The purpose is to ensure the maintenance of the existing array and the rollout of new stations in the near future. The project activities will start in the coming months, more details will be shared soon.