|Description:|| On Monday August 4th, new 10 Gb/s Ethernet transceivers were installed in the|
network switches in Exloo and Groningen, connecting the LOFAR stations
CS001, CS010, and CS016 to the correlator. The correlator itself was
recently upgraded from a Blue Gene/L to a Blue Gene/P. With only a few
months to prepare the upgrade, we worked hard to port the software from
the BG/L to the BG/P, and a prototype of the new correlator was able to
observe immediately after the wide-area network became functional. Even
to our own surprise, the software did what it was supposed to do.
This picture shows fringes in one of the 36 subbands (~60.5 MHz) of a
12-hour observation, taken in the night from last Tuesday to Wednesday.
The picture was made with Michiel Brentjens' uvplot tool. The red and
green components represent the real and imaginary parts of a
correlation. The intensity represents the amplitude; pixels are colored
white if the amplitude exceeds some threshold, typically due to RFI.
Blue pixels are flagged, as is the case with channel 0.
The picture shows the beating between a few strong celestial signals
(mainly Cas A and Cyg A) on a short baseline between two dipoles in
CS001. Frequency channels are on the horizontal axis, and time is on
the vertical axis. Each pixel represents a 762 Hz, 30 second
correlation. Delay compensation was turned off, to make more fringes
visible. Most of the bandpass is taken out (a ripple of a few percent,
invisible to the human eye, remains).
Although the BG/P does not differ that much from a BG/L, we had to make
significant changes to the software. The biggest difference comes from
the fact that the BG/P moved from 1 to 10 Gb/s Ethernet technology,
while the number of I/O nodes decreased from 768 to 140. As a
consequence, each I/O node has to process much more data than on the
BG/L (but they are not that much faster!) The I/O nodes now receive
data from 4 RSP boards instead of only one, and buffer the data for
about 5 seconds. Since the standard network software did not provide
sufficient bandwidth, we developed a new network protocol that forwards
this data at wire speed over the internal links to compute nodes. These
compute nodes transpose, filter, and correlate the data. The work is
not finished yet: the quality of the software is not yet exactly what we
call "production level", and the processing speed on the I/O nodes must
be improved to support the data rates that are required for LOFAR20.
But looking at the picture that shows the first fringes feels like victory!