LOFAR can return simple time/frequency beam-formed data instead of, or combined with, interferometric data. Array beams are calculated from the data streams from one or more stations in order to produce time-series' and dynamic spectra for high time and frequency resolution applications. Typical applications include pulsars and solar or planetary studies.
The minimum integration time is 5.12 μs. The maximum spectral channel
the width of one sub-band (195.3125 kHz with the 200 MHz clock).
Sub-bands may be split into a number of channels (this number must be a power of 2, so usual choices are 16 channels, 64 channels, 256 channels, up to a maximum
of 2048), to provide higher
spectral resolution. Increasing the spectral resolution
comes with a corresponding increase in the minimum integration time: This is calculated by taking the inverse of the frequency
resolution. For example, if 256 channels per sub-band are specified,
the minimum integration time will increase to 0.0013s.
In the current implementation, there are three Beam-Formed sub-modes which can be used individually or combined in the same observation:
1) The Coherent Stokes (CS) sub-mode produces a coherent sum of multiple stations (also known as a “tied-array” beam) by correcting for geometrical and instrumental delays. This produces a beam with restricted field-of-view, but with the full cumulative sensitivity of the combined stations.
This sub-mode is currently restricted to observations using only core stations; stations outside the core do not use the same clock and are not fully phased-up to the core stations.
The total number of simultaneous tied-array beams that can be formed is a function of the number of stations and number of subbands used: If using the 12 HBA sub-stations of the superterp and the full bandwidth of 162 subbands, up to 219 tied-array beams can be formed. This number reduces in proportion to the number of additional stations for the same bandwidth. If the bandwidth is reduced, the number of tied-array beams can be increased in proportion.2) The Incoherent Stokes (IS) sub-mode produces an incoherent combination of the various station beams by summing the powers after correction for only the geometrical delay. This produces beams with the same field-of-view as a station beam, but results in a decrease in sensitivity compared to a coherently-added tied-array beam.
One such incoherent array beam can be formed for each of the specified station beams - i.e., if all the stations being summed split their recorded bandwidth across say 8 pointing directions, then 8 incoherent array beams can be formed from these.
All LOFAR stations, including the international stations, can be summed in this sub-mode.
3) The Fly’s Eye (FE) sub-mode records the individual station beams (one or more per station) without summing. This is useful for diagnostic comparisons of the stations and other applications where station data need to remain separate. In combination with the Complex Voltage sub-mode, Fly’s Eye can be used to record the separate station voltages as input for oﬄine processing, but be aware that the data volumes for this are large and so the number of stations which can be included in such an observation is limited.
Polarizations (CS/FE), Polarizations (IS):
For the different modes explained above, polarizations can be selected:
Pulsar observations may be processed via the Known Pulsar Pipeline, as given in the following schematic and described in more detail by Stappers et al (2011). This uses standard pulsar analysis packages such as dspsr and presto.
A schematic overview of the overall Pulsar Pipeline, as it
runs online on the Correlator, followed by offline scientific processing on
the offline cluster. Offline pipeline processing can be run on data
directly out of the Correlator or on RFI-filtered data.
The Beam-Formed data written by the Correlator are stored on the LOFAR offline processing cluster in the HDF5 format (Hierarchical Data Format). Several conversion tools have been developed to convert these data into other formats, e.g. PSRFITS, suitable for direct input into standard pulsar data reduction packages, such as PSRCHIVE, PRESTO, and SIGPROC.
Among other things, these reduction packages allow for RFI masking, dedispersion, and searching of the data for single pulses and periodic signals. The standard pulsar pipeline can produce for example a dynamic spectrum ( 8-bits PSRFITS), dedispersed timeseries (32 bits PREPDATA .dat file), prepfold summaries and / or dspsr archives (.AR) files. Options can be given to select which part of the pipeline should run and with which parameters. Coherent dedispersion can be carried out online, also for multiple beams/dispersion measures. Online RFI excision to excise corrupted data from individual stations before they are processed in the correlator is under development but not offered for Cycle 6.
This pipeline is only offered for use with pulsar observations.
Please note that because of memory limitations, the standard Complex Voltage pipeline for LBA observations is not offered for observations below 30 MHz and above a DM of 100 pc cm-3, in case of a single beam. When using multiple beams, or a higher DM, other frequency/DM limitations apply. Please consult science support if you want to propose such an observation.
Since the Beam-Formed data serve a much larger community than pulsar astronomers, a dynamic spectrum tool is under development. This tool allows for the creation of dynamic spectra from the beam-formed data files and includes some functionality to re-bin data in time and/or frequency. It also includes the ability to only retain a useful part of the original data. Thus a user could use this tool to obtain a quick, low resolution, look at the data to identify regions of interest and then retain only these, discarding remaining, redundant, data. All dynamic spectra, whether processed or not, are stored in an HDF5 file format.
During Cycle 6, we may be able to offer *strictly limited* use of this tool upon request. If use is granted, the Observatory can run the tool to generate quicklook plots for the user to review and send their specifications for running the full tool to Observatory staff, who will then run the full tool *only once* per observation. No re-run will be permitted in the event of failure and multiple runs on the same dataset cannot be accommodated. All non-pulsar data will be made available to the user in the raw beam-formed data format via the Long-Term Archive. It is possible to request a version of the data converted to 8-bit to save on LTA and data transport requirements. Please also note that, currently, there are no RFI excision tools available for these data.
Processing times for typical pulsar observations are not yet robust and fully-characterised, and actual processing times can vary significantly. The following should be used to estimate processing times for the purpose of observing proposals. Not all cases are given here, so please apply a reasonable extrapolation if your particular setup is not noted. In all cases these times should apply in single- and multi-beam modes. Times are expressed in a ratio of processing/observing (P/O) times, so processing times should be calculated as this factor times the duration of the observaion, regardless of the number of cores and/or nodes required for processing.The numbers are based on using 18 cores per node, and per group of up to 80 nodes. If you less than 80 nodes, the (20 beams or complex voltage mode), scheduling assumes you use 80 nodes.
Stokes I, 32 channels per subband, time integration factor 8: Assume P/O = 0.5
Stokes I, 16 channels per subband, time integration factor 6, 32 MHz bandwidth: Assume P/O =0.35 / 80 beams (e.g. A single beam has a P/O of 0.35, but LOTAAS has a P/O of 1.4 for 216 beams)
Stokes IQUV, 16 channels per subband, no time integration: Assume P/O = 1.0
Complex Voltages: Assume P/O = 0.35 per beam
If you also request 8-bit conversion, please add an equivalent P/O for using an additional core (e.g. Complex Voltage becomes P/O 0.7 per beam, LOTAAS becomes 1.75)
Table 1 gives an indication of sensitivity for typical Beam Formed observations. Please use the numbers above for estimating processing time required.
Table 1: Sensitivity and Processing Performance Parameters for typical Beam-Formed Observations, assuming observations of 244 sub-bands.
1 This assumes the raw 32-bit floats written out by the Correlator. The Known Pulsar Pipeline converts these to 8-bit integers, and hence reduces the data volume by a factor of 4 in the case of Pulsar observations. If the scientific goal is to create dedispersed/folded pulse profiles, then the volume of the resulting data products is over an order of magnitude smaller.
2 Approximate minimum, period averaged, 100 MHz flux density for a detection with a signal-to-noise ratio > 10. For Coherent Stokes, this assumes the 6 Superterp stations. Using all Core stations, the raw sensitivity increases by a factor of 4 (i.e. the minimum flux becomes ~3mJy. For Incoherent Stokes, 48 incoherently added HBA sub-stations (24 tiles) are assumed. For Fly's Eye the sensitivity corresponds to that of a single HBA sub-station.
3 Processing time includes: a) conversion of 32-bit to 8-bit data; b) RFI flagging; c) dedispersion; d) folding; e) creation of diagnostics plots. Multiple beams/stations can be run in parallel. Standard observations of known sources often require only 1 beam.