Several data processing steps are necessary before data from a radio telescope such as LOFAR can be turned into a scientific image of the sky. The two most important steps in the processing are calibration of the data and solving the imaging equation. Calibration consists of correcting for imperfections in the data, such as turbulence in the ionosphere and differences in the cable lengths and clocks of the antennas. After calibration, the imaging step combines all data into an image. This involves complex mathematical calculations that transform the recorded correlated voltages into a map of the sky.
Data sets from telescopes have enormous volumes, making calibration and imaging a big data problem. They drive technological advancements of high-performance computing, and solutions are relevant to several other fields, including medical imaging and artificial intelligence. Advancements in calibration and imaging are crucial for delivering deep images of the sky within reasonable time frames, and therefore fundamental for making exciting astronomical discoveries. They are one of the biggest challenges for the Square Kilometre Array.
Ionospheric effects are stronger at low-frequencies. Because of this, the LOFAR telescope and the future SKA require direction-dependent self-calibration to take out ionospheric fluctuations from the data. Without advanced calibration methods, astronomical sources are smeared out and show unrealistic artefacts, and the resolution of the instrument will be limited.
State-of-the-art calibration strategies make a physical model of the instrument and the ionosphere. These models not only allow high quality images, but also contribute to ionospheric research and space weather analyses. Estimating model parameters from large observations requires advanced non-linear solvers such as DPPP and Sagecal. These tools implement regularisation and constraints to solve for the fluctuations in the total electron content (TEC) of the ionosphere.
The calibration step is tightly coupled to the imaging step. Recent imaging developments are the so-called image-domain gridder using graphical processing units (GPUs) and the use of smart multi-scale multi-frequency deconvolution methods. By combination these two steps in an iterative direction-dependent self-calibration loop, ultra-deep and realistic images of the sky can be made.