Daily Image
06-10-2015
TAQL gem
| Submitter: | Ger van Diepen |
| Description: | The Casacore Table Query Language (TaQL) is a versatile and reliable tool to manipulate data in Casacore tables, in particular MeasurementSets (MS). The various LOFAR data processing pipelines happily make use of TaQL, and so do other users of MeasurementSets(*). However, despite good documentation and tutorial sessions, the TaQL learning curve still seems to be a bit steep. Writing elaborate and error-prone Python scripts still seems to have a lower threshold than writing concise and powerful TaQL commands. The LOFAR Long Baseline Data Processing is a good example of this regrettable state of affairs. The DPPP-StationAdder routine adds the signals of the core stations to form a single superstation, thus increasing the sensitivity of the long LOFAR baselines. The routine creates new baselines between the superstation and the non-core stations by calculating the weighted average of the DATA and UVW coordinates of the baselines between the core and non-core stations. Alas, in its first version the data averaging was not done correctly, so the DATA and UVW values of the new baselines had to be repaired. Leah Morabito then wrote a 100-line Python script using numpy and pyrap to fix the MS. It did a nice job and repaired the data. But when we used TaQL to look at the results, we realized that fixing the data could also be done with the single TaQL command shown here. (So this ADJI shows a pretty command instead of the usual pretty picture.) Admittedly, the seemingly complex command may not yet inspire users to try something similar in TaQL. However, when taking a closer look, one can see that the command can be broken up into small logical entities that are much easier to grasp, making it possible to comprehend the whole(**). Note that the functions used in the command are quite similar to the numpy functions that we all love and use (e.g., summing an entire array or partial summing over axes). In this case, the TaQL commands are even more powerful because they know about the table structure of the MS. More background information may be found in the TaQL reference manual (casacore.github.io/casacore-notes/199.html) and the tutorial slides (www.astron.nl/~gvd/tutorial4.pdf). And of course the author will, within limits, be happy to oblige(***). (*) One of the strong features of the Casacore Table System is the easy access it provides to the contents of the widely used MeasurementSets. (**) It is important to know that the ANTENNA2 column contains the non-core stations, so grouping per (TIME,ANTENNA2) gives all baselines for a given time and non-core station. (***) The author is contemplating a catalog of example TaQL commands like this one, that can be easily customized. |
| Copyright: | Tammo Jan Dijkema, Ger van Diepen |
| Tweet |  |