Visualizing LOFAR beams, etc.

This purrlog is meant to demonstrate some new features of Siamese/Calico, namely sky-Jones visualization. It's also a short tour of Sarod's LOFAR beam models, which I've cleaned up and incorporated into Siamese properly.

For starters, make sure you have a recent (9/10/2009 or later) build of MeqTrees and the latest Cattery. Under Cattery/Siamese/MS, you will find some makems configuration files. Use WSRT_hba_makems.cfg with makems to make yourself an MS corresponding to WSRT with LOFAR high band frequencies (or use Stephen Bourke's mkant tool to make yourself a proper LOFAR MS if you want.)

Now, go into Cattery/Siamese/SBY, and type "make" to build a small library with Sarod's beam codes.

Now load up Cattery/Siamese/example-sim.py, and take a look at the compile-time options. The attached screenshot shows off the new visualization options.

Under "Measurement Equation Options", you'll find a new entry called "Include visualizers for sky-Jones terms". Enable this, and select RA-Dec for the Coordinate grid. This will give you visualizations in RA-Dec over the entire sky. (Other options are az-el over the entire sky, az-el above the horizon, or a section of the lm plane. You can play with these later to understand what they mean.)

Under "E Jones", you can find the new-look "Siamese.SBY.lofar_beams" module. This contains the following options:

* You can simulate an array of dipoles, and array of (properly beamforming) stations, or one proper station correlated with a bunch of dipoles ("CS1 mode".) Start by selecting "dipoles".

* Under "Dipole configuration", select either LBA or HBA (depending on which MS you use.) Leave dipole parameters at their defaults, you can always play with them later (note that Sarod's HBA model actually has even more parameters than the LBA, but for now they're hardcoded in Siamese/SBY/hba_beams.c instead of being exposed in the GUI.)

* "Beam library" should point to the library Siamese/SBY/lofar_beams_lib.so, which you should've built in the first step above.

* Select "Use same E-Jones for all stations". This saves a lot of computation by assuming that the same beam pattern applies to all stations in the array (which is a fair approximation for a small array.)

For "Sky model", select a gridded_sky with a single source. We're not really interested in sky models at this stage anyway. Now compile the script and run "simulate MS".

(NB: we need to re-run the simulation every time we build the tree, to make sure sky-Jones visualization works properly, i.e. picks up the right frequencies and everything. This is a bit annoying, since this kind of visualization shouldn't actually require us to simulate an MS. I hope to eliminate this defect later.)

Under bookmarks, you'll now find "E visualizer" and "|E^2| visualizer". Open these, and look in the TDL Exec menu (see screenshot.) You now have a sub-menu called "Visualize sky-Jones terms". Open this up and take a look at the options:

* Number of x/y pixels determines resolution. 100 is on the low end of reasonable. The more the slower.

* "Time span" determines the length of time over which a sky-Jones term is visualized, and "number of time planes" determines the time resolution. This only makes sense with time-variable Jones terms, of course. 24.0 and 25 respectively will give us one slice per hour.

* Same goes for frequency, here we ask for three slices at 155, 160 and 165 MHz.

Note that the result of visualization will be a 4D hypercube of Npix x Npix x Ntimes x Nfreq values, so don't go overboard with the resolution here. 100x100x25x3 LOFAR beams can be computed in well under a minute though.

Now click on "Visualize E (beam)" to tell your tree to go off and compute the visualization. Twiddle your thumbs for a bit, until the displays come up (you did open the bookmarks, right?)

The first image (|E^2| visualizer) shows you the total beam power over the whole sky. RA is the horizontal axis, Dec is vertical (ignore the fake axis labels for now, they are but a simple kludge.) This is a cylindrical projection unfortunately, so the images are badly distorted towards the poles (i.e. towards the top/bottom). Beg Tony to implement a spherical coordinate display!

The "time" slider lets you go forward and back in time over 24 hours, the "freq" slider does likewise for the frequency planes we put in.

The "E visualizer" image shows you the complex beam gain (i.e. one E-Jones element). Use the right-click menu to look at the other Jones elements ("Change selected vells"), or to switch to an amplitude-phase display. (Note especially the cross-terms.)

Play with the sliders and note the following features:

* This is a dipole beam, and it's tied to the ground. As you slide through time (and rotate the Earth), you can see the beam pattern sweep over the sky.

* The beam is 0 below the horizon (WSRT/LOFAR is in the Northern hemisphere, of course.)

* E-Jones cross terms are very interesting.

If you want to see something not very interesting but highly comforting, go back to Compile Options, and select "Az-El" for the sky-Jones visualization frame. Rebuild tree, re-rerun simulation and visualization. Observe that in the az-el frame the beam pattern is constant in time, as indeed it ought to be for a fixed pair of dipoles.

Now go back to Compile Options, and under E-Jones / Array composition select "stations". You will see a new sub-menu called "Station configuration", go in there and make sure a station layout file is specified. Siamese/SBY/AntennaCoords.hba and .lba provide some example HBA and LBA LOFAR station layouts. We will now simulate a beamforming station composed of dipoles.

Make sure your Coordinate grid option (under "ME options / Visualize sky-Jones") is still set to RA-Dec. Build the tree and re-run the simulation.

Now open up the "Visualize sky-Jones terms" menu under TDL Exec, and enter 200 for the number of x/y pixels (station beams are tighter, so we really need the extra resolution.) Make sure the same bookmarks are open, and click on "Visualize E (beam)". With the higher resolution, this may take a couple of minutes.

Use the "time" and "freq" sliders to play with the resulting displays (see attached images.) Note the following:

* We're now beamforming and trying to track a fixed point on the sky (the MS phase center, in fact.)

* The source is above the horizon somewhere in the middle of our 24-hour interval, and that's the only time the station beam looks sensible. The rest of the time the beamformer is essentially trying to phase up on something below the horizon, with "interesting" results.

For another interesting display, go back to Compile Options, and change your Coordinate grid option (under "ME options / Visualize sky-Jones") to "lm plane". Rebuild the tree and re-run the simulation.

We're now going to visualize the station beam in a 20deg x 20deg area around the phase center. Under TDL Exec, go into "Visualize sky-Jones terms". Since we switched our coordinate grid to "lm plane", we now have two new options here for the size of the lm patch that we want to visualize. Enter 1200, 1200 here. (You should also reset the x/y resolution back to 100 pixels, to keep things moving quickly.) Click on "Visualize E" to recompute everything.

Now play with the time sliders (and don't forget to middle-click for cross-sections.) Note again how the main lobe only looks reasonable when the source is above the horizon.

You can also try visualizing at the station beam in the az-el frame, this will show it sweeping over the sky nicely.

And in case you haven't picked up on this yet: this visualization is now automatically available for ANY SKY-JONES TERM. Including other E-Jones models, dipole projection (see attached), ionosphere, etc.

(Hint: to keep things fast when visualizing, keep the "Use the same x-Jones for all stations" checked, and don't go overboard in x/y/time/freq resolution.)