public:user_software:dynspec_tools

In this section, functionality, usage and parameters are presented. Two main categories of tools are available:

  • Tools whose the name start with «Beam…» do the Beam formed data (ICD3) conversion to dynspec data (ICD6), with or without rebinning.
  • Tools whose the name start with «Dynspec…» work on dynspec data (ICD6). They offer rebinning, recomputation of the polarization, subtraction and visualization.

DTC contains 3 sub-packages: CEP2, Stand-alone and Tools sub-packages. this table contains an overview on the tools:

Sub-Package Tool Function
CEP2 Beam2Dynspec-Complete Beam formed data complete conversion to Dynspec format
CEP2 Beam2Dynspec-Rebin Beam formed data conversion to Dynspec formet with a Time/Frequency rebin and/or selection
CEP2 Beam2Dynspec-Quick Dynspec Quicklook generation from Beam formed data
Stand-alone Beam2Dynspec-Standalone-Complete Stand-alone Station Beam formed data conversion to Dynspec formet with a Time/Frequency rebin and/or selection
Stand-alone Beam2Dynspec-Standalone-Rebin Stand-alone Station Beam formed data conversion to Dynspec formet with a Time/Frequency rebin and/or selection
Stand-alone Beam2Dynspec-Standalone-Quick Dynspec Quicklook generation from Beam formed data Stand-alone Station
Tools Dynspec-LinPol Convert a Dynspec with I,Q,U,V polarization to a Dynspec with I, Linear Polarization, Position Angle, Total Polarisation
Tools Dynspec-Rebin Allow to rebin/select in time and frequency a dynspec (avoid to reprocess all original data)
Tools Dynspec-Substract Generate a dynspec which is the subtraction of 2 dynspec: dynspec1-(k.dynspec2) where k is a number
Tools Dynspec-Visu Visualization tool for all dyspec

To use Dynspec on CEP2, execute

use Dynspec
use Pythonlibs
use LUS

For CEP4 there is a docker image. The run command may need to be adapted for write access on /data/.. :

docker-run.sh dynspec:latest /bin/bash
Beam2Dynspec-Complete

This tool does a complete conversion from Beam formed data (ICD3) to Dynspec data (ICD6). It should be used only on small datasets (at least if the resulting file is to be visualized with Dynspec-Visu). This tool is compatible with LOFAR observations generated by CEP, i.e. observations using the core or remote stations, but not for single international stations in standalone mode (See Stand-alone sub-package for that).

If the user types «Beam2Dynspec-Complete», the usage is explained:

MISSING Parameters
Usage: Beam2Dynspec-Complete --id=Lxxxxxx  --obsDIR= --outputDIR= --RAM=(x in Gb) --Npart= --RebinAll=(yes or no) [if Rebinall=yes default => 0: else --NSAP=]

The user has to provide the following parameters:

  • id=: This is the LOFAR Beam format observation ID. It usually starts with Lxxxxxx
  • obsDIR=: This is the observation directory which contains all the h5 and raw files of the observation. The full path must be provided. For CEP2 usage (for Observers of the RO), you can provide from the hoover node (from locus101 or 102, all nodes are accessible) just a path like: «/PATH/*/Lxxxxxx/», to allow the code find all files (h5 and raw) distributed on all CEP2 nodes.
  • outputDIR= This is the output directory where Dynspec(s) will be written. To avoid too large files, for an observation, you can have several SAP (Sub-Array Pointing) i.e several pointing, i.e several targets, several Beams. It has been decided to create one Dynspec per SAP (see ICD3:Beam formed format; and ICD6:dynspec format documents) i.e one dynspec per target.
  • RAM=: The amount of RAM the code is allowed to use. A value typically between 0.5-1 GB is recommended on CEP. This value has been estimated empirically, so it is not perfect. Be careful, large memory usage do not mean a speed improvement. Actually, the RAM quantity is linked to the size of the block per block array which will process to obtain the full array, and often it is better to have a lot of small block than a large one!
  • Npart=: Large observation are split is several parts (see ICD3 nomenclature). You have to provide the number of the part of the observation.
  • RebinAll=: equal to yes or no. If 'yes' is provided, it means that all SAP (Sub-Array Pointing) will be process. If 'no' is provided, you have to provide the ID of the SAP that you want to process (see NSAP=)

Optional:

  • Only if RebinAll=no –NSAP must be provided ⇒ –NSAP=: ID of the SAP that you wish to process.
Beam2Dynspec-Rebin

This tool does a conversion from Beam formed data (ICD3) to Dynspec data (ICD6). With this tool, it is possible to select the interesting time and frequency windows and also to rebin time and frequency channels to reduce the size of the dataset. The final dataset will usually be visualized easily. This tool is compatible with LOFAR observations generated by CEP, i.e. observations using the core or remote stations, but not for single international stations in standalone mode (See Stand-alone sub-package for that).

if the user types «Beam2Dynspec-Rebin», the usage is explained:

MISSING Parameters
Usage: Beam2Dynspec-Rebin --id=Lxxx  --obsDIR= --outputDIR=  --tmin= --tmax=  --tscale= --fmin= --fmax= --chanPerSubband= --RAM=(x in Gb) --Npart=  --RebinAll=(yes or no)  
[if Rebinall=yes default => 0: else --NSAP=]

The user has to provide the following parameters:

  • id=: This is the LOFAR Beam format observation ID. It usually starts with Lxxxxxx
  • obsDIR=: This is the observation directory which contains all the h5 and raw files of the observation. Full path must be provided. For CEP2 usage (for Observers of the RO), you can provide from the hoover node (from locus101 or 102, all nodes are accessible) just a path like: «/PATH/*/Lxxxxxx/», to allow the code find all files (h5 and raw) distributed on all CEP2 nodes.
  • outputDIR= This is the output directory where Dynspec(s) will be written. To avoid too large files, for an observation, you can have several SAP (Sub-Array Pointing) i.e several pointing, i.e several targets, several Beams. It has been decided to create one Dynspec per SAP (see ICD3:Beam formed format; and ICD6:dynspec format documents) i.e one dynspec per target.

Rebining and Selection parameters:

  • tmin=: This is a Time selection parameter. This is the minimum time selection (in second), i.e the starting selected time of the observation. It must be equal or greater than 0 and must smaller than tmax.
  • tmax=: This is a Time selection parameter. This is the maximum time selection in second), i.e the maximum selected time of the observation. It must be greater than tmin and must smaller than total observation duration.
  • tscale=: This is a Time rebinning parameter. It the value of the time scale (second per pixel), so it is the time scale. It must be greater than the current time scale of the observation.
  • fmin=: This is a frequency selection parameter. This is the minimum frequency selection (in MHz), i.e the starting selected frequency of the observation. It must be equal or greater than 0 and must smaller than fmax.
  • fmax=: This is a frequency selection parameter. This is the maximum frequency selection (in MHz), i.e the ending selected frequency of the observation. It must be greater than fmin and must smaller than the maximum frequency of the observation.
  • chanPerSubband=: This is a frequency rebinning parameter. It the value of the frequency scale (channel per subband), so it is the frequency scale. It must be smaller than the current number of pixel of the observation. In addition, it must be equal to 2^n where n is an integer (⇒2, 4, 8, 16, 32, 64, 128, etc…).

Note: Some parameters are automatically checked and adapted to the right value: for example if fmin<Observation frequency, the code will provide a WARNING and fix fmin=minimum observation frequency, the same for the maximum frequency. Similarly, the parameters are corrected if tmin<0 ⇒ tmin=0, and a warning is issued.

  • RAM=: The amount of RAM the code is allowed to use. A value typically between 0.5-1 GB is recommended on CEP. This value has been estimated empirically, so it is not perfect. Be careful, large memory usage do not mean a speed improvement. Actually, the RAM quantity is linked to the size of the block per block array which will process to obtain the full array, and often it is better to have a lot of small block than a large one!
  • Npart=: Large observation are split is several parts (see ICD3 nomenclature). You have to provide the number of part of the observation.
  • RebinAll=: equal to yes or no. If yes is provided, it means that all SAP (Sub-Array Pointing) will be process. If no is provided, you have to provide the ID of the SAP that you want to process (see NSAP=)

Optionnal:

  • Only if RebinAll=no: NSAP must be provided ⇒ NSAP=: ID of the SAP that you want to process.
Beam2Dynspec-Quick

This tool generates a quick look of the data. It is recommended to use just after the observation and visualize the data to determine which part of the data is useful, and which part can be removed. The user provides to this tool a percent of the time and of the frequency that he would like to visualize every seconds (the time scale is fixed to 1s) for each subbands (the frequency scale is 1 channel/subband). Every second and subband, the code will take x% of the data for 1 second and y% of the data for 1 subband and average them to obtain the value of the pixel. Usually 1-5 percent of the time and 10-20 percent of the frequency channel are often enough to obtain a good quicklook. This tool is able to generate a quicklook of several hundreds of GByte observation in less than 5 minutes. Quicklooks are produced in jpeg and h5 format (which can be visualize with the DTC visualisation tool). This tool is compatible with LOFAR observations generated by CEP, i.e. observations using the core or remote stations, but not for single international stations in standalone mode (See Stand-alone sub-package for that).

if the user types «Beam2Dynspec-Quick», the usage is explained:

MISSING Parameters
Usage: Beam2Dynspec-Quick --id=Lxxx --obsDIR= --outputDIR= --percentTimeData=[0->1] --percentSpectralData=[0->1] --transpose=[yes or no] --nofPart=

The user has to provide the following parameters:

  • id=: This is the LOFAR Beam format observation ID. It usually starts with Lxxxxxx
  • obsDIR=: This is the observation directory which contains all the h5 and raw files of the observation. Full path must be provided. For CEP2 usage (for Observers of the RO), you can provide from the hoover node (from locus101 or 102, all nodes are accessible) just a path like: «/PATH/*/Lxxxxxx/», to allow the code find all files (h5 and raw) distributed on all CEP2 nodes.
  • outputDIR= This is the output directory where Dynspec(s) will be written. To avoid too large files, for an observation, you can have several SAP (Sub-Array Pointing) i.e several pointing, i.e several targets, several Beams. It has been decided to create one Dynspec per SAP (see ICD3:Beam formed format; and ICD6:dynspec format documents) i.e one dynspec per target.
  • percentTimeData=[0→1]: This is a time selection parameter for the quicklook. This the data percent of time data that you want to use for quicklook generation.
  • percentSpectralData=[0→1]:This is a frequency selection parameter for the quicklook. This the data percent of frequency data that you want to use for quicklook generation.
  • transpose=[yes or no]: Usually, data are plotted with frequency in x and time in y. but it is possible by put yes, to invert the axis in the jpeg image.
  • nofPart=: It is the number of Part of the observation. The tool will generated a quicklook for each part automatically.
Beam2Dynspec-Standalone-Complete

This tool does a complete conversion from Beam formed data (ICD3) to Dynspec data (ICD6). It should be used only on small datasets (at least if the resulting file is to be visualized with Dynspec-Visu). The Stand-alone package is for Beam formed data obtained on single stations in stand-alone mode. It does exactly the same as «Beam2Dynspec-Complete» but for single station data produced with the «Pelican» system. (Note: We have to test with data from the new system «Lump» system).

if the user types «Beam2Dynspec-Standalone-Complete», the usage is explained:

MISSING Parameters
Usage: ./ICD3-Complete-Standalone.py --id=Dxxxxxxxxxxxxxxxx.0Zxx...xxx  --obsDIR= --outputDIR=  --RAM=(x in Gb)

The user has to provide the following parameters:

  • id=: This is the LOFAR Beam format observation ID. It usually starts with Dxxxxxxxxxxxxxxxx.0Zxx…xxx
  • obsDIR=: This is the observation directory which contains all the h5 and raw files of the observation. Full path must be provided. For CEP2 usage (for Observers of the RO), you can provide from the hoover node (from locus101 or 102, all nodes are accessible) just a path like: «/PATH/*/Lxxxxxx/», to allow the code find all files (h5 and raw) distributed on all CEP2 nodes.
  • outputDIR= This is the output directory where Dynspec(s) will be written. To avoid too large files, for an observation, you can have several SAP (Sub-Array Pointing) i.e several pointing, i.e several targets, several Beams. It has been decided to create one Dynspec per SAP (see ICD3:Beam formed format; and ICD6:dynspec format documents) i.e one dynspec per target.
  • RAM=: The amount of RAM the code is allowed to use. A value typically between 0.5-1 GB is recommended on CEP. This value has been estimated empirically, so it is not perfect. Be careful, large memory usage do not mean a speed improvement. Actually, the RAM quantity is linked to the size of the block per block array which will process to obtain the full array, and often it is better to have a lot of small block than a large one!
Beam2Dynspec-Standalone-Rebin

This tool does a conversion from Beam formed data (ICD3) to Dynspec data (ICD6). With this tool, it is possible to select the interesting time and frequency windows and also to rebin time and frequency channels to reduce the size of the dataset. The final dataset will usually be visualized easily. The Stand-alone package is for Single station Beam formed data observation only. The Stand-alone package is for Beam formed data obtained on single stations in stand-alone mode. It does exactly the same as «Beam2Dynspec-Rebin» but for single station data produced with the «Pelican» system. (Note: We have to test with data from the new system «Lump» system).

If the user types «Beam2Dynspec-Standalone-Rebin», the usage is explained:

MISSING Parameters
MISSING Parameters
Usage: ./ICD3-Rebin-Standalone.py --id=Dxxxxxxxxxxxxxxxx.0Zxx...xxx  --obsDIR= --outputDIR=  --tmin= --tmax=  --tscale= --fmin= --fmax= --chanPerSubband= --RAM=(x in Gb)]

The user has to provide the following parameters:

  • id=: This is the LOFAR Beam format observation ID. It usually starts with Lxxxxxx
  • obsDIR=: This is the observation directory which contains all the h5 and raw files of the observation. Full path must be provided. For CEP2 usage (for Observers of the RO), you can provide from the hoover node (from locus101 or 102, all nodes are accessible) just a path like: «/PATH/*/Lxxxxxx/», to allow the code find all files (h5 and raw) distributed on all CEP2 nodes.
  • outputDIR= This is the output directory where Dynspec(s) will be written. To avoid too large files, for an observation, you can have several SAP (Sub-Array Pointing) i.e several pointing, i.e several targets, several Beams. It has been decided to create one Dynspec per SAP (see ICD3:Beam formed format; and ICD6:dynspec format documents) i.e one dynspec per target.

Rebining and Selection parameters:

  • tmin=: This is a Time selection parameter. This is the minimum time selection (in second), i.e the starting selected time of the observation. It must be equal or greater than 0 and must smaller than tmax.
  • tmax=: This is a Time selection parameter. This is the maximum time selection in second), i.e the maximum selected time of the observation. It must be greater than tmin and must smaller than total observation duration.
  • tscale=: This is a Time rebinning parameter. It the value of the time scale (second per pixel), so it is the time scale. It must be greater than the current time scale of the observation.
  • fmin=: This is a frequency selection parameter. This is the minimum frequency selection (in MHz), i.e the starting selected frequency of the observation. It must be equal or greater than 0 and must smaller than fmax.
  • fmax=: This is a frequency selection parameter. This is the maximum frequency selection (in MHz), i.e the ending selected frequency of the observation. It must be greater than fmin and must smaller than the maximum frequency of the observation.
  • chanPerSubband=: This is a frequency rebinning parameter. It the value of the frequency scale (channel per subband), so it is the frequency scale. It must be smaller than the current number of pixel of the observation. In addition, it must be equal to 2^n where n is an integer (⇒2,4,8,16,32,64,128,etc…).

N.B: Some parameters are automatically checked and adapted to the right value: for example if fmin<Observation frequency, the code will provide a WARNING and fix fmin=minimum observation frequency, the same for the maximum frequency. Also if tmin<0 ⇒ tmin=0 with a warning.

  • RAM=: The amount of RAM the code is allowed to use. A value typically between 0.5-1 GB is recommended on CEP. This value has been estimated empirically, so it is not perfect. Be careful, large memory usage do not mean a speed improvement. Actually, the RAM quantity is linked to the size of the block per block array which will process to obtain the full array, and often it is better to have a lot of small block than a large one!
Beam2Dynspec-Standalone-Quick

This tool generates a quick look of the data. It is recommended to use just after the observation and visualize the data to determine which part of the data is useful, and which part can be removed. The user provide to this tool a percent of the time and of the frequency that he would like to visualize every seconds (the time scale is fixed to 1s) for each subbands (the frequency scale is 1 channel/subband). Actually, every second and subband, the code will take x% of the data for 1 second and y% of the data for 1 subband and average them to obtain the value of the pixel. Usually 1-5 percent of the time and 10-20 percent of the frequency channel are often enough to obtain a good quicklook. This tool is able to generate a quicklook of several hundreds of GByte observation in less than 5 minutes. Quicklook are in jpeg and h5 format (could be visualize with the DTC visualisation tool). It does exactly the same as «Beam2Dynspec-Quick» but for single station data produced with the «Pelican» system. (Note: We have to test with data from the new system «Lump» system).

If the user types «Beam2Dynspec-Quick», the usage is explained:

MISSING Parameters
Usage: Beam2Dynspec-Quick --id=Lxxx --obsDIR= --outputDIR= --percentTimeData=[0->1] --percentSpectralData=[0->1] --transpose=[yes or no]

The user has to provide the following parameters:

  • id=: This is the LOFAR Beam format observation ID. It usually starts with Lxxxxxx
  • obsDIR=: This is the observation directory which contains all the h5 and raw files of the observation. Full path must be provided. For CEP2 usage (for Observers of the RO), you can provide from the hoover node (from locus101 or 102, all nodes are accessible) just a path like: «/PATH/*/Lxxxxxx/», to allow the code find all files (h5 and raw) distributed on all CEP2 nodes.
  • outputDIR= This is the output directory where Dynspec(s) will be written. To avoid too large files, for an observation, you can have several SAP (Sub-Array Pointing) i.e several pointing, i.e several targets, several Beams. It has been decided to create one Dynspec per SAP (see ICD3:Beam formed format; and ICD6:dynspec format documents) i.e one dynspec per target.
  • percentTimeData=[0→1]: This is a time selection parameter for the quicklook. This the data percent of time data that you want to use for quicklook generation.
  • percentSpectralData=[0→1]:This is a frequency selection parameter for the quicklook. This the data percent of frequency data that you want to use for quicklook generation.
  • transpose=[yes or no]: Usually, data are plotted with frequency in x and time in y. but it is possible by put yes, to invert the axis in the jpeg image.
Dynspec-LinPol

After Dynspec generation with a tool from CEP2 or Stand-alone sub-package, users can convert the I, Q, U, V Dysnpec to I, Linear Polarization, Position Angle, Total Polarization.

If the user types «Dynspec-LinPol», the usage is explained:

MISSING Parameters
Usage: .Dynspec-LinPol --outDir= --ID=[what you want:Determine the output filename] --filename=[Name of the ICD6 file (*.h5)] --RAM=

The user has to provide the following parameters:

  • outDir=: This is the output directory where the new Dynspec with I,Lin,PA,Total polarization will be written.
  • ID=: Actually, determine the name of the output file. It is not the «real» ID of the observation, just a flag to distinguish the output file from the input one.
  • filename=:Full Path of the input dynspec file (*.h5).
  • RAM=: The amount of RAM the code is allowed to use. A value typically between 0.5-1 GB is recommended on CEP. This value has been estimated empirically, so it is not perfect. Be careful, large memory usage do not mean a speed improvement. Actually, the RAM quantity is linked to the size of the block per block array which will process to obtain the full array, and often it is better to have a lot of small block than a large one!
Dynspec-Rebin

This tool is for rebinng of a dynspec file (ICD6, producing another ICD6 file). It avoids having to go back to the original beam-formed data when further rebinning is required (i.e. the user wishes a lower time/frequency resolution or a smaller time/frequency windows). The Visualization tool is not able to image large dynspec files. After a first conversion (with or without rebin/selection), it is possible that the user needs to reduce the size of the dynspec for an easy visualization. This tool is very similar to the rebin tools contained in CEP2 and Stand-alone sub-packages, but it rebin dynspec (ICD6) not Beam formed (ICD3) data.

If the user types «Dynspec-LinPol», the usage is explained:

MISSING Parameters
Usage: Dynspec-Rebin --Pathfile=  --obsname=Lxxx[for metadata] --outputFile=[choose the complete output name]  --tmin= --tmax=  --tscale= --fmin= --fmax= --chanPerSubband= 
--RAM=(x in Gb)]

The user has to provide the following parameters:

  • Pathfile=: Path to the dynspec to rebin (*.h5)
  • obsname=:Lxxx observation ID. It is used for writting correct metadata
  • outputFile=: Name of the output dynspec. The user chosse the name of the output file.

Rebining and Selection parameters:

  • tmin=: This is a Time selection parameter. This is the minimum time selection (in second), i.e the starting selected time of the observation. It must be equal or greater than 0 and must smaller than tmax.
  • tmax=: This is a Time selection parameter. This is the maximum time selection in second), i.e the maximum selected time of the observation. It must be greater than tmin and must smaller than total observation duration.
  • tscale=: This is a Time rebinning parameter. It the value of the time scale (second per pixel), so it is the time scale. It must be greater than the current time scale of the observation.
  • fmin=: This is a frequency selection parameter. This is the minimum frequency selection (in MHz), i.e the starting selected frequency of the observation. It must be equal or greater than 0 and must smaller than fmax.
  • fmax=: This is a frequency selection parameter. This is the maximum frequency selection (in MHz), i.e the ending selected frequency of the observation. It must be greater than fmin and must smaller than the maximum frequency of the observation.
  • chanPerSubband=: This is a frequency rebinning parameter. It the value of the frequency scale (channel per subband), so it is the frequency scale. It must be smaller than the current number of pixel of the observation. In addition, it must be equal to 2^n where n is an integer (⇒2,4,8,16,32,64,128,etc…).

Note: Some parameters are automatically checked and adapted to the right value: for example if fmin<Observation frequency, the code will provide a WARNING and fix fmin=minimum observation frequency, the same for the maximum frequency. Also if tmin<0 ⇒ tmin=0 with a warning.

  • RAM=: The amount of RAM the code is allowed to use. A value typically between 0.5-1 GB is recommended on CEP. This value has been estimated empirically, so it is not perfect. Be careful, large memory usage do not mean a speed improvement. Actually, the RAM quantity is linked to the size of the block per block array which will process to obtain the full array, and often it is better to have a lot of small block than a large one!
Dynspec-Subtract

This tool is for Dynspec subtraction. Frequently, observations have 2 beam, one on the target, one off-axis to have a measure of the noise, RFI, etc…
This tool does a subtraction of 2 beams which can either be in the same or different dynspec files. The only condition is that the data must have the same format! Finally, it is possible to put a multiplicator for the subtraction like: New_dynspec = dynspec1-(k.dynspec2) where k is a number.
:!:For clarification, before dynspec conversion, beams are in different files. After, beams are in the same dynspec file, and their nomenclature (inside the dynspec file, see hdfview above, or the tutorial) is DYNSPEC_xxx.

If the user types «Dynspec-LinPol», the usage is explained:

MISSING Parameters
Usage: Dynspec-Substract --outDir= --id1=[what you want:Determine the output filename => id1-kxid2 or id1_dynspec1-kxdynspec2; depends if file1 & file2 are the same] 
--file1=[Name of the ICD6 file (*.h5)]  --dynspec1=[Name of the dynspec Group ex: DYNSPEC_000] --id2= --file2= --dynspec2= --k=[scaling factor: dynspec1-(k*dynspec2)] 
--RAM=

The user has to provide the following parameters:

  • outDir=: This is the output directory where the new subtracted Dynspec will be written.
  • id1=: what you want:Determine the output name ⇒ id1-kxid2 or id1_dynspec1-kxdynspec2. It depends if file1 and file2 are the same
  • file1=: path to the dynspec1 (*.h5)
  • dynspec1=: Name of the dynspec Group of the file1 (i.e the Beam inside the dynspec file) ex: DYNSPEC_000
  • id2=: what you want:Determine the output name ⇒ id1-kxid2 or id1_dynspec1-kxdynspec2. It depends if file1 and file2 are the same
  • file2= path to the dynspec2 (*.h5)
  • dynspec2= Name of the dynspec Group of the file2 (i.e the Beam inside the dynspec file) ex: DYNSPEC_001
  • k=: scaling factor: dynspec1-(k*dynspec2). Must be a number! Use 1 for a simple subtraction.
  • RAM=: The amount of RAM the code is allowed to use. A value typically between 0.5-1 GB is recommended on CEP. This value has been estimated empirically, so it is not perfect. Be careful, large memory usage do not mean a speed improvement. Actually, the RAM quantity is linked to the size of the block per block array which will process to obtain the full array, and often it is better to have a lot of small block than a large one!

This is the Visualization tool. It allows to visualize any Dynspec (ICD6) file, and inside the file any DYNSPEC_xxx (i.e Beam) and any Stokes parameters (also DYNSPEC_XXX which has Linear, PA and Total polarization). It is easy to use. There are still very minor visualization bugs (due to an error in matplotlib).
During the conversion or the data processing («Dynspec-Lin» or «Dynspec-Subtract») data are still real data ! But for visualization, we use the first tencile (10%) for visualization. At on time, we generate a cumulative distribution of the frequency channels, and compute the first tencile of this distribution. Then for all pixels at this time, the plotted value is I' = I Real/First tencile. For Q,U and V, it is plotted Q'=Q/I, U/I and V/I. So, for Q, U and V, a percentage of the polarisation is plotted (the same for linear, PA and Total Polarization).

An example of the use of Dynspec-Visu below:
Type in a terminal:

Dynspec-Visu


(click for larger version!)
A grey window appears (in case you are working on a remote server, be sure you have a graphical session opened i.e ssh -XY …).

Click on “File” in the upper left corner.
(click for larger version!)
A tab appears with a new command button: 'Load a Dynamic Spectrum'.

Click on 'Load a Dynamic Spectrum':
(click for larger version!)
and a selecting window appears.
:!: In the current version, the cancel button does not really work (it creates a bad objet in the graphical window). Avoid to use it.

After the file selection, The visualisation structure appears.
(click for larger version!)
At this step, you see on the top left (in the box “Dynamic Spectra”) the different dynamic spectra contained in your file (DYNSPEC_000, DYNSPEC_001, …).

Select a dynamic spectrum by double-clicking its name (e.g. “DYNSPEC_000”):
(click for larger version!)
In the “Metadata” window, the metadata of this dynamic spetrum appear (observation ID, Target, RA, DEC, LOFAR stations involved, frequency and time resolution, etc …). In the “Select Stoke” window just below, you see which Stokes parameters (I, Q, U, V, or I, lin, PA, total polarization) are available for the selected dynamic spctrum. And, in the “Resume selection” window, you can see that DYNSPEC_000 is selected, but no Stokes paramter has been chosen yet.

In the “Select Stokes” window, double-click on the Stokes component you are interested in (e.g. “Stokes: I” for the beginning):
(click for larger version!)
Now, the selected Stokes parameter appears in the “Resume Selection” window. With this, the display buttons (Non-continous and continous) become available in the “Display” section. When your observation covers a continuous frequency band, you will not see a difference. When your observation covers several frequency bands separated by gaps, the behaviour differs:

  • “No-Continuous” will show the data and the data gaps. This is useful, for example, Useful to visualize slopes in bursts.
  • “Continuous” will only show the data, and omit the data gaps. This can be useful when the data gaps are very large, and when the signal is broadband and/or slowly variable in time.


Click (:!: only once) on either “continuous” or “non-continuous”.
Be patient, data display can be long, especially of your file is > 1 GB. A progress bar indicates approximately what fraction the file is processed (don't be surprised if it does not move for some time, its granularity is low).
(click for larger version!)
On a laptop, a 300 MB file took 20 seconds to load for Stokes I, and 120 seconds for other Stokes parameters (which are normalized by Stokes I, requiring extra calculations). Congratulations, your dynamic spectrum is now loaded!

Click in the dynamic spectrum plot (in the window “Plotting The LOFAR Dynamic Spectrum Data”):
(click for larger version!)
The position of your cursor in time and frequency is indicated below the plot. Also, frequency and time profiles through this point appear on the windows “Plotting TimeS pectra” and “Plotting Frequency Spectra” to the right.

To change the contrast in the main window (“Plotting The LOFAR Dynamic Spectrum Data”), you can change the minimum and maximum values (boxes “Minimum” and “Maximum” just above the plot), and click on the respective “Update” buttons:
(click for larger version!)
:!: Don't believe the value with which the “Minimum” and “Maximum” boxes are initialized.
Now the scale of your dynamic spectrum is changed!

Currently, the color-scale is linear. In some cases, it is useful to use a non-linear color scale. For this, change the value of the “Gamma Correction” (the default value is 1. :!: Use a value close to 1!):
(click for larger version!)
The dynamic spectrum display changed! :!: Don't believe the value with which the “Gamma Correction” box is initialized.
Note: Updating the gamma correction can be slow.

To inverse the time and frequency axes, click on the “Transpose” button:
(click for larger version!)
Time and frequency axis are inverted. Some people prefer to display time-frequency instead of frequency-time.

Now select “DYNSPEC_005”, “Stokes: V” and display with “continuous”:
(click for larger version!)
Now, we visualize DYNSPEC_005 and Stokes V. Note that the scale now goes from -1 to 1. (For Stokes Q, U and V, the display shows Q/I, U/I and V/I.)

Below each of the three plot, a few buttons are displayed.

  • The rightmost one allows to save the image. Make sure to select your favorite image format (png, jpg, eps, pdf, …).
  • The second button from the right allows to rescale size of the plot within the window.
  • The third button from the right allows to zoom into the dynamic spectrum plot (click left, do not lose the mouse button, and drag the mouse to define the zone you want to zoom into).
    :!: In zoom mode, the image may jump when you move the mouse. This is a known bug of the current version of matplotlib.
  • When zoomed in, the middle button allows to switch to “pan” mode. #After zooming in, this can be used to shift the part of the image that is shown (left click on image, then move mouse without releasing the mouse button).
  • When in “zoom” mode or in “pan” mode, the second and third button from the right allow to browse back to the previous view (left arrow), and go back to the current one (right arrow).
  • The left button is used to unzoom and reset the scales.

It is recommended now to do the Tutorial.

Return to main page: dynspec

Continue with Tutorials:

  • Last modified: 2017-03-08 15:27
  • by 127.0.0.1