====== Note: page is outdated ======
This page is an early documentation, I tried to document BBS clearer on the **[[bbs|BBS page]]**.
====== Introduction ======
This page describes the syntax of the BBS parset file. BBS consists of three components: ''control'', ''kernel'', and ''solver''. Both ''kernel'' and ''solver'' need only a very small subset of the information that can be put in the parset (see examples below). The ''control'' parset file is usually much larger because it describes the processing that BBS has to do. For testing purposes, it is often useful to run all three executables on a single node. In that case it is possible to create a single combined parset, because each executable will ignore all keys it does not understand.
Below you will find a typical example parset file for each of the three executables. The rest of this page discusses all the valid keys in more detail.
===== Example control parset file =====
Observation = L2007_03463.gds # Global measurement description (GDS) file
Strategy.ChunkSize = 100 # Chunk size (timeslots) [SEE DOCUMENTATION]
Strategy.Steps = [solve, subtract, correct]
BBDB.Key = run00 # (Unique) name identifying the calibration session
BBDB.Host = cepmaster0 # Hostname or IP-address of postgresql server
BBDB.Port = 5432 # Port number where the postgresql server is listening
BBDB.Name = john # Name of the database to use
BBDB.User = postgres # Username for accessing the postgresql server
BBDB.Password = # Password for accessing the postgresql server
Step.solve.Operation = SOLVE # Operation to perform
Step.solve.Model.Sources = [] # Sources to include in the model (all if empty)
Step.solve.Model.DirectionalGain.Enable = T # Include separate complex gain terms for each source.
Step.solve.Model.Cache.Enable = T # Enable caching of intermediate results.
Step.solve.Solve.Parms = ["DirectionalGain:0:0:*", "DirectionalGain:1:1:*"] # Parameters to fit
Step.solve.Solve.CellSize.Freq = 0 # Solution cell size (channels)
Step.solve.Solve.CellSize.Time = 1 # Solution cell size (timeslots)
Step.solve.Solve.CellChunkSize = 10 # Cell chunk size (timeslots)
Step.solve.Solve.Options.MaxIter = 10 # Maximal number of iterations
Step.solve.Solve.Options.EpsValue = 1e-9 # Convergence criterion
Step.solve.Solve.Options.EpsDerivative = 1e-9 # Convergence criterion
Step.solve.Solve.Options.ColFactor = 1e-9 # Colinearity factor
Step.solve.Solve.Options.LMFactor = 1.0 # Levenberg-Marquardt factor
Step.solve.Solve.Options.BalancedEqs = F # Assume balanced equations?
Step.solve.Solve.Options.UseSVD = T # Use singular value decomposition?
Step.subtract.Operation = SUBTRACT # Operation to perform
Step.subtract.Model.Sources = [] # Sources to include in the model (phase centre if empty)
Step.subtract.Model.DirectionalGain.Enable = T # Include separate complex gain terms for each source.
Step.correct.Operation = CORRECT # Operation to perform
Step.correct.Model.Sources = ["CasA"] # Sources to include in the model (all if empty)
Step.correct.Model.DirectionalGain.Enable = T # Include separate complex gain terms for each source.
Step.correct.Output.Column = CORRECTED_DATA # Output column (no output if empty)
===== Example kernel parset file =====
ObservationPart.Filesystem = lioff021:/dev/sda10 # File system on which the part of the observation to process is located.
ObservationPart.Path = /data/L2007_03463_SB0.MS # Absolute path to the part of the observation to process.
BBDB.Key = run00 # (Unique) name identifying the calibration session
BBDB.Host = cepmaster0 # Hostname or IP-address of postgresql server
BBDB.Port = 5432 # Port number where the postgresql server is listening
BBDB.Name = john # Name of the database to use
BBDB.User = postgres # Username for accessing the postgresql server
BBDB.Password = # Password for accessing the postgresql server
ParmDB.Instrument = L2007_03463_SB0.instrument # Instrument model parameters
ParmDB.Sky = L2007_03463_SB0.sky # Sky model parameters
===== Example solver parset file =====
PortRange = [6500, 6599] # Port range to try on start-up
ConnectionBacklog = 25 # Maximal number of pending connections
BBDB.Key = run00 # (Unique) name identifying the calibration session
BBDB.Host = cepmaster0 # Hostname or IP-address of postgresql server
BBDB.Port = 5432 # Port number where the postgresql server is listening
BBDB.Name = john # Name of the database to use
BBDB.User = postgres # Username for accessing the postgresql server
BBDB.Password = # Password for accessing the postgresql server
====== Global settings ======
**BBDB** [Relevant to ''control'', ''kernel'', ''solver'']
>Information about the BlackBoard database. **NB. Do _not_ include when using the //calibrate// script**
>
>**Key** : //string// = default
>>Name that identifies the session.
>**Host** : //string// = localhost
>>Hostname or IP-address of the database server.
>**Port** : //integer// =
>>Port number on which the database server is listening. For Postgres databases the default is 5432.
>**Name** : //string//
>>Name of the database.
>**User** : //string//
>>Username to access the database server.
>**Password** : //string// =
>>Password to access the database server.
**Observation** : //string// [Relevant to ''control'']
>Global measurement descrition (GDS) file that describes the parts that compose the observation to be processed. **NB. Do _not_ include when using the //calibrate// script**
**ObservationPart** [Relevant to ''kernel'']
>Describes the part (MS) of the observation to be processed. **NB. Do _not_ include when using the //calibrate// script**
>
> **Filesystem** : //string//
>>File system on which the part of the observation to process is located. **NB. Must match the file system specified in the GDS-file exactly.**
>**Path** : //string//
>>Absolute path to the part of the observation to process. **NB. Must match the path specified in the GDS-file exactly.**
**ParmDB** [Relevant to ''kernel'']
>Information about the parameter databases (e.g. instrument model parameters, sky model parameters). **NB. Do _not_ include when using the //calibrate// script**
>
>**Instrument** : //string//
>>Path to the instrument model parameter database.
>**Sky** : //string//
>>Path to the sky model parameter database.
**PortRange** : //list of integers// = [6500, 6599] [Relevant to ''solver'']
>On start-up the specified range of ports will be searched for a free port on which to start listening. **NB. Do _not_ include when using the //calibrate// script**
**ConnectionBacklog** : //integer// = 10 [Relevant to ''solver'']
>Maximal number of pending connections. This value may have to be increased when starting a large number of ''kernel'' processes.
====== Strategy ======
The strategy defines the operations that need to be performed on the data. It consists of one or more (multi-)steps.
**InputColumn** : //string// = DATA
>Name of the column in the observation part that contains the input data.
**Baselines** : //string// = *&
>Baselines to read. The {{msselection.pdf | CASA baseline selection syntax}} should be used here. If this key is not specified, all cross-correlations will be selected.
Strategy.Baselines = *& # Select all cross-correlations (default).
Strategy.Baselines = CS*&&RS*;CS*& # Select all cross- and auto-correlations between core and remote stations,
# and all cross-correlations between core stations.
**Correlations** : //list of strings// = []
>Correlations to read.
>
>**NB. Specifying anything here will generate an exception because reading a subset of the available correlations is not supported yet.**
**TimeRange** : //list of strings// = []
>Time range to process, expressed as date/time string(s) (as returned by //msinfo//). All timeslots will be used if this field is left empty. Either a range or a start time can be provided.
Strategy.TimeRange = [27-Jul-2007/16:05:04]
Strategy.TimeRange = [27-Jul-2007/16:05:04, 28-Jul-2007/13:05:04]
**ChunkSize** : //integer//
>Chunk size in timeslots. A chunk represents an amount of input data that is loaded into memory and processed as a whole. This is useful when the amount of visibility data is too large to fit into main memory. A value of zero means all.
**UseSolver** : //bool// = F
>Will a global solver be used in this strategy?
**Steps** : //list of strings//
>The names of the steps that compose the strategy. It is an error to leave this field empty.
====== Step ======
A //single-step// describes one unit of work in the strategy. A step that is defined in terms of a number of other steps is known as a //multi-step//. The attributes of a multi-step should be interpreted as //default values// for the steps that compose the multi-step. These default values can always be overridden.
**Steps** : //list of strings//
>The names of the steps that compose this step (for multi-steps), or absent (for single steps). If specified, this key shall //not// be empty, or an exception will be thrown.
**Baselines** : //string// = *&
>Baselines to process. The [[http://casa.nrao.edu/Memos/msselection/index.html | CASA baseline selection syntax]] should be used here. If this key is not specified, all cross-correlations will be selected.
Step.subtract.Baselines = *& # Select all cross-correlations (default).
Step.subtract.Baselines = CS*&&RS*;CS*& # Select all cross- and auto-correlations between core and remote stations,
# and all cross-correlations between core stations.
**Correlations** : //list of strings// = []
>Correlations to process. If this key is not specified, all correlations will be selected.
Step.simulate.Correlations = [XX,YY]
Step.simulate.Correlations = [RR,RL]
**Model** : //[[engineering:software:tools#Model|Model]]//
>Model configuration to use.
**Operation** : //string//
>The operation to be performed in this step. One of 'PREDICT', 'ADD', 'SUBTRACT', 'CORRECT', 'SOLVE'. Only relevant for single steps, should be absent for multi-steps.
>PREDICT - Simulate visibilities.
>ADD - Add simulated visibilities to the input visibilities.
>SUBTRACT - Subtract predicted visibilities from the input visibilities.
>CORRECT - Correct the input visibilities.
>SOLVE - Fit model parameters.
//Single steps should define one of the following fields, depending on the value of **Operation**://
>**Correct** : //[[engineering:software:tools#Correct|Correct]]//
>>Arguments of the CORRECT operation.
>**Solve** : //[[engineering:software:tools#Solve|Solve]]//
>>Arguments of the SOLVE operation.
**Output**
>**Column** : //string// =
>>Column in the observation part wherein the output values of this step should be written. If left empty, no data will be written.
>
>**WriteFlags** : //bool// = F
>>If set to true the flags in the observation will be updated.
>
>**WriteCovariance** : //bool// = F
>>If set to true, covariance information will be written to the covariance column linked to the column specified at **Column**.
===== Example =====
Step.MultiStepExample.Steps = [solve, subtract] # Steps that compose the multi-step.
Step.MultiStepExample.Baselines = CS016*&CS008*,CS001*;CS001*&CS010*
Step.solve.Operation = SOLVE # Operation to perform
Step.solve.Model.Sources = [] # Sources to include in the model (all if empty)
Step.solve.Model.DirectionalGain.Enable = T # Include separate complex gain terms for each source.
Step.solve.Model.Cache.Enable = T # Enable caching of intermediate results.
Step.solve.Solve.Parms = ["DirectionalGain:0:0:*", "DirectionalGain:1:1:*"] # Parameters to fit
Step.solve.Solve.CellSize.Freq = 0 # Solution cell size (channels)
Step.solve.Solve.CellSize.Time = 1 # Solution cell size (timeslots)
Step.solve.Solve.CellChunkSize = 10 # Cell chunk size (timeslots)
Step.solve.Solve.Options.MaxIter = 10 # Maximal number of iterations
Step.solve.Solve.Options.EpsValue = 1e-9 # Convergence criterion
Step.solve.Solve.Options.EpsDerivative = 1e-9 # Convergence criterion
Step.solve.Solve.Options.ColFactor = 1e-9 # Colinearity factor
Step.solve.Solve.Options.LMFactor = 1.0 # Levenberg-Marquardt factor
Step.solve.Solve.Options.BalancedEqs = F # Assume balanced equations?
Step.solve.Solve.Options.UseSVD = T # Use singular value decomposition?
Step.subtract.Operation = SUBTRACT # Operation to perform
Step.subtract.Model.Sources = [] # Sources to include in the model (all if empty)
Step.subtract.Model.DirectionalGain.Enable = T # Include separate complex gain terms for each source.
Step.correct.Operation = CORRECT # Operation to perform
Step.correct.Model.Sources = ["CasA"] # Sources to include in the model (all if empty)
Step.correct.Model.DirectionalGain.Enable = T # Include separate complex gain terms for each source.
Step.correct.Output.Column = CORRECTED_DATA # Output column (no output if empty)
====== Model ======
Model configuration.
**Sources** : //list of strings// = []
>List of sources to include in the model. Shell style wildcards are allowed. An empty list will select all sources in the sky model parmdb.
**Cache.Enable** : //bool// = F
>If set to true, intermediate results will be cached. This is especially useful when solving, because it will prevent unnecessary recomputation at the start of each iteration.
>
> **NB. The current implementation does //not// limit the maximal amount of memory occupied by the cache. Therefore, caching is still considered //experimental//. It will work reliably in most cases, but it can cause the application to crash if the cache grows to the point where no more (virtual) memory can be allocated.**
**Phasors.Enable** : //bool// = F
>If set to true, complex parameters are expressed as (amplitude, phase) components instead of (real, imaginary) components. As a consequence the model is extended with a conversion for each complex parameter from (amplitude, phase) to (real, imaginary).
**Bandpass.Enable** : //bool// = F
\\
**NB: Bandpass correction is not supported anymore in the current implementation. A revised weighting is being worked on.**
>Multiply the sum of the source coherences by a //real-valued diagonal matrix//. The naming convention for the associated parameters is ''"Bandpass:0:0:"'' and ''"Bandpass:1:1:"''.
**Gain.Enable** : //bool// = F
>Multiply the sum of the source coherences by a //complex-valued Jones matrix//. The naming convention for the associated parameters is ''"Gain:{0,1}:{0,1}:{Real,Imag}|{Ampl,Phase}:"''. Depending on the value of **Phasors.Enable** either ''{Real,Imag}'' or ''{Ampl,Phase}'' is selected.
**DirectionalGain.Enable** : //bool// = F
>Multiply the source coherence of each source with a source (direction) specific //complex-values Jones matrix//. The naming convention for the associated parameters is ''"DirectionalGain:{0,1}:{0,1}:{Real,Imag}|{Ampl,Phase}::