Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
| public:documents:raw_olap_data_formats [2012-03-20 09:16] – Jan David Mol | public:documents:raw_olap_data_formats [2017-03-08 15:27] (current) – external edit 127.0.0.1 | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ===== Raw OLAP data formats ==== | + | ===== Raw OLAP data formats |
| OLAP produces several data formats, which are intended to be replaced by their final format, such as HDF5. | OLAP produces several data formats, which are intended to be replaced by their final format, such as HDF5. | ||
| - | ==== After 2011-10-24 ==== | + | ===== After 2011-10-24 |
| - | Files adhere to the following naming scheme: Liiiii_SAPsssss_Bbbb_Sz_bf.raw, | + | Files adhere to the following naming scheme: |
| - | - iiiii = SAS observation ID | + | - '' |
| - | - sssss = Station beam number (SAP) | + | - '' |
| - | - bbb = Tied-array beam number (TAB) | + | - '' |
| - | - z = Stokes number | + | - '' |
| The stokes numbers are to be interpreted as follows: | The stokes numbers are to be interpreted as follows: | ||
| Line 29: | Line 29: | ||
| < | < | ||
| struct block { | struct block { | ||
| - | float stokes[SAMPLES][SUBBANDS][CHANNELS]; | + | float sample[SUBBANDS][CHANNELS]; |
| }; | }; | ||
| </ | </ | ||
| + | The constants used can be derived from the parset: | ||
| + | |||
| + | < | ||
| + | SUBBANDS = len(parset[" | ||
| + | |||
| + | if (complex voltages || coherent stokes) { | ||
| + | |||
| + | CHANNELS = parset[" | ||
| + | if (CHANNELS == 0) CHANNELS = parset[" | ||
| + | |||
| + | } elif (incoherent stokes) { | ||
| + | |||
| + | CHANNELS = parset[" | ||
| + | if (CHANNELS == 0) CHANNELS = parset[" | ||
| + | |||
| + | } | ||
| + | </ | ||
| + | |||
| + | The sampling rate can be derived as follows: | ||
| + | |||
| + | < | ||
| + | # clock frequency (f.e. 200 MHz) | ||
| + | clock_hz = parset[" | ||
| + | |||
| + | # subband frequency (f.e. 195 kHz) | ||
| + | base_subband_hz = clock_hz / 1024 | ||
| + | |||
| + | # channel frequency (f.e. 763 Hz) | ||
| + | base_nrchannels = parset[" | ||
| + | base_channel_hz = base_subband_hz / base_nrchannels | ||
| + | |||
| + | if(complex voltages || coherent stokes) { | ||
| + | cs_temporalintegration = parset[" | ||
| + | |||
| + | sample_hz = base_channel_hz / cs_temporalintegration | ||
| + | |||
| + | } elif(incoherent stokes) { | ||
| + | |||
| + | is_temporalintegration = parset[" | ||
| + | |||
| + | sample_hz = base_channel_hz / is_temporalintegration | ||
| + | } | ||
| + | | ||
| + | </ | ||
| - | ==== Before 2011-20-24 ==== | + | ===== Before 2011-10-24 ===== |
| Data can be recorded as either complex voltages (yielding X and Y polarisations) or one or more stokes. In either case, a sequence of blocks will be stored, each of which consists of a header and data. The header is defined as: | Data can be recorded as either complex voltages (yielding X and Y polarisations) or one or more stokes. In either case, a sequence of blocks will be stored, each of which consists of a header and data. The header is defined as: | ||
| Line 169: | Line 213: | ||
| A BFRaw file starts with a file header containing the configuration: | A BFRaw file starts with a file header containing the configuration: | ||
| - | < | + | < |
| struct file_header | struct file_header | ||
| { | { | ||
| Line 199: | Line 243: | ||
| After the file header, there is a series of blocks until the end of file, configured using values from the file header: | After the file header, there is a series of blocks until the end of file, configured using values from the file header: | ||
| - | < | + | < |
| struct block | struct block | ||
| // 0x2913D852 | // 0x2913D852 | ||
| Line 236: | Line 280: | ||
| To convert a TimeStamp-compatible int64_t to a C-readable timestamp, use | To convert a TimeStamp-compatible int64_t to a C-readable timestamp, use | ||
| - | < | + | < |
| /* clockspeed is in Hz */ | /* clockspeed is in Hz */ | ||
| int64 nanoseconds = (int64) (timestamp * 1024 * 1e9 / clockspeed); | int64 nanoseconds = (int64) (timestamp * 1024 * 1e9 / clockspeed); | ||
| Line 258: | Line 302: | ||
| </ | </ | ||
| - | ==== Constants | + | === Constants === |
| Constants can be computed using the parset file. Below is a translation between the C constants used above and their respective parset keys: | Constants can be computed using the parset file. Below is a translation between the C constants used above and their respective parset keys: | ||