The DTC package is based on C++ codes (one for each functionality) interfaced with python scripts. Exception for the «Visualization», it is a python graphical interface coded in Tkinter.

The two main prerequisites are the (LOFAR-specific) DAL and hdf5 libraries. In addition, you'll need a few standard packages, as detailed in the following.

• Install hdf5: hdf5 (version 1.8.x), including the developper package (ubuntu 12.04: libhdf5-serial, libhdf5-serial-dev).
• The DAL (Data Access Library) is developed by ASTRON and must be installed on your laptop/server before installing DTC. Click on the link, click on INSTALL, and follow the instructions, e.g.
  • mkdir build
  • cd build
  • cmake .. # (indicate the directory for C++ libraries, e.g. “cmake -DCMAKE_INSTALL_PREFIX=~/usr/local/DAL ..” if installing without administrator privileges also it try to install in /usr/local automatically).
  • make
  • make install
  • add the path of the python DAL (which still is in the ../source_code/build directory) to PYTHONPATH: e.g. export PYTHONPATH=~/DAL/build/dal/:$PYTHONPATH
  • The DAL provides tools for python (pydal) and for C++. Both are in different locations!
  • Example: A user clones the DAL with «git» into a new directory called /home/user/DAL (we'll call this the source_code directory). He decides to install the DAL to /usr/local/DAL. After the installation, the C++ libraries are indeed in /usr/local/DAL/ (and have to be indicated during the installation of DTC using the -DDAL_ROOT_DIR parameter). However, the python DAL is in /home/user/DAL/build/ (must be added to his PYTHONPATH, for example by editing the .bashrc).

• h5py library (ubuntu 12.04: python-h5py).
• svn (ubuntu 12.04: subversion).
• python Tkinter (for ubuntu: python-tk package).
• python-dev
• python-matplotlib
• hdfview (recommended, not mandatory. It allows to view metadata of observation files, and adapt the command-line processing accordingly)

The DTC package is located on the LUS ASTRON svn server. It requires an account to checkout. Contact for a svn account: loose@astron.nl (or grit@astron.nl).

To install Prerequisites, follow links provided above in the Prerequisites section. Be sure that DAL, hdf5 libraies and h5py are well installed. To process the installation, cmake is also required.

Checkout form the LUS DTC package:

svn co  http://usg.lofar.org/svn/code/trunk/src/Dynspec-Toolkit dynspec 

then go to dynspec directory:

cd dynspec

Create a build directory:

mkdir build

go to build directory:

cd build

Run Cmake:

cmake -DDAL_ROOT_DIR=/PATH/to/DAL -DHDF5_ROOT_DIR=/PATH/to/HDF5 -DCMAKE_INSTALL_PREFIX=/Where/You/Want/to/Install   ..

N.B:

• on CEP hdf5 is in the path, and the dal is at /opt/cep/dal/current
• you must launch cmake on the CmakeList file which is in /Where/You/Want/to/Install/dynspec/ from /Where/You/Want/to/Install/dynspec/build/, for example
  

• cmake -DDAL_ROOT_DIR=/shared/pulsar/usr/local/DAL  -DCMAKE_INSTALL_PREFIX=/shared/pulsar/usr/local/dynspec   ..

• -DDAL_ROOT_DIR is the PATH which contains DAL's lib and include directories
• -DHDF5_ROOT_DIR is the PATH which contains HDF5's lib and include directories
• Those (above) are not required if DAL's or HDF5's lib and include are in your path (e.g in /usr/lib and /usr/include)

Then make:

make

Then install

make install

PS: Without the CMAKE_INSTALL_PREFIX, it will install by default into /usr/local. If you select another directory, add it to the search path (for example by editing .bashrc):

export PATH=/Where/You/Want/to/Install/bin:$PATH
export PYTHONPATH=/Where/You/Want/to/Install/lib/python2.6/site-packages/:$PYTHONPATH

e.g.

export PYTHONPATH=~/usr/local/dynspec/lib/python2.7/site-packages/:$PYTHONPATH

WARNING: Currently, DTC provides C++ exectuables and python scripts, which are all installed in the same directory. Their names differ, but are rather similar (for example, the python scripts do not have a .py extension). Be careful to use the python script and not the C++ executable.

Users must be aware about the file size. Usually, Beam formed observation are really voluminous (could be several TB). DTC converts Beam formed data to Dynspec data. Users can have an idea of the size of a complete conversion by summing the size of all file of a given SAP. Remember, for example on CEP each node has 16TB of data storage, but the real limit is the physical limit of one disk, so 2 TB (and certainly less, because the “real” disk is certainly not empty !!). So be careful when you do a complete conversion of a large observation. In practice, a complete conversion must be done ONLY for small dataset. In addition, the Visualization tool can visualize (without swapping) only files whose size is almost equal to its RAM memory. So, it is really recommended to rebin your data, except if you want a Dynspec original copy, but in this last case, dataset (for one SAP) must be smaller than 1TB. Last remark, if s user has a large observation with several SAP, it usually is more ingenious to process SAP by SAP, to avoid to fill up all the storage space in one run (and with one single file).

Return to main page, where you will find tutorials: dynspec