public:install_lofar_redhat5.8

This is an old revision of the document!

Intallation of LOFAR in Red Hat Enterprise 5.8

This is the description of the installation of LOFAR on the cluster of the IAA in Granada. It bundles an old version of Red Hat Enterprise Linux that does not contain many of the new libraries required by LOFAR.

Description of the system:

  • Operating system: Red Hat Enterprise Linux Server release 5.8 (Tikanga)
  • PBS queue system: TORQUE
  • Libraries available: HDF5 and old versions of boost and fftw3.

After some tests we determined that we needed to isntall, at least, the followin software and libraries:

  • scons
  • WCS
  • lapack
  • atlas
  • fftw3
  • Python
    • numpy
  • Boost
  • LOFAR related:
    • casacore
    • pyrap
    • casarest
    • LOFAR (LofIm)

Installation steps

The software was installed in the home of the use jsm (/home/users/dae/jsm). You should change the scripts accordingly.

Previous steps: 

## Create the installation directory
mkdir -p /home/users/dae/jsm/local/src
cd /home/users/dae/jsm/local/src

Download and copy the data: 

# scons
wget http://prdownloads.sourceforge.net/scons/scons-2.3.0.tar.gz
# WCS
wget ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib.tar.bz2
# lapack
wget http://www.netlib.org/lapack/lapack-3.4.2.tgz
# atlas
wget http://prdownloads.sourceforge.net/math-atlas/Developer%20%28unstable%29/3.11.11/atlas3.11.11.tar.bz2
# FFTW3
wget http://www.fftw.org/fftw-3.3.3.tar.gz
# Boost
wget http://prdownloads.sourceforge.net/boost/boost/1.54.0/boost_1_54_0.tar.gz
# LOFAR related software
# casacore
#svn co http://casacore.googlecode.com/svn/tags/casacore-1.5.0
cp -r /home/users/dae/jsm/local-v0/src/casacore-1.5.0 .
rm -rf casacore-1.5.0/build
# pyrap
#svn co http://pyrap.googlecode.com/svn/tags/pyrap-1.1.0
cp -r /home/users/dae/jsm/local-v0/src/pyrap-1.1.0 .
# casarest
#svn co https://svn.astron.nl/casarest/trunk/casarest/
cp -r /home/users/dae/jsm/local-v0/src/casarest .
rm -rf casarest/build

We had already got a copy of the LOFAR software with SVN in /home/users/dae/jsm/LOFAR: 

# LOFAR
cp -r /home/users/dae/jsm/LOFAR .

We used the following paths: 

# PATH
export PATH=/home/users/dae/jsm/local/bin:/home/users/dae/jsm/python/bin:/usr/local/casapy-stable-42.0.25430-001-64b:/usr/local/iraf/bin:/usr/local/mpich-install/bin:/usr/local/bin:/bin:/usr/bin
# LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/home/users/dae/jsm/local/lib:/home/users/dae/jsm/local/atlas/lib:/home/users/dae/jsm/python/lib:/usr/local/hdf5-1.8.11-linux-x86_64-shared/lib:/usr/local/mpich-install/lib:/usr/local/lib

Install Python and their important libraries: 

# Compile
mkdir -p /home/users/dae/jsm/python/src
cd /home/users/dae/jsm/python/src
# Download Python **** CHANGE
# Extract Python **** CHANGE
cd Python-2.7.5 
./configure --prefix=/home/users/dae/jsm/python --enable-shared
make 
make install
 
# Install zeromq (used by IPython parallel)
cd /home/users/dae/jsm/python/src
wget http://download.zeromq.org/zeromq-3.2.4.tar.gz
tar xzf zeromq-3.2.4.tar.gz
cd zeromq-3.2.4
./configure --prefix=/home/users/dae/jsm/local
make 
make install
 
# Install easy install and pip
cd /home/users/dae/jsm/python/src
wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
python ez_setup.py
wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
# rm -r /tmp/pip-build-jsm/pip
python get-pip.py
 
# Install packages
pip install virtualenv
pip install numpy
pip install h5py
pip install scipy
pip install matplotlib
pip install pandas
pip install pyfits
pip install pywcs
pip install astropy
pip install cython
 
# Install pyzmq
cd /home/users/dae/jsm/python/src
wget https://pypi.python.org/packages/source/p/pyzmq/pyzmq-13.0.2.tar.gz --no-check-certificate
tar xzf pyzmq-13.0.2.tar.gz
cd pyzmq-13.0.2
python setup.py configure --zmq=/home/users/dae/jsm/local/
python setup.py install
 
# Install IPython
cd /home/users/dae/jsm/python/src
pip install ipython[all]

From now on we need to enter an interactive PBS job to avoid our jobs to be killed after certain amount of time (the maximum allowed lenght for a job without using PBS is 10 minutes). The nodes do not have direct access to the Internet (that is why they were previously downloaded): 

### Enter interactive qsub session
qsub -I -X -N LOFAR_soft

## Update the PATH and the LD_LIBRARY_PATH
# PATH
export PATH=/home/users/dae/jsm/local/bin:/home/users/dae/jsm/python/bin:/usr/local/hdf5-1.8.11-linux-x86_64-shared/bin:/usr/local/casapy-stable-42.0.25430-001-64b:/usr/local/iraf/bin:/usr/local/mpich-install/bin:/usr/local/bin:/bin:/usr/bin
# LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/home/users/dae/jsm/local/lib:/home/users/dae/jsm/local/atlas/lib:/home/users/dae/jsm/python/lib:/usr/local/hdf5-1.8.11-linux-x86_64-shared/lib:/usr/local/mpich-install/lib:/usr/local/lib

Compile the libraries and tools: 

## scons
cd /home/users/dae/jsm/local/src
tar xfz scons-2.3.0.tar.gz
cd scons-2.3.0
python setup.py install --prefix=/home/users/dae/jsm/local
 
 
## ATLAS + LAPACK + BLAS
cd /home/users/dae/jsm/local/src
tar xfj atlas3.11.11.tar.bz2
mkdir -p ATLAS/build
cd ATLAS/build
../configure -b 64 -D c -DPentiumCPS=2932 --shared \
  --prefix=/home/users/dae/jsm/local/atlas \
  --with-netlib-lapack-tarfile=/home/users/dae/jsm/local/src/lapack-3.4.2.tgz
make build
make check
make ptcheck
make time
make install
ln -s /home/users/dae/jsm/local/atlas/lib/libcblas.a \
  /home/users/dae/jsm/local/atlas/lib/libblas.a
 
 
## WCS
cd /home/users/dae/jsm/local/src
tar xfj wcslib.tar.bz2
cd wcslib-4.19
./configure --prefix=/home/users/dae/jsm/local/
make
make install
 
 
## FFTW3
cd /home/users/dae/jsm/local/src
tar xfz fftw-3.3.3.tar.gz
cd fftw-3.3.3
./configure --enable-threads --enable-shared \
  --prefix=/home/users/dae/jsm/local/
make
make install
make clean
./configure --enable-threads --enable-shared  --enable-float \
  --prefix=/home/users/dae/jsm/local/
make 
make install
 
 
## Boost 
cd /home/users/dae/jsm/local/src
tar xfz boost_1_54_0.tar.gz
cd boost_1_54_0
./bootstrap.sh --prefix=/home/users/dae/jsm/local \
  --with-python=/home/users/dae/jsm/python/bin/python
./b2 
./b2 install

Compile casacore 

## casacore
cd /home/users/dae/jsm/local/src
cd casacore-1.5.0
mkdir build; cd build
cmake .. -DWCSLIB_LIBRARY=/home/users/dae/jsm/local/lib/libwcs.so \
         -DWCSLIB_INCLUDE_DIR=/home/users/dae/jsm/local/include/ \
         -DCMAKE_INSTALL_PREFIX=/home/users/dae/jsm/local/casacore \
         -DUSE_HDF5=ON -DUSE_FFTW3=ON \
         -DHDF5_ROOT_DIR=/usr/local/hdf5-1.8.11-linux-x86_64-shared \
         -DLAPACK_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/liblapack.a \
         -DBLAS_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a \
         -DBLA_VENDOR=ATLAS \
         -DFFTW3_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3.so \
         -DFFTW3_THREADS_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3_threads.so \
         -DFFTW3F_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3f.so \
         -DFFTW3F_THREADS_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3f_threads.so \
         -DFFTW3_INCLUDE=/home/users/dae/jsm/local/include/ \
         -DCMAKE_Fortran_COMPILER:FILEPATH=/usr/bin/gfortran
make
make install

Install pyrap (scons needed): 

## pyrap
cd /home/users/dae/jsm/local/src
cd pyrap-1.1.0
## PATCH pyrap *******
python batchbuild.py --boost-root=/home/users/dae/jsm/local \
  --casacore-root=/home/users/dae/jsm/local/casacore \
  --enable-hdf5 --hdf5-root=/usr/local/hdf5-1.8.11-linux-x86_64-shared \
  --lapack-root=/home/users/dae/jsm/local/atlas \
  --wcs-root=/home/users/dae/jsm/local \
  --prefix=/home/users/dae/jsm/local

Install casarest 

## casarest
cd /home/users/dae/jsm/local/src
cd casarest
mkdir build; cd build
cmake .. -DBOOST_ROOT=/home/users/dae/jsm/local \
         -DLAPACK_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/liblapack.a \
         -DLAPACK_INCLUDE=/home/users/dae/jsm/local/atlas/include/ \
         -DBLAS_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a \
         -DBLAS_INCLUDE=/home/users/dae/jsm/local/atlas/include/ \
         -DWCSLIB_LIBRARY=/home/users/dae/jsm/local/lib/libwcs.so \
         -DWCSLIB_INCLUDE_DIR=/home/users/dae/jsm/local/include/ \
         -DCASACORE_ROOT_DIR=/home/users/dae/jsm/local/casacore \
         -DCMAKE_INSTALL_PREFIX=/home/users/dae/jsm/local/casarest \
         -DHDF5_ROOT_DIR=/usr/local/hdf5-1.8.11-linux-x86_64-shared \
         -DLIB_EXTRA_SYNTHESIS=/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a \
         -DBUILD_ALL=1
make
make install

Install LOFAR: 

cd /home/users/dae/jsm/local/src
#cp -r ~/LOFAR .
cd LOFAR
### PATCH LOFAR ********
mkdir -p build/gnu_opt; cd build/gnu_opt
mkdir /home/users/dae/jsm/local/LofIm
cmake ../.. -DBUILD_SHARED_LIBS=ON \
  -DCMAKE_INSTALL_PREFIX=/home/users/dae/jsm/local/LofIm \
  -DUSE_LOG4CPLUS=OFF \
  -DBOOST_ROOT=/home/users/dae/jsm/local \
  -DHDF5_ROOT_DIR=/usr/local/hdf5-1.8.11-linux-x86_64-shared \
  -DWCSLIB_LIBRARY=/home/users/dae/jsm/local/lib/libwcs.so \
  -DWCSLIB_INCLUDE_DIR=/home/users/dae/jsm/local/include/ \
  -DLAPACK_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a\;/home/users/dae/jsm/local/atlas/lib/liblapack.a \
  -DBLAS_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a\;/home/users/dae/jsm/local/atlas/lib/liblapack.a  \
  -DFFTW3_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3.so \
  -DFFTW3_THREADS_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3_threads.so \
  -DFFTW3F_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3f.so \
  -DFFTW3F_THREADS_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3f_threads.so \
  -DFFTW3_INCLUDE=/home/users/dae/jsm/local/include/ \
  -DCASACORE_ROOT_DIR=/home/users/dae/jsm/local/casacore \
  -DCASA_CASA_LIBRARY=/home/users/dae/jsm/local/casacore/lib/libcasa_casa.so\;/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a\;/home/users/dae/jsm/local/atlas/lib/liblapack.a \
  -DUSE_OPENMP=ON \
  -DBUILD_PACKAGES=Offline\;LofarFT\;StaticMetaData\;SPW_Combine \
  -DCASAREST_ROOT_DIR=/home/users/dae/jsm/local/casarest/ \
  -DPYTHON_LIBRARY=/home/users/dae/jsm/python/lib/libpython2.7.so \
  -DPYTHON_EXECUTABLE:FILEPATH=/home/users/dae/jsm/python/bin/python \
  -DPYTHON_INCLUDE_DIR:PATH=/home/users/dae/jsm/python/include \
  -DPYRAP_INCLUDE_DIR=/home/users/dae/jsm/local/include/pyrap \
  -DPYRAP_LIBRARY=/home/users/dae/jsm/local/lib/libpyrap.so \
  -DCMAKE_Fortran_COMPILER:FILEPATH=/usr/bin/gfortran \
  -DF2PY_FCOMPILER=gnu95 2>&1 | tee log.txt
make
make install

Finish the installation (FIXME):

  • Configure .bashrc
  • Configure .casarc

Patches needed

pyrap and LOFAR need to be patched to allow them to compile in this old system.

Problems

  • Last modified: 2013-10-10 09:55
  • by Jose Sabater Montes