public:install_lofar_redhat5.8

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

public:install_lofar_redhat5.8 [2013-10-10 09:55]
Jose Sabater Montes created
public:install_lofar_redhat5.8 [2017-03-08 15:27]
Line 1: Line 1:
-====== Intallation of LOFAR in Red Hat Enterprise 5.8 ====== 
  
-This is the description of the installation of LOFAR on the cluster of the [[http://www.iaa.es/|IAA]] in [[http://es.wikipedia.org/wiki/Granada|Granada]]. It bundles an old version of Red Hat Enterprise Linux that does not contain many of the new libraries required by LOFAR. 
- 
-Description of the system: 
-  * Operating system: Red Hat Enterprise Linux Server release 5.8 (Tikanga) 
-  * PBS queue system: [[http://www.adaptivecomputing.com/products/open-source/torque/|TORQUE]] 
-  * Libraries available: HDF5 and old versions of boost and fftw3. 
- 
-After some tests we determined that we needed to isntall, at least, the followin software and libraries: 
-  * scons 
-  * WCS 
-  * lapack 
-  * atlas 
-  * fftw3 
-  * Python 
-    * numpy 
-  * Boost 
-  * LOFAR related: 
-    * casacore 
-    * pyrap 
-    * casarest 
-    * LOFAR (LofIm) 
- 
- 
-===== Installation steps ===== 
- 
-The software was installed in the home of the use ''jsm'' (''/home/users/dae/jsm''). You should change the scripts accordingly. 
- 
-Previous steps: 
-<code bash> 
-## Create the installation directory 
-mkdir -p /home/users/dae/jsm/local/src 
-cd /home/users/dae/jsm/local/src 
-</code> 
- 
-Download and copy the data: 
-<code bash> 
-# scons 
-wget http://prdownloads.sourceforge.net/scons/scons-2.3.0.tar.gz 
-# WCS 
-wget ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib.tar.bz2 
-# lapack 
-wget http://www.netlib.org/lapack/lapack-3.4.2.tgz 
-# atlas 
-wget http://prdownloads.sourceforge.net/math-atlas/Developer%20%28unstable%29/3.11.11/atlas3.11.11.tar.bz2 
-# FFTW3 
-wget http://www.fftw.org/fftw-3.3.3.tar.gz 
-# Boost 
-wget http://prdownloads.sourceforge.net/boost/boost/1.54.0/boost_1_54_0.tar.gz 
-# LOFAR related software 
-# casacore 
-#svn co http://casacore.googlecode.com/svn/tags/casacore-1.5.0 
-cp -r /home/users/dae/jsm/local-v0/src/casacore-1.5.0 . 
-rm -rf casacore-1.5.0/build 
-# pyrap 
-#svn co http://pyrap.googlecode.com/svn/tags/pyrap-1.1.0 
-cp -r /home/users/dae/jsm/local-v0/src/pyrap-1.1.0 . 
-# casarest 
-#svn co https://svn.astron.nl/casarest/trunk/casarest/ 
-cp -r /home/users/dae/jsm/local-v0/src/casarest . 
-rm -rf casarest/build 
-</code> 
- 
-We had already got a copy of the LOFAR software with SVN in ''/home/users/dae/jsm/LOFAR'': 
-<code bash> 
-# LOFAR 
-cp -r /home/users/dae/jsm/LOFAR . 
-</code> 
- 
-We used the following paths: 
-<code bash> 
-# PATH 
-export PATH=/home/users/dae/jsm/local/bin:/home/users/dae/jsm/python/bin:/usr/local/casapy-stable-42.0.25430-001-64b:/usr/local/iraf/bin:/usr/local/mpich-install/bin:/usr/local/bin:/bin:/usr/bin 
-# LD_LIBRARY_PATH 
-export LD_LIBRARY_PATH=/home/users/dae/jsm/local/lib:/home/users/dae/jsm/local/atlas/lib:/home/users/dae/jsm/python/lib:/usr/local/hdf5-1.8.11-linux-x86_64-shared/lib:/usr/local/mpich-install/lib:/usr/local/lib 
-</code> 
- 
-Install Python and their important libraries: 
-<code bash> 
-# Compile 
-mkdir -p /home/users/dae/jsm/python/src 
-cd /home/users/dae/jsm/python/src 
-# Download Python **** CHANGE 
-# Extract Python **** CHANGE 
-cd Python-2.7.5  
-./configure --prefix=/home/users/dae/jsm/python --enable-shared 
-make  
-make install 
- 
-# Install zeromq (used by IPython parallel) 
-cd /home/users/dae/jsm/python/src 
-wget http://download.zeromq.org/zeromq-3.2.4.tar.gz 
-tar xzf zeromq-3.2.4.tar.gz 
-cd zeromq-3.2.4 
-./configure --prefix=/home/users/dae/jsm/local 
-make  
-make install 
- 
-# Install easy install and pip 
-cd /home/users/dae/jsm/python/src 
-wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py 
-python ez_setup.py 
-wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py 
-# rm -r /tmp/pip-build-jsm/pip 
-python get-pip.py 
- 
-# Install packages 
-pip install virtualenv 
-pip install numpy 
-pip install h5py 
-pip install scipy 
-pip install matplotlib 
-pip install pandas 
-pip install pyfits 
-pip install pywcs 
-pip install astropy 
-pip install cython 
- 
-# Install pyzmq 
-cd /home/users/dae/jsm/python/src 
-wget https://pypi.python.org/packages/source/p/pyzmq/pyzmq-13.0.2.tar.gz --no-check-certificate 
-tar xzf pyzmq-13.0.2.tar.gz 
-cd pyzmq-13.0.2 
-python setup.py configure --zmq=/home/users/dae/jsm/local/ 
-python setup.py install 
- 
-# Install IPython 
-cd /home/users/dae/jsm/python/src 
-pip install ipython[all] 
-</code> 
- 
-From now on we need to enter an interactive PBS job to avoid our jobs to be killed after certain amount of time (the maximum allowed lenght for a job without using PBS is 10 minutes). The nodes do not have direct access to the Internet (that is why they were previously downloaded): 
- 
-<code> 
-### Enter interactive qsub session 
-qsub -I -X -N LOFAR_soft 
- 
-## Update the PATH and the LD_LIBRARY_PATH 
-# PATH 
-export PATH=/home/users/dae/jsm/local/bin:/home/users/dae/jsm/python/bin:/usr/local/hdf5-1.8.11-linux-x86_64-shared/bin:/usr/local/casapy-stable-42.0.25430-001-64b:/usr/local/iraf/bin:/usr/local/mpich-install/bin:/usr/local/bin:/bin:/usr/bin 
-# LD_LIBRARY_PATH 
-export LD_LIBRARY_PATH=/home/users/dae/jsm/local/lib:/home/users/dae/jsm/local/atlas/lib:/home/users/dae/jsm/python/lib:/usr/local/hdf5-1.8.11-linux-x86_64-shared/lib:/usr/local/mpich-install/lib:/usr/local/lib 
-</code> 
- 
-Compile the libraries and tools: 
-<code bash> 
-## scons 
-cd /home/users/dae/jsm/local/src 
-tar xfz scons-2.3.0.tar.gz 
-cd scons-2.3.0 
-python setup.py install --prefix=/home/users/dae/jsm/local 
- 
- 
-## ATLAS + LAPACK + BLAS 
-cd /home/users/dae/jsm/local/src 
-tar xfj atlas3.11.11.tar.bz2 
-mkdir -p ATLAS/build 
-cd ATLAS/build 
-../configure -b 64 -D c -DPentiumCPS=2932 --shared \ 
-  --prefix=/home/users/dae/jsm/local/atlas \ 
-  --with-netlib-lapack-tarfile=/home/users/dae/jsm/local/src/lapack-3.4.2.tgz 
-make build 
-make check 
-make ptcheck 
-make time 
-make install 
-ln -s /home/users/dae/jsm/local/atlas/lib/libcblas.a \ 
-  /home/users/dae/jsm/local/atlas/lib/libblas.a 
- 
- 
-## WCS 
-cd /home/users/dae/jsm/local/src 
-tar xfj wcslib.tar.bz2 
-cd wcslib-4.19 
-./configure --prefix=/home/users/dae/jsm/local/ 
-make 
-make install 
- 
- 
-## FFTW3 
-cd /home/users/dae/jsm/local/src 
-tar xfz fftw-3.3.3.tar.gz 
-cd fftw-3.3.3 
-./configure --enable-threads --enable-shared \ 
-  --prefix=/home/users/dae/jsm/local/ 
-make 
-make install 
-make clean 
-./configure --enable-threads --enable-shared  --enable-float \ 
-  --prefix=/home/users/dae/jsm/local/ 
-make  
-make install 
- 
- 
-## Boost  
-cd /home/users/dae/jsm/local/src 
-tar xfz boost_1_54_0.tar.gz 
-cd boost_1_54_0 
-./bootstrap.sh --prefix=/home/users/dae/jsm/local \ 
-  --with-python=/home/users/dae/jsm/python/bin/python 
-./b2  
-./b2 install 
-</code> 
- 
-Compile casacore 
-<code bash> 
-## casacore 
-cd /home/users/dae/jsm/local/src 
-cd casacore-1.5.0 
-mkdir build; cd build 
-cmake .. -DWCSLIB_LIBRARY=/home/users/dae/jsm/local/lib/libwcs.so \ 
-         -DWCSLIB_INCLUDE_DIR=/home/users/dae/jsm/local/include/ \ 
-         -DCMAKE_INSTALL_PREFIX=/home/users/dae/jsm/local/casacore \ 
-         -DUSE_HDF5=ON -DUSE_FFTW3=ON \ 
-         -DHDF5_ROOT_DIR=/usr/local/hdf5-1.8.11-linux-x86_64-shared \ 
-         -DLAPACK_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/liblapack.a \ 
-         -DBLAS_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a \ 
-         -DBLA_VENDOR=ATLAS \ 
-         -DFFTW3_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3.so \ 
-         -DFFTW3_THREADS_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3_threads.so \ 
-         -DFFTW3F_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3f.so \ 
-         -DFFTW3F_THREADS_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3f_threads.so \ 
-         -DFFTW3_INCLUDE=/home/users/dae/jsm/local/include/ \ 
-         -DCMAKE_Fortran_COMPILER:FILEPATH=/usr/bin/gfortran 
-make 
-make install 
-</code> 
- 
-Install pyrap (scons needed): 
-<code bash> 
-## pyrap 
-cd /home/users/dae/jsm/local/src 
-cd pyrap-1.1.0 
-## PATCH pyrap ******* 
-python batchbuild.py --boost-root=/home/users/dae/jsm/local \ 
-  --casacore-root=/home/users/dae/jsm/local/casacore \ 
-  --enable-hdf5 --hdf5-root=/usr/local/hdf5-1.8.11-linux-x86_64-shared \ 
-  --lapack-root=/home/users/dae/jsm/local/atlas \ 
-  --wcs-root=/home/users/dae/jsm/local \ 
-  --prefix=/home/users/dae/jsm/local  
-</code> 
- 
-Install casarest 
-<code bash> 
-## casarest 
-cd /home/users/dae/jsm/local/src 
-cd casarest 
-mkdir build; cd build 
-cmake .. -DBOOST_ROOT=/home/users/dae/jsm/local \ 
-         -DLAPACK_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/liblapack.a \ 
-         -DLAPACK_INCLUDE=/home/users/dae/jsm/local/atlas/include/ \ 
-         -DBLAS_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a \ 
-         -DBLAS_INCLUDE=/home/users/dae/jsm/local/atlas/include/ \ 
-         -DWCSLIB_LIBRARY=/home/users/dae/jsm/local/lib/libwcs.so \ 
-         -DWCSLIB_INCLUDE_DIR=/home/users/dae/jsm/local/include/ \ 
-         -DCASACORE_ROOT_DIR=/home/users/dae/jsm/local/casacore \ 
-         -DCMAKE_INSTALL_PREFIX=/home/users/dae/jsm/local/casarest \ 
-         -DHDF5_ROOT_DIR=/usr/local/hdf5-1.8.11-linux-x86_64-shared \ 
-         -DLIB_EXTRA_SYNTHESIS=/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a \ 
-         -DBUILD_ALL=1 
-make 
-make install 
-</code> 
- 
-Install LOFAR: 
-<code bash> 
-cd /home/users/dae/jsm/local/src 
-#cp -r ~/LOFAR . 
-cd LOFAR 
-### PATCH LOFAR ******** 
-mkdir -p build/gnu_opt; cd build/gnu_opt 
-mkdir /home/users/dae/jsm/local/LofIm 
-cmake ../.. -DBUILD_SHARED_LIBS=ON \ 
-  -DCMAKE_INSTALL_PREFIX=/home/users/dae/jsm/local/LofIm \ 
-  -DUSE_LOG4CPLUS=OFF \ 
-  -DBOOST_ROOT=/home/users/dae/jsm/local \ 
-  -DHDF5_ROOT_DIR=/usr/local/hdf5-1.8.11-linux-x86_64-shared \ 
-  -DWCSLIB_LIBRARY=/home/users/dae/jsm/local/lib/libwcs.so \ 
-  -DWCSLIB_INCLUDE_DIR=/home/users/dae/jsm/local/include/ \ 
-  -DLAPACK_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a\;/home/users/dae/jsm/local/atlas/lib/liblapack.a \ 
-  -DBLAS_LIBRARIES=/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a\;/home/users/dae/jsm/local/atlas/lib/liblapack.a  \ 
-  -DFFTW3_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3.so \ 
-  -DFFTW3_THREADS_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3_threads.so \ 
-  -DFFTW3F_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3f.so \ 
-  -DFFTW3F_THREADS_LIBRARY=/home/users/dae/jsm/local/lib/libfftw3f_threads.so \ 
-  -DFFTW3_INCLUDE=/home/users/dae/jsm/local/include/ \ 
-  -DCASACORE_ROOT_DIR=/home/users/dae/jsm/local/casacore \ 
-  -DCASA_CASA_LIBRARY=/home/users/dae/jsm/local/casacore/lib/libcasa_casa.so\;/home/users/dae/jsm/local/atlas/lib/libf77blas.a\;/home/users/dae/jsm/local/atlas/lib/libcblas.a\;/home/users/dae/jsm/local/atlas/lib/libatlas.a\;/home/users/dae/jsm/local/atlas/lib/liblapack.a \ 
-  -DUSE_OPENMP=ON \ 
-  -DBUILD_PACKAGES=Offline\;LofarFT\;StaticMetaData\;SPW_Combine \ 
-  -DCASAREST_ROOT_DIR=/home/users/dae/jsm/local/casarest/ \ 
-  -DPYTHON_LIBRARY=/home/users/dae/jsm/python/lib/libpython2.7.so \ 
-  -DPYTHON_EXECUTABLE:FILEPATH=/home/users/dae/jsm/python/bin/python \ 
-  -DPYTHON_INCLUDE_DIR:PATH=/home/users/dae/jsm/python/include \ 
-  -DPYRAP_INCLUDE_DIR=/home/users/dae/jsm/local/include/pyrap \ 
-  -DPYRAP_LIBRARY=/home/users/dae/jsm/local/lib/libpyrap.so \ 
-  -DCMAKE_Fortran_COMPILER:FILEPATH=/usr/bin/gfortran \ 
-  -DF2PY_FCOMPILER=gnu95 2>&1 | tee log.txt 
-make 
-make install 
-</code> 
- 
-Finish the installation (FIXME): 
-  * Configure .bashrc 
-  * Configure .casarc 
- 
-==== Patches needed ==== 
- 
-pyrap and LOFAR need to be patched to allow them to compile in this old system. 
- 
-===== Problems ===== 
  • Last modified: 2017-03-08 15:27
  • (external edit)