public:install_lofar_centos6.5

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
public:install_lofar_centos6.5 [2016-12-22 15:19] – [1. Setup of the environment] init modules first Frank Breitlingpublic:install_lofar_centos6.5 [2017-03-08 15:27] (current) – external edit 127.0.0.1
Line 1: Line 1:
-====== Installing LOFAR Software on a CentOS 6.5 (Final) system (cluster Newton of the AIP Potsdam======+====== Installing LOFAR Software on a CentOS 6.5 (Final) System - The Newton Cluster of the AIP Potsdam ======
  
 +This document is evolving and was created by
  --- //[[fbreitling@aip.de|Frank Breitling]] 2015/06/26 18:47//  --- //[[fbreitling@aip.de|Frank Breitling]] 2015/06/26 18:47//
  
  
-I was mainly following the previous installation notes provided by others: \\+A useful starting point were previous installation notes such as: \\
  
 http://www.lofar.org/operations/doku.php?id=public:user_software:lofar \\ http://www.lofar.org/operations/doku.php?id=public:user_software:lofar \\
 http://www.lofar.org/operations/doku.php?id=public:user_software:ubuntu_12_4 http://www.lofar.org/operations/doku.php?id=public:user_software:ubuntu_12_4
  
-However a few problems occured and had to be solved as described below. \\+However some additional issues occurred at the Newton cluster and had to be solved as described below. \\
 Ideas and support by **Arno Schoenmakers**, **Ger van Diepen** and **Marcel Loose** were very helpful and highly appreciated. \\ Ideas and support by **Arno Schoenmakers**, **Ger van Diepen** and **Marcel Loose** were very helpful and highly appreciated. \\
-===== 1. Setup of the environment =====+ 
 + 
 +===== 1. Setup of build environment =====
  
 The following packages are already provided on the cluster: The following packages are already provided on the cluster:
Line 17: Line 20:
   * BLAS   * BLAS
   * LAPACK   * LAPACK
-  * Boost 1.41.0+  * Boost 1.41.0 (but too old for AOFlagger v2.8.0 and later)
   * cfitsio 3.240   * cfitsio 3.240
   * HDF5   * HDF5
Line 23: Line 26:
  
 The cluster also provides a //module system// to load newer versions of the installed software. \\ The cluster also provides a //module system// to load newer versions of the installed software. \\
-We will activate it and replace the default version of gcc 4.4.7 with gcc 4.8.3 for C++11 support.+Its used below to replace the default version of gcc 4.4.7 with gcc 4.8.3 for C++11 support.
  
  
-=== 1.1 Pathes === +=== 1.1 Paths === 
-This instruction will install all packages in $HOME/local/. \\ + 
-And for the installation of Pyrap (python-casacore) we need a Python path with write access. \\ +The following changes are required: 
-So I added the following lines to $HOME/.bashrc:+ 
 +  * A more recent version of GCC with C++11 required by Casacore is activated through the module system. This also requires adding /opt/gcc-4.8.3/lib64 to the LD_LIBRARY_PATH since otherwise Pyrap will not find it and complain with
 <file> <file>
-source /usr/share/Modules/init/sh  # activate module systemneeds to be done first +File "/home/fkbreitl/local/lib64/python2.6/site-packages/python_casacore-2.1.2-py2.6-linux-x86_64.egg/casacore/tables/table.py"line 43, in <module
-module load gcc  # use gcc 4.8.3 (instead of 4.4.7for C++11 support+    from ._tables import Table 
 +ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /home/fkbreitl/local/lib/libboost_python.so.1.63.0) 
 +</file>
  
 +  * The packages will be installed in $HOME/local/. \\
 +  * Pyrap (python-casacore) needs a writable Python path. \\
 +
 +This adds the following lines to $HOME/.bashrc:
 +<file>
 +source /usr/share/Modules/init/sh  # activate module system, needs to be done first
 +module load gcc/4.8.3  # use gcc 4.8.3 (instead of 4.4.7) for C++11 support
 +export LD_LIBRARY_PATH=/opt/gcc-4.8.3/lib64:$HOME/local/lib:$LD_LIBRARY_PATH
 export PYTHONPATH=$HOME/local/lib64/python2.6/site-packages:$PYTHONPATH export PYTHONPATH=$HOME/local/lib64/python2.6/site-packages:$PYTHONPATH
-export LD_LIBRARY_PATH=$HOME/local/lib:$LD_LIBRARY_PATH 
 export PATH=$HOME/local:$PATH export PATH=$HOME/local:$PATH
 </file> </file>
Line 41: Line 54:
  
 === 1.2 CMake === === 1.2 CMake ===
 +
 The 4 Newton head nodes provide CMake 2.6 by default, but the LOFAR Software requires at least 2.6.6. A sufficiently recent version of CMake 3.0.0 is provided via [[https://en.wikipedia.org/wiki/Environment_Modules_%28software%29|Environment Modules]] (see also http://modules.sourceforge.net/).\\ The 4 Newton head nodes provide CMake 2.6 by default, but the LOFAR Software requires at least 2.6.6. A sufficiently recent version of CMake 3.0.0 is provided via [[https://en.wikipedia.org/wiki/Environment_Modules_%28software%29|Environment Modules]] (see also http://modules.sourceforge.net/).\\
 The environment modules also provide more recent HDF5 libraries. Both could be loaded by adding ''module load cmake hdf5'' to $HOME/.bashrc. \\ The environment modules also provide more recent HDF5 libraries. Both could be loaded by adding ''module load cmake hdf5'' to $HOME/.bashrc. \\
 But unfortunately the CMake 3.0.0 loaded on the head node newl1 is broken. Furthermore loading the HDF5 libs has no effect on the libs found by CMake, which are still the system's default. So we don't load any module and don't add this line to the .bashrc. Instead we build our own CMake as follows: But unfortunately the CMake 3.0.0 loaded on the head node newl1 is broken. Furthermore loading the HDF5 libs has no effect on the libs found by CMake, which are still the system's default. So we don't load any module and don't add this line to the .bashrc. Instead we build our own CMake as follows:
 +
 <file> <file>
 mkdir -p $HOME/local/src mkdir -p $HOME/local/src
Line 54: Line 69:
 make install make install
 </file> </file>
-===== 2. Installation of required packages =====+ 
 + 
 +=== 1.3 Python ==
 +To use matplotlib 2 one also needs Python 2.7 or later. 
 + 
 +<file> 
 +V_PY=2.7.13 
 +cd ~/local/src/ 
 +wget https://www.python.org/ftp/python/$V_PY/Python-$V_PY.tgz 
 +tar xf Python-$V_PY.tgz 
 +cd Python-$V_PY 
 +./configure --prefix=$HOME/local --enable-shared                                
 +time make -j 
 +make install 
 +</file> 
 + 
 + 
 + 
 +=== 1.4 Boost === 
 +Also a newer version of [[http://www.boost.org/build/doc/html/bbv2/installation.html | Boost]] is required since AOFlagger v2.8.0 and later: 
 + 
 +<file> 
 +cd $HOME/local/src 
 +V_BOOST=1_63_0 
 +wget wget https://sourceforge.net/projects/boost/files/boost/$V_BOOST/boost_$V_BOOST.tar.bz2 
 +tar xf boost_$V_BOOST.tar.bz2 
 +cd boost_$V_BOOST 
 +./bootstrap.sh 
 +time ./b2 install --prefix=${HOME}/local 
 +</file> 
 + 
 + 
 +===== 2. Installation of dependencies ===== 
 Next we need to build and install these other packages as follows: Next we need to build and install these other packages as follows:
 +
 +=== 2.1 WCSLIB ===
 +The [[http://www.atnf.csiro.au/people/mcalabre/WCS/ | WCSLIB]]:
 +
 <file> <file>
 cd $HOME/local/src/ cd $HOME/local/src/
 wget ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib.tar.bz2 wget ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib.tar.bz2
 tar xf packages/wcslib.tar.bz2 tar xf packages/wcslib.tar.bz2
-cd wcslib-5.6/+cd wcslib-5.15  # (-5.6)
 ./configure --prefix=$HOME/local/ ./configure --prefix=$HOME/local/
-make install+time make -j install 
 +</file>
  
  
 +=== 2.2 FFTW ===
 +The [[http://www.fftw.org/ | FFTW lib]]:
 +
 +<file>
 +V_FFTW=3.3.5  # 3.3.4
 cd $HOME/local/src/ cd $HOME/local/src/
-wget http://www.fftw.org/fftw-3.3.4.tar.gz +wget http://www.fftw.org/fftw-$V_FFTW.tar.gz 
-cd fftw-3.3.4 +cd fftw-$V_FFTW 
-configure --prefix=$HOME/local --enable-threads --enable-shared --enable-float +./configure --prefix=$HOME/local --enable-threads --enable-shared --enable-float --enable-sse --enable-sse2 --enable-avx 
-make -j12 install+time make -install
 </file> </file>
 +
  
  
 ===== 3. Installation the LOFAR software ===== ===== 3. Installation the LOFAR software =====
-Now we can continue building the LOFAR software. \\+Now we can build the LOFAR software. \\
  
 +We will need the following CMake switches:
  
-==3.1 Essential CMake switches ==+   1. -DCMAKE_INSTALL_PREFIX:PATH=$HOME/local 
-We will need these essential CMake switches:+   2. -DBOOST_ROOT=$HOME/local/src/boost_1_63_0 
 +   3. -DCMAKE_CXX_FLAGS="-lboost_thread -lboost_filesystem -L$HOME/local/lib" 
 +   # 4. -DBoost_NO_BOOST_CMAKE=YES (not necessary with new installed Boost)
  
-   1. -DUSE_OPENMP=YES +1.) Sets the installation path to $HOME/local.
-   2. -DCMAKE_INSTALL_PREFIX:PATH=$HOME/local +
-   3. -DBoost_NO_BOOST_CMAKE=YES +
-   4. -DENABLE_TABLELOCKING=NO (only for Casacore) +
-   5. -DUSE_LOG4CPLUS=NO (only for the LOFAR packages) +
-   6-DUSE_BACKTRACE=NO+
  
-1.) OpenMP makes use of multi-core CPUs. We want to use this since it speeds up our processing.+2.) Necessary to find the new Boost installation
  
-2.) Sets the installation path to $HOME/local/.+3.) Necessary to link against the new Boost installation
  
-3.) Is necessary when using some older versions of boost with cmake-2.8.6-rc2 or laterwhere the boost finding has changed. \\ +# 4.) Was required by some older versions of boost e.g. 1.41 with cmake-2.8.6-rc2, producing this problem: \\ 
-(See https://stackoverflow.com/questions/9948375/cmake-find-package-succeeds-but-returns-wrong-path.)\\ +https://stackoverflow.com/questions/9948375/cmake-find-package-succeeds-but-returns-wrong-path resulting in\\
-Otherwise make will show this error later:+
 <file> <file>
 make[2]: *** No rule to make target `/usr/lib64/lib64/libboost_program_options-mt.so.5', needed by `segment' Stop. make[2]: *** No rule to make target `/usr/lib64/lib64/libboost_program_options-mt.so.5', needed by `segment' Stop.
Line 99: Line 155:
 make: *** [all] Error 2 make: *** [all] Error 2
 </file> </file>
 +It is not necessary any more with the new boost installation.
  
-4.) On the AIP Lustre file system Casacore cannot acquire read locks leading to errors like:+ 
 +=== 3.2 Casacore === 
 +[[https://github.com/casacore/casacore | Casacore]] also requires the CASA data.  
 +This is for example provided with [[http://casa.nrao.edu/casa_obtaining.shtml | CASA]]. \\ 
 +Here we assume CASA is already installed and can be found in the $CASAPATH given below. For convenience this is a link to the CASA folder. 
 +Then the data is in $CASAPATH/data. \\ 
 +We will specify this via -DDATA_DIR, otherwise we need a ~/.casarc file with the line: ''measures.directory: $HOME/local/casa/casa/data'' .\\ 
 + 
 +''-DENABLE_TABLELOCKING=NO'' switches off table locking, since read locks could not be acquired from the AIP Lustre file system leading to errors like:
 <file> <file>
 msoverview: Version 20110407GvD msoverview: Version 20110407GvD
Line 108: Line 173:
 /lustre/fkbreitl/data/L206894/L206894.cal/L206894_SAP000_SB200_uv.MS.dppp_00001~00040/table.lock /lustre/fkbreitl/data/L206894/L206894.cal/L206894_SAP000_SB200_uv.MS.dppp_00001~00040/table.lock
 </file> </file>
-With table locking switched off these errors are avoided. For practical work there is no disadvantage in disabling table locking. \\ +For practical work there is no disadvantage in disabling table locking. \\
-This switch is only needed for Casacore! +
- +
-5.) Is necessary for the LOFAR packages if the Log4cplus libs are not installed. +
- +
-6.) Is necessary if building shared libraries (the default with LOFAR), but libiberty.so is not installed on your system, only libiberty.a.\\ +
-(See https://support.astron.nl/lofar_issuetracker/issues/8046). +
- +
-So we can build the rest of the packages as follows. +
- +
-=== 3.2 Casacore === +
-Casacore also requires the CASA data. This is for example provided with [[http://casa.nrao.edu/casa_obtaining.shtml | CASA]]. \\ +
-Here we assume CASA is already installed and $HOME/local/casa is a link to the CASA folder. Then the data is in $HOME/local/casa. \\ +
-We will specify this via -DDATA_DIR, otherwise we need a ~/.casarc file with the line: ''measures.directory: $HOME/local/casa/data'' .\\+
  
 <file> <file>
-cd ~local/src/+# 2.1, 2.2 
 +cd ~/local/src/
 git clone https://github.com/casacore/casacore git clone https://github.com/casacore/casacore
 mkdir build/casacore mkdir build/casacore
 cd build/casacore cd build/casacore
-time cmake -DBUILD_PYTHON=YES -DCMAKE_INSTALL_PREFIX=~/local -DUSE_FFTW3=YES -DENABLE_TABLELOCKING=NO \ +CASAPATH=$HOME/local/casa/casa 
-  -DUSE_OPENMP=YES -DDATA_DIR=~/local/casa/data -DUSE_HDF5=YES -DCMAKE_PREFIX_PATH=$HOME/local ../../casacore +time cmake ../../casacore -DBUILD_PYTHON=YES -DCMAKE_INSTALL_PREFIX=~/local -DUSE_FFTW3=YES -DENABLE_TABLELOCKING=NO \ 
-time make -j16+  -DUSE_OPENMP=YES -DDATA_DIR=$CASAPATH/data -DUSE_HDF5=YES -DCMAKE_PREFIX_PATH=$HOME/local 
 +time make -j
 make install make install
 </file> </file>
Line 146: Line 200:
  
 == 3.3.1 Alternative Pyrap installation == == 3.3.1 Alternative Pyrap installation ==
-Alternatively we can install Pyrap with //pip// if we install pip before:+Alternatively we can install [[https://github.com/casacore/python-casacore | Pyrap]] with //pip// if we install pip before:
 <file> <file>
 wget https://bootstrap.pypa.io/get-pip.py wget https://bootstrap.pypa.io/get-pip.py
Line 153: Line 207:
    --global-option=-L$HOME/local/lib --install-option=--prefix=$HOME/local    --global-option=-L$HOME/local/lib --install-option=--prefix=$HOME/local
 </file> </file>
 +
  
 === 3.4 Casarest === === 3.4 Casarest ===
Line 161: Line 216:
 mkdir build/casarest mkdir build/casarest
 cd build/casarest cd build/casarest
-cmake -DCASACORE_ROOT_DIR=$HOME/local -DBUILD_ALL=1 -DCMAKE_INSTALL_PREFIX:PATH=$HOME/local +cmake ../../casarest -DCMAKE_INSTALL_PREFIX:PATH=$HOME/local -DCASACORE_ROOT_DIR=$HOME/local -DCMAKE_PREFIX_PATH=/usr/include/cfitsio -DBUILD_ALL=1 \ 
-  -DCMAKE_PREFIX_PATH=/usr/include/cfitsio -DBoost_NO_BOOST_CMAKE=YES ../../casarest +  -DBOOST_ROOT=$HOME/local/src/boost_1_63_0 -DCMAKE_CXX_FLAGS="-lboost_thread -lboost_filesystem -L$HOME/local/lib" 
-time make -j16+# not necessary any more -DBoost_NO_BOOST_CMAKE=YES 
 +time make -j
 make install make install
 </file> </file>
Line 169: Line 225:
  
 === 3.5 AOFlagger === === 3.5 AOFlagger ===
-Since LOFAR Offline release 2.15.0 also an installation of the AOFlagger is required. \\ +The [[https://sourceforge.net/projects/aoflagger/ | AOFlagger]] is also required since the  
-A download link and installation instructions are given at https://sourceforge.net/p/aoflagger/wiki/installation_instructions/ \\ +[[http://www.lofar.org/wiki/doku.php?id=public:offline_release_2_15_0 | LOFAR Offline release 2.15.0]]. \\ 
-and http://www.lofar.org/wiki/doku.php?id=public:offline_release_2_15_0+For further details see the LOFAR release link or the  
 +[[https://sourceforge.net/p/aoflagger/wiki/installation_instructions/ | installation instructions]]. \\ 
 +The AOFlagger v2.8.0 and later requires a version of Boost later than 1.41.0. 
 <file> <file>
-cd ~local/src/ +V_AOF=2.9.0  # 2.7.1, 2.8.0 
-tar xf aoflagger-2.7.1.tar.bz2 +cd ~/local/src/ 
-mkdir aoflagger-2.7.1/build/ +wget http://downloads.sourceforge.net/project/aoflagger/aoflagger-$AOVER/aoflagger-$V_AOF.tar.bz2 
-cd aoflagger-2.7.1/build/ +tar xf aoflagger-$V_AOF.tar.bz2 
-cmake -DCMAKE_INSTALL_PREFIX:PATH=$HOME/local -DBUILD_SHARED_LIBS=YES ..+mkdir aoflagger-$V_AOF/build/ 
-time make -j 16+cd aoflagger-$V_AOF/build/ 
 +time cmake ../ -DCMAKE_INSTALL_PREFIX:PATH=$HOME/local -DBUILD_SHARED_LIBS=YES -DBOOST_ROOT=$HOME/local/src/boost_1_63_0 \ 
 +  -DCMAKE_CXX_FLAGS="-lboost_thread -lboost_filesystem -L$HOME/local/lib" 
 +time make -j
 make install make install
 </file> </file>
 +
  
 === 3.6 The LOFAR packages === === 3.6 The LOFAR packages ===
-Finally we can build the LOFAR packages. We can chose between a release or the latest version in trunk/ by commenting in and out the corresponding RELEASE lines. \\ +Finally we can build the LOFAR packages. We can chose between [[http://www.lofar.org/operations/doku.php?id=maintenance:lofar_release_notes | a release]],  
-An overview of available releases is given at http://www.lofar.org/operations/doku.php?id=public:lofar_offline_releases .+[[http://www.lofar.org/operations/doku.php?id=public:lofar_offline_releases | an offline release]]  
 +or the latest version in [[https://svn.astron.nl/LOFAR/trunk/]] by commenting in and out the corresponding RELEASE lines. \\ 
 + 
 +''-DUSE_BACKTRACE=NO'' is necessary for building shared libraries (the default), but libiberty.so is not installed.\\ 
 +(See https://support.astron.nl/lofar_issuetracker/issues/8046). 
 <file> <file>
-cd ~local/src/ +cd ~/local/src/ 
-RELEASE=LOFAR-Release-2_16 && svn checkout --ignore-externals -N https://svn.astron.nl/LOFAR/branches/$RELEASE+RELEASE=LOFAR-Release-2_19 && svn checkout --ignore-externals -N https://svn.astron.nl/LOFAR/branches/$RELEASE 
 +# 2_16_4, 2_16, 2_12
 #RELEASE=LOFAR && svn checkout --ignore-externals -N https://svn.astron.nl/LOFAR/trunk $RELEASE #RELEASE=LOFAR && svn checkout --ignore-externals -N https://svn.astron.nl/LOFAR/trunk $RELEASE
 svn update $RELEASE/CMake svn update $RELEASE/CMake
 mkdir -p build/gnu_opt mkdir -p build/gnu_opt
 cd build/gnu_opt cd build/gnu_opt
-time cmake -DCASACORE_ROOT_DIR=$HOME/local -DBUILD_PACKAGES="DP3 Calibration" -DCMAKE_INSTALL_PREFIX:PATH=$HOME/local +time cmake ../../$RELEASE -DCASACORE_ROOT_DIR=$HOME/local -DBUILD_PACKAGES="DP3 Calibration" -DCMAKE_INSTALL_PREFIX:PATH=$HOME/local -DUSE_BACKTRACE=NO 
-  -DUSE_OPENMP=YES -DUSE_LOG4CPLUS=NO -DUSE_BACKTRACE=NO -DBoost_NO_BOOST_CMAKE=YES ../../$RELEASE +-DUSE_LOG4CPLUS=NO was required in absence of the Log4cplus lib 
-time make -j16+time make -j
 make install make install
 </file> </file>
  • Last modified: 2016-12-22 15:19
  • by Frank Breitling