public:grid_srm_software_installation

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
public:grid_srm_software_installation [2013-06-04 08:22] – [1.1 Obtaining a personal grid certificate] Adriaan Rentingpublic:grid_srm_software_installation [2021-05-19 12:27] (current) – [Certificates for the Grid Certificate Authorities (CA)] Hanno Holties
Line 1: Line 1:
  
 ====== GRID storage access tools installation ====== ====== GRID storage access tools installation ======
 +
 +//This page describes the procedure to set up the tools for working directly on the srm storage. You may want to consider using the staging and download services provided by Astron if you just want to retrieve data from the archive: [[http://www.lofar.org/wiki/doku.php?id=public:lta_howto]].//
 +
 +//You might also be interested in the [[public:srmclientinstallation|portable package of GRID storage access tools]], which offers the same functionality but does not require root access.//
  
 The following documentation was developed for the installation of a GRID tools and certificates on an Ubuntu 12.04 system using a bash shell (using .bashrc as a configuration script). We have not tested the installation using csh derivatives but it is to be expected that not all scripts will work. Thanks to Martin van den Akker for providing notes of his installation procedure. The following documentation was developed for the installation of a GRID tools and certificates on an Ubuntu 12.04 system using a bash shell (using .bashrc as a configuration script). We have not tested the installation using csh derivatives but it is to be expected that not all scripts will work. Thanks to Martin van den Akker for providing notes of his installation procedure.
Line 10: Line 14:
   * [[http://glite.cern.ch|Home of the gLite software suite]] (gLite UI provides a typical client installation)   * [[http://glite.cern.ch|Home of the gLite software suite]] (gLite UI provides a typical client installation)
  
-NB Only the Ubuntu 12 installation described below has been tested by us. Please let us know if you have feedback or if you can contribute instructions on installations in different environments.+NB We have tested the installation as described below only on Ubuntu 12 and CentOS 7. Please let us know if you have feedback or if you can contribute instructions on installations in different environments. 
 +You might also first need to obtain a [[GRID certificate]]. 
 +===== Installation of software packages =====
  
-===== 1 Installation of certificates =====+Note: all installations require root permissions.
  
-==== 1.1 Obtaining a personal grid certificate ====+==== globus client software ====
  
-  * Request a user certificate via the [[https://tcs-escience-portal.terena.org/|Terena eScience portal]] (federated European organisations) or [[http://ca.dutchgrid.nl/start/jgridstart.jnlp|jGridstart]] (other Dutch organisations) and follow the instructions. A good starting point for finding your certificate authority if not available from these sites is: https://www.eugridpma.org/members/worldmap/ (Europe) or http://www.igtf.net/ (World wide). Contact Science Support if none of the above works. +The file transfer tools from the Globus package are needed, most importantly globus-url-copy:
-  * Install the certificate by following the instructions provided by your certificate authority. Typically this involves: +
-  - Save the mail with the signed certificate as ''$HOME/.globus/usercert.pem'' and make sure that the ''userkey.pem'' file in that directory contains the pricate key that matches this certificate. As these ''*.pem'' files are strictly personal, their permissions should be set properly by the following command:\\ ''> chmod 600 *.pem''  +
-  - Load the certificate into your web browser. See e.g. http://ca.dutchgrid.nl/info/browser for instructions. Installation can also be done using the jGridstart tool (see above) for supported browsers. +
-  - Apply for authorization to use LOFAR resources: +
-    * https://voms.grid.sara.nl:8443/voms/lofar for access to the LOFAR LTA storage. +
-  - NB If appropriate, you may be interested in applying for membership of other communities, e.g.: +
-    * https://voms.grid.sara.nl:8443/vomses for other GRID communities hosted in the Netherlands. +
-    * https://cic.gridops.org/index.php?section=home&page=volist for all Grid communities registered in Europe by discipline.+
  
-=== Notes on jGridStart usage ===+Ubuntu 
 +  sudo apt-get install globus-gass-copy-progs
  
-  * It's possible that on OSX Lion or Mountain Lion the jGridStart does not work. This is due to the java support for OSX moving from Apple to Oracle directly. To fix this check the following article http://support.apple.com/kb/HT5559 to get javaws on the commandline so you can use it like this: ''> javaws http://ca.dutchgrid.nl/start/jgridstart.jnlp''. You can also download the latest java for [[http://www.oracle.com/technetwork/java/javase/downloads/index.html|OSX from Oracle]] and  ''open /System/Library/CoreServices'' after that to update your CoreServices (you will just see a new Finder window, nothing seems to happen). +CentOS 
-  * Sometimes the "install into browser" mode of the jGridStart doesn't work properly. In that case you can have jGridStart generate a .p12 file and import that into your browser manually. This seems openssl related as generating the .p12 with openssl on the commandline also seems to fail if this happens.+  sudo yum install globus-gass-copy-progs
  
-===== 2 Installation of software packages =====+==== voms client software ====
  
-Note: all installations require root permissions.+The VOMS tools for logging in and user account management:
  
-==== 2.1 globus client software ====+Ubuntu 
 +  sudo apt-get install voms-clients
  
-The file transfer tools from the Globus package are needed, most importantly globus-url-copy:+CentOS 
 +  sudo yum install voms-clients-cpp
  
-  sudo apt-get install globus-gass-copy-progs+==== Certificates for the Grid Certificate Authorities (CA) ====
  
-==== 2.2 voms client software ====+Execute the following commands to install the certificates from the site of the European Grid Infrastructure (EGI) (root permissions are required).
  
-The VOMS tools for logging in and user account management:+Ubuntu
  
-  sudo apt-get install voms-clients+Note 2021-05-19: newer versions of Ubuntu require to first add the appropriate GPG key for the EGI repository:
  
-==== 2.3 Certificates for the Grid Certificate Authorities (CA) ====+<code> 
 +wget -q -O - https://dist.eugridpma.info/distribution/igtf/current/GPG-KEY-EUGridPMA-RPM-3 | sudo apt-key add -
  
-Execute the following commands to install the certificates from the +</code> 
-site of the European Grid Infrastructure (EGI) (root permissions are +<code>
-required).+
  
-  sudo add-repository 'deb http://repository.egi.eu/sw/production/cas/1/current egi-igtf core' +sudo add-apt-repository 'deb http://repository.egi.eu/sw/production/cas/1/current egi-igtf core' 
-  sudo apt-get update +sudo apt-get update 
-  sudo apt-get install ca-policy-egi-core+sudo apt-get install ca-policy-egi-core
  
-==== 2.4 srmtools ====+</code> 
 + 
 +CentOS 
 + 
 +<code> 
 +wget http://repository.egi.eu/sw/production/cas/1/current/repo-files/EGI-trustanchors.repo 
 +sudo mv EGI-trustanchors.repo /etc/yum.repos.d/ 
 +sudo yum install ca-policy-egi-core 
 + 
 +</code> 
 + 
 + 
 +==== srmtools ====
  
 The SRM tools are needed to communicate with the storage management system. The SRM tools are needed to communicate with the storage management system.
  
   - Download srmtools   - Download srmtools
-    *  {{:public:srm.tar.gz|srm.tar.gz}} +    *  {{public:srmclient-2.6.28.tar.gz|srmclient-2.6.28.tar.gz}}  (Java7, Java8)  
-  - Extract and install the srmtools, e.g. in ''/opt/''\\ This will create a subdirectory ''srm'' containing the required files.\\ Note that the srm package may be installed anywhere (e.g. in your home directory). +    * {{public:srmclient-2.2.25.tar.gz|srmclient-2.2.25.tar.gz}}  (Java6) 
-  - Set the relevant environment path variables, e.g. in .bashrc:\\ ''export SRM_PATH=<Install Directory>/srm''\\ ''export PATH=$SRM_PATH/bin:$PATH''+  - Extract and install the srmtools, e.g. in ''/opt/''\\ This will create a subdirectory, e.g. ''srmclient-2.6.28'', containing the required files.\\ Note that the srm package may be installed anywhere (e.g. in your home directory). 
 +  - Set the relevant environment path variables, e.g. in .bashrc (modify version number if applicable):\\ ''export SRM_PATH=<Install Directory>/srmclient-2.6.28/usr/share/srm''\\ ''export PATH=<Install Directory>/srmclient-2.6.28/usr/bin:$PATH''
  
-NB The srm client tools depend on JAVA. There is a known issue with openjava version 7. If you have this version of JAVA installed, or otherwise get JAVA exceptions when running an srm command, please install another JAVA VM. java-6-openjdk is known to work. It is possible to have multiple JAVA VM installations and it is thus not required to replace an existing installation: if the default VM installation does not work with srm, another VM may be configured to be used by setting the following environment parameter:+NB The srm client tools depend on JAVA. There is a known issue with openjava version 7. If you have this version of JAVA installed, or otherwise get JAVA exceptions when running an srm command, please install another JAVA VM. Java-7-oracle is known to work. It is possible to have multiple JAVA VM installations and it is thus not required to replace an existing installation: if the default VM installation does not work with srm, another VM may be configured to be used by setting the following environment parameter:
  
-  export JAVA_HOME=/usr/lib/jvm/java-6-openjdk-i386/jre+  export JAVA_HOME=/usr/lib/jvm/java-7-oracle/jre
  
-==== 2.5 Certificate Revocation List retrieval (optional) ====+The JAVA VM used by default in Ubuntu and CentOS can be selected using the following command: 
 + 
 +  sudo update-alternatives --config java 
 + 
 +==== Certificate Revocation List retrieval (optional) ====
  
 The fetch-crl tool retrieves Certificate Revocation Lists. The fetch-crl tool retrieves Certificate Revocation Lists.
  
 +Ubuntu
   sudo apt-get install fetch-crl   sudo apt-get install fetch-crl
 +
 +CentOS
 +  sudo yum install fetch-crl
  
 NB This is not required unless you intend to allow others to access your system by providing their grid certificate. NB This is not required unless you intend to allow others to access your system by providing their grid certificate.
  
-===== 3 Additional configuration =====+===== Additional configuration =====
  
-==== 3.1 VOMSES file for LOFAR ====+==== VOMSES file for LOFAR ====
  
-Add the following string for the LOFAR Virtual Organization (VO) to the vomses file.+Add the following string for the LOFAR Virtual Organization (VO) to the vomses file (any filename is fine).
  
   "lofar" "voms.grid.sara.nl" "30019" "/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl" "lofar"   "lofar" "voms.grid.sara.nl" "30019" "/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl" "lofar"
Line 88: Line 110:
 You can find this string also on the following website https://voms.grid.sara.nl:8443/voms/lofar/configuration/configuration.action in the text block under //VOMSES string for this VO//\\ The vomses file should be placed in one of the following default locations: ''/etc/vomses'', ''$HOME/.voms/vomses'', ''$HOME/.glite/vomses'' You can find this string also on the following website https://voms.grid.sara.nl:8443/voms/lofar/configuration/configuration.action in the text block under //VOMSES string for this VO//\\ The vomses file should be placed in one of the following default locations: ''/etc/vomses'', ''$HOME/.voms/vomses'', ''$HOME/.glite/vomses''
  
-==== 3.2 List of certificates for voms.grid.sara.nl.lsc ====+==== List of certificates for voms.grid.sara.nl.lsc ====
  
 Put the following strings: Put the following strings:
Line 101: Line 123:
 NB If this step is skipped or nor configured correctly ''voms-proxy-init'' will work but finish with warnings. NB If this step is skipped or nor configured correctly ''voms-proxy-init'' will work but finish with warnings.
  
-==== 3.3 Environment (optional) ====+==== Environment (optional) ====
  
 You may want to provide the following settings in ''.bashrc'' or another initialisation/startup script if the relevant files are not in the default locations (defaults provided below). You may want to provide the following settings in ''.bashrc'' or another initialisation/startup script if the relevant files are not in the default locations (defaults provided below).
Line 109: Line 131:
   export X509_CERT_DIR=/etc/grid-security/certificates   export X509_CERT_DIR=/etc/grid-security/certificates
   export X509_VOMS_DIR=/etc/grid-security/vomsdir   export X509_VOMS_DIR=/etc/grid-security/vomsdir
 +  export X509_USER_PROXY=$HOME/.proxy
   export VOMS_USERCONF=$HOME/.glite   export VOMS_USERCONF=$HOME/.glite
 +   
 +**Note:** For (t)csh, use *.csh init scripts and 'setenv <key> <value>' instead of 'export <key>=<value>'.
 ==== CRL cron job (optional) ==== ==== CRL cron job (optional) ====
  
-You may want to create a cron job to automatically retrieve certificate revocation lists (CRLs) by incoking the fetch-crl tool at regular intervals (at least once a year).+You may want to create a cron job to automatically retrieve certificate revocation lists (CRLs) by invoking the fetch-crl tool at regular intervals (at least once a year). 
 + 
 +===== Usage ===== 
 + 
 +This creates a proxy (valid for 48 hours, increase if needed) in your home directory: 
 +<file> 
 +voms-proxy-init -valid 48:00 -voms lofar:/lofar/user -out ~/.proxy 
 +</file> 
 + 
 +You can test that everything works by copying this file from surfsara to your working directory: 
 +<file> 
 +srmcp -server_mode=passive srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/lofar/ops/fifotest/file1M file://`pwd`/file1M 
 +</file> 
 + 
 +If your firewall allows incoming connections to non-standard ports, you can try this command without the server_mode option which will enable utilization of multiple streams to increase performance. 
 + 
 +If you have the [[public:grid_srm_software_installation#globus_client_software|gridftp client software]] installed and in your path, it provides superior performance as compared to the native JAVA gridftp client that is provided by srmcp. In order to utilize this, download {{:public:lta-url-copy.sh.gz|lta-url-copy.sh.gz}}, unzip it and use the command: 
 +<file> 
 +srmcp -use_urlcopy_script=true -urlcopy=./lta-url-copy.sh -server_mode=passive srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/lofar/ops/fifotest/file1M file://`pwd`/file1M 
 +</file> 
 +**Note:** You may have to force the use of TLS, (export GLOBUS_GSSAPI_FORCE_TLS=1 or in /etc/grid-security/gsi.conf set FORCE_TLS=true) to make this work.
  • Last modified: 2013-06-04 08:22
  • by Adriaan Renting