| Both sides previous revision Previous revision Next revision | Previous revision |
| public:srmclientinstallation [2015-05-07 14:24] – Joern Kuensemoeller | public:srmclientinstallation [2017-03-08 15:27] (current) – external edit 127.0.0.1 |
|---|
| **Note** that the previouly provided ''proxy-init.sh'' script from [[http://code.google.com/p/jlite/|jLite]] is now discouraged because its encryption strength of 512 bit is not considered sufficient any more. SRM sites require a minimum strength of 1024 bit. | **Note** that the previouly provided ''proxy-init.sh'' script from [[http://code.google.com/p/jlite/|jLite]] is now discouraged because its encryption strength of 512 bit is not considered sufficient any more. SRM sites require a minimum strength of 1024 bit. |
| |
| To allow usage of the LOFAR VO (Virtual Observatory), there are three additional steps to take: | To allow usage of the LOFAR VO (Virtual Organization), there are three additional steps to take: |
| |
| * Create a ''vomses'' file to allow ''voms-proxy-init'' to contact the relevant VOMS server | * Create a ''vomses'' file to allow ''voms-proxy-init'' to contact the relevant VOMS server |
| === Active gridftp === | === Active gridftp === |
| |
| In the examples above, srmcp is run with the option -server_mode=passive, which limits the transfer to a single stream. If you want to enable 'active' transfers, your firewall has to allow **incoming** access to the ports configured as the globus port range (typically ports 8443, 8444, 2811 and 20000-25000; look in client documentation for more details). | In the examples above, srmcp is run with the option -server_mode=passive, which limits the transfer to a single stream. If you want to enable 'active' transfers, your firewall has to allow **incoming** access to the ports configured as the globus port range (typically ports 20000-25000, also open 8443, 8444, 2811). |
| The IP ranges for remote gridftp servers that need to be able to connect to your machine are: | The IP ranges for remote gridftp servers that need to be able to connect to your machine are: |
| |
| * 145.100.32.0/22, i.e. 145.100.32.0 to 145.100.35.255, for SURFsara | * 145.100.32.0/22, i.e. 145.100.32.0 to 145.100.35.255, for SURFsara |
| * 134.94.*.* for FZJuelich. | * 134.94.32.0/22, i.e. 134.94.32.0 to 134.94.32.255, for FZJuelich. |
| |
| Active gridftp can improve performance of a single transfer as it will use multiple parallel connections for retrieving a file. For the FNAL/dCache client, 'active' transfers are initiated if the ''-server_mode=passive'' setting is omitted. For the Berkely client, parallel transfers will be initiated when the ''-parallelism'' parameter is set to a value larger than 1. Since most cases LOFAR datasets consist of a large number of files, a similar performance improvement can be achieved by splitting the set of files over multiple srm copy processes. This is usually easier to set up than the firewall requirements. Note that e.g. the dCache client does not have a default setting for the gridftp port range. Further, srmcp ignores the GLOBUS_TCP_PORT_RANGE environment variable. You have to specify the port range (that you opened in your firewall) via the ''globus_tcp_prt_range'' option of srmcp, e.g.: | Active gridftp can improve performance of a single transfer as it will use multiple parallel connections for retrieving a file. For the FNAL/dCache client, 'active' transfers are initiated if the ''-server_mode=passive'' setting is omitted. For the Berkely client, parallel transfers will be initiated when the ''-parallelism'' parameter is set to a value larger than 1. Since most cases LOFAR datasets consist of a large number of files, a similar performance improvement can be achieved by splitting the set of files over multiple srm copy processes. This is usually easier to set up than the firewall requirements. Note that e.g. the dCache client does not have a default setting for the gridftp port range. Further, srmcp ignores the GLOBUS_TCP_PORT_RANGE environment variable. You have to specify the port range (that you opened in your firewall) via the ''globus_tcp_prt_range'' option of srmcp, e.g.: |
| * Untar package in directory of your choosing:\\ <code>tar -xvzf lofar_grid_clients.tar.gz</code> | * Untar package in directory of your choosing:\\ <code>tar -xvzf lofar_grid_clients.tar.gz</code> |
| * Determine your java version:\\ <code>java -version</code> | * Determine your java version:\\ <code>java -version</code> |
| * Source init.sh (Java 7) or init_java6 (Java 6) in lofar_grid/, e.g. :\\ <code>. lofar_grid/init.sh</code> | * Source init.sh (Java 7 or 8) or init_java6 (Java 6) in lofar_grid/, e.g. :\\ <code>. lofar_grid/init.sh</code> |
| * Update the certificates with one of the provided scripts, e.g.: \\ <code>. update_certificates_eugridpma.sh</code> | * Update the certificates with one of the provided scripts, e.g.: \\ <code>. update_certificates_eugridpma.sh</code> |
| * Optional: Set proxy environment variable to custom location:\\ <code>export X509_USER_PROXY=<proxy_location></code> | * Optional: Set proxy environment variable to custom location:\\ <code>export X509_USER_PROXY=<proxy_location></code> |
| * Generate a proxy:\\ <code>voms-proxy-init -voms lofar:/lofar/user</code> | * Generate a proxy:\\ <code>voms-proxy-init -voms lofar:/lofar/user</code> |
| * Test data retrieval:\\ <code>srmcp -server_mode=passive srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/lofar/ops/fifotest/file1M file:///file1M</code> | * Test data retrieval:\\ <code>srmcp -server_mode=passive srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/lofar/ops/fifotest/file1M file://`pwd`/file1M</code> |
| * Done!\\ // NB If you modified any default location by the ''export'' command, you have to put it in a shell start-up script like '.bashrc' to make your changes permanent, of course (with full paths where appropriate).// | * Done!\\ // NB If you modified any default location by the ''export'' command, you have to put it in a shell start-up script like '.bashrc' to make your changes permanent, of course (with full paths where appropriate).// |
| * If you get any errors <del>related to CA certificates</del>, retry after running one of the provided scripts to update your certificates, e.g.\\ <code>. update_certificates_eugridpma.sh</code> The certificates change every now and then, and then you need to update them. | * If you get any errors <del>related to CA certificates</del>, retry after running one of the provided scripts to update your certificates, e.g.\\ <code>. update_certificates_eugridpma.sh</code> The certificates change every now and then, and then you need to update them. |
| |
| **Note:** For (t)csh, use *.csh init scripts and 'setenv <key> <value>' instead of 'export <key>=<value>'. | **Note:** For (t)csh, use *.csh init scripts and 'setenv <key> <value>' instead of 'export <key>=<value>'. |
| |
| ====== Troubleshoot ===== | ====== Troubleshoot ===== |
| |
| * There is a [[public:lta_faq|LTA FAQ page]] that should help with the common difficulties. | * There is a [[public:lta_faq|LTA FAQ page]] that should help with the common difficulties. |