public:lta_tricks

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
public:lta_tricks [2020-07-21 13:53] – Updated link for Python stager API releases Thomas Jürgespublic:lta_tricks [2023-07-17 08:51] (current) Robbie Luijben
Line 160: Line 160:
     Observation     Observation
 from awlofar.toolbox.LtaStager import LtaStager, LtaStagerError from awlofar.toolbox.LtaStager import LtaStager, LtaStagerError
 +
  
 </code> </code>
Line 266: Line 267:
     for project in (project1, project2) :     for project in (project1, project2) :
         stager.stage_uris(uris[project])         stager.stage_uris(uris[project])
 +
  
 </code> </code>
Line 331: Line 333:
 print(len(query)) print(len(query))
 # 1800 # 1800
 +
  
 </code> </code>
Line 366: Line 369:
 for product in query_products : for product in query_products :
     print("Product: %s, %s, %s, %s" % (product.dataProductIdentifier, product.dataProductIdentifierSource, product.dataProductType, product.releaseDate))     print("Product: %s, %s, %s, %s" % (product.dataProductIdentifier, product.dataProductIdentifierSource, product.dataProductType, product.releaseDate))
 +
  
 </code> </code>
Line 371: Line 375:
 ===== Python Module for Staging ===== ===== Python Module for Staging =====
  
-The python interaction with the LTA catalog can be complemented with the use of a specific module developed to give users more control over their staging requests. The module is made available **[[https://git.astron.nl/ro/lofar_stager_api/-/releases|here]]** and its functions are mostly self-explanatory.+The python interaction with the LTA catalog can be complemented with the use of a specific module developed to give users more control over their staging requests.
  
-**Alternatively to the .awe/Environment.cfg described above, user credentials can also be provided via a file ~/.stagingrc with credentials of your Lofar account**, similar to ./wgetrc: +Current released version 2.0 ([[http://git.astron.nl/astron-sdc/lofar_stager_api/-/releases/2.0|tagged on master branch]]) is to be used with the new LTA stager (stageit), it is not backwards compatible with the old LTA stager. Older versions of this script (i.e., 1.7 and older) have become obsolete. Please see **"Version 2.0 usage notes"**  listed below for documentation (or check the README file in the repository linked above).
-<code>+
  
-  user=XXX +  * User documentation for __//stageit //__  can be found at: [[https://support.astron.nl/confluence/display/SDCP/User+documentation|https://support.astron.nl/confluence/display/SDCP/User+documentation]] 
-  password=YYY+  * Version 2.0 release can be found at: [[https://git.astron.nl/astron-sdc/lofar_stager_api/-/releases/2.0|https://git.astron.nl/astron-sdc/lofar_stager_api/-/releases/2.0]]
  
-</code>+==== Version 2.0 usage notes ==== 
 + 
 +The module is made available [[http://git.astron.nl/astron-sdc/lofar_stager_api/-/releases/2.0|here]] (tagged on master branch). Simply checkout the tagged commit and use the script. Please see the last note in the list below with regards to required dependencies. 
 + 
 +__Notes:__ 
 + 
 +  * You need an access token to the stageit api. Please refer to the user guide linked above to sign up and login to stageit. After logging in, a token can be obtained in one of two ways: 
 +      * Visit [[https://sdc.astron.nl/stageit/api/staging/get-token|https://sdc.astron.nl/stageit/api/staging/get-token]] 
 +      * From anywhere in the application, click on your account name in the top right to access your profile. From your profile page, click the "Request token" button to receive a token. 
 +  * The token is valid indefinitely. Requesting a token multiple times will yield the same token. 
 +  * Make sure the token is available in your ~/.stagingrc file: 
 +      * api_token=YOUR_TOKEN_HERE 
 +      * remove the old username and password from the ''.stagingrc''  file 
 +  * The script is Python2 compatible, there is a Dockerfile available for Python2 testing in ''./tests/docker'' 
 +  * The requests library is a required dependency. If you care about Python2 compatability, you can use at most version 2.22.0 of requests. Otherwise, you can install any version (note: you can also ''pip install -r 'requirements.txt''', which will install version 2.22.0) 
 + 
 +Also note that some functions are not supported in the new LTA stager. The states that a request can be in have been simplified. As such, there is no need for these functions anymore. Upon use, they will display an error stating that the function is deprecated. Please look at the ''stager_access.py''  file for more information. 
 + 
 +==== Functionality ====
  
 For a description of what the user can do, we list here the functions that are available. For a description of what the user can do, we list here the functions that are available.
Line 398: Line 419:
  
 Below is an example of how to use this: Below is an example of how to use this:
- +<code> 
-<code>> python+> python
 Python 2.7.10 (default, Oct 23 2015, 19:19:21) Python 2.7.10 (default, Oct 23 2015, 19:19:21)
 [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Line 408: Line 429:
  
 + 12227 + 12227
-  - File count      ->     100+  - File count   ->     100
   - Files done      ->     40   - Files done      ->     40
   - Flagged abort      ->     false   - Flagged abort      ->     false
  • Last modified: 2020-07-21 13:53
  • by Thomas Jürges