Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
public:lta_tricks [2020-04-10 10:08] – [Examples] Reinoud Bokhorst | public:lta_tricks [2023-07-17 08:51] (current) – Robbie Luijben | ||
---|---|---|---|
Line 51: | Line 51: | ||
There is a server that gives the option to run your own queries on the database [[https:// | There is a server that gives the option to run your own queries on the database [[https:// | ||
- | A useful query might be this one, that gives you all files for a certain Obs Id (SAS VIC tree ID).< | + | A useful query might be this one, that gives you all files for a certain Obs Id (SAS VIC tree ID). |
+ | < | ||
SELECT fo.URI, dp." | SELECT fo.URI, dp." | ||
| | ||
Line 61: | Line 63: | ||
AND fo.data_object = dp." | AND fo.data_object = dp." | ||
AND dp." | AND dp." | ||
+ | |||
</ | </ | ||
Line 77: | Line 80: | ||
AND fo.data_object = dp." | AND fo.data_object = dp." | ||
AND dp." | AND dp." | ||
+ | |||
</ | </ | ||
Line 85: | Line 89: | ||
Once you have installed the client, set up your user name and password. These are the same as for MoM. Remember that this is just a different interface to the LTA catalogue: you will need the same credentials as for the web interface. | Once you have installed the client, set up your user name and password. These are the same as for MoM. Remember that this is just a different interface to the LTA catalogue: you will need the same credentials as for the web interface. | ||
- | After installing the LTA client, the file .awe/ | + | After installing the LTA client, the file .awe/ |
+ | < | ||
[global] | [global] | ||
database_user | database_user | ||
database_password | database_password | ||
+ | |||
</ | </ | ||
Line 120: | Line 127: | ||
pprint(result) | pprint(result) | ||
+ | |||
</ | </ | ||
Line 134: | Line 142: | ||
Pointing found RA 95.561458 DEC 24.584056 | Pointing found RA 95.561458 DEC 24.584056 | ||
..etc.. | ..etc.. | ||
+ | |||
</ | </ | ||
Line 151: | Line 160: | ||
Observation | Observation | ||
from awlofar.toolbox.LtaStager import LtaStager, LtaStagerError | from awlofar.toolbox.LtaStager import LtaStager, LtaStagerError | ||
+ | |||
+ | |||
</ | </ | ||
Line 197: | Line 208: | ||
stager = LtaStager() | stager = LtaStager() | ||
stager.stage_uris(uris) | stager.stage_uris(uris) | ||
+ | |||
</ | </ | ||
Line 255: | Line 267: | ||
for project in (project1, project2) : | for project in (project1, project2) : | ||
stager.stage_uris(uris[project]) | stager.stage_uris(uris[project]) | ||
+ | |||
+ | |||
</ | </ | ||
Line 305: | Line 319: | ||
stager = LtaStager() | stager = LtaStager() | ||
stager.stage_uris(uris) | stager.stage_uris(uris) | ||
+ | |||
</ | </ | ||
Line 318: | Line 333: | ||
print(len(query)) | print(len(query)) | ||
# 1800 | # 1800 | ||
+ | |||
+ | |||
</ | </ | ||
Line 346: | Line 363: | ||
for pipeline in query_pipelines : | for pipeline in query_pipelines : | ||
print(" | print(" | ||
- | |||
# Data products | # Data products | ||
Line 353: | Line 369: | ||
for product in query_products : | for product in query_products : | ||
print(" | print(" | ||
+ | |||
+ | |||
</ | </ | ||
+ | |||
===== Python Module for Staging ===== | ===== Python Module for Staging ===== | ||
- | The python interaction with the LTA catalog can be complemented with the use of a specific module developed to give users more control over their staging requests. The module is made available **[[https:// | + | The python interaction with the LTA catalog can be complemented with the use of a specific module developed to give users more control over their staging requests. |
- | **Alternatively to the .awe/Environment.cfg described above, user credentials | + | Current released version 2.0 ([[http:// |
- | user=XXX | + | |
- | password=YYY | + | * User documentation for __//stageit //__ can be found at: [[https:// |
- | </code> | + | * Version 2.0 release |
+ | |||
+ | ==== Version 2.0 usage notes ==== | ||
+ | |||
+ | The module is made available [[http:// | ||
+ | |||
+ | __Notes: | ||
+ | |||
+ | * You need an access token to the stageit api. Please refer to the user guide linked above to sign up and login to stageit. After logging in, a token can be obtained in one of two ways: | ||
+ | | ||
+ | | ||
+ | * The token is valid indefinitely. Requesting a token multiple times will yield the same token. | ||
+ | * Make sure the token is available in your ~/.stagingrc file: | ||
+ | * api_token=YOUR_TOKEN_HERE | ||
+ | * remove the old username and password | ||
+ | * The script is Python2 compatible, there is a Dockerfile available for Python2 testing in '' | ||
+ | * The requests library is a required dependency. If you care about Python2 compatability, | ||
+ | |||
+ | Also note that some functions are not supported in the new LTA stager. The states that a request can be in have been simplified. As such, there is no need for these functions anymore. Upon use, they will display an error stating that the function is deprecated. Please look at the '' | ||
+ | |||
+ | ==== Functionality ==== | ||
For a description of what the user can do, we list here the functions that are available. | For a description of what the user can do, we list here the functions that are available. | ||
Line 380: | Line 419: | ||
Below is an example of how to use this: | Below is an example of how to use this: | ||
- | + | < | |
- | < | + | > python |
Python 2.7.10 (default, Oct 23 2015, 19:19:21) | Python 2.7.10 (default, Oct 23 2015, 19:19:21) | ||
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin | [GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin | ||
Line 390: | Line 429: | ||
+ 12227 | + 12227 | ||
- | - File count -> 100 | + | - File count |
- Files done -> 40 | - Files done -> 40 | ||
- Flagged abort -> false | - Flagged abort -> false | ||
Line 397: | Line 436: | ||
- Status | - Status | ||
- User id -> 1919 | - User id -> 1919 | ||
+ | |||
</ | </ | ||