Differences

This shows you the differences between two versions of the page.

--- public:lta_tricks [2019-09-20 07:25] – Updated link to LOFAR stager API Python module to point to the ASTRON gitlab repo Thomas Jürges
+++ public:lta_tricks [2020-04-10 10:08] – [Examples] Reinoud Bokhorst
@@ Line 140: / Line 140: @@
 In case of errors, there may be the need to open some port on the firewall at your institution. Specifically, port 1521 should be open. Also make sure that the LTA client library can be found in your PYTHONPATH (see [[:lta:client_installation|LTA Client installation]] for more details). In case of trouble, get in contact with Science Operations and Support.
-== Examples ==
+==== Examples ====
 Once you have tested that your connection to the catalogue is working, you are ready to browse the archive and stage the data you need. Here we will list a few examples of python scripts that can be used to access the LTA. All of them will need to import some modules:
@@ Line 155: / Line 155: @@
 The lines above must be added to each of the scripts below for these to work.
-This simple script will allow you to find all data within a single project, for example LC2_035. Please change the project name to the code of a project of yours. If you also want to stage the data you found, just set the do_stage variable to True. Be careful with how many files you stage and what size they have: the same limits as for the web interface apply here.
+=== Ex: get staging URI's ===
+This script will allow you to find all data within a single project, for example LC2_035. Please change the project name to the code of a project of yours. If you also want to stage the data you found, just set the do_stage variable to True. Be careful with how many files you stage and what size they have: the same limits as for the web interface apply here.
 <code python>
@@ Line 197: / Line 199: @@
 </code>
+=== Ex: filter on subbands ===
 The following script will find subbands 301 and 302 for all targets within two different projects.
@@ Line 252: / Line 256: @@
         stager.stage_uris(uris[project])
 </code>
+=== Ex: filter on frequency and observation date ===
 Here, we find data between freq1 and freq2 taken within one project between day1 and day2
@@ Line 302: / Line 308: @@
 </code>
-== Example; query public data ==
+=== Ex: query public data ===
 Querying public data in projects you are not member of. First set project ALL, then construct a query and optionally limit the query to a certain project :
@@ Line 314: / Line 320: @@
 </code>
+=== Ex: get release dates ===
+<code python>
+from awlofar.main.aweimports import Observation, PipelineRun, DataProduct
+from common.database.Context import context
+project = 'LC2_035'
+# Query for private data of the project, you must be member of the project
+private_data = True
+# To see private data of this project, you must be member of this project
+if private_data :
+    context.set_project(project)
+    if project != context.get_current_project().name:
+        raise Exception("You are not member of project %s" % project)
+# Observations
+query_observations = Observation.select_all().project_only(project)
+for observation in query_observations :
+    print("Querying ObservationID %s, %s" % (observation.observationId, observation.releaseDate))
+# Pipelines
+query_pipelines = PipelineRun.select_all().project_only(project)
+for pipeline in query_pipelines :
+    print("Pipeline: %s, %s, %s" % (type(pipeline).__name__, pipeline.pipelineName, pipeline.releaseDate))
+# Data products
+query_products = DataProduct.select_all().project_only(project)
+query_products &= DataProduct.isValid == 1
+for product in query_products :
+    print("Product: %s, %s, %s, %s" % (product.dataProductIdentifier, product.dataProductIdentifierSource, product.dataProductType, product.releaseDate))
+</code>
 ===== Python Module for Staging =====