Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
public:lta_howto [2018-07-10 09:38]
Reinoud Bokhorst
public:lta_howto [2018-07-11 08:32] (current)
Reinoud Bokhorst
Line 6: Line 6:
  
 For background information and in case of problems, please refer to the [[public:​lta_faq|Frequently Asked Questions]] page. For background information and in case of problems, please refer to the [[public:​lta_faq|Frequently Asked Questions]] page.
 +
 +====== Release Notes ======
 +
 +
 +^ Release ^ Description ^
 +| July 2018 | 1) All data is now searchable, not only released data or data of projects you're a member of. Note that for downloading data (staging) the proprietary restrictions still apply. \\ 2) All projects can now be selected by all users, not only by project members. This provides the means to filter data based on project.\\ 3) The user interface has been changed to improve user friendliness. Key points are a more intuitive menu structure, improved project selection/​deselection and restyling. \\ 4) Implemented cone search algorithm based on the Haversine formula for angular distance calculation. The calculated angular distance to the reference coordinates is now displayed in the search results. \\ 5) Search keys added for "time resolution",​ "​Frequency resolution"​ and "nr. of subbands per subband group" \\ 6) Search keys removed: "sky model database",​ strategy name" |
 +
 +====== Public vs Proprietary data ======
  
 ====== User Access ====== ====== User Access ======
Line 13: Line 21:
 Please visit https://​webportal.astron.nl/​pwm/​public/​ForgottenPassword to request a new one. Please visit https://​webportal.astron.nl/​pwm/​public/​ForgottenPassword to request a new one.
  
-=== Public ​data ===+=== Searching / retrieving ​data ===
  
-The LTA catalogue can be searched ​using the anonymous ​account ​(AWWORLD)This account gives direct access ​to the project "All public data", ​and search queries will return results of the entire catalogue because metadata are public for all LTA content.+The LTA catalogue can be searched ​directly without needing any account. ​Access ​to all projects ​and search queries will return results of the entire catalogue because metadata are public for all LTA content.
  
-Staging and subsequent downloading of public data always requires an account ​(if you have it, this is your MoM account) ​with LTA user access. If you do not have an account yet, you can register with "​[[https://​lofar.astron.nl/​useradministration/​public/​setUpUserAccount.do|Create account]]"​. LTA access privileges will need to be granted by Science Operations and Support. Please, __once you have created your account, ​contact Science Operations and Support ​asking for LTA privileges__. ​+Staging and subsequent downloading of __**public**__ data always requires an account with LTA user access ​privileges. This automatically happens if you were a member of the original project proposal in Northstar/​MoM. If you do not have an account yet, you can register with "​[[https://​lofar.astron.nl/​useradministration/​public/​setUpUserAccount.do|Create account]]"​. LTA access privileges will need to be granted by Science Operations and Support. Please, __once you have created your account, ​submit a support request to the [[https://​support.astron.nl/​rohelpdesk|Radio Observatory helpdesk]] ​asking for LTA privileges__.
  
-Step-by-step guide to access public ​data:  +To stage and retrieve project-related ​data in the LTA which are __**proprietary**__ you need to have an account in [[https://​lofar.astron.nl/mom3|MoM]] that is enabled for the archive and coupled with the projects of interest. To this aim you can request Science Operations & Support to be added to the list of co-authors of the project. When you send such a request, you must add the project'​s ​PI in cc. After Science Operations ​and Support adds you to the project, you might get an email asking you to set a new password in [[https://​webportal.astron.nl/​pwm/​private/​Login|ASTRON Web Applications Password Self Service]]. ​ Please note that this will set a new password not just for the LTA //but for MoM (LOFAR/​WSRT) and Northstar as well//.
-  ​log in to https://lta.lofar.eu +
-  * click SELECT PROJECT in the top menu, then select 'All Public data' from the list of project groupings +
-  * click SEARCH in the top menu +
-  * click on "​here"​ (more advanced search option) +
-  * choose “All Observations and Pipelines” +
-  * find the relevant observation by either:  +
-    * choosing ​the project ​code from the dropdown list (if you know which project ​your observation of interest was taken under) +
-    * specify the 'observation id' of the observation(sand/or pipeline(s) that you are interested in +
-  * click the '​Search'​ button at the bottom +
-  * from the screen that follows, you should be able to stage the dataproducts+
  
-=== Project ​data ===+=== Step-by-step guide to search and retrieve ​data ===
  
-To directly access project-related data in the LTA you need to have an account in [[https://​lofar.astron.nl/mom3|MoM]] that is enabled for the archive. This automatically happens if you were a member ​of the original project proposal in Northstar/​MoM.+__Basic search__: 
 +  * log in to https://lta.lofar.eu 
 +  * click SEARCH DATA in the top menu 
 +  * specify the data product types of interest and a target name or coordinated 
 +  * click on "​Search"​ button at the bottom 
 +  * from the screen that follows, you should be able to stage the data products
  
-If you were not included among the co-authors of the project but are still a collaborator,​ you can request to be added by Science Operations and Support. When you send such a request, you must add the project'​s PI in cc. After Science Operations and Support adds you to the project, you might get an email asking you to set a new password in [[https://webportal.astron.nl/pwm/​private/​Login|ASTRON Web Applications Password Self Service]]. ​ Please note that this will set new password not just for the LTA //but for MoM (LOFAR/​WSRT) and Northstar as well//.+__Advanced search__: 
 +  * log in to https://lta.lofar.eu 
 +  * click SEARCH DATA in the top menu 
 +  * click on the side Advanced Search drop-down list 
 +  * specify the data product types of interest from the drop down list 
 +  * select products features and specify ​target name or coordinated 
 +  * click on "​Search"​ button at the bottom 
 +  * from the screen that follows, you should be able to stage the data products
  
 +__Project search (to restrict all data searches to that project only)__:
 +  * log in to https://​lta.lofar.eu/ ​
 +  * click BROWSE PROJECTS in the top menu
 +  * at this level membership can be checked, with the first column showing if you are a member of the project or for finding public projects. Available options are:
 +    * click on the project name to view the project details
 +    * use the '​search'​ button to select the project and go to the search page
 +    * use the 'show data' button to select the project and to show all data in it
 +  * from the screen that follows, you should be able to either search / select / stage the data products
 ====== How to find data in the archive ====== ====== How to find data in the archive ======
  
-Once your account is set up, or as anonymous user (AWWORLD) ​you can navigate the catalogue.+Once your account is set up, or as anonymous user you can navigate the catalogue. In the former case you can login by clicking on the top right LOGIN button shown below.
  
 === Page navigation === === Page navigation ===
Line 46: Line 64:
  
  
-{{:​public:​lta_menu.png|}} +{{:​public:​lta_menu.png?900|}}
- +
-A "​project"​ can be selected by registered users that have access to a specific project. Subsequent searches will then be done within that project. +
- +
-Searching the catalogue is easiest by selecting one of the otions in the "​Search"​ menu. Depending on the knowledge of the data you are looking for (if any), searches can be performed on e.g., target ("​Simple Search"​),​ date and/or observation-ID and more advanced parameters per data type ("​Advanced Search"​).+
  
-Another option is to choose "Show Latest",​ which will show the latest additions ​in the LTA. This is particularly useful for project related searches.+A search ​in the LTA catalogue can be initiated by clicking on the SEARCH DATA button on the menuAt this point a default basic search ​is setup, where users can select the data product type of interest and perform a cone search. An advanced search mode, with more advanced parameters per data type, can also be selected by clicking on the drop menu on the left side
  
 +A "​project"​ can be selected by clicking on the BROWSE PROJECTS button on the menu. This is particularly useful for project related searches, as well as for checking either public projects or user's co-author membership. At this stage several actions are allowed
 +    * Click on the project name to view the project details and eventually select it.
 +    * Use the '​search'​ button to select the project and go to the search page. Subsequent searches will then be done within that project.
 +    * Use the 'show data' button to select the project and to show all data in it.
  
 === Finding Data === === Finding Data ===
  
-{{:public:lta_observations_2017.png?900|}}+{{:public:lta_observations_2018.png?900|}}
  
 Depending on the search parameters, e.g., which data products were requested (observation,​ pipeline), lists of observations and/or pipelines will be returned (see observations example above). From this point there are several options: Depending on the search parameters, e.g., which data products were requested (observation,​ pipeline), lists of observations and/or pipelines will be returned (see observations example above). From this point there are several options:
  
-  - select observations/​pipelines and stage (prepare for download) all date related to the selection+  - select observations/​pipelines and stage (prepare for download) all data related to the selection
   - select observations and "show pipelines"​ related to the observations,​ then select pipelines and stage   - select observations and "show pipelines"​ related to the observations,​ then select pipelines and stage
   - select observations/​pipelines,​ "show dataproducts"​ related to the selection, possibly filter the data products (to have smaller selections) and then select and stage the data products   - select observations/​pipelines,​ "show dataproducts"​ related to the selection, possibly filter the data products (to have smaller selections) and then select and stage the data products
Line 86: Line 104:
 Once you have a list of dataproducts,​ observations or pipelines, you can use the check boxes to select which files you want to download. The first check box can be used to select or deselect all files or observations on a page. Once you have a list of dataproducts,​ observations or pipelines, you can use the check boxes to select which files you want to download. The first check box can be used to select or deselect all files or observations on a page.
  
-{{:​public:​lta_staging_1.png|}}+{{:​public:​lta_staging_1.png?900|}}
  
 The LOFAR Archive stores data on magnetic tape. This means that it cannot be downloaded right away, but has to be copied from tape to disk first. This process is called '​staging'​. The LOFAR Archive stores data on magnetic tape. This means that it cannot be downloaded right away, but has to be copied from tape to disk first. This process is called '​staging'​.
Line 92: Line 110:
 When you have made your selection of files, click on //stage//. This shows you the following message. It means that a request has been sent to the LTA staging service to start retrieving the requested files from the tape and make them available on disk. You will get a confirmation e-mail, to acknowledge that your staging request was received and the process was queued. When the files are staged, you will get a notification email informing you that your data are ready for retrieval. When you have made your selection of files, click on //stage//. This shows you the following message. It means that a request has been sent to the LTA staging service to start retrieving the requested files from the tape and make them available on disk. You will get a confirmation e-mail, to acknowledge that your staging request was received and the process was queued. When the files are staged, you will get a notification email informing you that your data are ready for retrieval.
  
-{{:public:lta_howto3.png|}}+{{:public:lta_howto_3.png?900|}}
  
 The e-mail that you get when the staging on disk is complete gives you a list of files and has several attachments. Amongst them are two files ''​html.txt''​ and ''​srm.txt'':​ The e-mail that you get when the staging on disk is complete gives you a list of files and has several attachments. Amongst them are two files ''​html.txt''​ and ''​srm.txt'':​
Line 241: Line 259:
 ===== Troubleshooting ===== ===== Troubleshooting =====
  
-  * There is a [[public:​lta_faq|LTA FAQ page]] that should help with the common difficulties. +  * There is a [[public:​lta_faq|LTA FAQ page]] that should help with the common difficulties. ​
-  * [[public:​lta_new|Update July 11, 2018]]+
  • Last modified: 2018-07-11 08:32
  • by Reinoud Bokhorst