| Both sides previous revision Previous revision Next revision | Previous revision |
| cep3:start [2017-05-11 12:34] – [CEP3 Cluster] Arno Schoenmakers | cep3:start [2021-02-19 14:21] (current) – Marco Iacobelli |
|---|
| ====== CEP3 Cluster ====== | ====== CEP3 Cluster ====== |
| |
| //**IMPORTANT NOTICE: CEP3 will be going down for maintenance and upgrade activities between May 15th and June 5th 2017!!!!**// | This is the entry point for all CEP3 related information. CEP3 is a processing cluster that will be configured as a managed cluster. |
| | |
| This is the entry point for all CEP3 related information. CEP3 is the new processing cluster that will be configured as a managed cluster. | |
| |
| ====== CEP3 usage policy ====== | ====== CEP3 usage policy ====== |
| |
| **The use of CEP3 cluster is allocated by the LOFAR Programme Committee and the ILT director during the regular proposal evaluation stages, or under Director's Discretionary Time. Access and use of CEP3 cluster is under the sole control of the Radio Observatory's Science Operations and Support Group (SSG). Access for Users will be granted only at the discretion of the Science Operations and Support Group. Users should conform to the access, resource allocation and data deletion policies issued by the SSG at all times.** | **The use of CEP3 cluster is allocated by the LOFAR Programme Committee and the ILT director during the regular proposal evaluation stages, or under Director's Discretionary Time. Access and use of CEP3 cluster is under the sole control of the ASTRON Science Data Centre Operations (SDCO) team. Access for Users will be granted only at the discretion of the SDCO staff. Users should conform to the access, resource allocation and data deletion policies issued by the SDCO at all times.** |
| |
| **To have access to CEP3 a formal request must be submitted to sos@astron.nl or be explicitly given in a proposal for LOFAR observing time. When submitting a request users should clearly include the following information:** | **To have access to CEP3 a formal request must be submitted via the ticketing system at ** **[[https://support.astron.nl/sdchelpdesk|https://support.astron.nl/sdchelpdesk]]**** or be explicitly given in a proposal for LOFAR observing time. When submitting a request users should clearly include the following information:** |
| * **A brief explanation of why access to CEP3 is required (e.g., you do not have access to suitable computing resources elsewhere)** | |
| | * **A brief explanation of why access to CEP3 is required (e.g., you do not have access to suitable computing resources elsewhere)** |
| * **Project to be worked on (i.e. commissioning, cycle or archived data (post) processing)** | * **Project to be worked on (i.e. commissioning, cycle or archived data (post) processing)** |
| * **Description of the kind of processing for which CEP3 access is requested** | * **Description of the kind of processing for which CEP3 access is requested** |
| * **General description of the data to be processed (e.g. data size)** | * **General description of the data to be processed (e.g. data size)** |
| * **Estimated processing time required** | * **Estimated processing time required** |
| | **Users awarded with access to CEP3 will be able to access the cluster for a limited period of time (__8 weeks by default__). The awarded period starts following the timeline communicated to the user via the SDCO notification and available at [[:cep3:earlyaccess|]]. At the beginning of a Cycle, users requesting CEP3 processing time in their observing proposals can derive this timeline by checking the observing schedule, which is available [[https://old.astron.nl/radio-observatory/observing/cycles-allocations-and-observing-schedules/cycles-allocations-and-observ|here]]****. Access timelines related to observing programs involving observations spread in time will be discussed between the PI and SDCO staff. In general, info about CEP3 access of users are detailed at [[:cep3:earlyaccess|]]** |
| |
| **Users awarded with access to CEP3 will be able to access the cluster for a limited period of time (__8 weeks by default__). The awarded period starts: from the moment the user's data is copied from CEP4 (after Radio Observatory pipeline processing) to CEP3 or following the timeline communicated to the user via the SSG notification and available at http://www.lofar.org/operations/doku.php?id=cep3:earlyaccess. | **__After the granted period on CEP3 has expired, all user's data products generated on the cluster will be automatically and promptly removed, to enable new users to have enough disk space to perform their data reduction.__ ** |
| At the beginning of a Cycle, users requesting CEP3 processing time in their observing proposals can derive this timeline by checking the observing schedule, which is available at http://www.astron.nl/radio-observatory/cycles/cycles. Access timelines related to observing programs involving observations spread in time will be discussed between the PI and Science Operations and Support. In general, info about CEP3 access of users are detailed at http://www.lofar.org/operations/doku.php?id=cep3:earlyaccess.** | |
| |
| **__After the granted period on CEP3 has expired, all user's data products generated on the cluster will be automatically and promptly removed, to enable new users to have enough disk space to perform their data reduction.__** | **Extensions to the __default 8-weeks__ period will be granted only in exceptional circumstances and only if properly justified through a formal request to be sent to the [[https://support.astron.nl/sdchelpdesk|SDC helpdesk]] no later than 1 week before the expiration of your access privileges. Monitoring of node usage during allocated time will be performed and the evaluation of extension requests will be based on such statistics.** |
| | |
| | |
| | |
| **Extensions to the __default 8-weeks__ period will be granted only in exceptional circumstances and only if properly justified through a formal request to be sent to sos@astron.nl no later than 1 week before the expiration of your access privileges. Monitoring of node usage during allocated time will be performed and the evaluation of extension requests will be based on such statistics.** | |
| ===== CEP3 setup and configuration ===== | |
| | |
| * [[cep3:system|CEP3 system setup (hardware, OS, packages, etc.)]] | |
| * [[cep3:Managementsoftware|CEP3 cluster/job management software (Slurm) information]] | |
| | |
| ===== CEP3 Science Operations and Support documentation ===== | |
| | |
| * [[cep3:scsup_setupuseraccess| How to enable user processing on CEP3]] | |
| * [[cep3:scsup_slurmnotes| Notes from Science Operations and Support on the issues found when managing reservations on CEP3]] | |
| * [[cep3:scsup_data_transfers| Data transfers on or to CEP3]] | |
| * [[cep3:scsup_account_management| Remove users/accounts from Slurm]] | |
| * [[cep3:scsup_email_template| Templates for notifying users]] | |
| |
| ===== CEP3 User documentation ===== | ===== CEP3 User documentation ===== |
| |
| * [[cep3:access|How to obtain access to CEP3 and its nodes]] | * [[:cep3:access|How to obtain access to CEP3 and its nodes]] |
| * [[cep3:usersoftware|What software and which version is available on CEP3]] | * [[:cep3:userenvironment|How to setup and understand the standard CEP3 user login environment]] |
| * [[cep3:userenvironment|How to setup standard CEP user login environment]] | * [[:cep3:usersoftware|What software is available on CEP3]] |
| * [[cep3:userdata|How to get user data on and off CEP3]] | * [[:cep3:userdata|How to get user data on and off CEP3]] |
| * [[cep3:access#processing_node_access|How to specify and run jobs on CEP3 (for access to nodes)]] | * [[:cep3:access#processing_node_access|How to specify and run jobs on CEP3 (for access to nodes)]] |
| |
| ===== CEP3 Usage Schedule ===== | ===== CEP3 Usage Schedule ===== |
| |
| * [[cep3:earlyaccess |CEP3 access and Users time slot]] | * [[:cep3:earlyaccess|CEP3 access and Users time slot]] |
| | * [[:cep3:whereismydata|Where is my data after the CEP3 upgrade of Jun '17]] |
| |
| ===== CEP3 Testing ===== | ===== CEP3 Setup, configuration and internals ===== |
| |
| * [[cep3:earlyaccesstests|CEP3 early access testing setup]] | * [[:cep3:systeminternalsetup|Setup and configuration of CEP3 cluster (internal use)]] |
| * [[cep3:earlyaccessnotes|CEP3 early access User and Usage Notes]] | |
| * [[cep3:casatests|CEP3 casapy tests: User suggestions and Fixing Notes]] | |
| ===== Meeting notes ===== | |
| |
| * [[cep3:tigerteammeetings|Notes of the daily CEP3 setup meetings (Aug/Sep 2014)]] | |
| * [[cep3:simulation_20140819|Notes of the CEP3 usage simulation on 19 August 2014]] | |
| * [[cep3:demoscsup_20140902|Notes from the Demo for Science Operations and Support on Sep. 3rd 2014]] | |
| * [[cep3:cep3review_20150129|Notes from the CEP3 review meetings during 2015]] | |
| |