This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Next revisionBoth sides next revision |
| public:cs1_data_policy [2007-07-18 08:04] – Hanno Holties | public:cs1_data_policy [2007-07-18 08:10] – Hanno Holties |
|---|
| |
| NB: The policy as provided here applies principally to the data in the "offline" domain. In this domain, the main central storage facility consists of five storage systems, each with a storage capacity of 1.2 Tbyte. A more complete description of the CS1 offline systems can be found [[public:cs1_storage|here]]. | NB: The policy as provided here applies principally to the data in the "offline" domain. In this domain, the main central storage facility consists of five storage systems, each with a storage capacity of 1.2 Tbyte. A more complete description of the CS1 offline systems can be found [[public:cs1_storage|here]]. |
| | |
| |
| |
| ===== CS1 observation data policy ===== | ===== CS1 observation data policy ===== |
| |
| The datasets created with the LOFAR CS1 system are copied to the offline storage cluster after observations have finished. The available datasets and their locations can be found in the [[public:dataproduct:catalog_of_observation|observation catalog]]. Data is kept on the central storage system for a period of at least a week. After the retention period, the data is either deleted or archived, depending on its status in the "Archive" column in the observation catalog. The oldest datasets will be (re) moved first. The exact time at which deletion or archiving is carried out will depend on the storage capacity required for near future observations and commissioning activities. | The datasets created with the LOFAR CS1 system are copied to the offline storage cluster after observations have finished. The available datasets and their locations can be found in the [[public:dataproduct:catalog_of_observation|observation catalog]]. Data is kept on the central storage system for a period of at least a week. After the retention period, the data is either deleted or archived, depending on its status in the "Archive" column in the observation catalog. The oldest datasets will be (re) moved first. The exact time at which deletion or archiving is carried out will, amongst other things, depend on the storage capacity required for near future observations and commissioning activities. |
| |
| Archived data will be moved to GRID based storage servers. Commissioning data will be owned by the /lofar/ops group and stored under the lfc directory /grid/lofar/ops/cs1_data as a tarball. The current policy for this data is that it will be archived on tape in two copies after an unknown period. Please beware that GRID based storage is experimental. At present, no guarantees can be given about data lifetime and retrieval although these issues are being addressed and data security is likely orders of magnitude better than on the central CS1 storage facilities. | Archived data will be moved to GRID based storage servers. Commissioning data will be owned by the /lofar/ops group and stored under the lfc directory /grid/lofar/ops/cs1_data as a tarball. The current policy for this data is that it will be archived on tape in two copies after an unknown period. Please beware that GRID based storage is experimental. At present, no guarantees can be given about data lifetime and retrieval although these issues are being addressed and data security is likely orders of magnitude better than on the central CS1 storage facilities. |