Daily Image
18-02-2011
LOFAR Pulsar Data Joins Archive Flow to Grid
| Submitter: | Joeri van Leeuwen, Hanno Holties |
| Description: | In the past months it has become ever more clear that the 500 Terabyte storage capacity on LOFAR Central Processing can be a major bottleneck for continued observations. To take some of the the pressure on the central storage, observation data is regularly transferred to the Petabyte BiG Grid storage at SARA since January 2011. After final development in Pulsar Busy Week 11, January 24-28, raw and reduced LOFAR beam-formed/pulsar data now also routinely stream to the Grid for storage and further processing. We would like to thank Adriaan Renting, Marcel Loose and Coen Schrijvers from SARA for their help. As the graph above shows, in this first week of operations, we streamed more accumulated LOFAR Pulsar "lofpul", EOR "loeor" and general "lofops" data to the Grid than the LHC experiments Atlas and Alice combined. Per storage stream, data is generally transferred at speeds of around 100MB/s. Up to at least 5 such streams can smoothly run independently over the 10Gbps line to SARA; in a week, around 30TB of pulsar data was packaged and stored in a tape pool. The currently reserved size of the archive is 1PB. For pulsar observing with LOFAR this alleviates storage constraints, allowing for more time on sky, essential for enabling a larger-scale pulsar survey. The current pulsar observations produce around 40GB/hr, but for this envisioned multi-beam high-sensitivity pulsar survey, data rates in excess of 10TB/hr are expected. |
| Copyright: | JvL |
| Tweet |  |