ILTO meeting March 23, 2010 (ILT Operations)

The ILT Operations meeting is held at the start of a LOFAR observing semester. The purpose of the meeting is to discuss operations procedures for the ILT, covering all aspects from observation scheduling to station maintenance. The audience for this meeting are (technical) personnel involved in running and maintaining the station.

The first meeting was held on March 23, 2010, at ASTRON in Dwingeloo. At this meeting, roll-out issues were discussed as well, as several international stations were either under construction or to be constructed.

Meeting presentation (Harm Munk)

Presentation by I. Thomas on the LOFAR station under construction at Nancay (France)

Presentation by W. Reich on the LOFAR station at Effelsberg (Germany)

Presentation by B. Ciardi on the LOFAR station at Garching/Unterweilenbach (Germany)

Presentation by E. Middelberg on the LOFAR station to be constructed at Jülich (Germany)

Presentation by G. Mann on the LOFAR station at Potsdam (Germany)

Presentation by L. Heldner on the LOFAR station to be constructed at Onsala (Sweden)

Presentation by D. McKay on the LOFAR station to be constructed at Chilbolton (U.K.)

The page containing the email addresses and telephone numbers shown during the meeting can be found here. Note: you might have to login to the Wiki again to get access to this page! (“Feature” of this Wiki.)

The meeting started with presentations on LOFAR stations roll-out from ILT- consortia: see the presentation below.

The afternoon started with a demonstration of a 5 minute LOFAR observation, from specification to inspection of the results. Most aspects of LOFAR operations from the point of view of the observer were shown.

The meeting then continued with a presentation on LOFAR Technical Operations. The following topics were discussed to some length:

The following allocation of time for individual stations will be used:

• 10% time for maintenance (~16 hrs/week, the equivalent of two workdays). In this mode, the station is controlled by ASTRON’s RO, but the local station controller can be requested to run tests on the station.
• 90% time available for
• 9% time available for stand alone use (the name “private local use” is used in the presentation, but this term might be confusing.) In this mode, the station is disconnected from CEP: data is stored locally at the station.

All station use, including stand-alone mode, is scheduled. (See “Station control” for targets of opportunity.) If stand-alone mode can be scheduled for all stations at the same time is doubtful. Experience will show if and how stand-alone mode is to be scheduled. If during an array observation, or during a maintenance period a station is not used, it can on request be made available for stand alone use. This extra stand alone use is on top of the allocated 9%: in other words, it will not be subtracted from the allocated 9%.

Station control resides either at ASTRON’s RO or at the local station controller. Control is always handed over from the current controller to the new controller. Station control is never taken over. This protocol is also used for target of opportunity (ToO) observations. The requestor must contact the current station controller to request the handover of station control. If the current station controller cannot be contacted in time, the station cannot be used for the ToO observation. Note that both ASTRON’s RO and a local station controller can request a station for ToO observations. To do: implement a handover mechanism.

All commissioning and KSP-related observations are executed in array mode, and handled by the LOFAR system software controlled by ASTRON’s RO.

Scheduling is done in different stages. The first schedule is computed at the start of an observing semester, resulting in an overview of projects that will be executed during that semester. Some observing constraints can be taken into account at this stage (rise-set times, day-night observing, maintenance periods). The final stage of scheduling is done on a weekly basis, and possibly even daily, additionally taking LOFAR system availability constraints (e.g. station statuses), capacity (e.g. long term storage), and stand-alone mode requests into account. ILT and station status Currently, station status (both hardware and software) is monitored from the ASTRON control room through a collection of LOFAR system software. Effort is required to make these data available to station personnel. This activity must be planned.

Putting a station off-line for maintenance and putting it back on line is always communicated by the ILT operator to station personnel and by station personnel back to the ILT operator. Operator communication Day to day communication between the observer at ASTRON and local station personnel is by means of telephone and email. To avoid potential confusion, all involved parties are requested to provide one phone number and one email address per category: observations, station hardware, station software, and network. This contact data is collected on the LOFAR Wiki on the operations page: http://www.lofar.org/wiki/doku.php?id=operator:operations. It is advised to use internal email distributions lists to propagate emails to the appropriate person(s).

A weekly conference call will be held to discuss operational matters on upcoming observations, maintenance and other events that influence the operational status of the ILT.

ASTRON is responsible for developing and distributing station software and firmware for LOFAR operations. This includes network switch configuration.

ASTRON will announce software, firmware and configuration upgrades ahead of time.

Station health checks are executed by station software and by test software that is started by ASTRON RO.

Maintenance of station hardware (antennas, coaxes, electronics, power, network hardware) is the responsibility of station personnel. Broken parts are to be send back to ASTRON for cause analysis and repairs. ASTRON will provide packaging instructions and, if necessary, packaging material. Spare parts will b stocked at ASTRON. As experienced is gained in operating LOFAR some parts might be stocked at the individual stations. Station personnel are also responsible for terrain maintenance.

ASTRON will provide a price list for spare parts.

Based on the experience gained with station construction, having documents available with best practices is very valuable. We will start a comparable collection on station operation and maintenance in a cookbook on the LOFAR Wiki, accessible from the operations page.