http://www.astron.nl/~stappers/wiki/ 2010-03-16T17:23:30+01:00 http://www.astron.nl/~stappers/wiki/ http://www.astron.nl/~stappers/wiki/lib/images/favicon.ico text/html 2010-01-21T16:59:22+01:00 Roy Smits http://www.astron.nl/~stappers/wiki/doku.php?id=documentation:fsrelated&rev=1264089562&do=diff1264089562 File System problems are grouped under Storage nodes and Compute nodes. The storage nodes are more involved. Storage Nodes Signs of a broken disk on a Storage node During an observation run pyTop. In every list there should be: puma2_dmadb and dada_dbdisk. If not, then something is broken on that node. The last number of the ip address tells you which node: 192.168.100.54 means node 4. text/html 2010-01-20T17:43:30+01:00 Gemma Janssen http://www.astron.nl/~stappers/wiki/doku.php?id=documentation:catalogs&rev=1264005810&do=diff1264005810 Because our computing system is distributed far and wide, there are at least three different places where we need to keep the catalog updated! These are wop26/wop27/puma.tms.nl --- i.e. in /puma/soft/psrinfo/ master --- i.e. in /home/pulsar/CVSHome/soft_swin/models/psrinfo/ thoth/other computers in A'dam --- i.e. in /net/thoth/scratch4/zespri/rigel/psrhome/soft_swin/models/psrinfo/ Each of these is slightly different unfortunately. When you add a new… text/html 2010-01-12T17:16:23+01:00 Ben Stappers http://www.astron.nl/~stappers/wiki/doku.php?id=logs:tapes&rev=1263312983&do=diff1263312983 Tape1 8020703 Tape2 2301034 Tape3 1578638 Tape4 6518318 Tape5 5281581 Tape6 3699476 Tape7 8255452 Tape8 2729373 Tape9 7630811 Tape10 4670767 Tape11 8106051 Tape12 6999992 Tape13 4768203 Tape14 8500103 Tape15 9576073 Tape16 9545453 Tape17 7899931 Tape18 2371262 Tape19 7867386 Tape20 5493746 Tape21 7324512 Tape22 1022591 Tape23 6111586 Tape24 2103513 Tape25 3977141 Tape26 3734060 Tape27 2805752 Tape28 8935635 Tape29 7900870 Tape30 3323022 … text/html 2010-01-11T14:45:57+01:00 Ramesh http://www.astron.nl/~stappers/wiki/doku.php?id=documentation:storagenode&rev=1263217557&do=diff1263217557 Observations fail Observations, when either CtrlPumaII in wop28 is stopped for some reason or the PumMa-II client on master is dead. So check the following, and restart if necessary. * tmsmgr@wop28: ps -aef | grep -i Ctrlpuma --> should show CtrlPumaII running. If not run TMSCtrl stop CtrlPumaII followed by TMSCtrl start CtrlPumaII * pulsar@master: ps -aef | grep puma2cmi --> should show puma2cmi.py running. If not run, /homw/pulsar/bin/puma2cmi.py * pulsar@master: check the log files: … text/html 2010-01-11T14:31:43+01:00 Ramesh http://www.astron.nl/~stappers/wiki/doku.php?id=documentation:masterfs&rev=1263216703&do=diff1263216703 Master File system problems The master node has the following partitions: master:~ # df /dev/sda5 8064240 6387440 1594872 81% / /dev/sda6 1510032 147564 1347128 10% /tmp /dev/sda7 15116836 8313452 6649804 56% /usr /dev/sda8 2016016 610948 1384588 31% /var /dev/sda10 212540504 176996332 33384876 85% /home All of the device, ie. /dev/sda[5,6,7,8,10] needs to error-free and read-writable for proper cluster oper… text/html 2010-01-11T14:11:53+01:00 Ramesh http://www.astron.nl/~stappers/wiki/doku.php?id=documentation:sshrelated&rev=1263215513&do=diff1263215513 cannot ssh/rsh to nodes Note: Information here solves the problem described immediately after this section. The issue is that the ypbind in the nodes are in a bad state - this blocks rsh, and keeps asking for password. The trouble with ssh can be fixed by restarting ssh daemon in the nodes. As root@master, run the following: