dragnet:psrfits2fil

A series of tests were ran on dragnet (drg01), directly (now through slurm), on fits files from a random LOTAAS observation. Number of cores,means number of different fits files processed simultaneously. The total time is an extrapolation of this benchmark.

Only one test was ran for each occasion. Repeating them showed differences in execution time, but I doubt that the results change qualitatively.

Input/Output on the same disk

1-core:354sec
Total time for 16 files:5664sec

3-cores:630sec
Total time for 16 files:3360sec

4-cores:720sec
Total time for 16 files:2880sec

5-cores:1106sec
Total time for 16 files:3539sec

8-cores:1735sec
Total time for 16 files:3360sec

16-cores:3778sec
Total time for 16 files:3778sec

When writing the filterbank in the same disk with the fits file, the best performance is achieved by having 4 psrfits2fil instances running.

Input/output on different disks

1-core:329sec
Total time for 16 files:5264sec

4-cores:523sec
Total time for 16 files:2092sec

8-cores:901sec
Total time for 16 files:1802sec

16-cores:1513sec
Total time for 16 files:1513sec

24-cores (hyperthreaded):2252sec scales perfectly!
32-cores (hyperthreaded):3346sec performance loss already evident;~340 sec slower than expected

Using all the cores in the dragnet nodes gives the best performance. Moreover, multithreading behaves exactly as having the extra cores.

Moving a fil file from /data2 to /data1 takes 1 minute.

  • Last modified: 2017-03-08 15:27
  • by 127.0.0.1