Daily Image

Click here or on the picture for a full size image.

Improving pipeline description methods

Submitter: Yan Grange
Description: As part of the European Open Science Cloud (EOSC) pilot project, the LOFAR prefactor pipeline has been reimplemented in the Common Workflow Language (CWL). This is a language in which workflows and pipelines can be described independently of the software system used to run them.

The implementation was written by Gijs Molenaar. The goal of the project is to demonstrate how we could make our pipelines available for non-expert users, for example in the European Open Science Cloud. Using a standard like CWL to describe pipelines offers the opportunity to use different methods to manage running pipelines and workflows by any user, while making it easier to adapt the pipeline for expert users.

The image shows a visualisation of the first part of the prefactor pipeline - which takes care of calibration using a calibrator source, simplified and based on the pipeline written in CWL and parsed with the standard CWL viewer.

The time runs from top to bottom. The blue blocks represent the pipeline in- and outputs. Each block is either a file or a configurable value. Each yellow block represents one processing step and each arrow in or out of the box corresponds to an in- or output data product of that box.

For more information, please visit (one of) the following pages:
the cwl pipeline
The same pipeline in the standard CWL viewer
EOSC pilot
Copyright: Grange, Shulevski, Molenaar
  Follow us on Twitter
Please feel free to submit an image using the Submit page.