Poster Abstract

P10.22 Mark Wieringa (CSIRO Astronomy and Space Science)

Theme: Data processing pipelines

ASKAP Pilot Survey Processing Pipeline Readiness

The ASKAP telescope is in its final commissioning and testing phase. With all 36 telescopes fully operational and commissioning of the final correlator modes in process, we are now counting down to full survey science capability. During 2019 we are doing 36-beam imaging of survey test fields and the rapid ASKAP continuum survey (RACS). The latter will be processed into ASKAP's sky model and first source catalogue. We will then begin the pilot surveys to test readiness of the pipelines for the start of the full surveys in 2020.
The data processing for ASKAP is done on the dedicated Galaxy supercomputer at the Pawsey Centre in Perth using the MPI-based ASKAPSoft package. Semi-automated processing by operators is performed using processing pipelines built on top of the basic flagging, calibration and imaging tasks. Over the last two years the individual tasks, the pipelines and the data access patterns have been optimized to try and cope with the rapidly increasing data volumes (up to 31 TB per 8h observation for spectral line data) in as close to real time as we can manage. Unless we can process the full spectral line datasets into science ready data in near real time, spectral line survey progress will be limited by available diskspace and processing backlogs. We will describe the path to the current performance of the pipelines for the pilot surveys and discuss our plans for further performance improvements.