P11.4 Hisanori Furusawa (National Astronomical Observatory of Japan)
Theme: Other
Data Processing in Subaru Strategic Survey Program with Hyper Suprime-Cam
The Subaru Strategic Program (SSP) with Hyper Suprime-Cam (HSC) is an ongoing wide-field imaging survey project at Subaru Telescope. The project aims at providing well-calibrated science-ready data products to the astronomical community. Data processing involves a quick quality assurance at the observing site (onsite QA), and full production run conducted in Japan. The onsite QA system extracts quality indices of raw data including seeing, transparency, and sky level etc. and ingests them to a QA database. We refer to this database to determine exposures are to be combined into the final products based on the data quality. The full production work happens once or twice a year, generating 500TB data sets in each of data releases. These products are transferred to the science data archive site for external access, that include the two public data releases. The data processing pipelines are run across on-premise PC clusters equipped with fast file systems such as Lustre and Spectrum Scale, so that the whole procedure could be done within a ~2 months time scale. We have also been updating the pipeline software, where algorithms, such as sky subtraction, ghost masking etc, are being improved.
