Poster Abstract

P.8 Jeremie Ansart (CNES)

Scientific Challenges: Full-size integration tests of the Euclid Science Ground Segment

Euclid is a high-precision survey space mission developed in the frame of the Cosmic Vision Program of ESA to study the nature of dark energy and dark matter. Its Science Ground Segment (SGS) will deal with around 80PB of data coming from Euclid satellite data, science pipeline processing, external ground-based observations or simulations. The SGS will eventually produce a catalog containing the description of around ten billion objects with hundreds of attributes each.
The SGS scientific software is made of so-called Processing Functions (PFs) written by hundreds of developers disseminated all around Europe and the US, and is deployed to nine computing centers known as Science Data Centers (SDCs). Given the amount of data and SDCs, it was decided to move the processing, not the data: The sky is divided into regions which are allocated to the various SDCs, where each PFs processes the local region.
To integrate the PFs and ensure they interact inside and across SDCs, an iterative and incremental development model has been adopted. At each iteration, components are introduced, or upgraded during "Scientific Challenges", which aim at testing the SGS integration at system level. Moreover, an ever increasing sky area is simulated and distributed over the SDCs, and the corresponding PFs are triggered in the production environment.
In this paper, we describe the process and tools set up to test the integration of millions of lines of codes written by hundreds of developers and executed automatically on nine computing centers.