Oral Abstract

Lightning talk (L10) J. B. Raymond Oonk (SURFsara)

Theme: Multi-wavelength astronomy

(P6.1) Pioneering the Exascale era with Astronomy

SURF is the Dutch national Supercomputing and Networking centre for Research. I will present our latest developments in high performance cloud computing for Astronomy. In particular I will focus on our new OpenStack-based cloudified infrastructure layer. This layer allows us and our users to efficiently build and deploy dedicated, managed platforms (e.g., Kubernetes, Spark, Jupyter hub, batch processing clusters and private nodes & clusters). It also enables on-demand, dynamic scaling of the underlying hardware (compute, storage & network) and seamless integration between this hardware and the platforms built on top.

I will show the use of this cloudification by providing a practical example on how we currently process the Surveys Key Science Project data (50 Petabytes) from the LOFAR radio telescope across Europe. Using the lessons learned from LOFAR, SURF in collaboration with other data centers and the radio astronomical community are preparing for the global exascale compute and storage challenges posed by the Square Kilometre Array. Here I will briefly touch upon some of these challenges that we are starting to address in the European Open Science Cloud initiative (EOSC-hub, ESCAPE) and the AENEAS project.

I will end with a glimpse towards the next decade in IT and discuss emerging exascale technologies and innovations. Together with industry and science we explore these technologies in our SURF Open Innovation Lab. Here I will provide some recent results from the convergence of big data and HPC, and connecting edge compute to the cloud.

Poster Abstract

P6.1 J. B. Raymond Oonk (SURFsara)

Theme: Local and global cloud infrastructure for processing and storage

Pioneering the Exascale era with Astronomy

SURF is the Dutch national Supercomputing and Networking centre for Research. I will present our latest developments in high performance cloud computing for Astronomy. In particular I will focus on our new OpenStack-based cloudified infrastructure layer. This layer allows us and our users to efficiently build and deploy dedicated, managed platforms (e.g., Kubernetes, Spark, Jupyter hub, batch processing clusters and private nodes & clusters). It also enables on-demand, dynamic scaling of the underlying hardware (compute, storage & network) and seamless integration between this hardware and the platforms built on top.

I will show the use of this cloudification by providing a practical example on how we currently process the Surveys Key Science Project data (50 Petabytes) from the LOFAR radio telescope across Europe. Using the lessons learned from LOFAR, SURF in collaboration with other data centers and the radio astronomical community are preparing for the global exascale compute and storage challenges posed by the Square Kilometre Array. Here I will briefly touch upon some of these challenges that we are starting to address in the European Open Science Cloud initiative (EOSC-hub, ESCAPE) and the AENEAS project.

I will end with a glimpse towards the next decade in IT and discuss emerging exascale technologies and innovations. Together with industry and science we explore these technologies in our SURF Open Innovation Lab. Here I will provide some recent results from the convergence of big data and HPC, and connecting edge compute to the cloud.