Oral Contribution (O0.5) Jun Han (National Astronomical Observatories, Chinese Academy of Sciences)
Theme: Local and global cloud infrastructure for processing and storage
The design and realization of mass data processing model for the Five hundred meters Aperture Spherical Radio Telescope
The Five hundred meters Aperture Spherical Radio Telescope(FAST) is the biggest radio telescope in the world and construction was completed in the year 2017. After about two years' commissioning period, it is beginning to enter formal observation stage gradually. It generates huge observation data every day especially with the usage of 19 multi-beam receiver. These data have to be stored in a public storage because of the huge amount of data and shared computers are provided to support reseachers' usage . As everybody knows, there always exits many conflict problems with shared resources with different work modes for different people, for example different software environments, versions and so on. Besides, there are many general preprocessing. The data process software should be run automately and could parallelly increase or reduce the calculated amount according to data process task. For solving these problems, we introduce docker technique to package work environments for researchers, and introduce kubernetes technique for the automatic scheduling of computations. We have designed and realized some data process models for different applications based on kubernetes private cloud and it greatly improves the work efficiency. It not only meets the needs of different reseachers' data process environments, but also could adjust computation automately.
