Poster Abstract

P.5 Albert Jan Boonstra (ASTRON)

Streamlining LOFAR System Health Management

Observational capabilities of radio telescopes increase over time, pushing scientific boundaries, but also leading to large and increasingly complex telescope instrumentation and systems. The LOFAR radio telescope for example consists of 51 dual-band phased array radio telescope stations, producing up to about 100 terabyte daily. For data quality inspection purposes each observation produces up to 10000 spectrogram and correlation plots, and a myriad of low polling rate monitoring data. Finding system errors and identifying them is becoming increasingly difficult because of the data scale and the complexity of the system. Manual inspection and interpretation of the data is becoming impractical. Automatic detection of anomalies is helpful, but creating complete models that fully describe the system with all its configuration flexibility and expected sky signals is very complex.

Giving the enormous amounts of data, it should be possible to use artificial intelligence to address this challenge. For LOFAR system health management we have looked into this, and have implemented a machine learning algorithm in our online systems to automatically cluster patterns or features in our visibility spectrogram data. Instead of having to browse through 10000 plots, there is now only a hand full of typical feature clusters. This greatly improves inspection efficiency. The clustering already provides good results, but it does not yet identify the type and cause of particular features. In the talk I will describe the chosen approach, first results, and an outlook for further work, including adding meaning to the data via annotation.