Data analysis and archiving system

Work on this part of the observing system has been concentrate on two different aspects.

With the aim to make historical data easier to access, and to reprocess some of the old data for analysis, data from the archive CD and DVD disks has been read in to disk. Tools were developed to make it easier to upload the data itself, and to keep track of what has been uploaded, what is pending, and potential problems with faulty media, or missing files. So far, a total of 2 TB of data spanning the period 2008-2011 has been read in. A tool is being developed to assess the data.

In connection with the new data acquisition system related to the new detector controller, a new data storage and analysis system is being planned. A short document was made indicating the initial design and computing requirements. The overall philosophy of the system is to split the need for fast, low capacity storage and slower, high capacity storage and combine it with the need for high capacity, variable computing power for post processing and the low computing power needs for archiving.

A few entry level servers have been bought to replace some aging ones (mainly the DNS/Secondary mail server), which will also serve as a implementation test for the the new data storage and analysis system.