...
Once the user has run their experiment, the data is stored on the internal DLC data store for 40 days. The metadata is then pushed to ICAT’s Oracle database, and the raw data is sent to the data centre for processing.
Data Ingestion
The process starts with the XML client (depicted below) transmitting file metadata to the File Aggregator. This undertakes the task of segmenting the incoming data into discrete chunks. Subsequently, it orchestrates requests to the Diamond file client, procuring these segmented portions. The metadata encapsulates pertinent details such as file chunk attributes, encompassing size and storage coordinates, and is stored within the Storage D component. This component is used to look up the location of the data for retrieval purposes. These segmented file fragments are then passed to a tape archive, facilitated by the CERN archival storage system. This archival strategy is employed for preserving data over extended durations, typically decades.
...