The end location of the data should be the tape drive, so some software is needed to move it from Echo S3 to the tape drives.
Currently, all data goes through the file aggregator, storageD.
This means that, if we were to transfer the data to tape without using storageD, The IDS wouldn’t know about the newly uploaded files and wouldn’t be able to download them.
High-level requirements
A script, deployed on a server somewhere, will need to:
check ICAT for files added between a set of dates (Start data - > end date)
store the location of these files
download the files via s3 and create a checksum
create a drop file and publish to StorageD
once all files have been pushed to storageD, redownload and check checksum
update file locations in icat to reflect tape location instead of S3 URL
Add Comment