The end location of the data should be the tape drive, so some software is needed to move it from Echo S3 to the tape drives.
Currently, all data goes through the file aggregator, storageD.
This means that, if we were to transfer the data to tape without using storageD, The IDS wouldn’t know about the newly uploaded files and wouldn’t be able to download them.
A separate piece of software, in addition to the upload api, is needed to make the transfer.
The following 2 designs have been considered.
Architectural Design A
Here, the data will need to be downloaded to local storage so the StorageD client can pick it up. This has a high network overhead but has the advantage of being relatively easy to develop.
Architectural Design B
This design explores the possibility of the storageD server downloading the data directly. This would save a download step. However, it will mean a change to the storageD code which may be problematic due to the state of the code.
Solution
After a chat with Alan Kyffin, we decided to go with design A because it is simpler to develop and maintain.
the code isn’t in a git repo
isn't in great shape
is being looked after by another team
The code was put in a GitLab repo this review.
Add Comment