Assumptions / Questions

Owned by Alex Kemp
Last updated: Jun 13, 2024
2 min read

Some assumptions had to be made to progress with the project. These assumptions can also serve as questions for Diamond to approve. They include upcoming features and are not limited to current functionality.

Number

Assumption

Number

Assumption

1

The upload should be restricted to 5GB per upload session

2

The user doesn’t have a maximum amount of data they can upload

3

The user should only be able to upload certain files, these are:

  • images files

    • JPEG (.jpg or .jpeg)

    • PNG (.png)

    • GIF (.gif)

    • BMP (.bmp)

    • TIFF (.tif or .tiff)

    • WebP (.webp)

    • HEIF (.heic or .heif)

  • hdf files

    • .hd5

    • .hdf5

  • txt files

    • .txt

    • .csv

    • .json

    • .md - Markdown file

    • .log - Log file

4

The resting place for the uploaded data will be the tape drive

5

All users in the investigation will be notified when new data is added

6

Data services will maintain the upload service as well as the separate software needed to transfer the software to tape.

The service also has the following platform dependencies:

  • rabbitmq

  • prometheus

  • grafana

7

The metrics to be gathered are:

  • number of datasets created

  • number of datafiles created

  • size of files uploaded

8

There is a DATASET_TYPE_ID in ICAT to distinguish the newly uploaded datasets

9

To distinguish datafiles, the description will be set to: 'User uploaded file'

10

Inherently, users can only upload to investigations they have access to/they can see in Datagateway. The API doesn’t perform any verification. It does validate the user is logged in. So the user would have to be inside the VPN, be logged in, and have a valid token to upload data

11

The uploaded data will be transferred to tape via storageD