Kick off meeting notes

Date

Jul 30, 2024

To understand a bit more about what this project might entail

A goal should be to help Diamond improve their metadata
Diamond 2 is going to need a workflow management system of some sort and they want an automated process
Provenance is needed to track raw to processed data, and should be captured somewhere (ICAT?)
ISIS currently processes raw data in DAaaS through MANTID
ISIS are currently rewriting their software for automated “reduction”, in K8s, presumably in Argo?
ISIS have 1 member of staff looking into it
CLF is using Octopus to process data through workflows
Argo VS Common Workflow Language
- Brian notes DAFNI had some issues with Argo
Questions around who is running Argo and whether or not SCD will need to host anything
Diamond is also working on micro-services
Diamond has 2 members of staff working on micro-services and workflows
Diamond processed data needs to end up in ICAT
EPAC want to process their data manually using MANTID but it is doubtful they will
Galaxy uses Workflow Hub
- used for user-defined workflows
So what are we working towards?:
- a standard way of defining workflows so that they can be re-used between facilities
- potentially storing any provenance data in ICAT
  - entities for which already exist in ICAT (added as part of the Octopus project)
Timelines: The project needs to be ideally defined by September
FIAR data should feature in this somewhere.

Alex to organise a meeting to find out what workflow architecture/development work is taking place at the facilities and to see where we can help.