Kick off meeting notes
- Alex Kemp
Date
Jul 30, 2024
Participants
@Alex Kemp
@Kevin Phipps
@Antony Wilson
@Louise Davies
Paul Quinn
Goals
To understand a bit more about what this project might entail
Discussion points
A goal should be to help Diamond improve their metadata
Diamond 2 is going to need a workflow management system of some sort and they want an automated process
Provenance is needed to track raw to processed data, and should be captured somewhere (ICAT?)
ISIS currently processes raw data in DAaaS through MANTID
ISIS are currently rewriting their software for automated “reduction”, in K8s, presumably in Argo?
ISIS have 1 member of staff looking into it
CLF is using Octopus to process data through workflows
Argo VS Common Workflow Language
Brian notes DAFNI had some issues with Argo
Questions around who is running Argo and whether or not SCD will need to host anything
Diamond is also working on micro-services
Diamond has 2 members of staff working on micro-services and workflows
Diamond processed data needs to end up in ICAT
EPAC want to process their data manually using MANTID but it is doubtful they will
Galaxy uses Workflow Hub
used for user-defined workflows
So what are we working towards?:
a standard way of defining workflows so that they can be re-used between facilities
potentially storing any provenance data in ICAT
entities for which already exist in ICAT (added as part of the Octopus project)
Timelines: The project needs to be ideally defined by September
FIAR data should feature in this somewhere.