2023-05-18 Meeting Notes

Impact of Vector Read update to Echo

Vector Read

https://stfc.atlassian.net/wiki/spaces/GRIDPP/pages/137265343/Non-striper+read+v+implementation+for+WN+s+xrootd+gateways

https://stfc.atlassian.net/wiki/spaces/X/pages/edit-v2/143786029

https://github.com/stfc/xrootd-ceph/pull/37/files

12–16 May 2023 Echo instability following readV rollout

Timeline on Batch farm

• 5th May: ` wn-2020-xma - wn-2022-lenovo` will be set to drain.

• 9th May:

  • Merge the sandbox http://aquilon.gridpp.rl.ac.uk/sandboxes/diff.php?sandbox=update_cvmfs_client into production.

  • Drained workers will have the package `egi-cvmfs` manually removed.

  • Manually remove the `xrootd-proxy.service` (/etc/systemd/system/xrootd-proxy.service)

  • Manage workers into http://aquilon.gridpp.rl.ac.uk/sandboxes/diff.php?sandbox=xrootd-patch (will include cherry-picked commit for the Docker patch). http://aquilon.gridpp.rl.ac.uk/sandboxes/diff.php?sandbox=docker-cvmfs-xrootd-combo This sandbox combines all changes and gives a clear view of all changes (DO NOT DEPLOY).

  • Confirm workers have successfully compiled:

  • Check: `healthcheck_wn_condor` outputs healthy status.

  • Check ` xrootd-gateway.service` outputs healthy Docker status.

  • Bring online tranche at a time confirming with data team xrootd is working as expected.

• 10-11th May: Let the updated workers run for a few days.

• 12th May: `wn-2017-dell (all 2017’s) - wn-2019-dell` will be set to drain.

• 15th May: Repeat above process for second half of workers.

• 16th May: Merge all required sandboxes into prod and manage farm back into `prod_batch` in AQ

Discussion on merging bufferedIO into master.
Also to discuss pushing changes to “upstream” (xrootd/xroot-ceph”

https://github.com/stfc/xrootd-ceph/pull/44

Needs testing for ‘correctness’

Also some discussion ongoing on xrootd “issues” on the xrootd-ceph sub-module:
https://github.com/xrootd/xrootd/pull/2008

SEGV investigations with -S multi-stream flags

XRD-53 - Getting issue details... STATUS

fix in ‘master’ of core xrootd; not yet added to a tagged xrootd release; to follow up with Ian Johnson

CMSD status

XRD-41 - Getting issue details... STATUS

CC document

https://stfc.atlassian.net/wiki/spaces/GRIDPP/pages/136446019/High-level+XrootD+redirection+for+Echo?focusedCommentId=136118425

Transfers of 0-byte files

XRD-62 - Getting issue details... STATUS

Observed Dune transfer failures using 0-byte files