2025-03-27 Meeting Notes
- Thomas, Jyothish (STFC,RAL,SC)
- Matt Doidge
Date
Mar 27, 2025
Participants
@Thomas, Jyothish (STFC,RAL,SC)
@Alexander Rogovskiy
@Ian Johnson
Lancs: Matt, Gerard, Steven
Glasgow:
Apologies:
@James Walder
CC:
Goals
List of Epics
New tickets
Consider new functionality / items
Detailed discussion of important topics
Site report activity
Discussion topics
Current status of Echo Gateways / WNs testing
Recent sandbox’s for review / deployments:
Item | Presenter | Notes |
|
|---|---|---|---|
Operational Issues |
|
Quincy rolled back for S3 gws lsst likely to move on single group multiple roles |
|
Compilation and rollout status of RAL XRootD versions | @Thomas, Jyothish (STFC,RAL,SC) | 5.7.3 released (awaiting other changes to gateways) 5.8 officially released streamed checksums in testing |
|
cms-aaa naming convention | @Thomas, Jyothish (STFC,RAL,SC) | cms-aaa is the only remaining personality to use proxy/ceph as the xrootd service names Separate naming convention would be more appropriate, to have main/supporting (not so urgent). CC created, and sandbox is prepared and has been tested on a test host |
|
cms-aaa jemalloc use | @Thomas, Jyothish (STFC,RAL,SC) | memory limits caused some issues under load |
|
cms-aaa overhaul |
| Xcache gateways |
|
Shoveler | @Katy Ellis |
|
|
On the fly Checksums | @Ian Johnson
| Moved streaming calculation in XrdCeph to use XRootD’s checksum calculation library rather than directly calling zlib adler32 function. Looking at moving optional checksum calculation into the “core” XRootD. |
|
Deletions | To check deletion timing split between client/cluster response under DC saturation - not started Spike of mini-DC deletions taking longer than expected/longer than other sites: gathering XRootD logs for investigation
|
| |
XRootD Writable Workernode Gateway Hackaton
| @Thomas, Jyothish (STFC,RAL,SC)
|
|
|
Plan: file query system to summarize XRootD Logs |
| Plan to create a system to store info from across all gateways to search a filename and get creation time, last write time, last successful stat and deletion time in case of ‘lost’ files. Possible graduate sideproject. Ian plans to extend the database schema from the deletion tests (capturing file write completions and deletions) into a more general event schema. |
|
100 GbE Gateway testing: | @James Walder @Thomas, Jyothish (STFC,RAL,SC) | UKSRC - Acting as source for SRCNet verification tests; not being stressed so far … Tier-1 .
|
|
UKSRC Storage Architecture |
| Tom B. Working on CephAdm setup for the cluster. JW attempting to reinstall the hosts. |
|
Tokens Status |
|
token leak cannot be prevented on redirector setups as http redirection moves the headers to url for redirects. |
|
packet marking |
| svc20 xrd 731 xrootd.pmark map2act cms default default needs to be added for path based redirection to work |
|
test stress testing framework |
| Script ready, first test to be arranged |
|
on GGUS:
Site reports
Lancaster:
Wheels are turning and cunning plans are afoot, but not much to actually report. New G/W hardware has arrived, but will be a few weeks before it’s plumbed in.
Glasgow -
internal reboot for ceph upgrades, log verbosity caused some issues, all ok now.
moving to next xrootd version.
Action items