...
Item | Presenter | Notes | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
Operational Issues | 5.6.9 deployment failed, deployed 5.5.4 with case insensitive headers Sam - sl7 doesn’t crash but is less stable, rocky8/9 crashes with xrdceph glibc version? | ||||||||||
CHEP Abstract ideas | Deadline extended to 17th; one CHEP paper for load balancing, one from Matt, gerard/lancs for ceph monitoring SKA high throughput abstract | ||||||||||
XrootD Workshop plan | Registrations are open (pending prettification) | ||||||||||
Rocky 8 and 9 migration planning | CC passed, 4 gateways are planned to be deployed today remaining upgrades to be done after next week batch farm will be undergoing simultaneous updates preprod farm upgraded gridftp ones will stay until june 3rd | ||||||||||
Shoveller | |||||||||||
Future developments ideas planning work | |||||||||||
Deletion studies through RDR | .Preliminary deletion rate plot from DC24 “dip-stick” sampling (ATLAS VO):
plot taken for data during DC steven looking for cephfs deletion studies lancs-timestamp/filesizes vs number of slow ops comparison? | ||||||||||
Deletions |
| what’s the theoretical limit? | |||||||||
Planning for ALICE CMSD redirection | keepalived setup is working 1 dev gateway is being added to this cluster | ||||||||||
Checksums fixes | '21 generation is being rolled out '21 are being swapped with '22 job types as the '22s have separate os and disk drives, additional HW for '21 in a month on the WNs the checksums are forwarded to the prod cluster the preprod farm contributed ~5% of the total checksums | ||||||||||
Prefetch studies and WN changes | Some more data from the overload event, namely efficiency and error rates of the on 2021 gen (prefetch on):   It would be interesting to compare this to the “prefetch off” configuration. So far meaningful comparison is not possible, since the number of WGProduction jobs is nowhere near the numbers during the overload event (23.04.2024):  stress test for WNs? preprod CEs targetting internal xrootd cluster | ||||||||||
Tokens Status | |||||||||||
CMSD Load balancing | |||||||||||
SKA Gateway box | /wiki/spaces/UK/pages/215941180 2 new servers racked up; awaiting netbox configuration. | ||||||||||
Xrootd testing framework |
...
Lancaster - a week of living on Reef hasn’t yielded many operational issues, the only one of note was that Reef wasn’t happy with the “trimming settings” for the MDS, the Pacific defaults needed to be cranked up - otherwise smooth sailing. Dealing with fallout from a lot metrics being renamed and changes to the logging which has reduced our monitoring capabilities a bit, but that’s a niggle. Next task is update all the clients.
Glasgow:
redirector black holing -5.6.9, some crashes on the server
Do we want to talk about Durham?
having interesting ceph problems (files with read lock) transfers hanging and locking files after a while. Paul switching to match lancs and glasgow cephfs to see if it fixes things
✅ Action items
⤴ Decisions
...