CMSD rollout

The rollout is planned in phases. When a gateway is added to the redirector alias (managed by echo manager), it is put in the cmsd sandbox.

Current status

Phase 1

Single active gateway in the cmsd cluster, manager01 gets added to the xrootd alias . This substitutes gw14 in the xrootd and webdav aliases and ensures correctness of the initial change. any protocol/cmsd interaction unexpected behaviour should be caught here. Load behaviour and overheads should also become apparent.

Phase 2

For small scale testing, add additional gws to rdr, this is to test small scale load balancing. having 2 gateways might lead to instabilities (load ping-pong or fallback to single gw in case the other is loaded). The gateways are from different generations to evaluate impact on each. gws are kept within the xrootd/webdav aliases to evaluate the impact of cross-alias requests and for quick rollback capability. Atlas sends some transfer to rdr

Phase 3

All gws are added to rdr and VOs switch to rdr one by one

Phase 4 (a)

gw14 is reintroduced to the historic aliases. rdr becomes the default alias, but the xrootd/webdav aliases keep the current functionality, with partial load put into rdr (<10%)

Phase 4 (b)

When the previous phases pass, all other gws will be removed from the xrootd/webdav alias. All prod traffic will flow trough the redirector. This tests the capacity of the cmsd managers to cope with full prod traffic

Rollback

Phase1: the rdr dns will be removed from xrootd and webdav aliases and the previous set of gateways will be reinstated.

Phase2/3: VOs will switch back to xrootd/webdav aliases

Phase4: either of the previous depending on implementation