The rollout is planned in phases. When a gateway is added to the redirector alias (managed by echo manager), it is put in the cmsd sandbox.
...
Current status
xrootd | webdav | cmsd | |
---|---|---|---|
yes | no | passive | |
gw4 | |||
gw5 | |||
gw6 | |||
gw7 | |||
gw14 | |||
gw15 | |||
gw16 | |||
svc01 | |||
svc02 | |||
svc97 | |||
svc98 | |||
svc99 | |||
rdr |
Phase 1
Single active gateway in the cmsd cluster, manager01 gets added to the xrootd alias . This substitutes gw14 in the xrootd and webdav aliases and ensures correctness of the initial change. any protocol/cmsd interaction unexpected behaviour should be caught here. Load behaviour and overheads should also become apparent.
...
For small scale testing, add rdr alias to xrootd and webdav aliases this ensures production traffic using both protocols pass trough the redirector. Rdr redirects to 3 gatewaysadditional gws to rdr, this is to test small scale load balancing. having 2 gateways might lead to instabilities (load ping-pong or fallback to single gw in case the other is loaded). The gateways are from different generations to evaluate impact on each. svc97 is purposefully gws are kept within the xrootd/webdav aliases to evaluate the impact of cross-alias requests . This is useful to know for future projects (e.g. cross-site cmsd) but would not reflect the final state of this change.and for quick rollback capability. Atlas sends some transfer to rdr
xrootd | webdav | cmsd | |
---|---|---|---|
yes | no | passive | |
gw4 | |||
gw5 | |||
gw6 | |||
gw7 | |||
gw14 | |||
gw15 | |||
gw16 | |||
svc01 | |||
svc02 | |||
svc97 | |||
svc98 | |||
svc99 | |||
rdr |
Phase 3
All gws are added to rdr and VOs switch to rdr one by one
xrootd | webdav | cmsd | |
---|---|---|---|
yes | no | passive | |
gw4 | |||
gw5 | |||
gw6 | |||
gw7 | |||
gw14 | |||
gw15 | |||
gw16 | |||
svc01 | |||
svc02 | |||
svc97 | |||
svc98 | |||
svc99 | |||
rdr |
Phase 4 (a)
gw14 is reintroduced to the historic aliases. rdr becomes the default alias, but the xrootd/webdav aliases keep the current functionality, with partial load put into rdr (<10%)
xrootd | webdav | cmsd | |
---|---|---|---|
yes | no | passive | |
gw4 | |||
gw5 | |||
gw6 | |||
gw7 | |||
gw14 | |||
gw15 | |||
gw16 | |||
svc01 | |||
svc02 | |||
svc97 | |||
svc98 | |||
svc99 | |||
rdr |
Phase
...
4 (b)
When the previous phases pass, all other gws will be removed from the xrootd/webdav alias. All prod traffic will flow trough the redirector. This tests the capacity of the cmsd managers to cope with full prod traffic
xrootd | webdav | cmsd | |
---|---|---|---|
yes | no | passive | |
gw4 | |||
gw5 | |||
gw6 | |||
gw7 | |||
gw14 | |||
gw15 | |||
gw16 | |||
svc01 | |||
svc02 | |||
svc97 | |||
svc98 | |||
svc99 | |||
rdr |
Rollback
If any issues happens during the rollout, Phase1: the rdr dns will be removed from xrootd and webdav aliases and the previous set of gateways will be reinstated.
Phase2/3: VOs will switch back to xrootd/webdav aliases
Phase4: either of the previous depending on implementation