Detail
When theforce-linkdown xdr-command info command has been used to simulate the loss of a network link to a remote data center, tracking the XDR related metrics indicates that a few records are shipped to the remote destination after the command is issued. Why does this happen?Answer
This is expected behaviour because the force-linkdown option, when used with the xdr-command info command, mimics the behaviour of XDR in a situation where a link to a remote DC is really down.
The sequence is as follows:
- The link to the remote DC is marked as suspicious.
- XDR shipping continues.
- There is a 30 second grace period (to give the link time to recover in the event of a temporary problem.)
- After 30 seconds the link is checked again.
- If the link is really down or it has been forced down with the
xdr-commandinfo command shipping stops.
It is therefore evident that XDR will continue to ship records during the 30 second grace period after the force linkdown has been issued.
Notes
- The
xdr-commandinfo command is used to force the link between datacenters down when there are multiple XDR destinations and only 1 requires maintenance. - When there is a single XDR destination the
enable-xdrconfiguration parameter can be changed dynamically to pause XDR shipping while still tracking changes in the digest log. - In Aerospike server version 5.0 and above with a re-written XDR, such command will not be necessary as shipping to different DCs will be done independently and rewinding to force reshipping at a point in time will be possible.