Detail
In a running Aerospike cluster where records are written with non-zero TTL, one of the nodes(Ex: Node A) is taken out for maintenance or is down for some reason. Now, replica node will become master and start serving client requests. Before the Node A is brought back, the TTL for some records are lowered and those got expired. What happens to the records on node A which has higher TTL when the node joins back the cluster?Answer
When a node with higher or 0 TTL went down and TTL has been lowered for records when the node was away, if the record gets expired before the node joins back, then there is chance that the old record with higher TTL will come back as there is no version of that record to compare.This is an expected behavior, so it is not recommended to change TTL when the cluster is not stable or when any of the nodes are away.
Notes
- If a node is away and the TTL was not changed in between but the record is expired by the time that node comes back, then those records get expired while startup as server will have LUT and TTL for a record.