Problem Description
Evictions were observed after bringing a node back into the cluster without evict.smd.
Jan 13 2025 15:44:01 GMT: INFO (nsup): (nsup.c:314) {test} got smd evict-void-time 432408123
Jan 13 2025 15:44:01 GMT: INFO (nsup): (nsup.c:524) {test} now (474479041) > evict-void-time - using nowExplanation
Since evict.smd was removed, evict-void-time started at 0. Then SMD syncs and we change the evict_void_time, but since it’s also older than ‘now’, we use ‘now’ for that first nsup run. This has nsup go through the evict code path for anything that is expiring in that run.
What happens without SMD is what we see here because of the following:
-
Node starts up – (evict.smd does not exist)
-
evict_void_time starts at 0, and since evict.smd does not exist we set both evict_void_time and smd_evict_void_time to 0
-
SMD then syncs, and we see a change of this value. Since the value changed we check if it’s “later than now”. If now is greater than evict_void_time then we use now (typical cases have evict_void_time in the future which is when we would not use now)
-
Due to the above, we start the evict code path and any record within that time gets early expired (reported as eviction)
This is mostly a reporting behavior where expirations are getting flagged as evictions.
Solution
Working around this by keeping evict.smd should prevent this specific behavior.
Keeping the SMD would help work around this behavior because of the following:
-
Node starts up – (evict.smd exists)
-
evict_void_time starts at 0, but since we have evict.smd we set evict_void_time = smd_evict_void_time
-
At this point, we initialize the eviction function and use the smd_evict_void_time
-
Then next nsup runs will go through expirations (until HWM breach and we trigger evictions)