Detail
This FAQ details the possible scenario of adding a node with data to an active roster.Answer
The pre-existing records on a node being added to a Strong Consistency roster may have different behavior.What we do with those records depends on a few things:
1) If the involved partitions had a node with a full copy in the roster (before re-rostering with those nodes), the records coming back would be ‘ignored' by migrations (we would do one way migrations). This is the most likely situation and it doesn’t necessarily mean that those records will be dropped.
The records that were brought back through the nodes added to the roster can end up in any of the following three situations:
- They get dropped (if the nodes coming back end up being neither roster master nor roster replica for the new roster).
- We end up with a single copy (if the nodes coming back end up being roster master or roster replica) or with two copies (RF 2) depending on what other nodes in the new roster end up owning the impacted partitions (and whether they did also own such partition in the previous roster).
- If writes did happen when we had a fresh partition in a roster (when re-rostering down), those would prevail as they would participate in migrations.
2) If the involved partitions did not have a full copy in the roster, we would do two way migrations.
Notes
When re-rostering down before migrations completed and when different nodes are missing on different racks at the time the re-rostering down happen as well as the re-rostering up, all scenarios can happen for different partitions… based on how much a partition had migrated and writes that would have happened in between…Recommendation is to avoid above scenarios and let all fill migrations complete prior to re-rostering.