How to Recover Shared Memory (shmem) Usage by adding new nodes

Context

Alternative to rolling cold start, adding new empty nodes may be a better option for recovering shared memory usage, especially in a cloud environment.

Method

• Set migrate-max-num-incoming=0 on the existing nodes in the cluster.

• Add the new nodes.
• Quiesce the old nodes.

• Restore migrate-max-num-incoming if migration stalls.
• Wait for migration to complete.
• Shut down the old nodes.

Notes

You can monitor migration with:

Admin> show stat like record_re tions_active -flip

Example output:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Test Namespace Statistics (2025-09-30 23:12:25 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            Node|migrate_record_receives|migrate_record_retransmits|migrate_rx_partitions_active|migrate_tx_partitions_active
10.88.0.7:3103  |                     18|                         0|                           0|                           0
10.88.0.7:3102  |                   1219|                         0|                           0|                           0
Number of rows: 2

• Any nonzero migrate_record_retransmits indicates migration parameters are too aggressive and may need tuning.

Applies To Earliest Version

Applies To Latest Version