Articles in this section

Aerospike Support Alert - May 5, 2026 - Aerospike XDR Stackoverflow

*** Aerospike Support Alert - May 5, 2026 ***

*** Aerospike XDR Stackoverflow ***

Issue: A critical issue has been identified in Aerospike’s XDR (Cross-Datacenter Replication) component that can lead to a stack overflow, resulting in node instability or crashes.

Affected Releases:

Major Release Vulnerable Version Range Minimum Fixed Version
Aerospike 8

8.1.1.0 - 8.1.1.2

8.1.0.0 - 8.1.0.3

8.0.0.3 - 8.0.0.14

8.1.2.0

8.1.2.0

8.0.0.15

Aerospike 7

7.2.0.8 - 7.2.0.16

7.1.0.14 - 7.1.0.22

7.0.0.11 - 7.0.0.26

7.2.0.17

7.1.0.23

7.1.0.23

Aerospike 6

6.4.0.30 - 6.4.0.33

6.3.0.35

6.2.0.37

6.1.0.47

6.4.0.34

6.3.0.36

6.3.0.36

6.3.0.36

Legacy (< 6.1) All versions prior to 6.1.0.47 N/A

Background: A critical  issue in Aerospike (specifically AER-6898) has been identified as the root cause of intermittent Aerospike node crashes. Impacted clusters may experience sudden node crashes, when a partition changes state.

Immediate Actions: If you are using the affected versions, you should upgrade immediately.

Detection and Safe Upgrade Path: Operators can identify nodes at risk of crashing unexpectedly and mitigate the risk during rolling restarts.

  1. Detection: Execute the xdr-dc-state info command. A node is currently in a problematic state if the retry queue size is greater than 50 and the role state is marked as P (prole).
  2. Safe Upgrade Procedure: 
    1. To prevent the problematic state from triggering the bug during an upgrade: 
    2. Temporarily disable XDR entirely by removing all namespaces from all datacenters. 
    3. Proceed with the node upgrade
    4. Re-add the namespaces. 
    5. This effectively clears the retry queue and bypasses the Issue.
  3. Alternative Bridge-Cluster Procedure:
    1. If stopping XDR is not operationally feasible, use a bridge-cluster approach instead of a rolling upgrade on the vulnerable cluster.
    2. Build a new fixed-version bridge cluster in parallel to the existing cluster.
    3. Configure one-way XDR from the existing cluster to the new bridge cluster.
    4. Allow initial population and ongoing change capture, then validate replication and client behavior on the new cluster.
    5. After validation, move client traffic and replication destinations to the new cluster in stages, then retire the old cluster.

An XDR Rewind may be needed to recover missing replication during the maintenance

 

Detailed Description: A logic error in the XDR subsystem results in a stack overflow. Specifically, when a node transitions from prole to master with a high retry-queue count, the system attempts to allocate a stack-based variable-length array (VLA) based on a negative integer. This results in an inaccurate stack allocation, immediately exceeding the 8MB stack limit which leads to potential memory corruption and a crash of the node.

If you have questions or concerns, don't hesitate to contact Aerospike Support by opening a ticket in the Aerospike Support Portal.

Thank you,

Aerospike Support

Was this article helpful?
0 out of 0 found this helpful