Problem Description
You see messages like this in the Aerospike logs:
WARNING (xdr-client): (cluster.c:522) bad proto version 83 - 0x00007eee0000ee00 192.168.1.1:3000
WARNING (xdr-client): (cluster.c:522) bad proto version 0 - 0x0000000000000000 192.168.1.2:3000
Explanation
A "bad proto version" error message indicates that a destination XDR server returned a false or misleading response to the XDR client.
Solution
The possible triggers of this error are, in decreasing order of likelihood:
-
XDR seeds on the source are configured and point to a non-Aerospike node. This can also occur if an Aerospike node has another process running on the configured port or the seed node port is misconfigured. To remediate, make sure all XDR seeds are pointing to reachable addresses of nodes of the intended cluster.
- While configuring XDR proxy, the source cluster xdr configuration should have connector set to true.
-
The destination nodes are publishing the wrong IP addresses. For example, the
access-addressoralternate-access-addressare configured with IP addresses that belong to other systems. See Network Configuration for more information. -
Network level corruption such as corrupt network packets from NIC driver issues. This is outside the scope of Aerospike and will need to be addressed by your networking or hardware team.
-
Rapid connection churns occurring under heavy workload and impacting destination's networking environment. This may indicate your cluster is not sized appropriately for your workload.
Notes
This could be due to a particular network interface reaching a hardware limit or some firmware issue.