Articles in this section

How to implement rack awareness dynamically on a cluster

Context

How it works

The following examples illustrate how rack awareness operates.

  • When configuring three racks with replication factor of 3 (RF3), each rack receives one replica for a given partition.
    • The three replicas are each on their own racks. Specific nodes are distributed as specified in the succession list order.
  • If you lose a rack, the number of replicas is eventually restored to match the value of your replication-factor. For instance:
    • your server is configued for RF3
    • you reduce the number of racks from 3 racks to 2
    • one rack hosts the master
    • the other rack hosts one replica
    • and the third replica moves to one of the two racks.
  • To avoid the possibility of having data missing from the cluster when a physical rack ( or Availability Zone) is down, configure rack awareness to use multiple racks defined by rack-id.

    Method

    1. On each node use asadm's manage config command to change the rack-id to the desired value:
      asadm -e "enable; manage config namespace <namespaceName> param rack-id to 1 with <host>"
      or use asinfo's set-config command: 
      asinfo -v "set-config:context=namespace;id=namespaceName;rack-id=1"
       
    2. Add the rack-id to the namespace stanza in the configuration file to ensure that the configuration persists following any restarts.
    3. Trigger a rebalance of the cluster to engage migrations with the new rack-id configurations.

    Notes

    Note: Tools package 6.0.x or later is required to use asadm's manage config commands. manage config requires asadm to be in enable mode by typing enable. Otherwise, use the equivalent asinfo - set-config command.

    Note: Make sure to persist your changes in the configuration file to protect against a rollback due to restart. Also, verify that there are no typos in the configuration file. This is the best practice for updating the configuration file for any dynamic change.

    Applies To Earliest Version

    4.9

    Applies To Latest Version

    Current Version
    Was this article helpful?
    0 out of 0 found this helpful