Putting a Node into Maintenance Mode

Describes how to put a node into maintenance mode.

About this task

If you put a node into maintenance mode, the node is marked unserviceable, but is still attached to the cluster.

Before putting a node into maintenance mode, ensure that:
  • All copies of the CLDB volume exist if the node is a CLDB node. You cannot put a CLDB master into maintenance mode until the CLDB service is stopped. You can shut down the CLDB service only if the CLDB is a secondary CLDB or you have enabled high availability for CLDB.

    You cannot put a node into maintenance mode if the node is running as the CLDB master and also supporting file-system services.

  • All running processing tasks (NodeManager and Spark, for example) that depend on the file system have been stopped.
WARNING
Do not put a node under maintenance if there are any volume under-replicated alarms because doing so might take some data completely offline.
To put a node into maintenance mode, perform the following actions:

Procedure

  1. From a terminal, issue the node maintenance command: 
    /opt/mapr/bin/maprcli node maintenance -nodes <IP|hostname> -timeoutminutes <minutes>

    If you run this command, specify a timeout (in minutes) long enough for you to perform necessary maintenance on the node.

    NOTE
    For the duration of the timeout, the cluster CLDB considers this node as unavailable. Under this scenario, the CLDB does not trigger data replication for this node until it exits maintenance mode. Furthermore, clients accessing containers on the node receive the appropriate error and retry other container copies. Note also that if a node is put under maintenance for more than five minutes, the file system shuts down on that node to prevent any other operations from occurring. This value of five minutes is hard-coded and cannot be changed. Even if you reboot the node, the maintenance mode persists until the timeout is reached.
  2. Stop Warden on the node: 
    service mapr-warden stop
    To bring the node back online, see Taking a Node Out of Maintenance Mode.