Configuring Secure Clusters for Cross-Cluster Mirroring and Replication

Describes configuring clusters for cross-cluster operations such as mirroring and replication.

About this task

Cross-cluster tickets are required on secure clusters that need to pull data from another secure cluster and on secure clusters that need to push data to another secure cluster. For example:
  • Volume mirroring is a pull operation. The destination cluster pulls the volume data from the source cluster. Since the destination cluster performs the operation, the destination cluster receives a ticket that is generated on the source cluster.
  • Table and streams replication is a push operation. The source cluster pushes table or stream data to the destination cluster. Since the source cluster performs the operation, the source cluster receives a ticket that is generated on the destination cluster.

You can configure secure clusters for cross-cluster mirroring and replication manually (as described in Manually Setting up Secure Clusters for Cross-Cluster Mirroring and Manually Setting up Secure Clusters for Cross-Cluster Replication). You can configure secure clusters automatically, by running the configure-crosscluster.sh utility. This utility configures the clusters for both mirroring and replication in both directions. For more information, see configure-crosscluster.sh.

Manually Setting up Secure Clusters for Cross-Cluster Mirroring

About this task

To set up secure clusters for cross-cluster mirroring:

Procedure

  1. Verify that the user for whom you are configuring access, exists in the registry on both the clusters and has the following permissions:
    • Permissions to create volumes on the source cluster.
    • Permissions to mirror volumes on the destination cluster.
    You can set up access for the mapr user, who already has permissions to create volumes and mirror volumes.
  2. Configure source cluster (clusterA) to communicate with the other clusters by editing the mapr-clusters.conf file on each node of clusterA to specify the hostname or IP address of the CLDB nodes on the other clusters.
    For example, suppose:
    • The /opt/mapr/conf/mapr-clusters.conf file on the source cluster (clusterA) contains the following: 
      clusterA.cluster.com secure=true perfnode50.lab:7222
    • The /opt/mapr/conf/mapr-clusters.conf file on the destination cluster (clusterB) contains the following: 
      clusterB.cluster.com secure=true perfnode100.lab:7222

    Perform the following steps to configure the nodes on the clusters:

    1. On any node in clusterA, append the first entry from clusterB’s mapr-clusters.conf file, the entry which is prefixed with the cluster name, to the end of clusterA’s mapr-clusters.conf file.
      Note that clusterA’s entry must be the first line of the mapr-clusters.conf file: 
      clusterA.cluster.com secure=true perfnode50.lab:7222
clusterB.cluster.com secure=true perfnode100.lab:7222
      The mapr-clusters.conf file for clusterA now contains two entries.
    2. Copy the updated /opt/mapr/conf/mapr-clusters.conf file to all the other nodes in clusterA.
    3. On any node in the destination cluster (clusterB), append the first entry from clusterA’s mapr-clusters.conf file, entry which is prefixed with the cluster name, to the end of the remote cluster’s mapr-clusters.conf file.
      Note that clusterB’s entry must be the first line of the mapr-clusters.conf file: 
      clusterB.cluster.com secure=true perfnode100.lab:7222
clusterA.cluster.com secure=true perfnode50.lab:7222
      The mapr-clusters.conf file for clusterB now contains two entries.
    4. Copy the updated /opt/mapr/conf/mapr-clusters.conf file to all the nodes in clusterB.
    See mapr-clusters.conf.
  3. Log in to any node on the source cluster (ClusterA) and perform the following steps:
    1. Generate a cross-cluster ticket for the destination cluster (clusterB) for the mapr user.
      For example, to generate a cross-cluster for the destination cluster (clusterB), run the following command on the source cluster (clusterA): 
      /opt/mapr/bin/maprlogin generateticket -type crosscluster -out /tmp/crossclusterticket -user destinationclusteruser
    2. Copy the cross-cluster ticket file to any node on the destination cluster (clusterB).
      For example: 
      scp /tmp/crossclusterticket mapr@<dest-ip>:/tmp/sourceClusterTicketFile
  4. Log in to the node on the destination cluster (clusterB) where the cross-cluster ticket was copied, and perform the following steps:
    1. Merge the cross-cluster ticket file with the /opt/mapr/conf/maprserverticket file on the node.
      For example, to merge, run the following command: 
      cat /tmp/sourceClusterTicketFile >> /opt/mapr/conf/maprserverticket
    2. Copy the /opt/mapr/conf/maprserverticket file to all the CLDB nodes on the destination cluster.
  5. Merge the ssl_truststore files by using the /opt/mapr/server/manageSSLKeys.sh tool.
    In this step, you use the copytruststore option of manageSSLKeys.sh to create a copy of the truststore. Then you copy it to the destination node using SCP, and finally run merge without any additional options. For example:
    1. On clusterA, create a new ssl_truststore by using the copytruststore option: 
      /opt/mapr/server/manageSSLKeys.sh copytruststore /tmp/clusterA_ssl_truststore <ssl.server.truststore.password>  <ssl.server.truststore.password>
      You can obtain the ssl.server.truststore.password password from the /opt/mapr/conf/store-passwords.txt file for the key ssl.server.truststore.password on the CLDB master node for clusterA.
    2. Copy the ssl_truststore from clusterA to clusterB: 
      scp mapr@<remote-ip>:/opt/mapr/conf/clusterA_ssl_truststore /tmp/clusterA_ssl_truststore
    3. Merge the ssl_truststore on clusterB: 
      /opt/mapr/server/manageSSLKeys.sh merge /tmp/clusterA_ssl_truststore /opt/mapr/conf/ssl_truststore <ssl.server.truststore.password on ClusterA>  <ssl.server.truststore.password on clusterB>
      You can obtain the password for ssl.server.truststore.password from the /opt/mapr/conf/store-passwords.txt file for the key ssl.server.truststore.password on the CLDB master node for clusterA.

      You can obtain the password for ssl.server.truststore.password from the /opt/mapr/conf/store-passwords.txt file for the key ssl.server.truststore.password on the CLDB master node for clusterB.

  6. Copy the merged ssl_truststore file to every node on clusterB.
  7. Generate ticket for root user to clusterA from clusterB by using the following command: 
    maprlogin password -cluster clusterA
  8. Optional: If your clusters are secure, configure your source cluster so that you can use the Control System to set up and administer table replication from the source to the destination cluster.
    These steps make it convenient to use the Control System for setting up and managing replication involving two secure clusters. However, before following them, perform these prerequisite tasks.
    NOTE
    • Ensure that both clusters are managed by the same team or group. The UIDs and GIDs of the users that are able to log in to the Control System on the source cluster must exactly match their UIDs and GIDs on the destination cluster. This restriction applies only to access to both clusters through the Control System, and does not apply to access to both clusters through the maprcli. If the clusters are managed by different teams or groups, use the maprcli instead of the Control System to set up and manage table replication involving two secure clusters.
    • Ensure that the proper file-system and table permissions are in place on both clusters. Otherwise, any user who can log into the Control System and has the same UID or GID on the destination cluster will be able to set up replication either from the source cluster to the destination cluster or vice versa. A user could create one or more tables on the destination cluster, enable replication to them from the source cluster, load the new tables with data from the source cluster, and start replication. A user could also create tables on the source cluster, enable replication to them from tables in the destination cluster, load the new tables with data from the destination cluster, and start replication.
    1. On the source cluster (clusterA), generate a service ticket by using the maprlogin command: 
      maprlogin generateticket -type service -cluster <destination cluster>
-user mapr -duration <duration> -out <output folder>

      Where -duration is the length of time before the ticket expires. You can specify the value in either of these formats:

      • [Days:]Hours:Minutes
      • Seconds
    2. To every node of the destination cluster (clusterB), add the service ticket to the file /opt/mapr/conf/mapruserticket file: 
      cat <path and filename of the service ticket> >> /opt/mapr/conf/mapruserticket
    3. Restart the web server by running the maprcli node services command. For the syntax of this command, see node services.
  9. Perform the steps to verify configuration for mirroring.

Results

You can now create mirror volumes on the destination cluster and set up a schedule to pull data from the volumes on the source cluster. However, you cannot create volumes on the source cluster that pull data from volumes in the destination cluster, because the setup described above is unidirectional. To configure the clusters for bidirectional mirroring, repeat the steps above, by switching the source and destination clusters.

For example, suppose there are two clusters, clusterA and clusterB, and you performed the steps above for clusterA as the source cluster and clusterB as the destination cluster. After you complete the steps above, your destination cluster, clusterB can pull data from volumes on clusterA. For clusterA to mirror data on clusterB, perform the steps above with clusterB as the source cluster and clusterA as the destination cluster.

Manually Setting up Secure Clusters for Cross-Cluster Replication

About this task

To set up secure clusters for cross-cluster replication:

Procedure

  1. Verify that the user, for whom you are configuring access, exists in the registry on the destination cluster.
  2. Log in to any node on the destination cluster and perform the following steps:
    1. Generate a cross-cluster ticket for the source cluster.
      For example, to generate a cross-cluster for the source cluster, run the following command on the destination cluster: 
      /opt/mapr/bin/maprlogin generateticket -type crosscluster -out /tmp/crossclusterticket -user destinationclusteruser
    2. Copy the cross-cluster ticket file to any node on the source cluster.
      For example: 
      scp /tmp/crossclusterticket mapr@<source-ip>:/tmp/sourceClusterTicketFile
  3. Log in to the node in the source cluster where the cross-cluster ticket was copied, and perform the following steps:
    1. Merge the cross-cluster ticket file with the /opt/mapr/conf/maprserverticket file on the node.
      For example, to merge, run the following command: 
      cat /tmp/destinationClusterTicketFile >> /opt/mapr/conf/maprserverticket
    2. Copy the /opt/mapr/conf/maprserverticket file to all the nodes on the source cluster.
  4. Configure the Gateway for table and streams replication.
    See Configuring Gateways for Table and Stream Replication for more information.
  5. Perform the steps to verify configuration for replication.

Results

You can now set up volumes on the source cluster to push data to replicas on the destination cluster. However, you cannot create replicas on the source cluster that get data from volumes in the destination cluster because the setup described above is unidirectional. To configure the clusters for bidirectional replication, repeat the steps above by switching the source and destination clusters.

For example, suppose there are two clusters, clusterA and clusterB, and you performed the steps above for clusterA as the source cluster and clusterB as the destination cluster. After you complete the steps above, your source cluster, clusterA can push data to replicas on clusterB. For clusterB to replicate data on clusterA, perform the steps above with clusterB as the source cluster, and clusterA as the destination cluster.

Verifying Cross-Cluster Configuration for Mirroring and Replication

About this task

You can verify the cross-cluster configuration for:

Procedure

  1. Mirroring by logging in to a node on the destination cluster as the user for whom access was configured, and creating a mirror volume on the destination cluster for a volume on the source cluster.
    You can create mirror volumes using the Control System and/or the CLI.
  2. Replication by logging in to a node on the source cluster as the user for whom access was configured and creating a replica in the destination cluster for a volume, table, and stream on the source cluster.
    You can create replicas using the Control System and the CLI. To set up replication on secure clusters for: