configure-crosscluster.sh

Use the configure-crosscluster.sh utility to set up cross-cluster security between two clusters.

You can use the configure-crosscluster.sh utility to set up cross-cluster security between two clusters. When you run this utility with the create subcommand, it establishes security between the local cluster and a remote cluster. After the setup, communication between the two clusters is bi-directional. You can run this utility on any node in the source cluster to grant secure access to users and servers (for replication, or mirroring) on the destination cluster.

The utility prompts for the passwords for both the local and remote clusters. All hosts on a cluster must have the same password. Alternatively, you can use ssh public key authentication between the current node and the other nodes in the local and remote clusters.

Prerequisites

Before running this utility, you must:

  • Enable wire-level security for both the local and remote clusters.

    Set secure=true in your cluster entry for /opt/mapr/conf/mapr-clusters.conf, for both the local and remote clusters.

  • Ensure that the local and remote cluster are either all non-FIPS-enabled nodes or all FIPS-enabled nodes. See FIPS Support later on this page.
  • Install the pssh (Parallel SSH) package from EPEL.
  • Install the expect package.

If you plan to use a user other than the mapr administrative user for mirroring or gateway/streams replication, that user must already exist on both the local and remote clusters.

Permissions

In release 7.0.0 and later, the configure-crosscluster.sh script returns an error if it is run by a user other than the cluster owner (as configured in ${MAPR_HOME}/conf/daemon.conf). For example, an error is generated if you run the script as the root superuser:
# /opt/mapr/server/configure-crosscluster.sh create all \
    -remoteip my_ip_addr.mycorp.net \
    -localtruststorepassword QHj9NmFyZ_j9_BUnyooEWa898a_xNvQY \
    -remotetruststorepassword 0KHy3gbF05XTYOTjLQPGXCibtGDpI_Di
User is root. This script must be run as the cluster owner mapr

Syntax

/opt/mapr/server/configure-crosscluster.sh create <cross-cluster-type>
    [ -localcrossclusteruser <user> ]
    [ -localhosts <path_to_file> ]
    [ -localport <port_number> ]
    [ -localtruststorepassword <password> ]
    [ -remotetruststorepassword <password> ]
    [ -localuser <user> ]
    [ -recover <id> ]
    [ -remotecrossclusteruser <user> ]
    [ -remotehosts <port_number> ]
    -remoteip <ip_address>
    [ -remoteport <port_number> ]
    [ -remoteuser <user> ]

Required Parameters

In release 7.0.0 and later, the trust store passwords are no longer in plain text, and the configure-crosscluster.sh script cannot automatically retrieve them. Therefore, you must provide the passwords:

localtruststorepassword
Password for the local trust store. Beginning with release 7.0.0, this parameter is required if the ssl-server.xml file does not contain the plain-text password.
remotetruststorepassword
Password for the remote trust store. Beginning with release 7.0.0, this parameter is required.

Type Parameters

The <cross-cluster-type> parameter specifies the type of entity for which cross-cluster access must be established. The value can be one of the following:

user
Used for direct data access for the given user. When you run the utility with the user parameter, it performs the following tasks on both the clusters:
  1. Updates the /opt/mapr/conf/mapr-clusters.conf file to include the first entry from the /opt/mapr/conf/mapr-clusters.conf file on the other cluster.
  2. Imports the certificate of the other cluster in the /opt/mapr/conf/ssl_truststore file, and copies the updated /opt/mapr/conf/ssl_truststore file to all the other nodes on the cluster.
server
Used for data-fabric server access such as mirroring and replication. When you run the utility with the server parameter, it performs the following tasks on both the clusters:
  1. Generates a cross-cluster ticket on this cluster for the other cluster, and copies the ticket to the CLDB node on the other cluster.
  2. Merges the ticket with the /opt/mapr/conf/maprserverticket file on the node on the other cluster, and copies the updated /opt/mapr/conf/maprserverticket file to all the other CLDB nodes on the other cluster.
all
Used for both user and server access. When you run the utility with the all parameter, it performs the following actions on both the clusters:
  1. Updates the /opt/mapr/conf/mapr-clusters.conf file to include the first entry from the /opt/mapr/conf/mapr-clusters.conf file on the other cluster.
  2. Imports the certificate of the other cluster in the /opt/mapr/conf/ssl_truststore file, and copies the updated /opt/mapr/conf/ssl_truststore file to all the other nodes on the cluster.
  3. Generates a cross-cluster ticket for the other cluster, copies the ticket to the CLDB node on the other cluster, merges the ticket with the /opt/mapr/conf/maprserverticket file on the node in the other cluster, and copies the updated /opt/mapr/conf/maprserverticket file to all other CLDB nodes on the other cluster.

Options

The configure-crosscluster.sh utility supports the following options:

localcrossclusteruser
Default value: local user
This option applies only to the server parameter. Specifies the name of the local cross-cluster user if different from the local user, for mirroring and replication of tables, and streams.
localhosts
Default value: No Default Value
Contains the full or relative path to the file containing the list of IP addresses or host names of the hosts to update in the local cluster. Specify one host per line in the file, excluding the current host. If you specify this option, the utility updates the configuration, both on the host on which you are running the utility, and on the other nodes specified in the file. If you do not specify this option, the utility copies both the:
  • Updated server security configuration in the /opt/mapr/conf/maprserverticket file to only the CLDB nodes in the local cluster.
  • Updated user security configuration in the /opt/mapr/conf/mapr-clusters.conf file, and the /opt/mapr/conf/ssl_truststore file, to all the nodes in the local cluster.
localport
Default value: 22
Indicates the port to use to connect (using ssh or scp) to local cluster hosts.
localuser
Default value: mapr
Specifies the name of the user for the local cluster.
recover
Default value: No Default Value
Defines the option to recover from the failure to copy files to nodes in the local or remote cluster, due to failed cluster nodes. Use the special ID keyword latest to run with the contents of the most recent run. See Troubleshooting and Recovery for more information.
remotecrossclusteruser
Default value: remote user
This option applies only to the server parameter. Specifies the name of the remote cross-cluster user, if different from the remote user.
remotehosts
Default value: No Default Value
Contains the full or relative path to the file containing the list of IP addresses or host names of the hosts to update in the remote cluster. Specify one host per line in the file. If you do not specify this option, the utility copies both the:
  • Updated server security configuration in the /opt/mapr/conf/maprserverticket file to only the CLDB nodes in the remote cluster.
  • Updated user security configuration in the /opt/mapr/conf/mapr-clusters.conf file, and the /opt/mapr/conf/ssl_truststore file, to all the nodes in the remote cluster.

For example, if you have a file myhosts.txt in the current directory with the following contents:

10.10.20.100
10.10.20.101
10.10.20.102

then, specify -remotehosts myhosts.txt for this parameter.

ATTENTION
All hosts specified in this file must be directly reachable from the local hosts from which you run the configure-crosscluster.sh utility.
remoteip
Default value: No Default Value
This option is mandatory. Specifies the host name or IP address of a host in the remote cluster.
remoteport
Default value: 22
Indicates the port to use to connect (using ssh or scp) to remote cluster hosts.
remoteuser
Default value: local user
Designates the name of the user for the remote cluster.

FIPS Support

In release 7.0.0 and later, the configure-crosscluster.sh script supports FIPS clusters subject to the following limitations:
  • Both the local and remote cluster must have nodes with the same FIPS setting. This means that one of the following statements is true:
    • All nodes in the local and remote cluster are secure non-FIPS nodes.
    • All nodes in the local and remote cluster are FIPS-enabled nodes.
  • configure-crosscluster.sh with the basic options does not support mixed configurations consisting of combinations of FIPS and non-FIPS nodes. However, mixed configurations can be supported using manual steps. See Configuring Cross-Cluster Security for a Mixed (FIPS and Non-FIPS) Configuration.

Verification

To verify that cross-cluster security is correctly set up, perform one of the following actions:

  • If you ran the utility using the cross-cluster type user or all, and the utility completed successfully, you should be able to run remote commands from the local node after obtaining a user ticket using the maprlogin utility. See Configuring Secure Clusters for Running Commands Remotely for more information.
  • If you ran the utility using the cross-cluster type server or all, and the utility completed successfully, you should be able to perform various service operations from the local to the remote cluster and vice versa, including mounting volumes over NFS, mirroring volumes, and replicating tables and streams. See Configuring Secure Clusters for Cross-Cluster Mirroring and Replication for more information.

Sample Session

To configure cross-cluster security, run the utility on a CLDB host with wire-level security enabled, :

# /opt/mapr/server/configure-crosscluster.sh create all -remoteip 10.10.30.96
Remote IP is 10.10.30.96
WARNING: Strict host key checking will be disabled for this script.
Local user unset, defaulting to mapr
Remote user unset, defaulting to local user mapr
Enter password for mapr user (mapr) for local cluster: 
Enter password for mapr user (mapr) for remote cluster: 
Local cross-cluster user unset, defaulting to local user mapr
Remote cross-cluster user unset, defaulting to remote user mapr
Verifying connectivity to 10.10.30.96 and presence of mapr-clusters.conf
MapR credentials of user 'mapr' for cluster 'myCluster.cluster.com' are written to '/tmp/maprticket_0'
Local host is running the CLDB
chyelin101.cluster.com secure=true qa-cnode101.lab:7222
Configuring cross-cluster communication for users
Certificate stored in file </tmp/mapr-xcs/29668/local_mapcert>
Certificate stored in file </tmp/mapr-xcs/29668/remote_mapcert>
Successfully exported certificate for remote cluster to /tmp/mapr-xcs/29668/remote_mapcert
Certificate was added to keystore
Certificate was added to keystore
Configuring cross-cluster communication for server-side operations
Generating cross-cluster ticket for user mapr on remote node
Generating cross-cluster ticket for mirroring for user mapr
MapR credentials of user 'mapr' for cluster 'myCluster.cluster.com' are written to '/tmp/mapr-xcs/29668/local_crosscluster_ticket'
SUCCESS
This script has logged in to both the local and remote clusters. Please log out of
the clusters if needed.

Cleanup

After running the utility, you must perform two cleanup actions:

  1. Log out of the local and remote clusters if needed, using the maprlogin logout command.
  2. Delete the /tmp/mapr-xcs directory, if it is present, after verifying that the cross-cluster setup is correct.

    If you run this utility without the -recover option, the utility creates temporary files in the /tmp/mapr-xcs directory under the current process ID. These directories contain sensitive information such as server tickets that are protected by Unix permissions. The utility preserves these tickets, so that you can perform troubleshooting and recovery actions, as needed. You must delete this directory after verifying that the cross-cluster setup is correct:

    $ /bin/rm -rf /tmp/mapr-xcs

Post-Configuration Tasks

After you run this utility, cross-cluster security should be successfully set up between the local and the remote cluster. If you specified either the user or all cross-cluster type when running the utility, to perform any operations on the remote cluster from the local node, login to the remote cluster to obtain a user ticket using the maprlogin command.

$ maprlogin password -cluster <remote-cluster-name>
NOTE
You must obtain a new ticket when your current ticket expires.

Examples of Using the Cross-Cluster Utility

For examples on how to run the configure-crosscluster.sh utility, see Configure-crosscluster.sh Examples.