Enabling Security on a New Cluster Installation

Describes how to enable security for the cluster, platform, ecosystem components, and network-based connections.

About this task

The following steps enable:
  • Security for the cluster nodes
  • Wire-level encryption for the platform and ecosystem components
  • Authentication for all network-based connections
  • (Optional) Data-at-rest encryption on the cluster

These steps DO NOT enable security for client nodes. For client-installation information, see Setting Up Clients and Services.

Use one of the following procedures based on the composition of nodes in your cluster:

Enabling Security When All Nodes Are Non-FIPS

About this task

Use these steps to enable security for a cluster in which all nodes are non-FIPS-enabled nodes:

Procedure

  1. If the cluster is running, shut it down.
  2. If you are re-running the configure.sh script because of an invocation error from a previous run, remove the following files from ${MAPR_HOME}/conf (if they are present) if you want to re-generate the CLDB key, server ticket, and certificates:
    • All key and trust stores. The files differ depending on whether the node is FIPS enabled. FIPS-enabled nodes use BCFKS key and trust stores, while secure non-FIPS nodes use JKS/JCEKS/P12 key and trust stores:
      • maprkeycreds.jceks
      • maprtrustcreds.jceks
      • ssl_keystore, ssl_keystore.p12
      • ssl_truststore, ssl_truststore.p12
      • ssl_userkeystore
      • ssl_usertruststore
    • All other files in ${MAPR_HOME}/conf that are generated and configured on the first CLDB node:
      • All PEM files: ssl_keystore-signed.pem and ssl_userkeystore-signed.pem
      • All files in the ${MAPR_HOME}/conf/tokens directory
      • maprserverticket
      • mapruserticket
      • The store-passwords.txt file containing the clear-text passwords, if not already removed
    For example:
    cd /opt/mapr/conf  
rm -rf ssl_* *.bcfks *.jceks tokens store-passwords.txt maprserverticket mapruserticket  ssl-client.xml ssl-server.xml
    WARNING
    The DARE master key is generated in the tokens/ directory only if data at rest encryption is enabled on the cluster using the -dare option with configure.sh. Deleting the tokens directory is only intended for new installs to re-attempt an initial configuration. If the cluster is already running with DARE enabled, deleting the tokens directory results in a complete cluster loss.
  3. Run the configure.sh script with the -secure -genkeys -dare options on the first CLDB node in your cluster: 
    /opt/mapr/server/configure.sh -secure -dare -genkeys -Z <Zookeeper_node_list> -C <CLDB_node_list> -N <cluster_name>
    where both <Zookeeper_node_list> and <CLDB_node_list> have the form hostname[:port_no][,hostname[:port_no]...] and -N <cluster_name> specifies the cluster name. For the hostname, specify an FQDN as described in Connectivity. Do not specify an alias or IP address. The -dare option is required only if you wish to enable data-at-rest encryption at the cluster-level.
    IMPORTANT
    You must not run configure.sh with the -genkeys option on any other node after running it with the -genkeys option on the first CLDB node. The files must be generated only once on the first CLDB node by running configure.sh with the -genkeys option, and then copied to other nodes in the cluster.
    TIP
    For a comprehensive listing of the Trust and Key Store files, see Understanding the Key Store and Trust Store Files.
  4. Copy files to the destination nodes as follows:
    • If your cluster consists of all secure non-FIPS-enabled nodes, use the following table as a guide to copy files to the destination nodes which are the nodes where the -genkeys option is not used to generate keys.
      Destination Node Type Copy these files under ${MAPR_HOME} to the destination node . . .
      CLDB and/or ZooKeeper Nodes
      • conf/maprhsm.conf
      • conf/maprkeycreds.conf
      • conf/maprkeycreds.jceks
      • conf/maprserverticket
      • conf/maprtrustcreds.conf
      • conf/maprtrustcreds.jceks
      • conf/private.key1
      • conf/public.crt1
      • conf/ssl_keystore
      • conf/ssl_keystore.p12
      • conf/ssl_keystore.pem
      • conf/ssl_keystore-signed.pem
      • conf/ssl_truststore
      • conf/ssl_truststore.p12
      • conf/ssl_truststore.pem
      • conf/ssl_userkeystore
      • conf/ssl_userkeystore.p12
      • conf/ssl_userkeystore.pem
      • conf/ssl_userkeystore-signed.pem
      • conf/ssl_usertruststore
      • conf/ssl_usertruststore.p12
      • conf/ssl_usertruststore.pem
      • conf/tokens (use a command such as scp -r to copy everything in this folder)
      All other cluster nodes, including MFS-only nodes
      • conf/maprhsm.conf
      • conf/maprkeycreds.conf
      • conf/maprkeycreds.jceks
      • conf/maprserverticket
      • conf/maprtrustcreds.conf
      • conf/maprtrustcreds.jceks
      • conf/private.key1
      • conf/public.crt1
      • conf/ssl_keystore
      • conf/ssl_keystore.p12
      • conf/ssl_keystore.pem
      • conf/ssl_keystore-signed.pem
      • conf/ssl_truststore
      • conf/ssl_truststore.p12
      • conf/ssl_truststore.pem
      • conf/ssl_userkeystore
      • conf/ssl_userkeystore.p12
      • conf/ssl_userkeystore.pem
      • conf/ssl_userkeystore-signed.pem
      • conf/ssl_usertruststore
      • conf/ssl_usertruststore.p12
      • conf/ssl_usertruststore.pem
      1If you are running Data Fabric 7.0.0.5 or later, the private.key and public.crt are not present and do not need to be copied to all other nodes. On Data Fabric 7.0.0.5, the /opt/mapr/conf/ssl_usertruststore performs this function and is present on all nodes.
  5. Run configure.sh on each existing node in the cluster using the same arguments as in Step 3 but without the -genkeys option. 
    /opt/mapr/server/configure.sh -secure -dare -Z <Zookeeper_node_list> -C <CLDB_node_list> -N <cluster_name>
    The -secure option indicates that security must be enabled on the node where the command is run. The -dare option indicates that data at rest encryption must be enabled on the node and must be specified only if it was specified in Step 3.
    IMPORTANT
    • You must also do this on any nodes that you add to the cluster in the future.
    • If you run configure.sh -secure on a node before you copy the necessary files to that node, the command fails.
  6. Optionally, enable encrypted quorum ZooKeeper communication. See zoo.cfg for more information.

Enabling Security When All Nodes Are FIPS

About this task

Use these steps to enable security for a cluster in which all nodes are FIPS-enabled:

Procedure

  1. If the cluster is running, shut it down.
  2. If you are re-running the configure.sh script because of an invocation error from a previous run, remove the following files from ${MAPR_HOME}/conf (if they are present) if you want to re-generate the CLDB key, server ticket, and certificates:
    • All key and trust stores. The files differ depending on whether the node is FIPS enabled. FIPS-enabled nodes use BCFKS key and trust stores, while secure non-FIPS nodes use JKS/JCEKS/P12 key and trust stores:
      • maprkeycreds.bcfks
      • maprtrustcreds.bcfks
      • ssl_keystore (symlink), ssl_keystore.bcfks
      • ssl_truststore (symlink), ssl_truststore.bcfks
      • ssl_userkeystore (symlink), ssl_userkeystore.bcfks
      • ssl_usertruststore (symlink), ssl_usertruststore.bcfks
    • All other files in ${MAPR_HOME}/conf that are generated and configured on the first CLDB node:
      • All PEM files: ssl_keystore-signed.pem and ssl_userkeystore-signed.pem
      • All files in the ${MAPR_HOME}/conf/tokens directory
      • maprserverticket
      • mapruserticket
      • The store-passwords.txt file containing the clear-text passwords, if not already removed
    For example:
    cd /opt/mapr/conf  
rm -rf ssl_* *.bcfks *.jceks tokens store-passwords.txt maprserverticket mapruserticket  ssl-client.xml ssl-server.xml
  3. Run the configure.sh script with the -secure -genkeys -dare options on the first CLDB node in your cluster: 
    /opt/mapr/server/configure.sh -secure -dare -genkeys -Z <Zookeeper_node_list> -C <CLDB_node_list> -N <cluster_name>
    where both <Zookeeper_node_list> and <CLDB_node_list> have the form hostname[:port_no][,hostname[:port_no]...] and -N <cluster_name> specifies the cluster name. For the hostname, specify an FQDN as described in Connectivity. Do not specify an alias or IP address. The -dare option is required only if you wish to enable data at rest encryption at the cluster-level.
    IMPORTANT
    You must run configure.sh with the -genkeys option only once on one CLDB node, since the resulting files should be generated only once and then copied to other nodes.
    NOTE
    The DARE master key is generated in the tokens/ directory only if data at rest encryption is enabled on the cluster using the -dare option with configure.sh.
    TIP
    For a comprehensive listing of the Trust and Key Store files, see Understanding the Key Store and Trust Store Files.
  4. Copy files to the destination nodes as follows:
    • If your cluster consists of all FIPS-enabled nodes, use the following table as a guide to copy files to the destination nodes (the nodes where the -genkeys option is not used to generate keys):
      Destination Node Type Copy these files under ${MAPR_HOME} to the destination node . . .
      CLDB and/or ZooKeeper Nodes
      • conf/maprhsm.conf
      • conf/maprkeycreds.bcfks
      • conf/maprkeycreds.conf
      • conf/maprserverticket
      • conf/maprtrustcreds.bcfks
      • conf/maprtrustcreds.conf
      • conf/private.key2
      • conf/public.crt2
      • conf/ssl_keystore.bcfks1
      • conf/ssl_keystore-signed.pem1
      • conf/ssl_keystore.p121
      • conf/ssl_keystore.pem1
      • conf/ssl_truststore.bcfks1
      • conf/ssl_truststore.p121
      • conf/ssl_truststore.pem1
      • conf/ssl_userkeystore.bcfks1
      • conf/ssl_userkeystore.pem1
      • conf/ssl_userkeystore-signed.pem1
      • conf/ssl_usertruststore.bcfks1
      • conf/ssl_usertruststore.pem1
      • conf/tokens (use scp -r to copy everything in this folder)
      All other cluster nodes, including MFS-only nodes
      • conf/maprhsm.conf
      • conf/maprkeycreds.bcfks
      • conf/maprkeycreds.conf
      • conf/maprserverticket
      • conf/maprtrustcreds.bcfks
      • conf/maprtrustcreds.conf
      • conf/private.key2
      • conf/public.crt2
      • conf/ssl_keystore.bcfks1
      • conf/ssl_keystore.p121
      • conf/ssl_keystore.pem1
      • conf/ssl_keystore-signed.pem1
      • conf/ssl_truststore.bcfks1
      • conf/ssl_truststore.p121
      • conf/ssl_truststore.pem1
      • conf/ssl_userkeystore.bcfks1
      • conf/ssl_userkeystore.p121
      • conf/ssl_userkeystore.pem1
      • conf/ssl_userkeystore-signed.pem1
      • conf/ssl_usertruststore.bcfks1
      • conf/ssl_usertruststore.p121
      • conf/ssl_usertruststore.pem1
      1Do NOT copy the ssl_ symlink files contained in the conf/ directory. The symlinks are:
      • ssl_keystore (symlink)
      • ssl_truststore (symlink)
      • ssl_userkeystore (symlink)
      • ssl_usertruststore (symlink)

      2If you are running Data Fabric 7.0.0.5 or later, the private.key and public.crt are not present and do not need to be copied to all other nodes. On Data Fabric 7.0.0.5, the /opt/mapr/conf/ssl_usertruststore performs this function and is present on all nodes.

  5. Run configure.sh on each existing node in the cluster using the same arguments as in Step 3 but without the -genkeys option. 
    /opt/mapr/server/configure.sh -secure -dare -Z <Zookeeper_node_list> -C <CLDB_node_list> -N <cluster_name>
    The -secure option indicates that security must be enabled on the node where the command is run. The -dare option indicates that data at rest encryption must be enabled on the node and must be specified only if it was specified in Step 3.
    IMPORTANT
    • You must also do this on any nodes that you add to the cluster in the future.
    • If you run configure.sh -secure on a node before you copy the necessary files to that node, the command fails.
  6. Optionally, enable encrypted quorum ZooKeeper communication. See zoo.cfg for more information.

Enabling Security for a Mix of FIPS and Secure Non-FIPS Nodes

About this task

A mixed cluster is a cluster consisting of both FIPS-enabled and secure non-FIPS enabled nodes. Since the key and trust store formats are different between FIPS-enabled and secure non-FIPS enabled nodes, the BCFKS stores from FIPS-enabled nodes cannot be copied directly to secure non-FIPS enabled nodes, or vice versa. The Hadoop Credential stores also cannot be copied between FIPS-enabled and secure non-FIPS enabled nodes.

For a mixed configuration, you must:
  • Generate the key and trust store, and user key and trust stores if required, on the secure non-FIPS node using the new ${MAPR_HOME}/server/manageSSLKeys.sh convert utility:
    • After adding a FIPS-enabled node to a cluster consisting of only non-FIPS enabled nodes, generate the BCFKS key and trust stores on the non-FIPS enabled node. Copy them to the ${MAPR_HOME}/conf directory of the FIPS-enabled node before running configure.sh.
    • After adding a secure non-FIPS enabled node to a cluster consisting of only FIPS-enabled nodes, copy the BCFKS key and trust stores from the FIPS-enabled node to a temporary location in the secure non-FIPS enabled node. Generate the JKS key and trust store on the secure non-FIPS enabled node.
  • Run the configure.sh with the -storepasswds option on the node being configured to generate the credential stores.

Enabling Security for the First CLDB Node

About this task

The following steps describe how to enable security for the first CLDB node in the cluster. Note that the data-fabric core platform is installed as secure by default on FIPS-enabled hosts. Security is enabled even if the -secure flag is not specified to the configure.sh script.

Procedure

  1. If the cluster is running, shut it down.
  2. If you are re-running the configure.sh script because of an invocation error from a previous run, remove the following files from ${MAPR_HOME}/conf (if they are present) if you want to re-generate the CLDB key, server ticket, and certificates:
    • All key and trust stores. The files differ depending on whether the node is FIPS enabled. FIPS-enabled nodes use BCFKS key and trust stores, while secure non-FIPS nodes use JKS/JCEKS/P12 key and trust stores:
      FIPS Secure Non-FIPS
      maprkeycreds.bcfks maprkeycreds.jceks
      maprtrustcreds.bcfks maprtrustcreds.jceks
      ssl_keystore (symlink), ssl_keystore.bcfks ssl_keystore, ssl_keystore.p12
      ssl_truststore (symlink), ssl_truststore.bcfks ssl_truststore, ssl_truststore.p12
      ssl_userkeystore (symlink), ssl_userkeystore.bcfks ssl_userkeystore
      ssl_usertruststore (symlink), ssl_usertruststore.bcfks ssl_usertruststore
    • All other files in ${MAPR_HOME}/conf that are generated and configured on the first CLDB node:
      • All PEM files: ssl_keystore-signed.pem and ssl_userkeystore-signed.pem
      • All files in the ${MAPR_HOME}/conf/tokens directory
      • maprserverticket
      • mapruserticket
      • The store-passwords.txt file containing the clear-text passwords, if not already removed
    For example:
    cd /opt/mapr/conf  
rm -rf ssl_* *.bcfks *.jceks tokens store-passwords.txt maprserverticket mapruserticket  ssl-client.xml ssl-server.xml
  3. Run the configure.sh script with the -secure -genkeys -dare options on the first CLDB node in your cluster: 
    /opt/mapr/server/configure.sh -secure -dare -genkeys -Z <Zookeeper_node_list> -C <CLDB_node_list> -N <cluster_name>
    where both <Zookeeper_node_list> and <CLDB_node_list> have the form hostname[:port_no][,hostname[:port_no]...] and -N <cluster_name> specifies the cluster name. For the hostname, specify an FQDN as described in Connectivity. Do not specify an alias or IP address. The -dare option is required only if you wish to enable data at rest encryption at the cluster-level.
    IMPORTANT
    You must run configure.sh with the -genkeys option only once on one CLDB node, since the resulting files should be generated only once and then copied to other nodes.
    NOTE
    The DARE master key is generated in the tokens/ directory only if data at rest encryption is enabled on the cluster using the -dare option with configure.sh.

Enabling Security for Additional Cluster Nodes

About this task

To enable security for additional cluster nodes, run configure.sh without the -genkeys option after copying the required files to the node. For a mixed configuration, first create the key and trust stores on the secure non-FIPS node using the ${MAPR_HOME}/server/manageSSLKeys.sh convert utility. Then copy these stores to the key and trust stores of the additional cluster node:
  • If you are connecting an additional secure non-FIPS cluster node to the first FIPS-enabled cluster node, copy the ssl_keystore.bcfks and ssl_truststore.bcfks from the ${MAPR_HOME}/conf directory of the first FIPS-enabled cluster node to the node being configured. Then run the manageSSLKeys.sh convert utility from the secure non-FIPS node. Copy the converted JKS key and trust stores to the additional secure non-FIPS cluster node (or simply specify the destination key/trust store as ${MAPR_HOME}/conf/ssl_keystore and ${MAPR_HOME}/conf/ssl_truststore respectively in the ${MAPR_HOME}/server/manageSSLKeys.sh convert utility).
  • If you are connecting an additional FIPS-enabled cluster node to the first secure non-FIPS cluster node, copy the JKS ssl_keystore and ssl_truststore from the ${MAPR_HOME}/conf directory of the first secure non-FIPS cluster node to a temporary directory of the first node. Then run the manageSSLKeys.sh convert utility from the first secure non-FIPS node. Copy the converted BCFKS key and trust stores to the ${MAPR_HOME}/conf directory of the additional FIPS-enabled cluster node.

Adding a FIPS-Enabled Server to a FIPS Cluster

About this task

To connect a FIPS-enabled server to a cluster consisting of at least one FIPS-enabled node.

Procedure

  1. Copy the following files from the existing FIPS-enabled server to the new FIPS server:
    Destination Node Type Copy these files under ${MAPR_HOME} to the destination node . . .
    CLDB and/or ZooKeeper nodes
    • conf/ssl_keystore.bcfks
    • conf/ssl_keystore.p12
    • conf/ssl_keystore.pem
    • conf/ssl_truststore.bcfks
    • conf/ssl_truststore.p12
    • conf/ssl_truststore.pem
    • conf/maprkeycreds.bcfks
    • conf/maprkeycreds.conf
    • conf/maprtrustcreds.bcfks
    • conf/maprtrustcreds.conf
    • conf/maprhsm.conf
    • conf/maprhsm.conf
    • conf/maprserverticket
    • conf/tokens (use scp -r to copy everything in this folder)
    All other cluster nodes, including MFS-only nodes
    • conf/ssl_keystore.bcfks
    • conf/ssl_keystore.p12
    • conf/ssl_keystore.pem
    • conf/ssl_truststore.bcfks
    • conf/ssl_truststore.p12
    • conf/ssl_truststore.pem
    • conf/maprkeycreeds.bcfks
    • conf/maprkeycreds.conf
    • conf/maprtrustcreds.bcfks
    • conf/maprtrustcreds.conf
    • conf/maprhsm.conf
    • conf/maprserverticket
    • conf/ca (use a command such as scp -r to copy everything in this folder)
    CAUTION
    Do NOT copy conf/ssl_keystore and conf/ssl_truststore. These are symbolic links to ssl_keystore.bcfks and ssl_truststore.bcfks, which will be generated by configure.sh.
    CAUTION
    When adding a non-FIPS node to a FIPS cluster, DO NOT copy the Hadoop ssl*.xml files to the other cluster nodes. The manageSSLKeys.sh script (invoked by configure.sh) uses the store type to determine if FIPS is enabled and assumes the system is FIPS-enabled if the store type is BCFKS. Copying the Hadoop ssl* files that are set to the BCFKS store type from a FIPS node to a non-FIPS node causes the configure.sh script to fail.
  2. Run configure.sh without the -genkeys option. For example, if the cluster name is fips0.cluster.com and the CLDB and ZooKeeper nodes are at m2-mapreng-vm166250, then the command is: 
    /opt/mapr/server/configure.sh -secure -N fips0.cluster.com \ 
    -C m2-mapreng-vm166250:7222

Adding a Secure Non-FIPS Server to a FIPS Cluster

About this task

Non-FIPS enabled nodes do not support the BCFKS trust store format. Copying the BCFKS trust store from a FIPS-enabled server to the non-FIPS enabled server that is being added will not work. Create the JKS trust store on the non-FIPS server by importing the same keys and certificates that are in the BCFKS key and trust stores on the existing FIPS-enabled server host. Different configuration procedures apply depending on whether you are configuring for the first cluster or for subsequent clusters.

Procedure

  1. Copy the following files from an existing FIPS-enabled node in the cluster to the new non-FIPS node being added:
    Destination Node Type Copy these files under ${MAPR_HOME} to the destination node . . .
    CLDB and/or ZooKeeper nodes
    • conf/ssl_keystore.p12
    • conf/ssl_keystore.pem
    • conf/ssl_truststore.p12
    • conf/ssl_truststore.pem
    • conf/maprkeycreds.conf
    • conf/maprtrustcreds.conf
    • conf/maprhsm.conf
    • conf/maprserverticket
    • conf/tokens (use scp -r to copy everything in this folder)
    All other cluster nodes, including MFS-only nodes
    • conf/ssl_keystore.p12
    • conf/ssl_keystore.pem
    • conf/ssl_truststore.p12
    • conf/ssl_truststore.pem
    • conf/maprkeycreds.conf
    • conf/maprtrustcreds.conf
    • conf/maprhsm.conf
    • conf/maprserverticket
    • conf/ca (use a command such as scp -r to copy everything in this folder)
    CAUTION
    When adding a non-FIPS node to a FIPS cluster, DO NOT copy the Hadoop ssl*.xml files to the other cluster nodes. The manageSSLKeys.sh script (invoked by configure.sh) uses the store type to determine if FIPS is enabled and assumes the system is FIPS-enabled if the store type is BCFKS. Copying the Hadoop ssl* files that are set to the BCFKS store type from a FIPS node to a non-FIPS node causes the configure.sh script to fail.
  2. Copy the following key store, trust store, userkey store, and usertrust store files from the FIPS-enabled server to a temporary directory of the secure non-FIPS enabled server being added:
    • ${MAPR_HOME}/conf/ssl_keystore.bcfks
    • ${MAPR_HOME}/conf/ssl_truststore.bcfks
    • ${MAPR_HOME}/conf/ssl_userkeystore.bcfks
    • ${MAPR_HOME}/conf/ssl_usertruststore.bcfks
  3. Run the manageSSLKeys.sh convert utility to convert the key and trust store (and userkey and usertruststore) from BCFKS format to JKS format. The destination key and trust store will be set to the same password as the source key/trust store. You can obtain the key and trust store passwords from the store-passwords.txt file. For example: 
    # /opt/mapr/server/manageSSLKeys.sh convert \ 
    -srcType bcfks -dstType JKS \ 
    -p VccOl_Qhg3Ix6tLaRJhzr_b53judiaKC \ 
    /tmp/ssl_keystore.bcfks /opt/mapr/conf/ssl_keystore 
# /opt/mapr/server/manageSSLKeys.sh convert \ 
    -srcType bcfks -dstType JKS \ 
    -p 1IB_wtxT5Lbj6OU8xFpWpQiZ0SjE6BrA \ 
    /tmp/ssl_truststore.bcfks /opt/mapr/conf/ssl_truststore
# /opt/mapr/server/manageSSLKeys.sh convert \ 
    -srcType bcfks -dstType JKS \ 
    -p VccOl_Qhg3Ix6tLaRJhzr_b53judiaKC \ 
    /tmp/ssl_userkeystore.bcfks /opt/mapr/conf/ssl_userkeystore 
# /opt/mapr/server/manageSSLKeys.sh convert \ 
    -srcType bcfks -dstType JKS \ 
    -p 1IB_wtxT5Lbj6OU8xFpWpQiZ0SjE6BrA \ 
    /tmp/ssl_usertruststore.bcfks /opt/mapr/conf/ssl_usertruststore
  4. Run the configure.sh script without the -genkeys option on the secure non-FIPS enabled server being added, using the -storepasswds option to specify the key and trust store passwords. Since the converted key and trust stores are set to the same password as the source, the passwords must be the same as the passwords you specified using the -p option in step 3. For example: 
    # /opt/mapr/server/configure.sh -secure \ 
    -N hpe186.cluster.com \ 
    -C m2-mapreng-vm167186:7222 \ 
    -Z m2-mapreng-vm167186:5181 \ 
    -storepasswds \ 
    VccOl_Qhg3Ix6tLaRJhzr_b53judiaKC:1IB_wtxT5Lbj6OU8xFpWpQiZ0SjE6BrA

Adding a FIPS Server to a Secure Non-FIPS Cluster

About this task

Use the following steps to connect a FIPS-enabled server to a cluster consisting of only secure non-FIPS enabled nodes:,

Procedure

  1. Copy the following files from an existing secure non-FIPS node in the cluster to the FIPS-enabled server being added:
    Destination Node Type Copy these files under ${MAPR_HOME} to the destination node . . .
    CLDB and/or ZooKeeper nodes
    • conf/ssl_keystore.p12
    • conf/ssl_keystore.pem
    • conf/ssl_truststore.p12
    • conf/ssl_truststore.pem
    • conf/maprkeycreds.conf
    • conf/maprtrustcreds.conf
    • conf/maprhsm.conf
    • conf/maprserverticket
    • conf/tokens (use scp -r to copy everything in this folder)
    All other cluster nodes, including MFS-only nodes
    • conf/ssl_keystore.p12
    • conf/ssl_keystore.pem
    • conf/ssl_truststore.p12
    • conf/ssl_truststore.pem
    • conf/maprkeycreds.conf
    • conf/maprtrustcreds.conf
    • conf/maprhsm.conf
    • conf/maprserverticket
    • conf/ca (use a command such as scp -r to copy everything in this folder)
    CAUTION
    When adding a non-FIPS node to a FIPS cluster, DO NOT copy the Hadoop ssl*.xml files to the other cluster nodes. The manageSSLKeys.sh script (invoked by configure.sh) uses the store type to determine if FIPS is enabled and assumes the system is FIPS-enabled if the store type is BCFKS. Copying the Hadoop ssl* files that are set to the BCFKS store type from a FIPS node to a non-FIPS node causes the configure.sh script to fail.
  2. On the secure non-FIPS enabled server in the existing cluster, run the manageSSLKeys.sh convert utility to convert the key and trust store (and userkey and usertruststore) from JKS to BCFKS format. You can obtain the key and trust store passwords from the store-passwords.txt file. For example: 
    # /opt/mapr/server/manageSSLKeys.sh convert \ 
    -srcType JKS -dstType bcfks \ 
    -p VccOl_Qhg3Ix6tLaRJhzr_b53judiaKC \ 
    /opt/mapr/conf/ssl_keystore /tmp/ssl_keystore.bcfks 
# /opt/mapr/server/manageSSLKeys.sh convert \ 
    -srcType JKS -dstType bcfks \ 
    -p 1IB_wtxT5Lbj6OU8xFpWpQiZ0SjE6BrA \ 
    /opt/mapr/conf/ssl_truststore /tmp/ssl_truststore.bcfks 
# /opt/mapr/server/manageSSLKeys.sh convert \ 
    -srcType JKS -dstType bcfks \ 
    -p VccOl_Qhg3Ix6tLaRJhzr_b53judiaKC \ 
    /opt/mapr/conf/ssl_userkeystore /tmp/ssl_userkeystore.bcfks 
# /opt/mapr/server/manageSSLKeys.sh convert \ 
    -srcType JKS -dstType bcfks \ 
    -p 1IB_wtxT5Lbj6OU8xFpWpQiZ0SjE6BrA \ 
    /opt/mapr/conf/ssl_usertruststore /tmp/ssl_usertruststore.bcfks
  3. Copy the converted .bcfks files from the secure non-FIPS server to the FIPS server being added as follows:
    Copy this converted file . . . To this location on the FIPS server . . .
    ssl_keystore.bcfks /opt/mapr/conf/ssl_keystore.bcfks
    ssl_userkeystore.bcfks /opt/mapr/conf/ssl_userkeystore.bcfks
    ssl_truststore.bcfks /opt/mapr/conf/ssl_truststore.bcfks
    ssl_usertruststore.bcfks /opt/mapr/conf/ssl_usertruststore.bcfks
  4. Run configure.sh without the -genkeys option on the FIPS enabled server being added, using the -storepasswds option to specify the key and trust store passwords. Since the converted BCFKS key and trust store is set to the same password as the source, the passwords must be the same as the passwords specified using the -p option in step 2. For example: 
    /opt/mapr/server/configure.sh -secure \ 
  -N hpe186.cluster.com \ 
  -C m2-mapreng-vm167186:7222 \ 
  -Z m2-mapreng-vm167186:5181 \ 
  -storepasswds \  
VccOl_Qhg3Ix6tLaRJhzr_b53judiaKC:1IB_wtxT5Lbj6OU8xFpWpQiZ0SjE6BrA