HDFS DataTap Wire Encryption

NOTE This article only applies to HDFS DataTaps.

Wire encryption means that the network packets between virtual node and remote HDFS service are encrypted. This includes:

  • RPC encryption: the RPC messages between the virtual node(s) and the HDFS namenode are encrypted.
  • Data Transfer encryption: The control message and data between the virtual node(s) and the HDFS data nodes are encrypted.

No additional configuration is required to support this feature; however, the HDFS configurations must be modified to enable the wire encryption. Further, the remote HDFS must be Kerberized for security.

To enable wire encryption on a CDH HDFS service:

  1. Enable RPC encryption on the remote HDFS service using the CDH Manager interface, as shown here.

  2. Enable data transfer encryption on the remote HDFS service using the CDH Manager interface, as shown here.

  3. Restart the remote HDFS service.

To enable wire encryption on an HDP HDFS service:

  1. In the Ambari interface, enable RPC encryption by selecting HDFS>Configs>Advanced>Custom core-site, and then adding hadoop.rpc.protection = privacy, as shown here.

  2. In the Ambari interface, enable Data Transfer encryption by selecting HDFS>Configs>Advanced>Custom hdfs-site, add then adding dfs.encrypt.data.transfer = true, as shown here.

  3. Restart the remote HDFS service.
NOTE Currently the dfs.encrypt.data.transfer.algorithm supports AES, CTR, or NoPadding, and the dfs.encrypt.data.transfer,cipher.key.bitlength can support 128, 192, or 256 bits..