Hadoop Client Compatibility
Describes compatibility between Hadoop 2.x and Hadoop 3.x clients and servers.
Hadoop Client-Server Compatibility Matrix
As indicated in Hadoop Protocol Versions, EEP 9.0.0 and later include Hadoop 3, while previous EEP versions used Hadoop 2. The following table
summarizes Hadoop 2 and Hadoop 3 client-server compatibility:
1Some configuration is required. Use the following step.
| Hadoop 3 Server | Hadoop 2 Server | |
|---|---|---|
| Hadoop 3 Client | Compatible | Not Compatible |
| Hadoop 2 Client | Compatible1 | Compatible |
Using a Hadoop 2 Client with a Hadoop 3 Server
To use a Hadoop 2 client with a Hadoop 3 server, add the following properties to the
<HADOOP_HOME>/etc/hadoop/mapred-site.xml
file:<property>
<name>yarn.app.mapreduce.am.staging-dir</name>
<value>/var/mapr/cluster/yarn/rm/staging</value>
</property>
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory on the server node}</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory on the server node}</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory on the server node}</value>
</property>The first property change is needed for the Hadoop 2 client contained in EEP 8.1.0. In EEP 8.1.0, the default path was changed to /var/mapr/cluster/yarn/hs. In
EEP
9.0.0 and later, the
default was changed back to /var/mapr/cluster/yarn/rm/staging.