Configuring ATS 1.0 or 1.5 for Hadoop 3.3

Describes how to configure the YARN Application Timeline Server (ATS) 1.0 and 1.5 for Hadoop 3.3.

EEP 9.0.0 introduced Hadoop 3.3, which uses ATSv2 by default. Hive 3 and the Tez UI currently do not support ATSv2, but you can configure ATSv1 for use with Hadoop 3, and doing so enables the Tez UI. Use the following steps:

Check to ensure that the mapr-timelineserver package didn't install, and remove it if it did.

On all nodes, add the following configuration to the yarn-site.xml file:

<property>
    <name>yarn.timeline-service.enabled</name>
    <value>true</value>
</property>
<property>
    <name>yarn.timeline-service.hostname</name>
    <value><hostname></value>
</property>
<property>
    <name>yarn.resourcemanager.system-metrics-publisher.enabled</name>
    <value>true</value>
</property>
<property>
    <name>yarn.timeline-service.http-cross-origin.enabled</name>
    <value>true</value>
</property>
 <property>
    <name>yarn.timeline-service.version</name>
    <value>1.0f</value>
    <description>Timeline server version. Should be 1.0f or 1.5f</description>
  </property>

For ATS 1.5, add these additional properties:

<property>
    <name>yarn.timeline-service.entity-group-fs-store.summary-store</name>
    <value>org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore</value>
</property>
<property>
    <name>yarn.timeline-service.store-class</name>
    <value>org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore</value>
</property>
<property>
    <name>yarn.timeline-service.entity-group-fs-store.active-dir</name>
    <value>/apps/ats/active/</value>
</property>
<property>
    <name>yarn.timeline-service.entity-group-fs-store.done-dir</name>
    <value>/apps/ats/done/</value>
</property>
<property>
    <name>yarn.timeline-service.leveldb-timeline-store.path</name>
    <value>/opt/mapr/hadoop/hadoop-3.3.4/ats/leveldb/</value>
</property>
<property>
    <name>yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes</name>
    <value>org.apache.tez.dag.history.logging.ats.TimelineCachePluginImpl</value>      
</property>

For ATS 1.5, create the following directories in the Hadoop file system:

Directory	Permission	uid and guid Owner
`yarn.timeline-service.entity-group-fs-store.active-dir`	1777	Cluster admin
`yarn.timeline-service.entity-group-fs-store.done-dir`	0700	Cluster admin

Use these commands:

hadoop fs -mkdir -p /apps/ats/active/
hadoop fs -mkdir -p /apps/ats/done/
hadoop fs -chmod 1777 /apps/ats/active/
hadoop fs -chmod 0700 /apps/ats/done/

For ATS 1.5, create the following directory in the local file system:
```
mkdir -p /opt/mapr/hadoop/hadoop-3.3.4/ats/leveldb/
```

Change the owner manually:

sudo chown -R mapr:root /opt/mapr/hadoop/hadoop-3.3.4/ats

For ATS 1.5, copy the following libraries from Tez:

cp /opt/mapr/tez/tez-<version>/tez-yarn-timeline-cache-plugin-<artifact_name>.jar /opt/mapr/hadoop/hadoop-<hadoop_version>/share/hadoop/yarn/lib/
cp /opt/mapr/tez/tez-<version>/tez-api-<artifact_name>.jar /opt/mapr/hadoop/hadoop-<hadoop_version>/share/hadoop/yarn/lib/
cp /opt/mapr/tez/tez-<version>/tez-common-<artifact_name>.jar /opt/mapr/hadoop/hadoop-<hadoop_version>/share/hadoop/yarn/lib/

If the cluster is secure, add the security property to yarn-site.xml:

<property>
    <name>yarn.timeline-service.http-authentication.type</name>
    <value>org.apache.hadoop.security.token.delegation.web.MaprDelegationTokenAuthenticationHandler</value>
</property>

On all nodes, restart all YARN services (RM, NM, and JHS):

maprcli node services -nodes <node name> -name resourcemanager -action restart
maprcli node services -nodes <node name> -name nodemanager -action restart
maprcli node services -nodes <node name> -name historyserver -action restart
maprcli node services -nodes <node name> -name timelineserver -action restart

On one node, start ATS manually:

/opt/mapr/hadoop/hadoop-<hadoop_version>/bin/yarn --daemon start timelineserver

To stop ATS:

/opt/mapr/hadoop/hadoop-<hadoop_version>/bin/yarn --daemon stop timelineserver