Configuring ATS 1.0 or 1.5 for Hadoop 3.3 (Required for Tez UI)

Describes how to configure the YARN Application Timeline Server (ATS) 1.0 and 1.5 for Hadoop 3.3.x. You must complete this process in order to use the Tez UI.

EEP 9.0.0 introduced Hadoop 3.3, which uses ATSv2 by default. Hive 3 and the Tez UI currently do not support ATSv2, but you can configure ATSv1 for use with Hadoop 3, and doing so enables the Tez UI.
NOTE
ATSv2 is incompatible with the Tez UI. ATSv2 should not be installed across the cluster nodes if you are enabling the Tez UI by configuring ATS v1.5.
IMPORTANT
Check that the mapr-timelineserver package, which corresponds to ATSv2, is not installed. If the package is installed, remove it before proceeding.
After removing the mapr-timelineserver package, remove the following properties from yarn-site.xml:
<!--TIMELINE SERVER SECTION-->
<property>
    <description>Indicate to clients whether Timeline service is enabled or not. If enabled, the TimelineClient library used by end-users will post entities and events to the Timeline server.</description>
    <name>yarn.timeline-service.enabled</name>
    <value>true</value>
</property>
<property>
    <name>yarn.timeline-service.hostname</name>
    <value><HOSTNAME></value>
</property>
<property>
    <name>yarn.timeline-service.version</name>
    <value>2.0f</value>
</property>
<property>
    <name>yarn.timeline-service.http-cross-origin.enabled</name>
    <value>true</value>
</property>
<property>
    <name>yarn.system-metrics-publisher.enabled</name>
    <value>true</value>
</property>
<property>
    <name>yarn.rm.system-metrics-publisher.emit-container-events</name>
    <value>true</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services.timeline_collector.class</name>
    <value>org.apache.hadoop.yarn.server.timelineservice.collector.PerNodeTimelineCollectorsAuxService</value>
</property>
<!--TIMELINE SERVER SECTION END-->
<!--TIMELINE SECURITY SECTION-->
<property>
    <name>yarn.timeline-service.http-authentication.type</name>
    <value>org.apache.hadoop.security.token.delegation.web.MaprDelegationTokenAuthenticationHandler</value>
</property>
<!--TIMELINE SECURITY SECTION END-->
Use the following steps:
  1. Install ATS manually using the appropriate command for your distribution:
    NOTE
    ATS 1.5 is the recommended version. ATS 1.5 is faster and more stable, with better scalability and fewer lost events.

    To manage ATS 1.5 with Warden, you must install and configure mapr-timelineserv1.

    • RHEL, Rocky, or Oracle Linux:
      yum install mapr-timelineserverv1
    • Ubuntu:
      apt install mapr-timelineserverv1
    • SLES:
      zypper install mapr-timelineserverv1
  2. Run configure.sh to configure and automatically start ATS 1.0:
    configure.sh -R -TL <hostname>
  3. On all nodes where you are installing the ATS, ensure the following properties are present in the yarn-site.xml file.
    IMPORTANT
    For ATS 1.5, update the value of yarn.timeline-service.version to 1.5f and proceed with the configuration of ATS 1.5 on all nodes.
    <property>
    <name>yarn.timeline-service.enabled</name>
    <value>true</value>
</property>
<property>
    <name>yarn.timeline-service.hostname</name>
    <value><hostname></value>
</property>
<property>
    <name>yarn.resourcemanager.system-metrics-publisher.enabled</name>
    <value>true</value>
</property>
<property>
    <name>yarn.timeline-service.http-cross-origin.enabled</name>
    <value>true</value>
</property>
 <property>
    <name>yarn.timeline-service.version</name>
    <value>1.0f</value>
    <description>Timeline server version. Should be 1.0f or 1.5f</description>
  </property>
  4. For ATS 1.5, add these additional properties:
    <property>
    <name>yarn.timeline-service.entity-group-fs-store.summary-store</name>
    <value>org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore</value>
</property>
<property>
    <name>yarn.timeline-service.store-class</name>
    <value>org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore</value>
</property>
<property>
    <name>yarn.timeline-service.entity-group-fs-store.active-dir</name>
    <value>/apps/ats/active/</value>
</property>
<property>
    <name>yarn.timeline-service.entity-group-fs-store.done-dir</name>
    <value>/apps/ats/done/</value>
</property>
<property>
    <name>yarn.timeline-service.leveldb-timeline-store.path</name>
    <value>/opt/mapr/hadoop/hadoop-<version>/ats/leveldb/</value>
</property>
<property>
    <name>yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes</name>
    <value>org.apache.tez.dag.history.logging.ats.TimelineCachePluginImpl</value>      
</property>
  5. For ATS 1.5, create the following directories in the Hadoop file system:
    Directory Permission uid and gid Owner
    yarn.timeline-service.entity-group-fs-store.active-dir 1777 Cluster admin
    yarn.timeline-service.entity-group-fs-store.done-dir 0700 Cluster admin
    Use these commands: 
    hadoop fs -mkdir -p /apps/ats/active/
hadoop fs -mkdir -p /apps/ats/done/
hadoop fs -chmod 1777 /apps/ats/active/
hadoop fs -chmod 0700 /apps/ats/done/
  6. For ATS 1.5, create the following directory in the local file system:
    sudo mkdir -p /opt/mapr/hadoop/hadoop-<version>/ats/leveldb/
  7. Change the owner manually:
    sudo chown -R mapr:root /opt/mapr/hadoop/hadoop-<version>/ats
  8. For ATS 1.5, copy the following libraries from Tez:
    sudo cp /opt/mapr/tez/tez-<version>/tez-yarn-timeline-cache-plugin-<artifact_name>.jar /opt/mapr/hadoop/hadoop-<hadoop_version>/share/hadoop/yarn/lib/
sudo cp /opt/mapr/tez/tez-<version>/tez-api-<artifact_name>.jar /opt/mapr/hadoop/hadoop-<hadoop_version>/share/hadoop/yarn/lib/
sudo cp /opt/mapr/tez/tez-<version>/tez-common-<artifact_name>.jar /opt/mapr/hadoop/hadoop-<hadoop_version>/share/hadoop/yarn/lib/
  9. If the cluster is secure, add the security property to yarn-site.xml:
    <property>
    <name>yarn.timeline-service.http-authentication.type</name>
    <value>org.apache.hadoop.security.token.delegation.web.MaprDelegationTokenAuthenticationHandler</value>
</property>
  10. On all nodes, restart all YARN services (RM, NM, and JHS):
    maprcli node services -nodes `hostname -f` -name resourcemanager -action restart
maprcli node services -nodes `hostname -f` -name nodemanager -action restart
maprcli node services -nodes `hostname -f` -name historyserver -action restart
  11. To manually configure the Tez UI server so that it can be managed by Warden and the Control System, copy and rename the warden.tezui.conf.template file in the Tez conf dir:
    mv /opt/mapr/tez/tez-<version>/conf/warden.tezui.conf.template /opt/mapr/conf/conf.d/warden.tezui.conf
  12. To start, stop, or restart ATS manually, use the following commands as needed:
    To start ATS:
    maprcli node services -nodes `hostname -f` -name timelineserverv1 -action start
    To stop ATS:
    maprcli node services -nodes `hostname -f` -name timelineserverv1 -action stop
    To restart ATS:
    maprcli node services -nodes `hostname -f` -name timelineserverv1 -action restart