Hadoop (EEP 7.0.0) Release Notes

The notes below relate to the HPE Ezmeral Data Fabric distribution of Apache Hadoop.

These release notes contain only data-fabric-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Release Date September 2020
Version Interoperability See EEP Components and OS Support.
GitHub Source https://github.com/mapr/hadoop-common/
GitHub Release Tag
Maven Artifacts https://repository.mapr.com/maven/
Package Names Navigate to https://package.ezmeral.hpe.com/releases/MEP/ and select your EEP and OS to view the list of package names.

New in this Release

This release:

  • Decouples Hadoop from core and splits Hadoop into the following packages:
    • mapr-hadoop-util
    • mapr-hadoop-client
    • mapr-hadoop-core
    The mapr-mapreduce2 package has been removed.
  • Decouples the following YARN resources from core and moves them to the MEP repository:
    • mapr-nodemanager
    • mapr-resourcemanager
    • mapr-historyserver
    • mapr-timelineserver
  • Provides support for YARN CGroups.
  • Adds a CLI feature to manage the node.labels file that updates and autogenerates the labels file.
  • Provides support for JDK 11.
  • Adds saving ACE for Hadoop FS commands.


This data-fabric release includes the following fixes on the base Apache release. For details, refer to the commit log for this project in GitHub.

Commit Date (YYYY-MM-DD) Comment
e70d0068212 2020-09-04 MAPRHADOOP-110: Added java8_support option to run distributed shell on jdk8
27e5feefb53 2020-08-28 MAPRYARN-232: fix bug with label expression.
82608b1e9d7 2020-08-27 MAPRHADOOP-99: Added support for jmxagent at JMX_REMOTEHOST
22cd170dc77 2020-08-25 MAPRHADOOP-109: create output directory for MapReduce tasks
8d49274d29e 2020-08-25 CORE-235: 'yarn.nodemanager.linux-container-executor.group' settings are not configured properly for non mapr admin user
9af92b94658 2020-08-25 MAPRHADOOP-110: Distributedshell application fix issues MAPRHADOOP-84 and MAPRHADOOP-107
7ff579baeb1 2020-08-14 MAPRHADOOP-108: Hadoop configure.sh doesn't work on Mac OS
0097c56aeda 2020-08-13 MAPRHADOOP-84: Fixed illegal reflective access warning on Windows node
2d51e2ff1c2 2020-08-13 MAPRHADOOP-107: Change GC to ParallelGC for YARN services and containers
670e6816a71 2020-08-13 MAPRHADOOP-110: Fix OOM issue in DistributedShell.
b95185e9953 2020-08-12 MAPRHADOOP-87: added retry property to get token authentication exception
63cabe9cf05 2020-08-12 MAPRYARN-267: Doesn't work link from RM to AM and HS at the non-secure cluster
9acc2c994af 2020-08-10 Backported YARN-8844. TestNMProxy unit test is failing. (Eric Yang via wangda)
40d5ea49fd6 2020-08-10 Backported YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. (cherry picked from commit 00058167431475c6e63c80207424f1d365569e3a)
7f396a21a1c 2020-08-10 Backported HADOOP-11875. [JDK9] Adding a second copy of Hamlet without _ as a one-character identifier.
cc73db526b4 2020-08-10 Backported HADOOP-11875. [JDK9] Adding a second copy of Hamlet without _ as a one-character identifier.
cb153bc81ed 2020-08-10 Backported Fixed import at LdapGroupsMapping
5063d1f7170 2020-08-10 Backported HADOOP-15936. [JDK 11] MiniDFSClusterManager & MiniHadoopClusterManager compilation fails due to the usage of '_' as identifier. Contributed by Zsolt Venczel.
55cc8d04ed0 2020-08-10 Backported HADOOP-16299. [JDK 11] Build fails without specifying -Djavac.version=11
f93adaec25a 2020-08-10 Set java 11 as default for compile
68e65b80365 2020-08-10 Backported HADOOP-15764. Addendum patch: Fix NPE in SecurityUtil.
cf292164bc4 2020-08-10 Backported HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka.
cee2b9a3dbd 2020-08-10 Backported HADOOP-15756. [JDK10] Migrate from sun.net.util.IPAddressUtil to the replacement. Contributed by Akira Ajisaka.
3d1f99ab0b2 2020-08-10 MAPRYARN-265: Blkio CGroups - blkio.weight replaced by new blkio.bfq.weight start from centos8
ec7a13d362a 2020-08-06 MAPRDB-2274: Fixed hadoop_symlinks.sh script for Mac
83186016c2d 2020-08-05 MAPRYARN-241: handle JMX options
d51c8f5d9cd 2020-08-03 MAPRHADOOP-106: Added configure-hadoop.bat to Hadoop project
f36240f896c 2020-07-31 MAPRHADOOP-103: Renamed ext-conf/get-jars-list.sh into ext-conf/mapr-eco-config.sh
cbe7c9d10d6 2020-07-31 MAPRYARN-266: Jobs fails when yarn.use-central-logging-for-mapreduce-only = true
b66ca7bff1c 2020-07-30 MAPRHADOOP-105: Hadoop build fail to generate headers on Windows
a96c5e9fd78 2020-07-29 Backported HADOOP-14586. StringIndexOutOfBoundsException breaks org.apache.hadoop.util.Shell on 2.7.x with Java 9. Contributed by Uwe Schindler.
fb9c3aaef18 2020-07-24 MAPRHADOOP-103: Added get-jars-list.sh for use by Core mapr-config.sh (CORE-221)
143c1ec6a9f 2020-07-24 MFS-2465: added aces prototypes to hdfs.h unitTested skipBuild review @ Rajesh
9cc3222e9c8 2020-07-23 MAPRHADOOP-102: error -1 while copying aces from MapR-FS to local FS
a6f621fa884 2020-07-23 MAPRHADOOP-101: Add ext-conf to the hadoop tar file
f8b702588e6 2020-07-21 Backported HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297
5ba299d1b8a 2020-07-21 Backported YARN-8426:Upgrade jquery-ui to 1.12.1 in YARN. Contributed by Sunil Govindan
0e206b9fd7d 2020-07-21 Backported HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan.
c952b19130d 2020-07-16 MAPRHADOOP-97: Hadoop applications throw warnings
fc6d22f86eb 2020-07-16 MAPRHADOOP-96: Add support for cp/mv commands if symlink was created for directory
0d41507075b 2020-07-13 MAPRYARN-257: Sls tests finish work when run from command line
ad69d2a86cd 2020-07-09 MFS-6696: hadoop fs -mv should preserve source file ACEs to destination file ACEs within cross volumes
8c6fb1dae6e 2020-07-08 MAPRYARN-261: yarn admin users not able to see logs for running jobs
591c85a5ea2 2020-07-08 MAPRHADOOP-84: Illegal reflective access of ShimLoader when using jdk11 for containers
5fda517dbd7 2020-07-08 MAPRYARN-261: yarn admin users not able to see logs for running jobs
ed860bbd99c 2020-07-07 CORE-443: Fixed UMask to 022 for YARN services
14ae8ff7ea7 2020-07-07 MAPRYARN-223: Set max idle time for http and https connection
cb1f7ad20cb 2020-07-06 MAPRYARN-261: Yarn admin users not able to see logs for running jobs
b84ceb51267 2020-07-01 MAPRHADOOP-83: Updated tomcat version to 9.0.36
33e2f935f28 2020-06-19 Backported YARN-8147. TestClientRMService#testGetApplications sporadically fails. Contributed by Jason Lowe
847b75e0b08 2020-06-19 Backported HADOOP-13220. Follow on fixups after upgraded mini-kdc using Kerby. Contributed by Jiajia Li
13312bb98af 2020-06-19 Backported HADOOP-16935. Backport HADOOP-10848. Cleanup calling of sun.security.krb5.Config. (#1912)
82c9d989086 2020-06-19 Backported HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
9cef4b0168e 2020-06-17 CORE-434: Linking more Hadoop jar files under Core, for Zookeeper to use
fa1eefe0ef6 2020-06-11 MAPRYARN-258: Publish system metrics to ATS in batches
f70d35cc38d 2020-06-10 MAPRYARN-254: Webproxy failed to validate certificate
067edee9c63 2020-06-05 MAPRHADOOP-90: Removed hadoop-pipes from Hadoop project
1e675edcebb 2020-06-03 MAPRHADOOP-87: added retry property for renew token authentication exception
50898cdb44e 2020-05-28 MAPRHADOOP-83: Upgrade Tomcat to 9.0.34 at the hadoop-kms.
139f9229675 2020-05-28 Backported HADOOP-16439. Upgrade bundled Tomcat in branch-2.
f25fa6db3bb 2020-05-26 MAPRYARN-232: fixed docs and refactored cli feature to manage the node.lables file
2ef67211a1f 2020-05-22 MAPRMR-16: Remove symlink creation to mapr libraries from the hadoop-util post install
a4e4914a409 2020-05-22 MAPRHADOOP-84: Illegal reflective access of ShimLoader when using Java11
740ecbf7c45 2020-05-21 MAPRMR-16: Added script for symlinks creating
9a87ce6c498 2020-05-14 MAPRHADOOP-81: Update htrace-core to 4.2.0-incubating
e4c36d6e350 2020-05-14 Backported HDFS-9187. Fix null pointer error in Globber when FS was not constructed via FileSystem#createFileSystem (cmccabe)
f4367bb816d 2020-05-14 Backported HDFS-9080. Update htrace version to 4.0.1 (cmccabe)
53341245fe5 2020-05-14 Backported HDFS-8008. Support client-side back off when the datanodes are congested. Contributed by Haohui Mai.
1f54dc59c12 2020-05-14 Backported HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk (cmccabe)
65fbf4e65ae 2020-05-14 Backported HDFS-8100. Refactor DFSClient.Conf to a standalone class and separates short-circuit related conf to ShortCircuitConf.
33b4376755f 2020-05-14 Backported HDFS-7854. Separate class DataStreamer out of DFSOutputStream. Contributed by Li Bo.
20076f7d9ad 2020-05-08 MAPRYARN-250: Added timeout to rescan intermediate_done dirs
094f99dc84f 2020-05-07 SPARK-616: Removed MapRTicketGenerator
3f8150da966 2020-05-07 MAPRHADOOP-82: Fixed exception after guava upgrade
eea73281d19 2020-05-07 SPARK-616: Move to use MapRTicketGenerator from core by default
0622a9745dd 2020-05-07 MAPRHADOOP-82: Update Guava version to 28.2-jre
9cce8949947 2020-05-07 Backported HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
f48d8562069 2020-05-07 Backported HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2.
ae7dd42e74e 2020-05-07 Backported HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2.
ad083f5de6e 2020-05-07 Backported HADOOP-14382 Remove usages of MoreObjects.toStringHelper. Contributed by Andrew Wang
47e9d711513 2020-05-04 MAPRYARN-249: Clear disks after preemption
92c3887fb7f 2020-04-30 Backported YARN-6175. Negative vcore for resource needed to preempt
a527fe7fe56 2020-04-24 MAPRHADOOP-76: Review and clean up unnecessary files from Hadoop 2.7.4
9b138200dec 2020-04-24 Backported YARN-9783. Remove low-level zookeeper test to be able to build Hadoop against zookeeper 3.5.5. Contributed by Mate Szalay-Beko.
2b38194bc49 2020-04-24 Backported HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 (#1656). Contributed by Norbert Kalmár, Mate Szalay-Beko
015f61e3b7e 2020-04-15 MAPRHADOOP-72: Fixed hadoop for work with java 1.8
33ac369d359 2020-04-15 Backported HADOOP-15767. [JDK10] Building native package on JDK10 fails due to missing javah. Contributed by Takanobu Asanuma.
8a870750f7d 2020-04-15 Backported HADOOP-14056. Update maven-javadoc-plugin to 2.10.4.
a901c178b9f 2020-04-15 Backported HDFS-11610. sun.net.spi.nameservice.NameService has moved to a new location. Contributed by Akira Ajisaka.
6e97361b594 2020-04-15 Backported HADOOP-12760. sun.misc.Cleaner has moved to a new location in OpenJDK 9. Contributed by Akira Ajisaka.
c75f864bcf3 2020-04-15 HADOOP-15304. [JDK10] Migrate from com.sun.tools.doclets to the replacement. Contributed by Akira Ajisaka.
8ab08d9a70b 2020-04-10 MAPRHADOOP-73: Add ability to manage node labels through RM Rest API
16fe1cfc914 2020-03-30 MAPRHADOOP-27: Configure hadoop on the edge node
e66229651ca 2020-03-24 MAPRYARN-246: RM Hangs with a FATAL error when space is provided in the queue name
622decf8b00 2020-03-24 Backported YARN-3400. [JDK 8] Build Failure due to unreported exceptions in RPCUtil (rkanter)
a1947649fe9 2020-03-24 MAPRYARN-232: Add CLI / maprcli feature to manage the node.lables file updates or autogenerate the labels file
e4d6f999430 2020-03-23 Backported HADOOP-13773. Wrong HADOOP_CLIENT_OPTS in hadoop-env on branch-2. Contributed by Fei Hui
7066457622e 2020-03-19 MAPRYARM-241: Added JMX configuration to YARN RM/NM
c50bc6d60ce 2020-03-18 MAPRYARN-154: Cleanup extra init mapreduce directory
cbbb05dc11b 2020-03-11 MAPRYARN-244: RM hangs after calling System.exit()
85ab8d513de 2020-03-10 MFS-4656: added hdfs api to set ticket
bbd3cd4deee 2020-03-04 Backported YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian.
ebb6b4b11bb 2020-02-27 MAPRHADOOP-28: Hadoop - Distribute Notice.txt across components starting with MEP 7.0
8657ed9fae9 2020-02-27 Backported YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana
58746fe320e 2020-02-21 MAPRHADOOP-70: hadoop fs -cp -p and distCp should copy source file aces to destination file aces
fff79ee878b 2020-02-19 MAPRMR-14: DistCp error - duplicate files in input path
39e3b919f1e 2020-02-13 MAPRMR-13: Distcp link file copy doesn't work
825a1af3f29 2020-02-05 MAPRYARN-183: Nodemanager log level change for message: Can not find metadata for a job. Returning service metadata
e08862759b1 2020-01-30 MAPRYARN-224: Added logic to clean up Spill files directory
12b429601c3 2020-01-23 MAPRHADOOP-69: Disable TLSv1.1 by default and uses TLSv1.2 by default
2edb37aede0 2020-01-23 MAPRMR-12: Added support blocksperchunk for large files in distcp
18d193a7509 2020-01-17 MAPRYARN-211: Fix problem that NodeManager don't start when CPU lower than 4 and MFSCPU=4
9d1b3450a79 2020-01-17 MAPRYARN-185: Added CGroup support. Backported YARN-3365, YARN-3443, YARN-3366, YARN-2619, YARN-1912, YARN-3684, YARN-160, YARN-2194, YARN-2194, YARN-3982, YARN-3853, YARN-4253, YARN-1856, YARN-3542, YARN-4578, YARN-5849, YARN-6500, YARN-5301, YARN-6757, YARN-4744, YARN-6433, YARN-7818
0b68b6052f9 2020-01-17 MAPRYARN-224: NM should always remove spill directories or completed applications
28bd7d86327 2020-01-15 MAPRHADOOP-43: Evaluate impact of Protobuf upgrade for 6.2 to Protobuf 3.11.1 on Hadoop
ce5ba4702bd 2020-01-13 MAPRYARN-227: Infinite authentication loop when try to access AM UI from RM UI
6a34523c0cf 2019-12-23 MAPRYARN-223: Make jetty connection max idle time configurable
499840501ef 2019-12-18 MAPRHADOOP-64: upgraded tomcat to 6.0.53 version
2bbef34188b 2019-12-16 MAPRHADOOP-63: Removed jackson-databind dependency because it has security vulnerability
adb89d7b1c2 2019-12-10 MAPRHADOOP-61: Setting custom ticket location makes Kerberos fail for services
c8641f01c44 2019-11-25 MAPRYARN-221: ContainerLocalizer process hangs
c9f7560dd8e 2019-11-25 MAPRYARN-168: NM/RM process do not start with XX:ErrorFile option (fix for Ubuntu and SLES)
631e2c09e64 2019-11-14

MAPRYARN-218: Log aggregation has not completed or is not enabled

MAPRYARN-219: AggregatedLogDeletionService: Logs of all application containers are not removed when logs of single container are not accessible

51b197c0ef3 2019-11-11 MFS-3295: Fix build breakage due to an earlier commit. Including some header files needed for uid_t.
9778fb86988 2019-11-09 MAPRYARN-217: No metadata found for application by other users
36ef2eb3677 2019-11-08 MFS-3295: Added a new API hdfsConnectAsUid() for connecting to mapr cluster using a uid.
18288f2b6ed 2019-11-08 MAPRYARN-216: Node local log aggregation does not support hostname:port name format in command "yarn logs ... -nodeAddress"
0bc3a053882 2019-11-06 MAPRYARN-215: NodeLocalMetadataWriter init failed when restart NodeManager and mfs simultaneously
719246c1b94 2019-11-06 MAPRYARN-214: HistoryServer UI doesn't show local aggregated Logs
67608235160 2019-10-31 MAPRYARN-210: Node-local log aggregation
b9e18c50841 2019-10-09 CORE-297: Added security headers for RM, NM, JHS, TLS
2f41fd0ec1f 2019-09-23 MAPRYARN-208: Update Dockerfile, hadoop_env_checks and start-build-env files
f41206b97f8 2019-09-23 Backported HADOOP-11843. Make setting up the build environment easier. Contributed by Niels Basjes.
6aed14c8703 2019-09-18 MAPRHADOOP-59: Added debug logs to aggregated log deletion logic.
4200186ea25 2019-09-18 MAPRHADOOP-58: Changed version of commons-configuration because of CVE-2014-0114.
614d51ece3d 2019-09-04 MAPRHADOOP-54: Fixed getLabelsOnNode REST API method
3137436bea3 2019-08-14 MAPRHADOOP-57: Updated commons-net and commons-io versions
28a970b6c25 2019-08-02 Backported HADOOP-15865. ConcurrentModificationException in Configuration.overlay() method. Contributed by Oleksandr Shevchenko.
935de182482 2019-07-30 MAPRHADOOP-36: adding path to libjvm.so native lib to LD_LIBRARY_PATH env variable
b9514d38e04 2019-07-19 MAPRYARN-202: FairScheduler improvements. Contains partly or fully. YARN-1287. Consolidate MockClocks. YARN-4090. Make Collections.sort() more efficient by caching resource usage YARN-7382. NoSuchElementException in FairScheduler after failover causes RM crash
f2bdb27a4cd 2019-07-19 MAPRYARN-162: added preemptedDisks and allocatedDisks fields
1a29a9bca14 2019-07-12 MAPRHADOOP-37: add outDir null check
e94706b9266 2019-07-09 MAPRHADOOP-54: Add ability to fetch node labels through RM Rest API
bb7d41d794c 2019-07-04 MAPRYARN-200: Backport YARN-6797: TimelineWriter does not fully consume the POST response
1b02dfa19d0 2019-06-27 MAPRHADOOP-52: update dependency (upgrade version commons-compress and netty)
d0315775926 2019-06-27 Backported HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash.
bbfad1bdc9f 2019-06-26 MAPRYARN-191: Fixed some ClientRMService node/labels related methods.
28e0de41fc3 2019-06-21 MAPRHADOOP-49: Hadoop configuration after installation incomplete
f8560fdd218 2019-06-20 MAPRYARN-195: Fixed compare method at DominantResourceFairnessPolicy, change weight calculation for CPU, MEMORY, DISK.
ecc0d96cba4 2019-06-20 Backported YARN-8436. FSParentQueue: Comparison method violates its general contract. (Wilfred Spiegelenburg via Haibo Chen)
30ab0e70ef3 2019-06-18 MAPRMR-8: Added property for the configuration max number of blocks in the split for CombineFileInputFormat
dd01c093203 2019-06-14 MAPRHADOOP-37: Add copyAce logic on running Map Tasks
428acd4d377 2019-06-07 MAPRYARN-192: Refactored label wrapper to handle edge conditions
632659bfa7f 2019-06-07 MAPRYARN-191: Edited node labels page to work with MapR specific.
6bdf170a840 2019-06-07 MAPRYARN-191: Fixed yarn cluster -lnl to properly show label information.
8d95c10257a 2019-06-06 MAPRYARN-192: Added AND support to label expression
51919781c70 2019-06-04 MAPRYARN-154: Added creation of nm-staging volume to LocalVolumeAuxService.
97028f090e8 2019-06-03 MAPRYARN-168: Added -XX:ErrorFile default location to YARN_OPTS.
a44b8b3c195 2019-05-31 MAPRYARN-182: Added AM/PM flag to last heartbeat report time.
b43cb6a34a8 2019-05-30 MAPRHADOOP-37: added copyAce method to AbstractMapRFileSystem.
1cd2861e7d2 2019-05-29 MAPRYARN-187: Repair SLS to work with hadoop-2.7.4-mapr.
ab38d390906 2019-05-28 Backported YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter)
a283450c281 2019-05-24 MAPRHADOOP-42: Excluded mapr specific properties from map-reduce unit tests
7d3f70347e3 2019-05-23 Backported YARN-6078. Containers stuck in Localizing state. Adapted to 2.7 hadoop. (cherry picked from commit bd1a53c)
2e3f1b93ab3 2019-05-23 Backported YARN-6004. Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines. (Chris Trezzo via mingma) (cherry picked from commit 61424da)
9c9a8a89d49 2019-05-22 MAPRHADOOP-42: Excluded mapr specific properties from unit tests
a043d169aeb 2019-05-13 MAPRYARN-183: changed the log levels to WARN from ERROR at LocalVolumeAusService.java
e18c6c7bf33 2019-05-13 MAPRHADOOP-40: Set default mapr_home in ssl-client/server conf
a7489d421c6 2019-05-13 MAPRHADOOP-34: Property mapreduce.shuffle.ssl.enabled was add to the mapred-site.xml for secure clusters
12909327cc6 2019-05-07 MAPRHADOOP-39: Option -R doesn't clean up yarn-site.xml
81af4616c5a 2019-05-03 MAPRYARN-173: Fixed wrong constant usage in TestCapacityScheduler. Fixed using maxVcores instead of maxDisks in refreshMinimumAllocation in AbstractYarnScheduler. Created testRefreshMaximumResourceAllocationShouldReturnSameResourcesAsInArgument.
7ffee8be41e 2019-04-26 MAPRYARN-179: Fixed FairSharePolicy comparator.
0de1dad6d74 2019-04-24 MAPRHADOOP-36: Add condition for getting MaprFS instead of LocalFS for writing logs in case of enabled DFS logging.
bebab711a51 2019-04-24 MAPRMR-7: added new variable to FileSystem.Cache.Key
9c32b7a4296 2019-04-23 Backported HDFS-9612. DistCp worker threads are not terminated after jobs are done. (Wei-Chiu Chuang via Yongjun Zhang)
86199b5232f 2019-04-23 Backported HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang.
26dbab3311b 2019-04-23 MAPRYARN-175: yarn job fails to initialize when output directory is set to target cluster
32ae1a69d69 2019-04-18 Backported YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. Fix using existing Resource.java API in 2.7.0 hadoop.
0627a2c1ec6 2019-04-18 Backported YARN-6769. Make schedulables without demand less needy in FairSharePolicy#compare. (Yunfan Zhou via Yufei Gu)
e9cb861304a 2019-04-18 Backported YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee)
61cd9d9efdf 2019-04-10 MAPRYARN-171: jobs comparison(with disk usage) was fixed.
934258896f5 2019-03-22 MAPRYARN-161: History server Deletion thread stops once it finds an invalid application directory
1b4b61f19b5 2019-03-22 MAPRYARN-167: Timeline had incorrect configuration
00384e786b6 2019-03-21 MAPRYARN-169: Moved hadoop-maprfs package to hadoop-common
fae981d7127 2019-03-20 MAPRYARN-165: Options for setting oom_score_adj for NM child containers
239f2753da5 2019-03-15 MAPRHADOOP-32: Configure yarn-site before change permission
ef6d389c39f 2019-03-13 MFS-2179: Add few more C++ apis to hdfs header
11414518543 2019-03-13 Backported YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena
efb21906cd0 2019-03-13 Backported YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when state-store error occurs. Contributed by Jian He
9e7298ee372 2019-03-13 Backported YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He.
bfa630eba18 2019-03-13 Backported YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong (cherry picked from commit 1ba31fe9e906dbd093afd4b254216601967a4a7b)
6087ca36843 2019-03-07 MAPRYARN-155. changed "dot" in dfs logging property to "underscore".
d0daa13f506 2019-02-04 COMSECURE-117: Cleanup external token only for finished application
feb85e70e82 2018-11-07 Backported MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen (cherry picked from commit f1b74a3d9ff71bc014dbfd29a6996071b81d14c5)
b5120c88b17 2018-11-07 Backported MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu (cherry picked from commit bf70c5ae2824a9139c1aa9d7c14020018881cec2)
6014bac6e00 2018-10-23 Backported YARN-4153. TestAsyncDispatcher failed at branch-2.7. Contributed by Zhihai Xu
fc37a3014ce 2018-10-23 Backported YARN-3999. RM hangs on draing events. Contributed by Jian He
b26e91baae3 2018-10-23 Backported YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles)
c944bad4679 2018-10-16 Backported MAPREDUCE-6817. The format of job start time in JHS is different from those of submit and finish time. (Haibo Chen via kasha)
ad4fbefa295 2018-10-16 Backported MAPREDUCE-6620. Jobs that did not start are shown as starting in 1969 in the JHS web UI (haibochen via rkanter)
767c4bc75c2 2018-10-11 MAPR-32181: Added max delete size for ATS deletion thread wake up
88f57119dfc 2018-09-29 Backported YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding
7fe347bd5a5 2018-09-25 MAPR-32319: Containers fail during creating a symlink which started with hyphen for a resource file
9c8adf15343 2018-09-17 Backported MAPREDUCE-7101. Add config parameter to allow JHS to alway scan user dir irrespective of modTime. (Thomas Marquardt via wangda)
5ebae0609bd 2018-09-17 Backported MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du
18fb150b10a 2018-09-17 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B
6ffc9e2dd6f 2018-09-17 MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee)
ce37c0d2d17 2018-09-17 MAPR-YARN-124. NullPointerException in IFile$Reader for Apache's shuffle implementation
c561fd8b999 2018-09-17 Backported MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith.
2c25b288632 2018-09-17 MAPRYARN-123: Job with label runs on node without proper label.
5d4137018e6 2018-09-17 MAPRYARN-125: Possibility to enable/disable multi split locations
858f75b226f 2018-09-17 Backported MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException
dfd4d41340f 2018-09-17 Backported MAPREDUCE-7139. TestShuffleProvider#testShuffleProviders and TestTaskAttemptContainerRequest#testAttemptContainerRequest fail since a static common container launch context does't recreate
530f3db816a 2018-09-17 Backported MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler
9ac645828f1 2018-09-17 MAPRYARN-15: Set default value for 'java.security.auth.login.config' if unset before. Needed for unit tests. On a cluster this property set by JVMProperties.java.
603c3195d44 2018-09-17 Backported MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li Additional commit from 2.8.0. (cherry picked from commit ee4ee6af6a5a6299d27462adb6944206039bbbae)
a03346d409f 2018-09-17 Backported MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda)
03d1f08536b 2018-09-17 Backported MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma
07b71529e9f 2018-09-17 Backported MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko
81a931b6c00 2018-09-12 Backported YARN-8197. Fixed AM IP Filter and Webapp proxy to redirect app tracking-URLs correctly when UI is secure. Contributed by Sunil Govindan.
467c77e8096 2018-09-12 Backported YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot)
c70287ef4dd 2018-09-12 MAPRYARN-114: DistCp job fails during symlink copying The cause of the problem is comparing a size of a target file with a size of a symlink to ensure that copy was successful. They are not equal. We should compare the size of the source of symlink and target file which will be copied from a source of a symlink.
7cdb92251d7 2018-09-12 Backported HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel)
c2ec6367fec 2018-09-12 Backported HADOOP-12851. S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders. Contributed by Larry McCay.
a583bb41624 2018-09-12 Backported HADOOP-12548. Read s3a creds from a Credential Provider. Contributed by Larry McCay.
74a66284ec8 2018-09-12 Backported HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel)
a6b4c204373 2018-09-12 Backported HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu)

Known Issues and Limitations

  • None.

Resolved Issues

  • None.