Hadoop 3.3.5.500 (EEP 9.3.1) Release Notes

The notes below relate specifically to the HPE Ezmeral Data Fabric distribution of Apache Hadoop. You may also be interested in the Apache Hadoop changelog and the Apache Hadoop home page.

These release notes contain only HPE-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Version 3.3.5.500
Release Date October 2024
Version Interoperability See EEP Components and OS Support.
GitHub Source https://github.com/mapr/hadoop-common/
GitHub Release Tag 3.3.5.500-eep-931
Maven Artifacts https://repository.mapr.com/maven/
Package Names Navigate to https://package.ezmeral.hpe.com/releases/MEP/ and select your EEP and OS to view the list of package names.
Documentation

New in this Release

Hadoop 3.3.5.500 introduces the following enhancements or HPE platform-specific behavior changes:
  • CVE updates
  • Bug fixes
  • Backported critical fixes from Hadoop 3.4.0.
  • Implemented MultiDispatcher and changed the handler for the RMStateStore event to a separate event dispatcher.

Fixes

This HPE release includes the following fixes on the base Apache release:
GitHub Commit Number Date (YYYY-MM-DD) HPE Fix Number and Description
34a7f810349 2024-10-08 MAPRYARN-452: Updated Avro version to 1.11.4 due CVE-2024-47561
2446cd2454c 2024-10-05 MAPRYARN-449: Fixed typo in warning
a75cfcef9bc 2024-10-04 MAPRYARN-449: Changed aggregated logs ACE's for each application. Added property for force initializing correct ACE based on configuration
72e9e4dafdf 2024-09-27 MAPRHADOOP-486: Added colon support in file name for non-RawLocalFileSystem
9ffa696ebef 2024-09-26 Backport HADOOP-19031. Enhance access control for RunJar. (#6427). Contributed by He Xiaoqiao.
73e4aa9a17d 2024-09-17 Backport HADOOP-19262. Upgrade wildfly-openssl:1.1.3.Final to 2.1.4.Final to support Java17+ (#7032)
e56327a8cba 2024-08-30 Backport HADOOP-17581. Fix reference to LOG is ambiguous after HADOOP-17482. (#2764)
343182b74a9 2024-08-30 Backport HADOOP-17482: Remove Commons Logger from FileSystem Class (#2633)
689b7feba2d 2024-08-27 MAPRHADOOP-405: Added missing default MR configuration
4a0c50e451b 2024-08-19 MAPRYARN-357: Added retry logic for creating log aggregation app directories
d734081dd60 2024-08-19 MAPRYARN-442: Default node-attribute directory moved to MFS Yarn volume
570d1371072 2024-08-16 MAPRHADOOP-477: Check mapr-stream library exists before creating symlink
e1d49414a55 2024-08-15 MAPRYARN-447: Set to https protocol by default for daemonlog command
b35f9b0e265 2024-08-14 MAPRYARN-447: Set to yarn.admin.acl cluster admin by default
c7bcb518c6d 2024-08-12 MAPRYARN-443: Fixed build after dnsjava update
d7ab1fe7cf2 2024-08-09 MAPRYARN-443: Updated Jetty to 9.4.55.v20240627
f3bfabf8dcb 2024-08-09 MAPRYARN-443: Updated Netty to 4.1.112.Final
a5b97abf2e8 2024-08-09 Backport HADOOP-19237. Upgrade to dnsjava 3.6.1 due to CVEs (#6961) (#6971)
d68cf6dfeba 2024-08-09 Backport HADOOP-17317. [JDK 11] Upgrade dnsjava to remove illegal access warnings (#2442) (#6981)
f8a6d8c7710 2024-08-07 MAPRYARN-445: Application attempt resource usage is tracked incorrectly for node labels enabled
1285296c1b4 2024-08-05 MAPRYARN-437: Default node-labels directory moved to MFS Yarn volume
90fc0534f50 2024-08-01 MAPRYARN-424: Fixed issue with processing the latest event during stopping MultiDispatcherExecutor
e732582c96a 2024-08-01 MAPRYARN-424: Added MultiDispatcher and moved RMStateStore event from AsyncDispatcher to MultiDispatcher
c5d3c9b55d8 2024-07-31 MAPRYARN-441: Fixed duplicates in node-labels map
11eabeca5d3 2024-07-30 MAPRYARN-440: Added SIGQUIT before SIGKILL to generate dump and increased default HADOOP_STOP_TIMEOUT to 60 sec
002eb3674ba 2024-07-29 MAPRYARN-438: YARN scheduler regenerating map of all labels for all nodes on every scheduling decision
1d05db81ad5 2024-07-17 Backport HADOOP-19216. Upgrade Guice from 4.0 to 5.1.0 to support Java 17 (#6913). Contributed by Cheng Pan.
fbbcb3972da 2024-07-16 Backport HADOOP-19084. Prune hadoop-common transitive dependencies (#6574)
4808818a759 2024-07-16 MAPREDUCE-7457: Added support to limit count of spill files (#6155) Contributed by Mudit Sharma.
4ac41f88082 2024-07-16 Backport HADOOP-18824. ZKDelegationTokenSecretManager causes ArithmeticException due to improper numRetries value checking (#6052)
0762dfc8fe2 2024-07-16 Backport HADOOP-18751. Fix incorrect output path in javadoc build phase (#5688)
5a278abe243 2024-07-16 Backport MAPREDUCE-7428. Fix failing MapReduce tests due to the JUnit upgrades in WebServicesTestUtils (#5243)
5c63fd9d221 2024-07-16 Backport MAPREDUCE-7428. Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp (#5209)
d6ba705be98 2024-07-16 Backport YARN-11386. Fix issue with classpath resolution (#5183)
f9fd637eb89 2024-07-16 Backport HADOOP-18120. Hadoop auth does not handle HTTP Headers in a case-insensitive way. Contributed by Janos Makai.
032efa72dfd 2024-07-16 Backport HADOOP-17901. Performance degradation in Text.append() after HADOOP- 16951 (#3411)
9c7440010a8 2024-07-16 Backport YARN-10739. GenericEventHandler.printEventQueueDetails causes RM recovery to take too much time. Contributed by Qi Zhu.
74f508c3754 2024-07-16 Backport YARN-9618. NodesListManager event improvement. Contributed by Qi Zhu.
b9475e8b487 2024-07-16 Backport YARN-10380: Import logic of multi-node allocation in CapacityScheduler (#2494)
53c091952ea 2024-07-16 Backport HADOOP-16951: Tidy Up Text and ByteWritables Classes.
66cfb2f9289 2024-07-16 Backport YARN-10233. Fix YARN UI2 Daemon Logs
40d2f538f82 2024-07-16 Backport YARN-10219. Fix YARN Native Service Placement Constraints with Node Attributes.
174b35b12e1 2024-07-16 Backport HADOOP-16958. NPE when hadoop.security.authorization is enabled but the input PolicyProvider for ZKFCRpcServer is NULL. Contributed by Ctest.

Known Issues and Limitations

Hadoop 3.3.5.500 has the following known issues and limitations:

  • Multiple labels are not supported within the same node or queue.

Resolved Issues

None.