Spark - 2110 (EEP 6.3.5) Release Notes

This section provides reference information, including new features, patches, and known issues for Spark 2.4.4.

The notes below relate specifically to the HPE Ezmeral Data Fabric Distribution for Apache Hadoop. For more information, you may also wish to consult the open-source Spark 2.4.4 Release Notes.

These release notes contain only HPE-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Spark Version
Release Date October 2021
MapR Version Interoperability See Component Versions for Released EEPs and EEP Components and OS Support.
Source on GitHub
GitHub Release Tag
Maven Artifacts
Package Names Navigate to and select your EEP and OS to view the list of package names.
  • Beginning with EEP 6.0.0, the keyStore and trustStore password can be removed from spark-defaults.conf and set in /opt/mapr/conf/ssl-client.xml.
  • Beginning with EEP 6.0.0, after an upgrade, the previous version's configuration files are saved in the /opt/mapr/spark directory.
  • MapR 6.1.0 with EEP 6.0.0 and later support simplified security. If you enable security on your MapR cluster, MapR scripts automatically configure Spark security features.

Hive Support

This version of Spark supports integration with Hive, but has the following exceptions:

New in This Release


This HPE release includes the following new fixes since the latest data-fabric Spark release. For details, refer to the commit log for this project in GitHub.

GitHub Commit Number Date (YYYY/MM/DD) HPE Fix Number and Description
ce96abb 2021/05/24 MapR [SPARK-874] DStream: Spark processes queued time=t+interval batch even if time=t batch failed
1d80bac 2021/06/02 [SPARK-30225][CORE] Correct read() behavior past EOF in NioBufferedFileInputStream
53ae06b 2021/06/04 MapR [SPARK-847] Spark can't read data from symlink
87516c8 2021/06/09 MapR [SPARK-894] Hadoop artifacts should be taken from the cluster
35bb433 2021/06/10 MapR [SPARK-892] Customer's question about several CVE's impact
e0dd5d4 2021/08/12 MapR [SPARK-900] Introduce configuration option for symlink support
c6be19f 2021/09/15 MapR [SPARK-941] Problem with count via spark-shell
c442957 2021/09/17 MapR [SPARK-914] CVE-2020-13956, WS-2017-3734 vulnerabilities in http-client

Known Issues

  • None.

Resolved Issues

  • None.