Spark - 2104 (EEP 6.3.4) Release Notes

This section provides reference information, including new features, patches, and known issues for Spark 2.4.4.

The notes below relate specifically to the HPE Ezmeral Data Fabric Distribution for Apache Hadoop. For more information, you may also wish to consult the open-source Spark 2.4.4 Release Notes.

These release notes contain only HPE-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Spark Version
Release Date April 2021
MapR Version Interoperability See Component Versions for Released EEPs and EEP Components and OS Support.
Source on GitHub
GitHub Release Tag
Maven Artifacts
Package Names Navigate to and select your EEP and OS to view the list of package names.
  • Beginning with EEP 6.0.0, the keyStore and trustStore password can be removed from spark-defaults.conf and set in /opt/mapr/conf/ssl-client.xml.
  • Beginning with EEP 6.0.0, after an upgrade, the previous version's configuration files are saved in the /opt/mapr/spark directory.
  • MapR 6.1.0 with EEP 6.0.0 and later support simplified security. If you enable security on your MapR cluster, MapR scripts automatically configure Spark security features.

Hive Support

This version of Spark supports integration with Hive, but has the following exceptions:

New in This Release


This HPE release includes the following new fixes since the latest data-fabric Spark release. For details, refer to the commit log for this project in GitHub.

GitHub Commit Number Date (YYYY/MM/DD) HPE Fix Number and Description
f2fae47 2021/01/08 MapR [SPARK-833] Spark thriftserver can't start after the latest commit
55646ea 2021/01/08 MapR [SPARK-812] Update Ant version to 1.10.9 to avoid CVE-2020-1945
888a055 2021/01/27 [SPARK-32723][WEBUI] Upgrade to jQuery 3.5.1
54ffbd2 2021/04/05 MapR [SPARK-841] Backport SPARK-32723
6ee1b8a 2021/04/05 MapR [SPARK-862] Excessive messages for spark processes

Known Issues

  • SPARK-865: If you run a command to configure HBase after Spark configuration, then you must manually copy the hbase-site.xml configuration file from the HBase configuration directory to the Spark configuration directory.

Resolved Issues

  • None.