Spark - 2101 (EEP 7.0.1) Release Notes

This section provides reference information, including new features, patches, and known issues for Spark

The notes below relate specifically to the HPE Distribution for Apache Hadoop. For more information, you may also wish to consult the open-source Spark 2.4.7 Release Notes.

These release notes contain only HPE-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Spark Version 2.4.7
Release Date January 2021
MapR Version Interoperability See Component Versions for Released EEPs and EEP Components and OS Support.
Source on GitHub
GitHub Release Tag 2.4.7-mapr-2101
Maven Artifacts
Package Names Navigate to and select your EEP and OS to view the list of package names.
  • Beginning with EEP 6.0.0, the keyStore and trustStore password can be removed from spark-defaults.conf and set in /opt/mapr/conf/ssl-client.xml.
  • Beginning with EEP 6.0.0, after an upgrade, the previous version's configuration files are saved in the /opt/mapr/spark directory.
  • MapR 6.1.0 with EEP 6.0.0 and later support simplified security. If you enable security on your data-fabric cluster, HPE scripts automatically configure Spark security features.
  • Beginning with MapR 6.2 and EEP 7.0, Spark supports SSL for WebUI.

Hive Support

This version of Spark supports integration with Hive, but has the following exceptions:

New in This Release


This HPE release includes the following new fixes since the latest data-fabric Spark release. For details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
e1b4639 2020/09/02 MapR [SPARK-778] Delete ssl keys for spark job during spark package remove
4999284 2020/09/02 MapR [SPARK-759] Allow PAM and MapR-Sasl to be enabled at the same time
729a9a5 2020/09/07 MapR [SPARK-796] java.lang.ClassNotFoundException: Class com.mapr.fs.MapRFileSystem not found for spark-hive integration jobs
1d4ab93 2020/09/12 MapR [SPARK-797] Do not overwrite existing warden configuration files
ed0be85 2020/09/17 MapR [SPARK-795] - extended logging for Spark Streaming and Structured Streaming
547b943 2020/10/07 MapR [K8S-1882] Error on Spark SSL key generation
35f3ff5 2020/10/21 MapR [SPARK-805] Spark standalone worker can't start with enabled SSL for UI
3f7a240 2020/10/26 MapR [SPARK-703] Fix org.codehaus.jackson vulnerability
5660d09 2020/11/03 MapR [SPARK-804] sparkContext.loadFromMapRDB throwing RuntimeException in scala code
7373f82 2020/11/19 MapR [SPARK-812] Update Ant version to avoid CVE-2020-1945
2e2127a 2020/11/22 MapR [SPARK-819] CVE-2019-0205: Update Thrift version to v0.13.0
c409736 2020/11/30 [SPARK-33405][BUILD][2.4] Upgrade commons-compress to 1.20
4507ee8 2020/11/30 MapR [SPARK-822] WS-2019-0379: update commons-codec jar for spark
c3e72fd 2020/12/09 MapR [SPARK-824] CVE-2020-11612: Netty vulnerabilities
5394b4a 2020/12/29 MapR [SPARK-832] Multiple Vulnerabilities including Spark and hive jars found
187e7f2 2021/01/05 MapR [SPARK-833] Spark thriftserver can't start after the latest commit
3dd6575 2021/01/08 MapR [SPARK-812] Update Ant version to 1.10.9 to avoid CVE-2020-1945
816db59 2021/01/14 MapR [SPARK-776] IllegalArgumentException: Null user is thrown when using some methods from SparkContext
c3e4963 2021/01/15 MapR [SPARK-839] Unstable problem with security keys for job on yarn-cluster mode

Known Issues

  • None.

Resolved Issues

  • None.